Very slow FATFS write on ESP32 v ESP8266.

Here we can discuss about the problem found
Post Reply
AndyGadget
Posts: 111
Joined: Mon Feb 15, 2021 1:44 pm
Has thanked: 41 times
Been thanked: 81 times

Very slow FATFS write on ESP32 v ESP8266.

Post by AndyGadget »

Running the same code on ESP8266 and ESP32 shows a write time over 50 times slower on ESP32 when writing a 100 line file to FATFS.
The ESP32 timing is the same with a can.stop command.
I have attached the test file I was using for this.

ESP8266
Started at 22:53:26 on 24/07/21
Reading file
Writing file
Read time : 2.069 seconds.
Write time : 0.393 seconds.

ESP32
Started at 22:52:31 on 24/07/21
Reading file
Writing file
Read time : 1.4 seconds.
Write time : 16.382 seconds.

Code: [Local Link Removed for Guests]

wlog chr$(10)
wlog "Started at " + Time$ + " on " + Date$

EOL$ = chr$(10)
FName$ = "/testfile.txt"
dim MyArray$(100)

gosub ReadFile
gosub WriteFile
readtime$ = str$((TEndRead - TStartRead) /1000)
writetime$ = str$((TEndWrite - TStartWrite)/1000)

wlog "Read time  : " + readtime$ + " seconds."
wlog "Write time : " + writetime$ + " seconds."
end

ReadFile:
wlog "Reading file"
TStartRead = millis

for FCnt = 1 to 100
  FLine$ = file.read$(FName$,FCnt)
  MyArray$(FCnt) = FLine$
next FCnt

TEndRead = millis
return

WriteFile:
wlog "Writing file"
x =  file.delete(FName$)
TStartWrite = millis

for FCnt = 1 to 100
  FLine$ = MyArray$(FCnt) + EOL$
  file.append FName$,FLine$
next FCnt

TEndWrite = millis
return
You do not have the required permissions to view the files attached to this post.
cicciocb
Site Admin
Posts: 403
Joined: Mon Feb 03, 2020 1:15 pm
Location: Toulouse
Has thanked: 90 times
Been thanked: 294 times
Contact:

Re: Very slow FATFS write on ESP32 v ESP8266.

Post by cicciocb »

Hi Andy,
thanks for raising this point.

I just checked and I confirm this behaviour on the ESP32.

The C++ code is exactly the same for both chip but the ESP32 uses the FAT32 instead of the SPIFFS for the ESP8266.
Normally the FAT32 should be faster than the SPIFFS ......

I'll try to understand why this happens.

EDIT :

It seems a known problem :
[External Link Removed for Guests]
AndyGadget
Posts: 111
Joined: Mon Feb 15, 2021 1:44 pm
Has thanked: 41 times
Been thanked: 81 times

Re: Very slow FATFS write on ESP32 v ESP8266.

Post by AndyGadget »

Just a bit of additional info :
FILE.COPY on the same file takes half a second.
AndyGadget
Posts: 111
Joined: Mon Feb 15, 2021 1:44 pm
Has thanked: 41 times
Been thanked: 81 times

Re: Very slow FATFS write on ESP32 v ESP8266.

Post by AndyGadget »

Francesco, just thinking a bit more about this :
Does the FATFS emulation use the equivalent of a FILE OPEN before a write and a FILE CLOSE afterwards, or is it simply a series of ad-hoc writes to a reserved block of memory given a symbolic filename?
I'm wondering whether having a specific FILE.OPEN command then a series of writes ending in a FILE.CLOSE would speed things up.
I'm talking about situations where doing this with IOBUFFERS would be too unwieldly.
(But there again, you've probably already thought of that ;) )
cicciocb
Site Admin
Posts: 403
Joined: Mon Feb 03, 2020 1:15 pm
Location: Toulouse
Has thanked: 90 times
Been thanked: 294 times
Contact:

Re: Very slow FATFS write on ESP32 v ESP8266.

Post by cicciocb »

[Local Link Removed for Guests] wrote: [Local Link Removed for Guests]Thu Sep 09, 2021 12:13 pm Francesco, just thinking a bit more about this :
Does the FATFS emulation use the equivalent of a FILE OPEN before a write and a FILE CLOSE afterwards, or is it simply a series of ad-hoc writes to a reserved block of memory given a symbolic filename?
I'm wondering whether having a specific FILE.OPEN command then a series of writes ending in a FILE.CLOSE would speed things up.
I'm talking about situations where doing this with IOBUFFERS would be too unwieldly.
(But there again, you've probably already thought of that ;) )
Yes, In the code there is a file open / save / close.
Probably using an approach based on file.open could speedup but this will involve other problems
Post Reply