FILE.TAIL([{lines}] would be good.
Re: FILE.TAIL([{lines}] would be good.
Thanks all, for the great responses. I'm definitely going to look at the RTC storage, as that seems simplest to implement. And thanks for the code samples for scanning the file, interesting ideas!
Thank You!
Thank You!
- Fernando Perez
- Posts: 378
- Joined: Mon Feb 15, 2021 10:09 pm
- Location: Santander (Spain)
- Has thanked: 195 times
- Been thanked: 267 times
Re: FILE.TAIL([{lines}] would be good.
Thanks to you and I assure you that Annex returns in satisfaction the time you dedicate.
But I reopen this topic because I need Cicciocb or someone who has used the "bas.rtcmem$" statement to clarify a very relevant question for me:
Does the data written to the RTC ram survive a complete system shutdown (power disconnection) or just a reset?
But I reopen this topic because I need Cicciocb or someone who has used the "bas.rtcmem$" statement to clarify a very relevant question for me:
Does the data written to the RTC ram survive a complete system shutdown (power disconnection) or just a reset?
- Fernando Perez
- Posts: 378
- Joined: Mon Feb 15, 2021 10:09 pm
- Location: Santander (Spain)
- Has thanked: 195 times
- Been thanked: 267 times
Re: FILE.TAIL([{lines}] would be good.
I thought so, but I use this:
And I save and execute the file.
Then I comment on the first line and repeat the save and run procedure.
If I do a reset, it works, but if I disconnect the ESP8266 and reconnect it, the Wifi connection breaks and wlog does not show anything.
Code: [Local Link Removed for Guests]
bas.RtcMem$ = "Pepe y pepito"
a$ = bas.rtcmem$
wlog a$
Then I comment on the first line and repeat the save and run procedure.
Code: [Local Link Removed for Guests]
'bas.RtcMem$ = "Pepe y pepito"
a$ = bas.rtcmem$
wlog a$
-
- Posts: 143
- Joined: Mon Feb 08, 2021 10:10 pm
- Location: Scotland
- Has thanked: 47 times
- Been thanked: 51 times
Re: FILE.TAIL([{lines}] would be good.
Ok - I think I had better keep quiet and wait for an expert to answer the question - sorry.
- cicciocb
- Site Admin
- Posts: 2069
- Joined: Mon Feb 03, 2020 1:15 pm
- Location: Toulouse
- Has thanked: 441 times
- Been thanked: 1364 times
- Contact:
Re: FILE.TAIL([{lines}] would be good.
[Local Link Removed for Guests] wrote: [Local Link Removed for Guests]Tue Apr 27, 2021 5:47 pm Ok - I think I had better keep quiet and wait for an expert to answer the question - sorry.
Bugs don't worry, everyone makes mistakes the important is to recognise them
Yes, this function survives at the reset but not at the power off.
However, for non-volatile information, the best is always to save into a file
Re: FILE.TAIL([{lines}] would be good.
Hi All,
Thanks again for all the ideas, and I've come up with one that seems pretty quick, regardless of file size or the need to pre-count lines.
So I basically work back from the file size until I find a carriage return (13), but could easily modify to be line-feed (10), and I have my last line. Cost in loops, is just the length of the last line. Appears to take around 87 milliseconds (Or 147 milliseconds on the test file Fernando Perez posted, his lines are longer...). I'm sure it hits the string handling a bit hard, as I'm adding the next character to the front of the string. I'm sure it's not optimal for long lines either. But I am now thinking of changing the log file to have a fixed length, then I can do this:
Where 27 is the length of the string (plus the CR), which takes 48 milliseconds on my file (modified for a fixed line length).
Thanks all, for making me use my noodle.
Thanks again for all the ideas, and I've come up with one that seems pretty quick, regardless of file size or the need to pre-count lines.
Code: [Local Link Removed for Guests]
line$=""
fchr$=""
pos=FILE.SIZE(logfile$)-1
while fchr$<>chr$(13)
pos=pos-1
line$=fchr$+line$
fchr$=FILE.READ$(logfile$,pos,1)
wend
wlog line$
Code: [Local Link Removed for Guests]
line$ = FILE.READ$(logfile$,file.size(logfile$)-27,27)
Thanks all, for making me use my noodle.
- Fernando Perez
- Posts: 378
- Joined: Mon Feb 15, 2021 10:09 pm
- Location: Santander (Spain)
- Has thanked: 195 times
- Been thanked: 267 times
Re: FILE.TAIL([{lines}] would be good.
I discard the use of bas.rtcmem$ for not surviving between full shutdowns and go back to the filesystem.
I have written this SUB:
and I was surprised that in a short file it takes a few milliseconds, but (and I don't understand why) when I increase the number of rows in the file to 1,000, 3,000 and 6,000, the time increases to 2.5 seconds and stays that way regardless of the size of the file.
I will test your code to check speed in very large files
I have written this SUB:
Code: [Local Link Removed for Guests]
line$ = ""
t = 0
wlog "start..."
lastLine "/msft_250.csv", line$
wlog "250 lines processed in "; millis - t; " millliseconds"
lastLine "/msft_1000.csv", line$
wlog "1000 lines processed in "; millis - t; " millliseconds"
lastLine "/msft_3000.csv", line$
wlog "3000 lines processed in "; millis - t; " millliseconds"
lastLine "/msft_6000.csv", line$
wlog "6000 lines processed in "; millis - t; " millliseconds"
wlog chr$(13);"...finish"
END
SUB lastLine(fileName$, line$)
LOCAL a$, lastByte
line$ = ""
t = millis ' for debug, delete in final version
lastByte = file.Size(fileName$) - 1
ioBuff.dim(0, 1)
do
file.read_ioBuff(0), filename$, lastByte, 1
a$ = ioBuff.ToString$(0)
line$ = a$ + line$
lastByte = lastByte - 1
loop until a$ = chr$(10)
ioBuff.destroy(0)
wlog line$ ' for debug, delete in final version
END SUB
You do not have the required permissions to view the files attached to this post.
- Fernando Perez
- Posts: 378
- Joined: Mon Feb 15, 2021 10:09 pm
- Location: Santander (Spain)
- Has thanked: 195 times
- Been thanked: 267 times
Re: FILE.TAIL([{lines}] would be good.
This is the result:
If you want my advice, you'd better use the variable length file line system, so the code will be reusable by other people and in other situations.
Code: [Local Link Removed for Guests]
line$ = ""
t = 0
wlog "start..."
logfile$ = "/msft_250.csv" : gosub lastline
wlog "250 lines processed in "; millis - t; " millliseconds"
logfile$ = "/msft_1000.csv" : gosub lastline
wlog "1000 lines processed in "; millis - t; " millliseconds"
logfile$ = "/msft_3000.csv" : gosub lastline
wlog "3000 lines processed in "; millis - t; " millliseconds"
logfile$ = "/msft_6000.csv" : gosub lastline
wlog "6000 lines processed in "; millis - t; " millliseconds"
wlog "...finish"
END
lastLine:
t = millis ' for debug, delete in final version
line$ = ""
fchr$ = ""
pos = FILE.SIZE(logfile$) - 1
while fchr$ <> chr$(10)
pos = pos - 1
line$ = fchr$ + line$
fchr$ = FILE.READ$(logfile$, pos, 1)
wend
wlog line$ ' for debug, delete in final version
return
You do not have the required permissions to view the files attached to this post.
- Electroguard
- Posts: 867
- Joined: Mon Feb 08, 2021 6:22 pm
- Has thanked: 278 times
- Been thanked: 324 times
Re: FILE.TAIL([{lines}] would be good.
Touching on some earlier comments...
Non-volatile memory should be non-volatile, else it defeats the purpose of being non-volatile memory.
From my own very volatile biological memory of more than a year ago I seem to remember that RTCMEM$ did actually retain contents through a power-up, but that they got jumbled up, which suggested that it was actually non-volatile but that there was a code hiccup in the handling of the memory, perhaps inherited from arduino.
I know I had to abandon a useful snippet for saving variables such as reboots=nn (rolling count of the number of device reboots) into RTCMEM$ using word.setparam$ and word.getparam$, but it was easy enough just to do the same to a file instead of the unreliable volatile memory, so was more of a disappointment than a problem.
rtcmem$ = bas.rtcmem$
restarts$ = word.getparam$(rtcmem$,"Restarts")
restarts$ = str$(val(restarts$) + 1)
word.setparam rtcmem$, "Restarts", restarts$
bas.rtcmem$ = rtcmem$
...
SAVE:
word.setparam rtcmem$, "Script", bas.filename$
word.setparam rtcmem$, "Breadcrumb", breadcrumb$
word.setparam rtcmem$, "Time", time$
word.setparam rtcmem$, "Date", date$
word.setparam rtcmem$, "Ramfree", str$(ramfree)
word.setparam rtcmem$, "Flashfree", str$(flashfree)
word.setparam rtcmem$, "Restarts", restarts$
word.setparam rtcmem$, "Cause", cause$
' word.setparam rtcmem$, "Err num", str$(bas.errnum)
' word.setparam rtcmem$, "Err msg", bas.errmsg$
' word.setparam rtcmem$, "Err line", str$(bas.errline)
bas.rtcmem$ = rtcmem$
return
Non-volatile memory should be non-volatile, else it defeats the purpose of being non-volatile memory.
From my own very volatile biological memory of more than a year ago I seem to remember that RTCMEM$ did actually retain contents through a power-up, but that they got jumbled up, which suggested that it was actually non-volatile but that there was a code hiccup in the handling of the memory, perhaps inherited from arduino.
I know I had to abandon a useful snippet for saving variables such as reboots=nn (rolling count of the number of device reboots) into RTCMEM$ using word.setparam$ and word.getparam$, but it was easy enough just to do the same to a file instead of the unreliable volatile memory, so was more of a disappointment than a problem.
rtcmem$ = bas.rtcmem$
restarts$ = word.getparam$(rtcmem$,"Restarts")
restarts$ = str$(val(restarts$) + 1)
word.setparam rtcmem$, "Restarts", restarts$
bas.rtcmem$ = rtcmem$
...
SAVE:
word.setparam rtcmem$, "Script", bas.filename$
word.setparam rtcmem$, "Breadcrumb", breadcrumb$
word.setparam rtcmem$, "Time", time$
word.setparam rtcmem$, "Date", date$
word.setparam rtcmem$, "Ramfree", str$(ramfree)
word.setparam rtcmem$, "Flashfree", str$(flashfree)
word.setparam rtcmem$, "Restarts", restarts$
word.setparam rtcmem$, "Cause", cause$
' word.setparam rtcmem$, "Err num", str$(bas.errnum)
' word.setparam rtcmem$, "Err msg", bas.errmsg$
' word.setparam rtcmem$, "Err line", str$(bas.errline)
bas.rtcmem$ = rtcmem$
return