cause of a memory error?

[Stuart]

This piece of code:

Code: [Local Link Removed for Guests]

sub listvars
wlog "========================================="
wlog "   PROGRAM NAMES IN FILE: "+ fname$
wlog "========================================="
wlog "no of variable and subprogram names: "+str$(varcount)
for i = 1 to varcount
tablength = 20-len(varname$(i,1))
tstext$=varname$(i,1)+space$(tablength)+varname$(i,2)
wlog tstext$
next
end sub

causes an out of memory error whilst listing the data in an array. Is there anything in it that accumulates memory without releasing it? I would have thought not but it always causes a failure when the varname$ array is large. It is part of my parser program which manages to scan large files but can't manage the listing of the result from them.

[cicciocb]

Yes, the WLOG command.

Put a pause after the wlog or use PRINT instead

[Stuart]

The problem with Print for a linux user like me is that I don't know where the output comes out. At least wlog appears in the output panel in the browser, which is why i use it. How do I direct Print to a file? or a terminal window? or even a printer?

[cicciocb]

You can use any terminal emulator or the AnnexToolkit that contains a terminal emulator inside.

Or .... put a pause after the wlog.

edit :
or you can use the online web terminal [Local Link Removed for Guests]

[Stuart]

I suppose the reason that I use wlog rather than print is that wlog works over the wifi, one of the great reasons for using the annex ide, and as far as I can tell despite efforts, print doesn't work unless there is a usb connection. I can use udp, though that is a fiddle.

It seems that Annex can lose connection if you give the ESP32 too much processing to do, it doesn't seem to make enough time for itself to keep the background work up to date. Some advice on how much pausing to give it and how often might make staying connected easier.

My parsing app, of which this is an extract and which I wrote for Annex as that seemed a good idea, simply reads lines from a file, scans for variable names, and writes them to an array for later printing. It doesn't do anything clever like sorting them and doesn't need to be very efficient. But it gets indigestion after a few hundred lines. I see that, for each line read and processed, it usually uses and does not release a big chunk of memory. This varies from none at all to over 220 bytes, averaging over 200 bytes per line of code read. Yet the only space I would expect it to retain (if it finds something it needs to store) would be enough to save any first-time-found variable names, and their line numbers. Lets say for first occurrences of a variable, 15 bytes and for each occurrence say max 7 bytes for the line number. Unlikely to be more than 50 bytes per line and often none at all.

[cicciocb]

Using the WLOG command at high rate can saturate the memory as the module buffer the message in memory and then send it via the websockets.

So, try to limit the rate of the messages sent using wlog.
When pushing too much messages, the buffer (and the memory) will be full rapidly

[Zim]

Hi cicciocb

how much of a pause would you recommend after each "wlog"?

Thanks
Zim

[Electroguard]

Going back some years, scripts sending too much out to the browser would overload the output buffer and cause the script to error.
The 'fix' was to send info out in smaller mouthfuls.
So instead of listing all the a$=a$+"info" lines then html a$ at the end, it was better to send to the browser after about a dozen a$=a$+"info" lines:
a$=a$+"thats a good mouthful"
html a$
pause 250
a$=""
a$=a$+"start of next mouthful"
That pause of 250 milliseconds was enough for the browser to eat everything in smaller mouthful without getting indigestion.
So if that sort of delay worked for mouthfuls of html output, it may possibly also be ok for mouthfuls of wlog output.
But I wouldn't have thought it would be necessary after every wlog... try sending mouthfuls of wlogs in batches.

[cicciocb]

There is no particular requirement as all depends on the free available memory, the quality of the WIFI connection an the size of the information to send. Because the transmission is acknowledged from the receiving side, if the receiver doesn't receive correctly, the same package is sent several time or can stall for a given time.

So, as a general rule, is better to send few big block of data then many little packets; for "big size" I means ~4KB.

The optimum size should be 1460bytes (it fits inside a single packet where MTU is typically 1500)

In the first versions of Annex, the 250ms pause between each package was a "limitation" but this limit has been removed so that now you can now send multiple packages in sequence BUT not for a long time.

So, again, as a general rule, try to put a pause after sending data (I mean using WLOG, HTML, CSS, ....)

Try to consider WLOG as a debug mean that must be removed from the program (the command OPTION.WLOG 0 is your friend)

As a good alternative, you can still use the serial port (but I know that this requires a physical connection with the module) or UDP (that is a connectionless transmission)