ANNEX-CAM: vfs_fat: open: no free file descriptors

Here we can discuss about the problem found
Post Reply
User avatar
PeterN
Posts: 366
Joined: Mon Feb 08, 2021 7:56 pm
Location: Krefeld, Germany
Has thanked: 171 times
Been thanked: 203 times
Contact:

ANNEX-CAM: vfs_fat: open: no free file descriptors

Post by PeterN »

Hi Francesco

I am in the process of extending a demo script for the ANNEX CAM and have a serious problem.
When I try to output some locally stored CAM images via html, the system crashes.
The images are displayed on ONE single page and when the browser tries to open them in parallel, ANNEX crashes with a note about missing file descriptors.
Since the browser usually caches the images, the problem was somewhat hidden until I prevented the browser from caching.
Annex initializing...
PSRAM enabled
module Type 201 SD_Card 2
Camera Enabled
OK
No SD cards found. Reverts to FFAT

AnnexCAM DET WiFi 1.43.3
(C) ciccioCB 2021
Personal use only, commercial use strictly prohibited
Chip revision 1 Freq 240
STARTING...
Connecting.......
Connected to NELAN1_2.4GHz
IP address: 192.168.0.162
Number of program lines :320
Basic File Loaded: /cam6.bas
Program Running
0
0
0
ws[/ws][1][1] connect
file requested /demo2.css
file requested /PIC_1.jpg
file requested /PIC_2.jpg
file requested /PIC_3.jpg
file requested /PIC_4.jpg
file requested /PIC_5.jpg
file requested /PIC_6.jpg
E (35023) vfs_fat: open: no free file descriptors
File Not Found
You can see the problem if pressing the button "make FOTO" in my code.
This will force to show the stored fotos in a new order without caching.

Of course, the problem is intensified when there are two browsers.

Code: [Local Link Removed for Guests]

VERSION$="V6.4"
option.wdt 0
option.wlog 0

CALL$ = "DG8JA"
'CALL$ = "DB9JG"

PIC_NUM = 6  ' number of old pics to show

INFO$= CALL$ + "-CAM " +VERSION$ 

SHOW_PARAMS      = 0
SHOW_PARAMS_COL$ = "butblue"
SHOW_PARAMS_TXT$ = "SETTINGS=OFF"

MAKE_FOTO_COL$   = "butblue"
MAKE_FOTO_TXT$   = "make FOTO"
FOTO$="/PIC_1.jpg"

MAKE_UNIQUE$ = str$(millis)  

pin.mode 33, output
pin.mode 4, output
pin(33) = 0  'rote LED AUS
pin(4)  = 1  'WEISSE LED AN
wifi.power 20
rate = 50
'0    FRAMESIZE_QQVGA,    // 160x120
'1    FRAMESIZE_QQVGA2,   // 128x160
'2    FRAMESIZE_QCIF,     // 176x144
'3    FRAMESIZE_HQVGA,    // 240x176
'4    FRAMESIZE_QVGA,     // 320x240
'5    FRAMESIZE_CIF,      // 400x296
'6    FRAMESIZE_VGA,      // 640x480
'7    FRAMESIZE_SVGA,     // 800x600
'8    FRAMESIZE_XGA,      // 1024x768
'9    FRAMESIZE_SXGA,     // 1280x1024
'10   FRAMESIZE_UXGA,     // 1600x1200
'11   FRAMESIZE_QXGA,     // 2048*1536

' set the max resolution in terms of memory
'print camera.setup(9)
If CALL$ = "DG8JA" then
  print camera.params("vflip", 1)
  print camera.params("hmirror", 1)
else
  print camera.params("vflip", 0)
  print camera.params("hmirror", 0)
endif

pause 100
pin(4)=0   'WEISSE LED WIEDER  AUS

'gosub res_800x600
gosub res_1024x768
'gosub res_1280x1024

' setting variables
quality     = 10
brightness  = 0
contrast    = 0
saturation  = 0
awb         = 1
awb_gain    = 1
wb_mode     = 0
aec         = 0
aec2        = 1
ae_level    = 0
agc         = 0
agc_gain    = 0
gainceiling = 0

A$          = ""
DateTime$   = ""
LED_ON_OFF_COL$ = "butblue"
LED_ON_OFF_TXT$ = "LED=OFF"


if not (file.exists(FOTO$)) then print CAMERA.PICTURE(FOTO$) ' save the first picture


''''''''''''''''''
onHtmlReload setpage
gosub        setpage
onHtmlChange paramchange

timer0 1000, UPDATE
wait

'#########################################################
UPDATE:
option.wdtreset
T$ = time$
DateTime$ = "20" + date$(2) + "_" + T$
DateTime$ = replace$(DateTime$,"/","-")
DateTime$ = replace$(DateTime$,":","-")
return

'############################################
setpage:
cssexternal "/demo2.css"
cls
autorefresh 1000
a$ = ""
'A$ = A$ + |<h1> AnnexCAM Demo</h1>|
A$ = A$ + |<table><td>|
A$ = A$ + |<img id='camera' src="picture" alt="AnnexCam" style="width:100%;"><br>|
A$ = A$ + |<td>|
if SHOW_PARAMS = 1 then
  A$ = A$ + |<table>|
  A$ = A$ + slider$(quality, 10, 63, "slider") + "<td>jpg-Compression (10 .. 63) "
  A$ = A$ + |<tr><td>|
  A$ = A$ + slider$(brightness, -2, 2, "slider2") + "<td>Brightness(-2 .. 2) "
  A$ = A$ + |<tr><td>|
  A$ = A$ + slider$(contrast, -2, 2, "slider") + "<td>Contrast(-2 .. 2) "
  A$ = A$ + |<tr><td>|
  A$ = A$ + slider$(saturation, -2, 2, "slider2") + "<td>Saturation(-2 .. 2) "
  A$ = A$ + |<tr><td>|
  'A$ = A$ + checkbox$(awb) + "AWB"
  'A$ = A$ + |<tr><td>|
  'A$ = A$ + checkbox$(awb_gain) + "<td>AWB gain"
  
  ' this is a checkbox using the variable 'awb_gain' (data-var='awb_gain')
  A$ = A$ + |<div class="onoffswitch">|
  A$ = A$ + |    <input type="checkbox" data-var='awb_gain' onchange='cmdChange(event)' class="onoffswitch-checkbox" id="mycheck1" checked>|
  A$ = A$ + |    <label class="onoffswitch-label" for="mycheck1">|
  A$ = A$ + |        <span class="onoffswitch-inner"></span>|
  A$ = A$ + |        <span class="onoffswitch-switch"></span>|
  A$ = A$ + |    </label>|
  A$ = A$ + |</div>|
  A$ = A$ + |<td> AWB gain|
  
  
  A$ = A$ + |<tr><td>|
  A$ = A$ + slider$(wb_mode, 0, 4, "slider") + "<td>WB mode(0 .. 4) "
  A$ = A$ + |<tr><td>|
  'A$ = A$ + checkbox$(aec) + "AEC "
  'A$ = A$ + |<tr><td>|
  A$ = A$ + slider$(ae_level, -2, 2, "slider2" ) + "<td>AE Level"
  A$ = A$ + |<tr><td>|
  
  ' this is a checkbox using the variable 'aec2' (data-var='aec2')
  A$ = A$ + |<div class="onoffswitch">|
  A$ = A$ + |    <input type="checkbox" data-var='aec2' onchange='cmdChange(event)' class="onoffswitch-checkbox" id="mycheck2" checked>|
  A$ = A$ + |    <label class="onoffswitch-label" for="mycheck2">|
  A$ = A$ + |        <span class="onoffswitch-inner2"></span>|
  A$ = A$ + |        <span class="onoffswitch-switch"></span>|
  A$ = A$ + |    </label>|
  A$ = A$ + |</div>|
  A$ = A$ + |<td> AEC 2| 
  A$ = A$ + |</table>|
  A$ = A$ + button$("640x480"  , res_640x480, "butred") + "<br>"
  A$ = A$ + button$("800x600"  , res_800x600, "butred") + "<br>"
  A$ = A$ + button$("1024x768" , res_1024x768, "butred")+ "<br>"
  A$ = A$ + button$("1280x1024", res_1280x1024, "butred")+"<br>"
'  A$ = A$ + button$("1600x1200", res_1600x1200, "butred") + "<br>"
  A$ = A$ + textbox$(RES$, "tbox")+  "<br><br>"
  A$ = A$ + button$("V Flip", v_flip, "butblue") + " _ " + button$("H mirror", h_mirror, "butblue") 
  A$ = A$ + |</table>|
else
  A$ = A$ + |<table>|
'  A$ = A$ + " <br>"
  A$ = A$ + |<table>|
'  A$ = A$ + " <br>"
  A$ = A$ + |</table>|
  A$ = A$ + |</table>|
  
endif

A$ = A$ + textbox$(INFO$, "tbox") + " "+ textbox$(DateTime$, "tbox") + " "
A$ = A$ + textbox$(RES$, "tbox") + "<br>"
A$ = A$ + button$(LED_ON_OFF_TXT$, LED_ON_OFF, LED_ON_OFF_COL$)+ " "
A$ = A$ + button$(SHOW_PARAMS_TXT$, SHOW_PARAMS_ON_OFF, SHOW_PARAMS_COL$)+ " "
A$ = A$ + button$(MAKE_FOTO_TXT$, MAKE_FOTO, MAKE_FOTO_COL$)

' Force the browser one time to fetch the NEW  foto 
' not from his cache, by adding a fresh unique "?dummy=xxxxx" to the image name 
A$ = A$ + "<br><br>" 

for I = 1 to PIC_NUM
  A$= A$+|<img src='/PIC_|+ str$(I) +|.jpg?dummy=|+ MAKE_UNIQUE$ + |' width=32%>_|
next I


'A$ = A$ + "<br><br>" + replace$(image$(FOTO$),"jpg","jpg?dummy="+ MAKE_UNIQUE$)
html a$

jscall "set_pictimer(" + str$(rate) + ");"


'autorefresh 1000
a$ = ""
return

'############################################
LED_ON_OFF:
pin(4)=1-pin(4)
if pin(4)= 0 then
  LED_ON_OFF_COL$ = "butblue"
  LED_ON_OFF_TXT$ = "LED=OFF"
else
  LED_ON_OFF_COL$ = "butred"
  LED_ON_OFF_TXT$ = "LED=ON"
endif
gosub setpage
return


'############################################
MAKE_FOTO:
MAKE_FOTO_TXT$="--WAIT---"
MAKE_FOTO_COL$="butred"
gosub setpage

FOTO$="/PIC_"+STR$(PIC_NUM+1)+".jpg"
if file.exists(FOTO$) then print file.delete(FOTO$)
for I = (PIC_NUM) to 1 step -1
 print file.rename("/PIC_"+STR$(I)+".jpg","/PIC_"+STR$(I+1)+".jpg")
next I
FOTO$="/PIC_1.JPG"

cls
autorefresh 0
print CAMERA.PICTURE(FOTO$) ' save the picture



MAKE_FOTO_TXT$="make FOTO"
MAKE_FOTO_COL$="butblue"
MAKE_UNIQUE$ = str$(millis) 

gosub setpage
return


'############################################
SHOW_PARAMS_ON_OFF:
SHOW_PARAMS = 1-SHOW_PARAMS
IF SHOW_PARAMS = 0 then
  SHOW_PARAMS_COL$ = "butblue"
  SHOW_PARAMS_TXT$ = "SETTINGS=OFF"
else
  SHOW_PARAMS_COL$ = "butred"
  SHOW_PARAMS_TXT$ = "SETTINGS=ON"
endif
gosub setpage
refresh
return

'############################################
res_320x240:
print camera.params("framesize", 4)
RES$="320x240"

return

'############################################
res_640x480:
print camera.params("framesize", 6)
RES$="640x480"
return

'############################################
res_800x600:
print camera.params("framesize", 7)
RES$="800x600"
return

'############################################
res_1024x768:
print camera.params("framesize", 8)
RES$="1024x768"
return

'############################################
res_1280x1024:
print camera.params("framesize", 9)
RES$="1280x1024"
return

'############################################
res_1600x1200:
print camera.params("framesize", 10)
RES$="1600x1200"
return


'############################################
v_flip:
v = camera.getvalue("vflip")
print camera.params("vflip", 1 - v)
return

'############################################
h_mirror:
v = camera.getvalue("hmirror")
print camera.params("hmirror", 1 - v)
return

'############################################
paramchange:
print HtmlEventVar$
v$ = HtmlEventVar$
if v$ = "quality"     then print camera.params("quality", quality)
if v$ = "brightness"  then print camera.params("brightness", brightness)
if v$ = "contrast"    then print camera.params("contrast", contrast)
if v$ = "saturation"  then print camera.params("saturation", saturation)
if v$ = "awb"         then print camera.params("awb", awb)
if v$ = "awb_gain"    then print camera.params("awb_gain", awb_gain)
if v$ = "wb_mode"     then print camera.params("wb_mode", wb_mode)
if v$ = "aec"         then print camera.params("aec", aec)
if v$ = "aec2"        then print camera.params("aec2", aec2)
if v$ = "ae_level"    then print camera.params("ae_level", ae_level)

if v$ = "agc"         then print camera.params("agc", agc)
if v$ = "agc_gain"    then print camera.params("agc_gain", agc_gain)
if v$ = "gainceiling" then print camera.params("gainceiling", gainceiling)

return
User avatar
cicciocb
Site Admin
Posts: 1900
Joined: Mon Feb 03, 2020 1:15 pm
Location: Toulouse
Has thanked: 407 times
Been thanked: 1271 times
Contact:

Re: ANNEX-CAM: vfs_fat: open: no free file descriptors

Post by cicciocb »

Hi Peter,
this message is generated because the browser try to open simultaneously too many files.
I cannot really consider that this is a real bug but more a limitation of Annex.

The number of files opened at the same time is limited in Annex32 at 5 and, as per the information I found on internet, most of the browsers opens up-to 6
connections on the same server.

I've already been faced with this kind of problem and the unique solution I found at that time was to develop a special javascript that limited the access at one file at the time. This was, in particular, for ESP8266 that has much more limited resources.
You can find the code in the mpu6050_demo.html example (in this case the javascript libraries are loaded in sequence).

I could try to find a way to avoid the crash increasing the number of files to be open simultaneously but this will probably not be enough, in particular if you open several pages at the same time. Additionally this will waste more precious RAM.

So probably the unique viable solution is to implement some javascript in the html page or force the number of simultaneous connection of the browser using some command line options (not sure if this is still possible).

I'll let you know.
cicciocb
User avatar
PeterN
Posts: 366
Joined: Mon Feb 08, 2021 7:56 pm
Location: Krefeld, Germany
Has thanked: 171 times
Been thanked: 203 times
Contact:

Re: ANNEX-CAM: vfs_fat: open: no free file descriptors

Post by PeterN »

Thanks for the enlighting answer CiccioCB

Yes the problem occurred for the first time, when I increased the number of "old" images on the Website from 3 to 6.
For sure it would be a gain for the reliability of the system if a too high number of request could not crash the system.
If browsers do limit their max connections to 6 perhaps the invest of RAM for one more connection may be worth it.

To understand the problem indeed helps to find a workaround! You mentioned some.
I will simply modify my code to display only three old images for now.

A quick but not finished thought is to insert a header that forces the browser to restrict itself to http/1.1, as this seems to reduce concurrent connections to a maximum of 2. Not sure if this works or the browser will try it anyway.

Thanks again
User avatar
cicciocb
Site Admin
Posts: 1900
Joined: Mon Feb 03, 2020 1:15 pm
Location: Toulouse
Has thanked: 407 times
Been thanked: 1271 times
Contact:

Re: ANNEX-CAM: vfs_fat: open: no free file descriptors

Post by cicciocb »

Hi Peter,
I played a little bit around this interesting subject and I wrote a little snippet that loads the images one by one in sequence.
This method avoid to crash and stuck the modules sending a lot of image download requests.
This snippet simply shows in the output page all the pictures present in the folder /image

Code: [Local Link Removed for Guests]

' javascript code to load images in sequence avoiding to stuck the module
' the images must be specified as <img file="/image/xx.jpg">
' do not use the attribute src="xx.jpg" but file="xx.jpg"
' when the html page is completed, execute the command
' jscall "load_pics();"
A$ = ||
A$ = A$ + |var idx = -1;|
A$ = A$ + |var pic_list;|
A$ = A$ + |function load_pics() {|
A$ = A$ + |  pic_list = document.querySelectorAll("img");|
A$ = A$ + |  console.log("nb of pictures " + pic_list.length);|
A$ = A$ + |  for (var i=0; i<pic_list.length; i++) {|
A$ = A$ + |    pic_list[i].addEventListener('load', pic_loaded);|
A$ = A$ + |    pic_list[i].addEventListener('error', pic_error);|
A$ = A$ + |  }|
A$ = A$ + |  pic_loaded();|
A$ = A$ + |}|
A$ = A$ + |function pic_loaded() {|
A$ = A$ + |  idx++;|
A$ = A$ + |  if (idx < pic_list.length) {|
A$ = A$ + |    pic_list[idx].src = pic_list[idx].getAttribute("file") + "?" + Math.random();|
A$ = A$ + |    console.log("image " + idx + " " + pic_list[idx].src);|
A$ = A$ + |  }|
A$ = A$ + |}|
A$ = A$ + |function pic_error() {|
A$ = A$ + |  alert("Picture load error: " + pic_list[idx].src);|
A$ = A$ + |}|
jscript a$

'create an html page containing all the pictures found in the directory /images
a$ = ""
f$ = file.dir$("/images")
while (f$ <> "")
  wlog f$
  A$= A$+|<img file="/images/|+ f$ + |" width=20%>|
  f$ = file.dir$
wend

cls
html a$

'loads all the images in sequence
jscall "load_pics();"
User avatar
PeterN
Posts: 366
Joined: Mon Feb 08, 2021 7:56 pm
Location: Krefeld, Germany
Has thanked: 171 times
Been thanked: 203 times
Contact:

Re: ANNEX-CAM: vfs_fat: open: no free file descriptors

Post by PeterN »

Thank you very much Francesco
I will try to include this asap.
My CAM-Script renames all old images if a new one is taken. This asures to have them sorted LIFO by name AND time . To overcome the caching of (outdated) images in the browser, I made the requests unique by adding a ?dummy=[millis] to the images URL.

Will the browser cache the Images called by your javascript as well? This would show (old) images in the wrong sort as they have the same name but new content if a new one has been added.
User avatar
cicciocb
Site Admin
Posts: 1900
Joined: Mon Feb 03, 2020 1:15 pm
Location: Toulouse
Has thanked: 407 times
Been thanked: 1271 times
Contact:

Re: ANNEX-CAM: vfs_fat: open: no free file descriptors

Post by cicciocb »

Peter,
the script add a random number at the end of the url to avoid the cache.
It is the same principle already applied in the previous MPU6050 demo html.
User avatar
PeterN
Posts: 366
Joined: Mon Feb 08, 2021 7:56 pm
Location: Krefeld, Germany
Has thanked: 171 times
Been thanked: 203 times
Contact:

Re: ANNEX-CAM: vfs_fat: open: no free file descriptors

Post by PeterN »

Sorry Francesco ... I did not widen the editor and missed the " + Math.random();|"
Post Reply