Very low FPS when trying the AsyncCam example

[chewbaz1]

Hello

I tried using the AsyncCam example but I get very poor fps (1-2fps). When i use the standard example from espressif I get normal fps. Changing resolution doesnt help. I have an AI Thinker board and it gets very warm with this example. I am using PlatformIO and have tried upload with both AI thinker setting and Wrover setting but fps is bad on both. When I tried using arduino IDE i got a lot of errors therefore I use platformio instead.

Power: 5V

I would be happy if I recieved help with this issue.

Thanks

[ery346]

It is a very good library but I am also having the same problem

[agus0]

Same problem. cant made a HTTP+JS web joystick for RC Car via websockets with embebed stream video

[Matt-Stedman]

Same here. I can get a v. good frame rate if I'm on StreamMjpeg, but the async example is depressingly slow!

[yoursunny]

When i use the standard example from espressif I get normal fps.

Do they offer an example with AsyncTCP?
If so, can you post a link?

[ForrestFire0]

I can confirm that the frame rate on the Async example from this repo is quite poor :(. Even when lowering the resolution.

To capture a static jpeg is actually quite fast. I modified the example to have a download button and the download is almost instant.

I attempted to characterize the performance of the library - I found in esp32cam-asyncweb.h this line auto frame = Camera.capture().release(); I slapped a println there and disappointingly found that it is only capturing a frame once per second.

Using platformio with the following settings:

[env:esp32cam]
platform = https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5
board = esp32cam
framework = arduino
upload_speed = 921600
monitor_speed = 115200
monitor_rts = 0
monitor_dtr = 0

lib_deps =
    AsyncTCP
    ESP Async WebServer
    yoursunny/esp32cam

Using this board.

[bsdshneg]

+1
very slow video ~1fps

[yoursunny]

I added some debug logging in MjpegResponse and can have some insights in where it's spending the most time.
In my location the 2.4GHz network is very noisy, such that the home router indicates "RX rate 6 Mbit/s" from the ESP32.
I tested with 640x480 resolution for 30 seconds and got 1.393932 fps.

full serial logs

[   32941] MjpegResponse(0x3ffdb8ec) created
[   32942] MjpegResponse(0x3ffdb8ec) capturing
[   33417] MjpegResponse(0x3ffdb8ec) frame has 7352 octets
[   33532] MjpegResponse(0x3ffdb8ec) sent to client
[   33917] MjpegResponse(0x3ffdb8ec) capturing
[   34417] MjpegResponse(0x3ffdb8ec) frame has 7357 octets
[   34513] MjpegResponse(0x3ffdb8ec) sent to client
[   34517] MjpegResponse(0x3ffdb8ec) capturing
[   34917] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   34999] MjpegResponse(0x3ffdb8ec) sent to client
[   35005] MjpegResponse(0x3ffdb8ec) capturing
[   35417] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   35701] MjpegResponse(0x3ffdb8ec) sent to client
[   35706] MjpegResponse(0x3ffdb8ec) capturing
[   35917] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   36010] MjpegResponse(0x3ffdb8ec) sent to client
[   36417] MjpegResponse(0x3ffdb8ec) capturing
[   36917] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   37266] MjpegResponse(0x3ffdb8ec) sent to client
[   37269] MjpegResponse(0x3ffdb8ec) capturing
[   37417] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   37505] MjpegResponse(0x3ffdb8ec) sent to client
[   37514] MjpegResponse(0x3ffdb8ec) capturing
[   37917] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   38361] MjpegResponse(0x3ffdb8ec) sent to client
[   38417] MjpegResponse(0x3ffdb8ec) capturing
[   38917] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   39029] MjpegResponse(0x3ffdb8ec) sent to client
[   39033] MjpegResponse(0x3ffdb8ec) capturing
[   39417] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   39514] MjpegResponse(0x3ffdb8ec) sent to client
[   39514] MjpegResponse(0x3ffdb8ec) sent to client
[   39917] MjpegResponse(0x3ffdb8ec) capturing
[   40417] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   40513] MjpegResponse(0x3ffdb8ec) sent to client
[   40519] MjpegResponse(0x3ffdb8ec) capturing
[   40917] MjpegResponse(0x3ffdb8ec) frame has 7148 octets
[   41058] MjpegResponse(0x3ffdb8ec) sent to client
[   41417] MjpegResponse(0x3ffdb8ec) capturing
[   41917] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   42172] MjpegResponse(0x3ffdb8ec) sent to client
[   42173] MjpegResponse(0x3ffdb8ec) capturing
[   42417] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   42501] MjpegResponse(0x3ffdb8ec) sent to client
[   42917] MjpegResponse(0x3ffdb8ec) capturing
[   43417] MjpegResponse(0x3ffdb8ec) frame has 7166 octets
[   43754] MjpegResponse(0x3ffdb8ec) sent to client
[   43918] MjpegResponse(0x3ffdb8ec) capturing
[   43921] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   44006] MjpegResponse(0x3ffdb8ec) sent to client
[   44013] MjpegResponse(0x3ffdb8ec) capturing
[   44418] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   44562] MjpegResponse(0x3ffdb8ec) sent to client
[   44563] MjpegResponse(0x3ffdb8ec) capturing
[   44904] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   44947] MjpegResponse(0x3ffdb8ec) sent to client
[   45002] MjpegResponse(0x3ffdb8ec) capturing
[   45418] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   45520] MjpegResponse(0x3ffdb8ec) sent to client
[   45522] MjpegResponse(0x3ffdb8ec) capturing
[   45918] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   46014] MjpegResponse(0x3ffdb8ec) sent to client
[   46020] MjpegResponse(0x3ffdb8ec) capturing
[   46418] MjpegResponse(0x3ffdb8ec) frame has 7151 octets
[   46501] MjpegResponse(0x3ffdb8ec) sent to client
[   46918] MjpegResponse(0x3ffdb8ec) capturing
[   47418] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   47607] MjpegResponse(0x3ffdb8ec) sent to client
[   47918] MjpegResponse(0x3ffdb8ec) capturing
[   48418] MjpegResponse(0x3ffdb8ec) frame has 7125 octets
[   48502] MjpegResponse(0x3ffdb8ec) sent to client
[   48918] MjpegResponse(0x3ffdb8ec) capturing
[   49418] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   49514] MjpegResponse(0x3ffdb8ec) sent to client
[   49918] MjpegResponse(0x3ffdb8ec) capturing
[   50418] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   50561] MjpegResponse(0x3ffdb8ec) sent to client
[   50566] MjpegResponse(0x3ffdb8ec) capturing
[   50918] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   51041] MjpegResponse(0x3ffdb8ec) sent to client
[   51059] MjpegResponse(0x3ffdb8ec) capturing
[   51418] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   51512] MjpegResponse(0x3ffdb8ec) sent to client
[   51918] MjpegResponse(0x3ffdb8ec) capturing
[   52418] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   52801] MjpegResponse(0x3ffdb8ec) sent to client
[   52809] MjpegResponse(0x3ffdb8ec) capturing
[   52918] MjpegResponse(0x3ffdb8ec) frame has 7151 octets
[   53207] MjpegResponse(0x3ffdb8ec) sent to client
[   53418] MjpegResponse(0x3ffdb8ec) capturing
[   53553] MjpegResponse(0x3ffdb8ec) frame has 7161 octets
[   53664] MjpegResponse(0x3ffdb8ec) sent to client
[   53918] MjpegResponse(0x3ffdb8ec) capturing
[   54418] MjpegResponse(0x3ffdb8ec) frame has 7765 octets
[   54529] MjpegResponse(0x3ffdb8ec) sent to client
[   54736] MjpegResponse(0x3ffdb8ec) capturing
[   54918] MjpegResponse(0x3ffdb8ec) frame has 7160 octets
[   54998] MjpegResponse(0x3ffdb8ec) sent to client
[   55418] MjpegResponse(0x3ffdb8ec) capturing
[   55918] MjpegResponse(0x3ffdb8ec) frame has 7765 octets
[   56000] MjpegResponse(0x3ffdb8ec) sent to client
[   56001] MjpegResponse(0x3ffdb8ec) capturing
[   56418] MjpegResponse(0x3ffdb8ec) frame has 7765 octets
[   56495] MjpegResponse(0x3ffdb8ec) sent to client
[   56497] MjpegResponse(0x3ffdb8ec) capturing
[   56918] MjpegResponse(0x3ffdb8ec) frame has 7765 octets
[   57019] MjpegResponse(0x3ffdb8ec) sent to client
[   57020] MjpegResponse(0x3ffdb8ec) capturing
[   57418] MjpegResponse(0x3ffdb8ec) frame has 7150 octets
[   57507] MjpegResponse(0x3ffdb8ec) sent to client
[   57918] MjpegResponse(0x3ffdb8ec) capturing
[   58418] MjpegResponse(0x3ffdb8ec) frame has 7765 octets
[   58524] MjpegResponse(0x3ffdb8ec) sent to client
[   58918] MjpegResponse(0x3ffdb8ec) capturing
[   59418] MjpegResponse(0x3ffdb8ec) frame has 7160 octets
[   59695] MjpegResponse(0x3ffdb8ec) sent to client
[   59755] MjpegResponse(0x3ffdb8ec) capturing
[   59918] MjpegResponse(0x3ffdb8ec) frame has 7765 octets
[   60270] MjpegResponse(0x3ffdb8ec) sent to client
[   60291] MjpegResponse(0x3ffdb8ec) capturing
[   60418] MjpegResponse(0x3ffdb8ec) frame has 7143 octets
[   60630] MjpegResponse(0x3ffdb8ec) sent to client
[   60918] MjpegResponse(0x3ffdb8ec) capturing
[   61418] MjpegResponse(0x3ffdb8ec) frame has 7164 octets
[   61580] MjpegResponse(0x3ffdb8ec) sent to client
[   61628] MjpegResponse(0x3ffdb8ec) capturing
[   61918] MjpegResponse(0x3ffdb8ec) frame has 7164 octets
[   62021] MjpegResponse(0x3ffdb8ec) sent to client
[   62418] MjpegResponse(0x3ffdb8ec) capturing
[   62918] MjpegResponse(0x3ffdb8ec) frame has 7161 octets
[   63058] MjpegResponse(0x3ffdb8ec) sent to client
[   63130] MjpegResponse(0x3ffdb8ec) capturing
[   63418] MjpegResponse(0x3ffdb8ec) frame has 7152 octets
[   63789] MjpegResponse(0x3ffdb8ec) deleted after 43 frames at fps 1.393932

Some spreadsheet calculation reveals the average wait time:

1. Waiting to start capture: 133ms
2. Camera performing the capture: 224ms
3. Sending the frame to the client: 336ms

In the current codebase, the MjpegController type can only make use of one Frame, even if multiple frames are created via Config::setBufferCount.
If this limitation is resolved, the duration for "camera performing the capture" and "sending the frame to the client" can be overlapped, which would greatly improve the fps.

Separately, it should be possible to refactor MjpegResponse type to reduce or eliminate the "waiting to start capture" duration.
While this is a smaller improvement, it is much easier to achieve.

[yoursunny]

I tried to reduce the SEND-CAPTURE wait in 00d1ddb.
Then I retested 640x480 resolution for 30 seconds, same as above.

full serial logs

[  152698] MjpegResponse(0x3ffdc148) created
[  152698] MjpegResponse(0x3ffdc148) capturing
[  153189] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  154022] MjpegResponse(0x3ffdc148) sent to client
[  154023] MjpegResponse(0x3ffdc148) capturing
[  154032] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  154640] MjpegResponse(0x3ffdc148) sent to client
[  154640] MjpegResponse(0x3ffdc148) capturing
[  154689] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  155811] MjpegResponse(0x3ffdc148) sent to client
[  155811] MjpegResponse(0x3ffdc148) capturing
[  156142] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  157066] MjpegResponse(0x3ffdc148) sent to client
[  157066] MjpegResponse(0x3ffdc148) capturing
[  157142] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  158189] MjpegResponse(0x3ffdc148) sent to client
[  158189] MjpegResponse(0x3ffdc148) capturing
[  158689] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  159553] MjpegResponse(0x3ffdc148) sent to client
[  159553] MjpegResponse(0x3ffdc148) capturing
[  159689] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  161121] MjpegResponse(0x3ffdc148) sent to client
[  161121] MjpegResponse(0x3ffdc148) capturing
[  161189] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  161959] MjpegResponse(0x3ffdc148) sent to client
[  161959] MjpegResponse(0x3ffdc148) capturing
[  162146] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  162577] MjpegResponse(0x3ffdc148) sent to client
[  162577] MjpegResponse(0x3ffdc148) capturing
[  162689] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  163189] MjpegResponse(0x3ffdc148) sent to client
[  163189] MjpegResponse(0x3ffdc148) capturing
[  163237] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  164896] MjpegResponse(0x3ffdc148) sent to client
[  164897] MjpegResponse(0x3ffdc148) capturing
[  165189] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  165713] MjpegResponse(0x3ffdc148) sent to client
[  165713] MjpegResponse(0x3ffdc148) capturing
[  165743] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  166330] MjpegResponse(0x3ffdc148) sent to client
[  166330] MjpegResponse(0x3ffdc148) capturing
[  166338] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  167039] MjpegResponse(0x3ffdc148) sent to client
[  167039] MjpegResponse(0x3ffdc148) capturing
[  167162] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  168189] MjpegResponse(0x3ffdc148) sent to client
[  168189] MjpegResponse(0x3ffdc148) capturing
[  168689] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  170725] MjpegResponse(0x3ffdc148) sent to client
[  170725] MjpegResponse(0x3ffdc148) capturing
[  170776] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  171578] MjpegResponse(0x3ffdc148) sent to client
[  171578] MjpegResponse(0x3ffdc148) capturing
[  171689] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  172427] MjpegResponse(0x3ffdc148) sent to client
[  172427] MjpegResponse(0x3ffdc148) capturing
[  172514] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  173189] MjpegResponse(0x3ffdc148) sent to client
[  173189] MjpegResponse(0x3ffdc148) capturing
[  173196] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  173680] MjpegResponse(0x3ffdc148) sent to client
[  173680] MjpegResponse(0x3ffdc148) capturing
[  173689] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  174072] MjpegResponse(0x3ffdc148) sent to client
[  174072] MjpegResponse(0x3ffdc148) capturing
[  174189] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  174563] MjpegResponse(0x3ffdc148) sent to client
[  174563] MjpegResponse(0x3ffdc148) capturing
[  174689] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  175366] MjpegResponse(0x3ffdc148) sent to client
[  175366] MjpegResponse(0x3ffdc148) capturing
[  175479] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  176189] MjpegResponse(0x3ffdc148) sent to client
[  176189] MjpegResponse(0x3ffdc148) capturing
[  176419] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  177189] MjpegResponse(0x3ffdc148) sent to client
[  177189] MjpegResponse(0x3ffdc148) capturing
[  177271] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  177594] MjpegResponse(0x3ffdc148) sent to client
[  177594] MjpegResponse(0x3ffdc148) capturing
[  177689] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  178856] MjpegResponse(0x3ffdc148) sent to client
[  178856] MjpegResponse(0x3ffdc148) capturing
[  179163] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  180030] MjpegResponse(0x3ffdc148) sent to client
[  180030] MjpegResponse(0x3ffdc148) capturing
[  180186] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  181577] MjpegResponse(0x3ffdc148) sent to client
[  181577] MjpegResponse(0x3ffdc148) capturing
[  181689] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  182973] MjpegResponse(0x3ffdc148) deleted after 29 frames at fps 0.957886

According to the spreadsheet, each step takes average time as follows:

1. Waiting to start capture: 0ms
2. Camera performing the capture: 140ms
3. Sending the frame to the client: 857ms

The change is effective in eliminating the SEND-CAPTURE delay, but the duration spent within SEND state is much longer.
I can't tell whether it's due to WiFi interference or something in the code.