Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No frames received by frigate #84

Open
MStapelfeldt opened this issue Jul 16, 2023 · 10 comments
Open

No frames received by frigate #84

MStapelfeldt opened this issue Jul 16, 2023 · 10 comments
Labels
bug Something isn't working help wanted Extra attention is needed question Further information is requested

Comments

@MStapelfeldt
Copy link

MStapelfeldt commented Jul 16, 2023

Hello,
I've set up a camera with your code in frigate yesterday and it was working very good. Today the stream stopped working and I couldn't get it back on.
Here is the log output from Frigate:

2023-07-16 08:16:31.875044507  [2023-07-16 10:16:31] frigate.app                    INFO    : Starting Frigate (0.12.1-367d724)
2023-07-16 08:16:31.929656630  [2023-07-16 10:16:31] frigate.app                    INFO    : Creating directory: /tmp/cache
2023-07-16 08:16:31.933044352  [2023-07-16 10:16:31] peewee_migrate                 INFO    : Starting migrations
2023-07-16 08:16:31.941418138  [2023-07-16 10:16:31] peewee_migrate                 INFO    : There is nothing to migrate
2023-07-16 08:16:31.969271657  [2023-07-16 10:16:31] detector.cpu1                  INFO    : Starting detection process: 301
2023-07-16 08:16:31.975125045  [2023-07-16 10:16:31] frigate.app                    INFO    : Output process started: 304
2023-07-16 08:16:31.979259325  [2023-07-16 10:16:31] detector.cpu2                  INFO    : Starting detection process: 303
2023-07-16 08:16:31.998096109  [2023-07-16 10:16:31] frigate.app                    INFO    : Camera processor started for fri_Esp32: 311
2023-07-16 08:16:32.021537103  [2023-07-16 10:16:32] frigate.app                    INFO    : Capture process started for fri_Esp32: 315
2023-07-16 08:16:52.086622452  [2023-07-16 10:16:52] watchdog.fri_Esp32             INFO    : No frames received from fri_Esp32 in 20 seconds. Exiting ffmpeg...
2023-07-16 08:16:52.088567869  [2023-07-16 10:16:52] watchdog.fri_Esp32             INFO    : Waiting for ffmpeg to exit gracefully...
2023-07-16 08:16:52.093141952  [2023-07-16 10:16:52] frigate.video                  ERROR   : fri_Esp32: Unable to read frames from ffmpeg process.
2023-07-16 08:16:52.093332992  [2023-07-16 10:16:52] frigate.video                  ERROR   : fri_Esp32: ffmpeg process is not running. exiting capture thread...
2023-07-16 08:17:02.097484948  [2023-07-16 10:17:02] watchdog.fri_Esp32             ERROR   : Ffmpeg process crashed unexpectedly for fri_Esp32.
2023-07-16 08:17:02.097495578  [2023-07-16 10:17:02] watchdog.fri_Esp32             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-07-16 08:17:02.097506094  [2023-07-16 10:17:02] ffmpeg.fri_Esp32.detect        ERROR   : Finishing stream 0:0 without any data written to it.
2023-07-16 08:17:02.097513952  [2023-07-16 10:17:02] ffmpeg.fri_Esp32.detect        ERROR   : [swscaler @ 0x5604f5d5dbc0] [swscaler @ 0x5604f5d6ab80] deprecated pixel format used, make sure you did set range correctly
2023-07-16 08:17:02.097574766  [2023-07-16 10:17:02] ffmpeg.fri_Esp32.detect        ERROR   : [swscaler @ 0x5604f5d5dbc0] [swscaler @ 0x5604f5da7100] deprecated pixel format used, make sure you did set range correctly
2023-07-16 08:17:02.097648467  [2023-07-16 10:17:02] ffmpeg.fri_Esp32.detect        ERROR   : Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)
2023-07-16 08:17:22.117537376  [2023-07-16 10:17:22] watchdog.fri_Esp32             INFO    : No frames received from fri_Esp32 in 20 seconds. Exiting ffmpeg...
2023-07-16 08:17:22.117983568  [2023-07-16 10:17:22] watchdog.fri_Esp32             INFO    : Waiting for ffmpeg to exit gracefully...
2023-07-16 08:17:22.125588883  [2023-07-16 10:17:22] frigate.video                  ERROR   : fri_Esp32: Unable to read frames from ffmpeg process.
2023-07-16 08:17:22.126004954  [2023-07-16 10:17:22] frigate.video                  ERROR   : fri_Esp32: ffmpeg process is not running. exiting capture thread...
2023-07-16 08:17:32.129013484  [2023-07-16 10:17:32] watchdog.fri_Esp32             ERROR   : Ffmpeg process crashed unexpectedly for fri_Esp32.
2023-07-16 08:17:32.129404989  [2023-07-16 10:17:32] watchdog.fri_Esp32             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-07-16 08:17:32.130931440  [2023-07-16 10:17:32] ffmpeg.fri_Esp32.detect        ERROR   : Finishing stream 0:0 without any data written to it.
2023-07-16 08:17:32.131238952  [2023-07-16 10:17:32] ffmpeg.fri_Esp32.detect        ERROR   : [swscaler @ 0x55b246be4d00] [swscaler @ 0x55b246bf1cc0] deprecated pixel format used, make sure you did set range correctly
2023-07-16 08:17:32.131665545  [2023-07-16 10:17:32] ffmpeg.fri_Esp32.detect        ERROR   : [swscaler @ 0x55b246be4d00] [swscaler @ 0x55b246c2e240] deprecated pixel format used, make sure you did set range correctly
2023-07-16 08:17:32.131809452  [2023-07-16 10:17:32] ffmpeg.fri_Esp32.detect        ERROR   : Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)
2023-07-16 08:17:52.150725272  [2023-07-16 10:17:52] watchdog.fri_Esp32             INFO    : No frames received from fri_Esp32 in 20 seconds. Exiting ffmpeg...
2023-07-16 08:17:52.151302136  [2023-07-16 10:17:52] watchdog.fri_Esp32             INFO    : Waiting for ffmpeg to exit gracefully...
2023-07-16 08:17:52.159430060  [2023-07-16 10:17:52] frigate.video                  ERROR   : fri_Esp32: Unable to read frames from ffmpeg process.
2023-07-16 08:17:52.159882611  [2023-07-16 10:17:52] frigate.video                  ERROR   : fri_Esp32: ffmpeg process is not running. exiting capture thread...
2023-07-16 08:18:02.173452684  [2023-07-16 10:18:02] watchdog.fri_Esp32             ERROR   : Ffmpeg process crashed unexpectedly for fri_Esp32.
2023-07-16 08:18:02.173606798  [2023-07-16 10:18:02] watchdog.fri_Esp32             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-07-16 08:18:02.173713202  [2023-07-16 10:18:02] ffmpeg.fri_Esp32.detect        ERROR   : Finishing stream 0:0 without any data written to it.
2023-07-16 08:18:02.174574170  [2023-07-16 10:18:02] ffmpeg.fri_Esp32.detect        ERROR   : [swscaler @ 0x560d6648d3c0] [swscaler @ 0x560d6649a380] deprecated pixel format used, make sure you did set range correctly
2023-07-16 08:18:02.174921871  [2023-07-16 10:18:02] ffmpeg.fri_Esp32.detect        ERROR   : [swscaler @ 0x560d6648d3c0] [swscaler @ 0x560d664d5c80] deprecated pixel format used, make sure you did set range correctly
2023-07-16 08:18:02.174927072  [2023-07-16 10:18:02] ffmpeg.fri_Esp32.detect        ERROR   : Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)

And this is the frigate.yaml

  fri_Esp32:
    ffmpeg:
      global_args: -hide_banner -loglevel warning
      # Hardware acceleration does not work for me as described in the Frigate documentation on my Raspberry Pi 4 32Bit OS.
      hwaccel_args: []
      #  - -c:v
      #  - h264_mmal
      inputs:
        - path: rtsp://192.168.178.51:554/mjpeg/1
          roles:
            - detect
     #       - rtmp # deactivate rtmp, otherwise you only see a green image
    detect:
      width: 800
      height: 600
      fps: 5

The camera is also set to 800x600px, PSRAM active (has 4mb) AI Thinker. The web stream works and I also can view RTSP with other apps. After Rest, power disconnect and restarting Frigate, Camera shows again with log

2023-07-16 08:40:31.926576961  [INFO] Starting Frigate...
2023-07-16 08:40:33.875048360  [2023-07-16 10:40:33] frigate.app                    INFO    : Starting Frigate (0.12.1-367d724)
2023-07-16 08:40:33.925293123  [2023-07-16 10:40:33] frigate.app                    INFO    : Creating directory: /tmp/cache
2023-07-16 08:40:33.928701036  [2023-07-16 10:40:33] peewee_migrate                 INFO    : Starting migrations
2023-07-16 08:40:33.935573935  [2023-07-16 10:40:33] peewee_migrate                 INFO    : There is nothing to migrate
2023-07-16 08:40:33.963221959  [2023-07-16 10:40:33] detector.cpu1                  INFO    : Starting detection process: 299
2023-07-16 08:40:33.968608095  [2023-07-16 10:40:33] frigate.app                    INFO    : Output process started: 302
2023-07-16 08:40:33.973220884  [2023-07-16 10:40:33] detector.cpu2                  INFO    : Starting detection process: 301
2023-07-16 08:40:33.991433277  [2023-07-16 10:40:33] frigate.app                    INFO    : Camera processor started for fri_Esp32: 309
2023-07-16 08:40:34.016494839  [2023-07-16 10:40:34] frigate.app                    INFO    : Capture process started for fri_Esp32: 314
2023-07-16 08:40:54.074415105  [2023-07-16 10:40:54] watchdog.fri_Esp32             INFO    : No frames received from fri_Esp32 in 20 seconds. Exiting ffmpeg...
2023-07-16 08:40:54.075525753  [2023-07-16 10:40:54] watchdog.fri_Esp32             INFO    : Waiting for ffmpeg to exit gracefully...
2023-07-16 08:40:54.081943746  [2023-07-16 10:40:54] frigate.video                  ERROR   : fri_Esp32: Unable to read frames from ffmpeg process.
2023-07-16 08:40:54.083147444  [2023-07-16 10:40:54] frigate.video                  ERROR   : fri_Esp32: ffmpeg process is not running. exiting capture thread...
2023-07-16 08:41:04.090249151  [2023-07-16 10:41:04] watchdog.fri_Esp32             ERROR   : Ffmpeg process crashed unexpectedly for fri_Esp32.
2023-07-16 08:41:04.090256089  [2023-07-16 10:41:04] watchdog.fri_Esp32             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-07-16 08:41:04.090288047  [2023-07-16 10:41:04] ffmpeg.fri_Esp32.detect        ERROR   : Finishing stream 0:0 without any data written to it.
2023-07-16 08:41:04.090693117  [2023-07-16 10:41:04] ffmpeg.fri_Esp32.detect        ERROR   : [swscaler @ 0x55a26e454cc0] [swscaler @ 0x55a26e461c80] deprecated pixel format used, make sure you did set range correctly
2023-07-16 08:41:04.090697400  [2023-07-16 10:41:04] ffmpeg.fri_Esp32.detect        ERROR   : [swscaler @ 0x55a26e454cc0] [swscaler @ 0x55a26e491c00] deprecated pixel format used, make sure you did set range correctly
2023-07-16 08:41:04.090701456  [2023-07-16 10:41:04] ffmpeg.fri_Esp32.detect        ERROR   : Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)

But I see stream...
Any Idea what I can do to find the issue and get it working all the time?

Additional info:
Detection also works, but I can't load the record but can view the snapshot of capture.
Fri-cam32_1
Fri-cam32_2

@rzeldent
Copy link
Owner

Ho Mortiz,

Nice addition to use qualification to detect people!

To oanswer you question; Streaming the video is a lot of data and what I see often is that the ESP32 cannot handle that much data. A drop in connectivity or attempting to send more data than is possible causes the RTSP stream to freeze and the RTSP client times out/stops functioning. VLC for example has some buffering build in to be able to display a continuous stream.

To detect people I tried to detect using tensorflow lite on the ESP itself to overcome this problem. Take a look at the repository at https://github.com/rzeldent/esp32cam-persondetection. This works but the frame rate is not very high. However, you only need one ESP and no server or post processing. The training set can be found at https://studio.edgeimpulse.com/studio/29733. It works but the framerate is not very high. However, might be an idea to also add a repository to use the ESP32CAM as a security/detection camera.

Optionally, take a look at the Sipeed MAIX BIT with the K210 neural processor. I bought these around 30 euros (a long time ago) but they can do detection (yolo) at 50 frames/sec! Also the tensorflow model can be transformed to the kmodel so this runs very fast. However, I think they do not have WiFi (but can be resolved by adding an ESP8266).

Let me know if this helps or gave you inspiration...

@MStapelfeldt
Copy link
Author

MStapelfeldt commented Jul 16, 2023

Hi, the detection is all done by frigate. ESP32 only sends the bare rtsp stream to it. But that's where the problem is: I can't get a stable rtsp stream... When it's there, everything is working perfectly, but this part

2023-07-16 08:17:22.117537376  [2023-07-16 10:17:22] watchdog.fri_Esp32             INFO    : No frames received from fri_Esp32 in 20 seconds. Exiting ffmpeg...
2023-07-16 08:17:22.117983568  [2023-07-16 10:17:22] watchdog.fri_Esp32             INFO    : Waiting for ffmpeg to exit gracefully...
2023-07-16 08:17:22.125588883  [2023-07-16 10:17:22] frigate.video                  ERROR   : fri_Esp32: Unable to read frames from ffmpeg process.
2023-07-16 08:17:22.126004954  [2023-07-16 10:17:22] frigate.video                  ERROR   : fri_Esp32: ffmpeg process is not running. exiting capture thread...
2023-07-16 08:17:32.129013484  [2023-07-16 10:17:32] watchdog.fri_Esp32             ERROR   : Ffmpeg process crashed unexpectedly for fri_Esp32.
2023-07-16 08:17:32.129404989  [2023-07-16 10:17:32] watchdog.fri_Esp32             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-07-16 08:17:32.130931440  [2023-07-16 10:17:32] ffmpeg.fri_Esp32.detect        ERROR   : Finishing stream 0:0 without any data written to it.
2023-07-16 08:17:32.131238952  [2023-07-16 10:17:32] ffmpeg.fri_Esp32.detect        ERROR   : [swscaler @ 0x55b246be4d00] [swscaler @ 0x55b246bf1cc0] deprecated pixel format used, make sure you did set range correctly
2023-07-16 08:17:32.131665545  [2023-07-16 10:17:32] ffmpeg.fri_Esp32.detect        ERROR   : [swscaler @ 0x55b246be4d00] [swscaler @ 0x55b246c2e240] deprecated pixel format used, make sure you did set range correctly
2023-07-16 08:17:32.131809452  [2023-07-16 10:17:32] ffmpeg.fri_Esp32.detect        ERROR   : Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)
```

rerpeating every 20 seconds, makes it unpossible.

If the problem is the amount of data, what can I adjust? I have 4mb PSRAM (8mb version coming soon), less then 5m between router and esp32, image set to 800x600px. That's my current config:

ESP32
SDK Version: v4.4.4
CPU model: ESP32-D0WD-V3 rev. 3
CPU speed: 240 Mhz
CPU cores: 2
RAM size: 263.77 KB
PSRAM size: 4 MB
Flash size: 4 MB
Diagnostics
Uptime: 00:09:31
Chip temperature:
 39.44 °C
RTSP sessions: 0
Free heap: 144.13 KB
Max free block: 103.99 KB
Peripheral
Board type: AI THINKER
LED intensity:
 0 [0-100]
Network
Host name: esp32-b0a732f1d8e4.local
Mac address: ************
Wifi mode: STA
Access point: ****************
Signal strength: -60 dbm
IPv4 address: 192.168.178.***
IPv6 address: 0000:0000:0000:0000:0000:0000:0000:0000
Connected to the access point
Camera
Frame rate: 200 ms (5.0 f/s)
Frame size: SVGA (800x600)
JPEG quality: 14 [1-100]
Enable PSRAM: Enabled
Number of frame buffers: 1
Brightness: 0 [-2,2]
Contrast: 0 [-2,2]
Saturation: 0 [-2,2]
Special effect: Normal
White balance: Auto
AWB gain: Auto
WB mode: Auto
Exposure control: Auto
Auto exposure control (dsp): Enabled
Auto Exposure level: 0
Manual exposure value: 300
Gain control: Auto
AGC gain: 0
Gain ceiling: 2X
Black pixel correct: Manual
White pixel correct: Auto
Gamma correct: Enabled
Lens correction: Enabled
Horizontal mirror: Normal
Vertical flip: Normal
Downsize enable: Enabled
Color bar: Camera

Maybe VGA or less (higher number) JPEG Quality can help?

@rzeldent
Copy link
Owner

Hi Moritz,

Decreasing the resolution will reduce the number of bytes sent so should work. If the compression is increased this results in a lower bitrate but also more cpu power used and this will reduce the effectiveness.

So would go for lowering the image size, good placement/antenna.

@rzeldent
Copy link
Owner

Any luck / progress?

@MStapelfeldt
Copy link
Author

No not really. I tried an other board with 8mb psram, antenna, resolution down to 640x480 and many different qualities. Also trying not with RTSP, but jmpeg stream url over go2rtc... It's never stable over more then a short time.
The RTSP stream is working ok with onvif viewer and onvier app, but not possible to integrate into frigate NVR. 😭

@rzeldent
Copy link
Owner

rzeldent commented Feb 2, 2024

Maybe this helps: https://stackoverflow.com/questions/3367187/what-is-lib-swscale-used-for-by-ffmpeg-programers
Do you really need swscale? Maybe choosing the same width/size skips swscale...

@rzeldent rzeldent added bug Something isn't working help wanted Extra attention is needed question Further information is requested labels Feb 2, 2024
@KeyLaager
Copy link

Did you find a solution @MStapelfeldt? I am running into the same issue, I had the stream working for about 5 minutes on frigate then nothing. I can see the JPEG HTTP stream but RTSP will not load on either frigate or VLC.

@MStapelfeldt
Copy link
Author

Unfortunately not... I gave up on espcam and switched to cheap v380pro cam. https://a.aliexpress.com/_Exi5Wlj
Set them up to onfiv and block Internet after setup. They work as a charm for the money.

@KeyLaager
Copy link

KeyLaager commented Feb 20, 2024 via email

@jsnyder
Copy link

jsnyder commented Apr 26, 2024

Still similar in recent usage for me. It can sometimes run for some extended periods when parameters are chosen to keep things in sustainable ranges for framerate/datarate etc.. but ultimately it flakes out fairly using the recommended mjpeg camera rtsp settings. I will say that I'm currently getting a bit better behavior out of a stream from one with an ov5640 (which also does a better job with getting color right and having lower noise levels). I'm wondering if the lower noise might keep the frame sizes down or if there's something with that setup that's just a bit more consistent. Still it does need to be watched and things sometimes need to be restarted to get the stream going again. I have that set at 1600x1200 and 1fps on a seeed w/ the ov5640 swapped for the original ov2640. Frigate's reported storage requirements / hr seem to reflect that stream having imagery that compresses less well too.

I allowed the gain to go pretty high so that may not reflect everyone else's setups. For the life of me, I can't get acceptable image quality indoors out of the ov2640 without having quite a bit of light. the ov5640 is more forgiving.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Extra attention is needed question Further information is requested
Projects
None yet
Development

No branches or pull requests

4 participants