View my account

Recording Video in 60fps with threading and queue (Python)

Follow
Satsuna1352

Hello, my camera is Realsense D435.

I asked how to get 60fps RGB video stream before.

Now I want to record the RGB stream into a video, but I want to separate the step of getting RGB image and the step of writing them into video.

So I use threading and queue to implement it.

The concept is that the camera catch RGB image in 60fps, putting them into rgb_queue, and the other thread called RecordingJob get the images from rgb_queue, writing them into video continuously.

 

Unfortunately, when I run the code, the error Frame didn't arrived within 5000 usually happens.

For example, I would like to record a 5-min video(18000frames), but the code stuck at the 173th frame, the RecordingJob only write 160 frames into video, and 13 frames are still in rgb_queue.

q

If RecordingJob just get the images from queue and doesn't write them into video, the code can be run without error.

If I just get an image and write it into video directly in 60fps, the code can be executed without error, too.

​

I wanna know how to solve the problem, thank you in advance for your apply.

 

My code is below.(It seems that I cannot make an attachment?)

https://drive.google.com/open?id=1bTpFLGkQ5ewTzsyYADcCXCRgYC7Zf7ir

0

Comments

13 comments

Date Votes
  • Alexandra Ciuriuc

    Hi Satsuna1352,

     

    Thank you for your interest in the Intel RealSense D435 camera.

    Can you please tell me if you see the same behavior with the RealSense samples/tools? 

    Also, please make sure you are using the latest librealsense and firmware version.  

    https://downloadcenter.intel.com/product/128255/Intel-RealSense-Depth-Camera-D435

    https://github.com/IntelRealSense/librealsense/releases

     

    Regards,

    Alexandra

     

     

    0
    Comment actions Permalink
  • Satsuna1352

    Hi, I'm using the latest version firmware.

    And the error doesn't happen while running other Realsense samples.

    0
    Comment actions Permalink
  • Alexandra Ciuriuc

    Hello Satsuna1352,

     

    We are able to run your code but we do not get the same problem. We get over 1000 frames without a problem. The “frames didn’t arrive” issue usually occurs because of latencies somewhere in the system, not related to RealSense. It may be that your computer is processing the threads too slowly.

    Can you please tell me what are the specs of your system?

     

    Regards,

    Alexandra

    0
    Comment actions Permalink
  • Satsuna1352

    CPU: Intel i5-7200U

    RAM: 8GB DDR4

    GPU: NVIDIA GeForce 940MX

    Would you please check how many frames still in rgb_queue when the program finishes?

    Maybe there are some frames not being got by RecordingJob

    0
    Comment actions Permalink
  • Alexandra Ciuriuc

    Hello Satsuna1352.

     

    As you may see in the picture below, there are 18000 frames . 

     

    Did you try running the code on a different computer?

    We are still looking into your issue and we will get back to you later. 

     

    Regards,

    Alexandra

    0
    Comment actions Permalink
  • Satsuna1352
    • Edited

    Hi, Alexandra

    Sorry for my late reply, I ran the code on another computer.

    Sometimes it could record 18000 frames successfully, but sometimes it failed.

    The computer spec is below:

    CPU: Intel i5-7400

    RAM: 8GB DDR4

    GPU: NVIDIA GeForce GTX 1050Ti

    May I have your computer spec?

    Besides, the program will output "time.txt" as video recording finishes, can I get your txt file?

    I wanna see if the time difference of every 2 frames is always 16.6ms

    0
    Comment actions Permalink
  • Alexandra Ciuriuc

    Hi Satsuna1352,

     

    Were there any applications running on your system when you ran the code?

     

    Regards,

    Alexandra

    0
    Comment actions Permalink
  • Satsuna1352

    No, I ran the code right after turning on my laptop.

    I also opened the task manager and monitored the performance, the CPU usage, memory usage, hard disk usage didn't get higher drastically when running the code.

    0
    Comment actions Permalink
  • Alexandra Ciuriuc

     

    Hi Satsuna1352,

     

    Please find below a snapshot of my time.txt file. We are still looking into your issue. 

     

     

    Regards,

    Alexandra

    0
    Comment actions Permalink
  • Satsuna1352

    Hi, Alexandra,

    Based on your snapshot, the time difference of every 2 frames is 10~20ms.

    And it meets the 60 fps (16.6ms difference of every 2 frames).

    Attached is my snapshot, most time differences are reasonable (about 17ms), but you can see that the highlight space is an 22ms difference.

    Would you mind giving the whole time.txt so that I can check the time difference is always stable?

    0
    Comment actions Permalink
  • Alexandra Ciuriuc

    Hello Satsuna1352,

     

    Your issue may be due to inefficiencies in the Python Threading module. Try the Multiprocessing module which may have better performance.

     

    Regards,

    Alexandra

    0
    Comment actions Permalink
  • Satsuna1352

    Hello Alexandra,

    I've tried the Multiprocessing module and it actually works better than Threading.

    Thank you for your reply.

    In addition, I have a problem about how to get the timestamp of every frame, should I open another issue?

    0
    Comment actions Permalink
  • Alexandra Ciuriuc

    Hello Satsuna1352,

     

    I'm glad to hear it works now!

    Yes, please open a new case for this issue. 

     

    Regards,

    Alexandra

    0
    Comment actions Permalink

Please sign in to leave a comment.

Didn't find what you were looking for?

New post