View my account

How to get realsense record in the background while streaming camera data

Follow
Victor Vlu998
Hi everyone, I learned how to record and playback using realsense SDK. The question now is whether it is possible to process the camera stream meanwhile record at the background? The idea is similar to what realsense viewer can do: you can turn on off record in the middle of display the module stream. I tried to use rs2::sensor to get streaming data and rs2::pipeline to save to file at the same. But it turns out one would block another. I have read through realsense SDK documents and I didn't find anything related to this. I also wonder if this is achievable in ROS. I want to have my d455 camera record in SDK format while run realsense2_camera node. Thanks, Victor
0

Comments

7 comments

Date Votes
  • MartyX Grover

    Hi Victor Vlu998  A simple method that you could use may be to pause and unpause recording, so that you only capture the sections of the stream that you want.  When the pipeline is closed at the end of the streaming session, the recording would be saved.

     

    C++

    https://github.com/IntelRealSense/librealsense/tree/master/examples/record-playback#record-and-playback

    Python

    https://intelrealsense.github.io/librealsense/python_docs/_generated/pyrealsense2.recorder.html#pyrealsense2.recorder.pause

    https://github.com/IntelRealSense/librealsense/issues/4249

     

     

    In regard to pausing and resuming rosbag record in ROS, my research did find a ROS node script that somebody created in 2020 to provide this function.  That would record in ROS rosbag format and not the RealSense SDK's own bag format though.

    https://gist.github.com/vbschettino/13c1d0102fa184811b31397cb9a4e85d

    0
    Comment actions Permalink
  • Victor Vlu998
    • Edited

    Hi MartyG Thank you for your answer.

    I am not sure that pausing and unpause the recording is what I'm looking for.

    I want to have a complete record file (SDK format bag) to play it back using rtabmap meanwhile I want to do some other processes when it is recording. In that sense, pausing and unpause could affect the record file I get, which gives me the wrong frame rate (Please correct me if I am wrong)

    I would like to know if there are other ways to enable recording as a background process that would not affect the record file and live streaming data. For example, maybe I could replay and record at the same time using rs2::pipeline.

     

    Thanks,

    Victor

     

     

     

     

    0
    Comment actions Permalink
  • MartyX Grover

    The combination of a bag recorded in the RealSense SDK and RTABMAP may not be ideal.  This is because there can be differences between a rosbag file recorded in ROS and the bag format that the SDK uses.

    https://github.com/IntelRealSense/librealsense/issues/3020

     

    For what you are aiming to achieve, your best chance may be to use multithreading like the case linked to below where a RealSense user had one thread for the live-captured images and another thread to write the captured images to video.

    https://support.intelrealsense.com/hc/en-us/community/posts/360033158094-Recording-Video-in-60fps-with-threading-and-queue-Python-

     

    If you are using C++ instead of Python then there is a C++ multithreading discussion with scripting in the link below.

    https://github.com/IntelRealSense/librealsense/issues/6865

     

    If you prefer to use Python, the link below - also with scripting - may be helpful.

    https://github.com/IntelRealSense/librealsense/issues/6039

    0
    Comment actions Permalink
  • Victor Vlu998
    • Edited

    MartyG I do agree with the idea of multithreading to queue camera stream and save it into the file. But I am looking for more specifics.

    I would call the files recorded in ROS "ROS bags" and files recorded in SDK "SDK bags".  Is there any way I could convert one to another?

    What I am really looking for is a common interface to record files into SDK bags and generate ROS data at the same time live. I know we can use the SDK pipeline to record files into SDK bags and use realsense_ros to playback from SDK bags. But they are two different steps now. My concept is to bridge these two steps together so I could have realsense_ros launch from the SDK pipeline while getting the SDK bags. 

    Because SDK pipeline and realsense_ros node use the same camera data structure internally it is logical to bypass the camera stream to launch realsense_ros node before entering SDK bag. The question is if the SDK can do it or not.

     

    Thanks, 

    Victor

    0
    Comment actions Permalink
  • MartyX Grover
    • Edited

    I am not aware of a mechanism for converting an SDK bag to a rosbag and vice versa.  Some RealSense users have been successful in taking the approach of playing back bags to retrieve their data using the rosbag API instead of the librealsense pipeline though.  The links below provide examples of doing so.

    https://github.com/IntelRealSense/librealsense/issues/2215

    https://github.com/IntelRealSense/librealsense/issues/4057#issuecomment-499610742

    0
    Comment actions Permalink
  • Michael Mcgrath

    Hi Marty,  I'm working with Victor on a project.  We have a processing pipline with algorithms following the sensor data aquisition.  Our preference is to use the realsense "D455 sensor data bag" capture so we can replay the recordings and tune the processing algorithms.  From my perspective we also want to use the realsense viewer to look at the D455 data in detail.

    Clearly there are Realsense API functions to either record to the bag file or replay from the saved bag file - what we are looking for is a mechanism to forward data through for processing AND also support simultaneous capture of  a bag recording. (a "Tee" pipe approach).  I'm surprised this isn't already available...

    Any suggestions on API's or the APi functions we could build around to support this?

    Thanks Michael

    0
    Comment actions Permalink
  • MartyX Grover
    • Edited

    Hi Michael Mcgrath  I studied your question carefully.  If you need the functions of the RealSense SDK and ROS simultaneously, then perhaps you could base your project on ROS and create C++ or Python node scripts to access the SDK functions from ROS.  A couple of links that discuss node scripts for each language are provided below.

    C++

    https://github.com/IntelRealSense/realsense-ros/issues/1168#issuecomment-619527388

    Python

    https://github.com/IntelRealSense/realsense-ros/issues/1086

     

    You can define some post-processing filters in the ROS launch and set values for some options by loading a json camera configuration file during the ROS launch process.

    You can also look at live and rosbag camera data visually in ROS' RViz interface.

    0
    Comment actions Permalink

Please sign in to leave a comment.

Didn't find what you were looking for?

New post