View my account

Annotating data for custom dataset

Follow
Gunjan Narulkar
Hello, I'm trying to replicate this paper: https://arxiv.org/abs/1908.03364 using Realsense D435. It means that I'll need to create dataset and for that, I'll need: 1. A way to capture data at scale using D435 2. Annotate the data at scale I've studied the resources and clearly there are solutions up to come extent. For capturing the videos with depth set up, we need to take care of synchronizing depth and image streams programmatically (and using images, video can always be generated if required), resulting in a potential 4 channel equivalent of data stream - Am I correct? But I'm yet to find deeper discussion on annotating such 4 channel information - how to even begin? And what can be done for scaling the process in terms of involving participants to tag images? Any help will be great.
0

Comments

5 comments

Date Votes
  • MartyG
    • Edited

    I will start by highlighting that there have been RealSense visual assistance projects that require less user interaction.  These include sound-based obstacle sonar (Sonic-Sight) and having vibrating pads on body areas that vibrate when an obstacle is approaching that body part (SAW).

    Sonic-Sight

    https://github.com/IntelRealSense/librealsense/issues/3629#issuecomment-478445085 

    SAW

    https://github.com/IRSAW/IRSAW 

    https://youtube.com/channel/UCSyCwEuRWnjZUtWZXB1CPlg/videos 

    I hope that the above information will act as a useful springboard for further discussion in this case.

    0
    Comment actions Permalink
  • Gunjan Narulkar

    @MartyG, Thanks for both the links - as you indicated, neither comes close to what I need precisely, but both provide my brain some fodder to ruminate further on evaluate my options before leaning hard on any specific course of action. I really appreciate the directions, which I'll explore and get back. I believe my answer will be sitting somewhere in middle of how SAW is approaching the predefined scene segmentation and how other links are approaching the solution in that thread you mentioned. It'll possibly take me this week to do justice with the provided information and come up with intelligent questions.

    the main question I'm trying to answer to myself is - how to even use depth information while annotation? And then comes the tooling. But I'm sure your current links will help me here, so I'll get back. How ever, would it be possible for you to direct me to any aligned Depth Image annotation tool related projects? I know I can't be the only person asking this question over the years of Deep Learning revolution :) I'll understand if there is none, if that's the case. Thanks again.

    0
    Comment actions Permalink
  • MartyG

    You may save yourself a lot of labor if you use the OpenVINO Toolkit computer vision software as the foundation of your project.  It has data annotation and auto annotation functions.

    https://github.com/openvinotoolkit/cvat 

    https://youtube.com/watch?v=BKLyHOEACFw 

    https://youtube.com/watch?v=jbqOa8DX7Jg 

    Intel have their own implementation of OpenVINO.

    https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit.html 

    And the RealSense SDK has an OpenVINO compatibility wrapper.

    https://github.com/IntelRealSense/librealsense/tree/master/wrappers/openvino 

     

    1
    Comment actions Permalink
  • Gunjan Narulkar

    Hi Marty,

    Thanks to your links and kind guidance, I was able to do below:
    1. Image capture: Use PyRealSense and get D435 spring to action and capture images from required channels and save them.
    2. Annotation: OpenVino looks amazing! I'll check it out soon, but I think I've what I needed :)
    3. Inference: Digging one of your links, I found an example of MobileSSD using Open CV working for me!
    Now my next question is around implementation - one obvious way is schedule as batch, capture every minute and apply inference and sleep. But is there any better, more matured and elegant way of implementing it "real time"?

    On top of it, there are question around making this process interrupt driven on jetson nano - push a button to start and stop this process - and finally, how to cover the results (currently printed on console) to visually impaired user - any thoughts you may have based on your experience will go a long way!

    0
    Comment actions Permalink
  • MartyG
    • Edited

    The "wake - capture and process - sleep" cycle that you propose is a common one that is used when you only need to capture and perform an action periodically.  It has been made easier to set up trigger timing by the introduction of an external synchronization (genlock) feature in the RealSense SDK for camera models with global shutters, which includes the D435 model that you use..

    https://www.intelrealsense.com/depth-camera-external-sync-trigger/ 

    https://dev.intelrealsense.com/docs/external-synchronization-of-intel-realsense-depth-cameras 

    0
    Comment actions Permalink

Please sign in to leave a comment.

Didn't find what you were looking for?

New post