SoftwareDevice to get frames wirelessly into unity

Currently I use a ROSBridge which connects to where depth and color images are published from a RealSense Camera (along with camera stream intrinsics). I wrote a wrapper for RealSense SDK's SoftwareDevice to have the depth and color images added as VideoFrames to depth and color Software Sensors. I use the Syncer class to synchronize the frames from ROS and get FrameSets to make pointclouds from (using the RsProcessingPipeline and PointCloudRenderer). I do this so that the system that runs the VR this pointcloud is eventually rendered won't have to rely on the 12 foot camera cable from out streaming site. 

Currently I'm running into an issue with the Syncer where although I receive 5 fps from the  ROSBridge () I only get 1 fps out of the Syncer despite identical frame number and time stamps. The Syncer releases single frames if it cannot find a match and currently I just check if I have a FrameSet of 2 before I go on, but I was wondering if there is a better way to use the Syncer or if I need to make a custom processing block? 

Realsense -> RosBridge - > Unity (realsense sdk)