View my account

When trying to dynamically load multiple Realsense 465s in Unity, only one is loaded and the error "Frame didn't arrive within 5000" occurs.

Follow
Maruyama

Hello.

I have the problem as shown in the title when I load multiple units at runtime in Unity.
For example, when I run a program that connects 3 units and loads them at runtime, I cannot get information from the first and second Realsense units, but I can get information from the third Realsense unit.
When two devices are connected and a program is executed to load at runtime, the first Realsense will not be able to retrieve information, but the second Realsense will.
The terminal that cannot display the information will give the following error

 

ExternalException: rs2_pipeline_wait_for_frames(pipe:000001EC94855880)
Rethrow as Exception: Frame didn't arrive within 5000
Intel.RealSense.ErrorMarshaler.MarshalNativeToManaged (System.IntPtr pNativeData) (at <cb539c8577184d8bb3db94250e9792dc>:0)
Intel.RealSense.Pipeline.WaitForFrames (System.UInt32 timeout_ms) (at <cb539c8577184d8bb3db94250e9792dc>:0)
RsDevice.WaitForFrames () (at Assets/RealSenseSDK2.0/Scripts/RsDevice.cs:159)
System.Threading.ThreadHelper.ThreadStart_Context (System.Object state) (at <1c8569827291471e9db0dcd976e97952>:0)
System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) (at <1c8569827291471e9db0dcd976e97952>:0)
System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) (at <1c8569827291471e9db0dcd976e97952>:0)
System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) (at <1c8569827291471e9db0dcd976e97952>:0)
System.Threading.ThreadHelper.ThreadStart () (at <1c8569827291471e9db0dcd976e97952>:0)
UnityEngine.<>c:<RegisterUECatcher>b__0_0(Object, UnhandledExceptionEventArgs)
Intel.RealSense.NativeMethods:rs2_pipeline_wait_for_frames(IntPtr, UInt32, Object&)
Intel.RealSense.Pipeline:WaitForFrames(UInt32)
RsDevice:WaitForFrames() (at Assets/RealSenseSDK2.0/Scripts/RsDevice.cs:159)
System.Threading.ThreadHelper:ThreadStart()

 

Shows the experimental environment.

Environment:
Unity: 2021.3.26f1
Realsense: 465
Number of devices: 3 (2)

program：

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UI;
using Intel.RealSense;
using TMPro.EditorUtilities;
using System.Linq;

public class MultiSpawnDemo : MonoBehaviour
{
    [SerializeField] private GameObject _rsPipePrefab;
    [SerializeField] private Camera _cameraPrefab;
    [SerializeField] private Transform _localCameraParent;
    [SerializeField] private RawImage _colorTest;
    [SerializeField] private RawImage _depthTest;

    private Context _context;
    private List<RsStreamTextureRenderer> _rsDepthRenderers = new List<RsStreamTextureRenderer>();
    private List<RsStreamTextureRenderer> _rsColorRenderers = new List<RsStreamTextureRenderer>();
    private Texture2D[] _processedDepthTexturesArray;
    private Texture2D[] _processedColorTexturesArray;

    private int _id = 0;
    private int TestID
    {
        set
        {
            _id = value;
            _colorTest.texture = _processedColorTexturesArray[value];
            _depthTest.texture = _processedDepthTexturesArray[value];
        }
    }


    void Start()
    {
        _context = new Context();

        var deviceList = GetDevices();
        InitTexture2DSavingArray(deviceList.Count);

        StartCoroutine(SpawnDevicesCoroutine(deviceList));
    }

    private IEnumerator SpawnDevicesCoroutine(DeviceList deviceList)
    {
        yield return null;
        //SpawnDevices(deviceList);
        StartCoroutine(SpawnDevicesAsync(deviceList));
        yield return null;
    }

    private void Update()
    {
        if (Input.GetKeyDown(KeyCode.Space))
        {
            _id = (_id + 1) % _processedDepthTexturesArray.Length;
            TestID = _id;
        }
    }

    private void InitTexture2DSavingArray(int devicesCount)
    {
        _processedDepthTexturesArray = new Texture2D[devicesCount];
        _processedColorTexturesArray = new Texture2D[devicesCount];
    }

    private IEnumerator SpawnDevicesAsync(DeviceList deviceList)
    {
        for (int i = 0; i < deviceList.Count; i++)
        {
            Debug.Log($"Spawn: {i}");

            var label = RsUtilities.GetDeviceSerialNumber(deviceList[i]);
            Debug.Log(label);

            RsDevice spawnedDevice = null;
            var newPrefab = Instantiate(_rsPipePrefab);
            var pipeline = newPrefab.GetComponent<RsProcessingPipe>();

            spawnedDevice = newPrefab.GetComponentInChildren<RsDevice>(true);


            spawnedDevice.DeviceConfiguration.RequestedSerialNumber = label;

            InitRsRenderer(spawnedDevice);

            yield return new WaitForSeconds(5.5f);
        }
    }

    private void SpawnDevices(DeviceList deviceList)
    {
        for (int i = 0; i < deviceList.Count; i++)
        {
            var label = RsUtilities.GetDeviceSerialNumber(deviceList[i]);
            Debug.Log(label);

            RsDevice spawnedDevice = null;
            var newPrefab = Instantiate(_rsPipePrefab);
            spawnedDevice = newPrefab.GetComponentInChildren<RsDevice>(true);

            spawnedDevice.DeviceConfiguration.RequestedSerialNumber = label;

            InitRsRenderer(spawnedDevice);
        }
    }

    private void InitRsRenderer(RsDevice device)
    {
        var depthRenderer = device.gameObject.transform.GetChild(0).GetComponent<RsStreamTextureRenderer>();
        var colorRenderer = device.gameObject.transform.GetChild(2).GetComponent<RsStreamTextureRenderer>();

        _rsDepthRenderers.Add(depthRenderer);
        _rsColorRenderers.Add(colorRenderer);


        for (var i = 0; i < _rsDepthRenderers.Count; i++)
        {
            int a = i;
            Debug.Log($"a: {a}");
            _rsDepthRenderers[a].textureBinding.AddListener((texture) => SetDepthTextureList(texture, ref _processedDepthTexturesArray, a));
            _rsColorRenderers[a].textureBinding.AddListener((texture) => SetColorTextureList(texture, ref _processedColorTexturesArray, a));
        }
    }

    private void SetDepthTextureList(Texture texture, ref Texture2D[] targetTextureList, int ID)
    {
        targetTextureList[ID] = texture as Texture2D;
    }

    private void SetColorTextureList(Texture texture, ref Texture2D[] targetTextureList, int ID)
    {
        targetTextureList[ID] = texture as Texture2D;
    }


    private DeviceList GetDevices()
    {
        DeviceList deviceList = _context.QueryDevices();
        if (deviceList.Count == 0)
        {
            Debug.LogWarning("No device connected, please connect a RealSense device");
            return null;
        }

        return deviceList;
    }
}

Realsense's prefab will be the next parent-child relationship.

RsProccesingPipe
 -RsDevice
  --RsStreamTextureRenderer(DepthTexture)
  --RsStreamTextureRenderer(ColorTexture)

 

I have tried the following from errors and past troubleshooting with no resolution.

  1. Read each unit one by one with an interval of at least 5 seconds between reads
  2. All devices were connected directly to USB ports without USB extension cords.

 

Please let me know.
Thanks.

0

Comments

1 comment

Date Votes

Please sign in to leave a comment.

Didn't find what you were looking for?

New post