With the expanding market with XR content, be it augmented, mixed, or virtual reality, it is becoming increasingly difficult to deliver high quality content without restricting users to dedicated spaces or have setups that non-technical users may struggle to use. Here at Boston Labs we have had an exciting opportunity to try out an NVIDIA product called CloudXR which can remove those limitations.
Traditional XR experiences
Typically XR content is delivered via three usable platforms, a tethered headset, an all-in-one system or a mobile device. The tethered headset (e.g. HTC VIVE) is a headset which would be plugged into a VR capable system and be bound to the GPU as well as the base stations provided for the play area. This gives users the best performance as the system includes a powerful desktop or workstation CPU and GPU but results in a very limited area to work in as you must remain close by. Workstation class GPU’s such as NVIDIA’s RTX / Quadro series are commonly available and deliver an excellent visual experience.
All in one systems (e.g. Oculus Quest 2) are stand-alone devices which allow the user to use XR content without the need for a full VR kit setup and PC. This gives a lot more freedom at the cost of quality as there is no dedicated GPU. These types of platforms rely on batteries and therefore have low power consumption mobile processors and graphics which, whilst impressive for their power usage and portability, cannot compare to dedicated desktop component.
Mobile devices (Android and iOS) give the most freedom as they are widely available and most people already have one. These devices typically cost very little for a capable device. They also can be placed in the hand meaning they do not need to restrict the users view of the real world but can be combined with a cardboard shell to be head mounted. The result is typically the lowest quality of the three options as the processing power is very limited, and performance will vary wildly from device to device. Naturally, such devices have similar or less powerful mobile processors to all-in-one devices and as a result, capabilities are somewhat restricted. However, one huge advantage is that most devices have their own cellular connection so 5G is often readily available to get access to content on the move.
Figure 1 - Mobile, All in One and Tethered XR experiences
CloudXR
CloudXR is a ground-breaking technology which allows XR content to be streamed remotely to a local device. This means that content can be rendered on one device then the pixels and controls sent over the network or internet are displayed on another device for user consumption. This enables a high-quality workstation experience that can be viewed on any of the platforms. This, as you can guess, breaks down a lot of boundaries that would not have been possible previously as before you were limited to one of three conventional ways.
A fully performant GPU is used for rendering environments in real time which are then viewed on all in one or mobile devices; in short you get the full performance of the datacentre server or workstation in the palm of your hand. That could mean having an NVIDIA RTX GPU with its professional visualisation features, utilising Intel Xeon or AMD EPYC/Zen based processors and ultra-fast NVMe storage immediately accessible to any XR device. All that would be required is the end device to connect to the server and to have a strong WiFi or 5G cellular connection to receive the experience.
This can solve a lot of problems that are caused by the regular means of XR. An example being having a tethered headset; not only are users limited to a use space but also need to worry about the cable which runs from the GPU. Whilst it is already possible to buy a dedicated wireless WiGig adapter for the VIVE that allows for a wireless experience, this requires additional setup and costs. Not only that but the WiGig range is short (typically less than 15M from the workstation), the receiver is quite bulky on your headset, it needs a sensor connected to the workstation via PCIe card, a stand-alone battery pack and needs installing HTC VIVE wireless software. By using CloudXR, the content can be streamed to a stand-alone or mobile device with no other modifications than loading an app, removing the need for the complex setup and space limitations.
CloudXR also has the added benefit of using software that was once only dedicated for tethered devices on both all-in-one and mobile. This is also breaking a lot of traditional boundaries by giving the full GPU experience of applications with features such as Ray Tracing that would have been impossible on native devices.
The requirements for the setup are simple - a fully capable XR system with a Pascal or later generation of NVIDIA GPU. Software includes the CloudXR Client & Server packages (for respective devices) as well as either SteamVR with XR applications or Autodesk software such as VRED.
Figure 2 – CloudXR Concept
Before CloudXR - MUVR
Our original solution to having a multiple user XR experience was the MU-VR; an all-in-one system which allowed for up to four people in the same location to be able to interact in the same instance. This was achieved using Virtual Machines (VMs). VMs emulate a computer system based on a computer architecture and provide the same functionality as a physical system. This has the added benefit of users sharing one system’s resources and making management easier due to everything being contained in one place. Benefits from sharing the hardware mean you can allocate resources to different users when needed. Passing through enterprise grade GPUs, a way of allowing VMs to use a GPU without sharing with others on the same system, enabled 4 VR headsets to be deployed in parallel. Then when loaded up with compatible software and suitable networking, users can interact with each other in VR or run entirely private, separate sessions.
With the release of CloudXR the reliance on tethered headsets is loosened and the number of users per server can be expanded greatly. Additionally, with no light houses needed for tracking headsets and controllers or WiGig line of sight data connectivity with “all-in-one” or mobile devices, limited play areas are no longer restricted to fixed, predefined squares.
Figure 3 – Our original solution; the MU-VR
From MUVR to Cloud-XR: Our Experience
Working with CloudXR has been a great experience from end to end. Our team were able to test out the flexibility of the SDK through numerous applications and were pleased with the results.
Whilst the full stack may look complex, the items in green are all provided by the NVIDIA CloudXR SDK in a simple-to-use package. Deployment is comparable in difficulty to traditional desktop VR setup but with the slight added complexity of configuring a network layer of Ethernet, WiFi or 5G.
Figure 4 - The CloudXR Stack
As part of our test, the team deployed the full stack in line with NVIDIA's guidelines. They then compared the experiences received from different VR devices, both native GPU and with CloudXR remote GPU acceleration.
The devices tested include HTC VIVE Pro (tethered), an Oculus Quest 2 (all-in-one) and a OnePlus 7T Pro (mobile) HMD with a variety of VR experiences.
From this point, when we refer to the terms tethered, all-in-one and mobile devices are the devices above respectively.
Don't forget to look out for Part 2, where our team will look at latency and bandwidth, the graphical performance and explore our Boston Solution - the MU-CXR! Keep your eyes peeled on our channels for the second part of our in depth review. In the meantime, don't hold back on contacting our team if you would like to know more about the NVIDIA CloudXR by emailing us at [email protected] or calling us on 01727 876100.
Written by:
Peter Wilsher
Junior Field Application Engineer