What’s inside a VR camera?

Through the years I’ve developed several VR cameras. Starting from “RIGs” and DIY-like solutions to fully-featured professional cameras. Here I will tell you about key components which make up a VR camera.

What is “VR Camera”?

If you glue two cameras side by side, you get a VR camera. It can shoot spatial video. Swap rectilinear lenses for a fisheye and you get VR180 camera. Now cut and glue the cameras back to back to get VR360 camera. As you can see, VR camera is made of two regular cameras. It needs 2 lenses, 2 image sensors, and some other components I will discuss later.

I know there are cameras with more lenses/sensors. Canon released a lens to shoot VR180 using only one sensor. Because Canon can. But for the sake of simplicity, let me skip all the exceptions.

Lenses

The first thing you notice is the lenses. If you see more than one, it is probably a VR camera (or a modern smartphone).

Spatial Videos are shot using common rectilinear lenses. There’s not much to say about them.

VR180 and VR360 on the other hand, use fisheye lenses. Rectilinear lenses are technically limited in FOV. Fisheye lenses are way more wider. Typical FOV of fisheye lens is more than 180 degrees. With great FOV comes great drawback – strait lines don’t look strait with fisheye lens. To view fisheye images, you need to convert those to rectilinear projection first.

VR180 cameras use a pair of 180-185 degree fisheye lenses. 180-185 degree fisheye lenses are widely used in security cameras. There are tonnes of different fisheye lenses available off-the-shelf.

180 degree FOV is not sufficient for VR360 cameras, however. VR360 needs a fair amount of overlap to stitch images properly. Lenses that have >200 degrees FOV aren’t cheap, and there are not as many options available on the market.

Image sensors

Ideal sensor for VR180/VR360 should have 1:1 aspect ratio. But square sensors are very rare. Best solution is to crop 4:3 sensor to 3:3. Cropping 16:9 to 9:9 will be a huge waste of resolution. Another important parameter is sensor’s frame rate. 4:3 sensors are more likely meant for photography and have low maximum FPS. For video you need at least 30 fps. 50 or 60 fps is better.

Spatial camera doesn’t need square sensor because it captures standard 16:9 videos. 16:9 sensors are designed for video capture and have high frame rate.

IMU

IMU – inertial measurement unit. This is another key component of any VR camera. Each of you use this chip. It helps your phone change screen orientation when you turn your phone. VR camera needs it for the same reason. The data from the IMU gets embedded into the video file. A VR headset can then use it to match direction of view of the video and direction of view of the headset itself.

Brain

There are two options. A specialized webcam IC is the first and cheapest. It is only suitable for making webcam.

The other option is an SoC (System on Chip). With an SoC, camera can do recording and streaming. Modern SoCs can record up to 8K30. Pretty impressive, huh?

Wrapping up

VR camera is an impressive piece of tech! High-end components needed to build a good VR camera. And that’s just the tip of the iceberg.

🔗 Link