Let's talk about Virtual Reality

VR is awesome! Right? 
To quote Mark Zuckerberg: "VR has the potential to be the most social platform".
​It opens a whole new world to visualisation: Gaming, Immersion, Movies, Experiences. A few months ago, I have had the opportunity to try the HTC VIVE on a stealth game. It was literally one of the best experiences of my life. It really made an impact on me. Since then, I have started learning deeply how to develop applications on it and have done a lot of research on how VR works exactly. 

Today, I will be explaining how Virtual Reality works, and will try to go deep into it from an engineering perspective.

---

Virtual Reality tricks your brain into believing you are in a 3D world. The first way VR does this is with the stereoscopic display. This works by displaying two slightly different angles of the scene to each eye, stimulating depth. the image above is an example of the stereoscopic display. In order to create a life like experience, other techniques are developed. For instance, parallax defines the farther objects to the player seem to move slower, making it more realistic.
As you can see in the picture, the angle of the weapon is slightly different on each side, as is the crosshair. Once you put the headset, everything lines up perfectly, as if you actually a real weapon in your hand.

For VR to work convincingly, the person needs to feel completely immersed in an environment, but also have the ability to interact with it. It's the interaction - known as telepresence - that sets VR apart from other virtual world systems and 3D cinema. Moreover, there is the notion of feedback to be taken into account.
​
When interacting with objects in real life, we can touch, pick up and feel the objects. It is referred as force feedback. In order to create this in the VR world, things such as vibrating controllers have been implemented. They are called haptic systems. 

A very important part of VR is about tracking the movement and maintaining the illusion.

Virtual Reality just wouldn't be Virtual Reality without the ability to look around. I mean, otherwise, the whole thing would not be immersive.
Tracking a user's motion is nothing new in the world of computer technology. The mouse that you are currently using is already a device for tracking hand motion, a very simple one compared to an HUD like the Oculus Rift or the HTC Vive. A mouse could be described in engineering terms as a "two degrees of freedom" tracking device, since you can move it left-to-right and front-to-back across a desk. Mathematically speaking, it represents the x and y axis.

Most VR headsets have at least three degrees of freedom: you can "pitch" your head by tilting up and down , "roll " your head from shoulder to shoulder, or "yaw" your head from side to side (for example to say "no"). A few even let you move around a physical room in three dimension (such as the HTC Vive), for a total of six degrees of freedom.

The image above represents the tracking system of the HTC Vive.
The trackers take some time to setup, but do the work efficiently.

As you can imagine, tracking someone or something moving around in a combination of six different ways is more complicated than a mouse. One device for recording the kind of information needed is called an Inertial Measurement Unit (IMU).

• An IMU uses an accelerometer, a device like a miniature weight on springs for measuring forces in three dimensions, and also records linear movement as well as gravity.
• A gyroscope is used for tracking angular movements.
• Last but not least, a magnetometer - like a three dimensional compass - gives the orientation of the IMU relative to the earth.

Combining the measurements of the different devices in a process called sensor fusion, an IMU can estimate how a device is rotated or moved over time.
​
For VR to work, the illusion must be extraordinarily slick. Humans are extremely sensitive to visual inconsistencies; even small snags can cause "VR sickness" (also called Cyber-sickness), an affliction like motion-sickness. So images must update very quickly. That requires beefy computing hardware capable of generating 90 or more frames per second, also called FPS (standard TV, and most video games, target only 30 updates per second). And the sensors that track the user's head must be able to talk to the computer at least that fast: any delay can cause an unpleasant dragging sensation. Despite the difficulties, engineers are convinced that such problems have, at last, been banished.

Different VR platforms also have different specifications on the headsets themselves. The HTC Vive and Oculus Rift both have 90Hz displays, while the Playstation VR has a 60Hz display. It is a rule of thumb that you want your frames per second to match your monitor’s refresh rate, so it is recommended that the Vive and Rift both maintain 90 FPS while the PS VR maintains 60 FPS. Mobile is a different story, as different phones have different resolutions, but maintaining at least 60 FPS is the goal.

Expanding more on how FPS and refresh rate work, FPS and the refresh rate of a monitor are two separate things independent from each other. Frames per second is how fast your GPU can display images, per second. 60 FPS means that the GPU is outputting 60 images every second. The refresh rate of a monitor is how fast the monitor can display images per second, measured in hertz (Hz). This means that if you are playing a game and the FPS is 120 but your monitor refresh rate is 60 Hz, you will only be able to display 60 FPS. You are essentially losing half of your frames, which is not a good thing as “tearing” can occur.

Tearing is the phenomenon of objects in a game breaking up into a few pieces and being displayed in two different locations along the X axis giving a tearing effect.


The battle of VR headsets

1. HTC Vive
​
 + Most complete and best overall VR experience
 + Software partnership with Valve
 - Requires a high-end GPU
 - Expensive

As of right now, the HTC Vive is the most complete VR experience on the market. As well as a headset and two base stations (which are used for tracking the headset's movement) the Vive also includes two motion controllers in the box. This is important, since it allows the Vive to offer a much more immersive experience than using a traditional controller. 

2. Oculus Rift

 + VR pioneers
 + Facebook backing
 - No room-scale

The current VR arms race is all thanks to one man: Oculus founder Palmer Luckey. As a teenager, Luckey collected VR tech and was fascinated with making his own headset in his garage. Numerous prototypes and a $2Billion Facebook buyout later, Oculus is still the biggest name in VR.
Now the consumer version of the Oculus Rift is finally out and we can get our hands on the headset that started it all.

3. Samsung Gear VR

+ Good build quality
+ Works with many of the most popular phones
- A bit bulky
- Expensive compared to other mobile VR offerings

Powered by technology from Oculus, the Samsung Gear VR was effectively the first VR headset on the market. To use it, you simply grab a Samsung phone, download apps and games from the Oculus store, and clip it into the headset.
The original model supported the Note 4, but subsequent models have expanded compatibility to a number of phones including the Note 5, Galaxy S6 and Galaxy S7.

Below, you will find the specs needed for a computer to support VR.

Actually, the majority of people who own desktops are unable to use virtual reality, as their computers are not powerful enough. Steam recommends an Intel i5 Haswell or newer and either an Nvidia GTX 970 or AMD Radeon R9 290 for a smooth experience.

I hope this gave you some insight on Virtual Reality and how it works.