Video headsets are critical elements of augmented reality. They come in two basic designs: Optical See-Through (OST) where the wearer perceives the real world directly, and Video See-Through (VST) where cameras are used to pick up the environment. Microsoft Hololens is an OST headset, and Occulus Rift is VST. In both cases, computer-generated imagery is superimposed on the real world. Three applications illustrate the power of this technology.

Augmented Surgery

In conventional surgery, doctors have to shift their attention between the patient and x-rays posted in the operating room. With the new VOSTARS system, this is a thing of the past and the surgeon can focus their complete attention on the surgical field:

The VOSTARS (‘Video Optical See-Through Augmented Reality surgical System’) medical visor is a head-mounted display (HMD) system that is capable of superimposing the patient’s x-ray images in perfect 3D unison with their anatomy. The visor also presents a patient’s anaesthetic data, heart rate, body temperature, blood pressure, and breathing rates, conveniently into the surgeon’s field of vision, in a drive to increase accuracy by focusing on the operation and reduce time by never having to look away.

In a practical sense, this means the surgery goes faster and is more accurate. The speed is important because it reduces the time the patient is knocked out with anesthetics by some 11%.  VOSTARS uses a custom headset that’s actually a hybrid of OST and VST.

The system isn’t’ complete yet, but trials are expected in mid-2018. It goes without saying that it’s a life-critical system that will be thoroughly tested before it’s rolled out for general use.

Losing Weight by Playing Games

Not all life-threatening conditions can be addressed by surgery. Obesity is a potential killer that’s been on the rise since the late 1970s. It mystifies the medical establishment, giving rise to a plethora of diet and exercise plans as well as pharmaceuticals.

The Rise of Obesity

One the difficult issues with weight loss is starter systems for the morbidly obese, people who are so fat they have a hard time running or lifting weights without crushing their joints. One non-obvious way to address this problem is with virtual reality games that involved lots of motion.

A fellow named Job Stauffer managed to lose 50 pounds – from 300 t0 250 – playing a VR game called Soundboxing. This is a game where the user uploads music and engages in a shoot-’em up game.  Check the video to see him doing his routine.

The idea occurred to Staufer because he works in the gaming business. It’s a bit ironic that the technology that probably helped him put on the pounds is now helping him take them off.

Giving Sight to the Blind

As cool as as surgery and fitness are, they address easily fixable problems. Deafness and blindness are a whole different story. Hearing aids have improved a lot thanks to signal processing systems that change the frequency of perceived sounds onto ranges where the suffers hearing is best.

A similar trick works for many blind people. Most blind people retain the ability to see images up close: they’re not so much stone blind as visually impaired. A new VST headset allows severely visually impaired people to function nearly as well as sighted people.

Yvonne Felix is a sculptor has a specialized headset that has given her vision back:

Ms. Felix, who has Stargardt disease, wears a device from a company called eSight. The eSight 3—which weighs less than a quarter of a pound and is operated by hand-held remote—captures the world through a camera system and then displays it on OLED screens that sit very close to the eyes.

It’s lightweight, remote controlled, and in its early days. It’s not hard to imagine future versions that will track the eyes, eliminating the need for the remote; and it will probably become a bit less intrusive as well. But it’s already capable of giving sight to the blind, which is about as cool as technology gets.