The History of Virtual and Augmented Reality

The idea of virtual and augmented reality has a much more prominent place in history than you might think. In 1935, a short story called “Pygmalion’s Spectacles” by American science-fiction writer Stanley G. Weinbaum told the tale of a professor who invented a pair of goggles that enabled a user to trigger “a movie that gives one sight and sound … taste, smell, and touch… . You are in the story, you speak to the shadows (characters) and they reply, and instead of being on a screen, the story is all about you, and you are in it.”

Weinbaum’s writing predates computers and nearly predates the invention of television. If Weinbaum were to travel to the present and see just how closely his vision of virtual reality resembles that of the emerging technology of today, he would probably be shocked.

Both virtual reality and augmented reality have an incredibly rich and diverse history, far too deep to fully cover here. However, a general overview of some of the various incarnations of these technologies may provide some insight into where the technologies may be headed in the future.

The father of virtual reality

In 1955, a cinematographer named Morton Heilig, considered the father of virtual reality, imagined a multisensory theater called “The Cinema of the Future.” Heilig created the Sensorama, an arcade-style mechanical cabinet built to stimulate the senses, for which he then developed a number of short films. It included many of the features prevalent in modern-day virtual reality headsets, such as a stereoscopic 3D display, stereo speakers, and haptic feedback through vibrations in the user’s chair.

Shortly after inventing the Sensorama, Heilig also patented the Telesphere Mask, the first-ever head-mounted display (HMD), which provided stereoscopic 3D visuals and stereo sound. This (relatively) small HMD more closely resembles today’s consumer virtual reality headsets than the bulky seated form factor of the Sensorama. The patent image bears a striking resemblance to many of the headsets available today.

Augmented reality gets a name

In 1990, Tom Caudell, an employee at Boeing Computer Services Research, was asked to create a replacement for Boeing’s current system of large plywood boards with wiring instructions for each aircraft being built. Caudell and his co-worker David Mizell proposed a head-mounted display for construction workers that superimposed the position of cables through the eyewear and projected them onto multipurpose, reusable boards. Instead of having to use different boards for each aircraft, the custom wiring instructions could instead be worn by the workers themselves. Caudell and Mizell coined the term augmented reality for this technology.

Industrial strength augmented reality

Tom Caudell and David Mizell not only coined the term augmented reality, but also kicked off the use of augmented reality in an industrial setting.

Industrial manufacturing is poised to be one of the most significant areas of augmented reality expansion in the near future. Whereas companies developing consumer-facing augmented reality applications must deal with a number of complications (including an unknown user base and unknown environments to perform in), those variables can be managed or even removed within the tightly controlled environment of an industrial manufacturing workspace.

Manufacturers can develop targeted hardware and applications that can help their workforce train more quickly, work faster, access data more easily, and help avoid errors. All these benefits can lead to massive improvements in a company’s bottom line, making AR usage within industrial settings a match made in heaven.

Early virtual reality failures

In 1993, Sega, a videogame company riding high on the release of its massively popular Sega Genesis, announced the Sega virtual reality headset for the Sega Genesis at the Consumer Electronics Show (CES). Sega originally intended to deliver the device for $200 in the fall of 1993, a moderately affordable price point at the time. However, the system was plagued by development difficulties and was never released to the public. Sega’s CEO at the time, Tom Kalinske, said that the Sega VR was shelved due to testers developing painful headaches and motion sickness — an unfortunate first foray into consumer gaming virtual reality.

At the same time, another gaming industry veteran decided to release its take on virtual reality gaming. The Nintendo Virtual Boy was released as the first portable unit capable of displaying stereoscopic 3D graphics. With the Virtual Boy, Nintendo had hoped to capture a unique technology and cement Nintendo’s reputation as an innovator by encouraging more creativity in game development outside the traditional 2D screen space.

However, development issues also plagued the Virtual Boy. Initial tests of color LCDs were said to have caused jumpy images, leading to Nintendo retaining the red LEDs that the Virtual Boy was eventually released with. Additionally, the Virtual Boy started as a head-mounted system including tracking. However, with concerns about motion sickness and the risk of developing lazy-eye conditions in children, Nintendo changed the head-mounted system to a tabletop format. Critics panned the system. It was never able to meet sales targets and disappeared from the market within a year.

These early failures, coupled with other failed attempts at creating mass-consumer virtual reality devices, pushed virtual reality advances back into research laboratories and academia for a few decades.

Virtual reality breaks through

In 2010, a tech entrepreneur named Palmer Luckey was frustrated with the existing virtual reality head-mounted displays on the market. Almost all were expensive, extremely heavy, had a small field of view (the total viewing area a user can see), and high latency (delays between user interaction and the display refreshing to reflect those interactions) leading to a very poor end-user experience.

Channeling these frustrations, Luckey built a series of prototype HMDs, focusing on creating a low-cost, low-latency, large-field-of-view, and comfortably weighted headset. His sixth-generation unit was named Oculus Rift, and he offered it on the project funding website Kickstarter as Rift Development Kit 1 (DK1).

The Kickstarter campaign was a massive success, raising $2.4 million, almost 980 percent of the original target. More important, the Kickstarter campaign served to propel interest in virtual reality in the consumer market to an all-time high.

Augmented reality hits the mainstream

Augmented reality was treated to a surprise surge in popularity from a rather unexpected source: the mobile phone. Similar to virtual reality, augmented reality had trudged along in relative obscurity for a few decades since its inception. Interest had increased slightly with the rise of virtual reality in recent years, and new developments from companies such as Microsoft, Meta, and Magic Leap had shown promise, but nothing was available for mass consumption, and it was unclear when anything would be.

In 2017, augmented reality underwent its largest boost in public awareness since its inception, as both Apple and Google released their own takes on augmented reality for their various handheld mobile devices running either iOS or Android. Though neither has released exact numbers, estimates place the number of users with ARKit- or ARCore-capable devices to have reached over a quarter billion by the end of 2017.

Augmented reality, long toiling in relative obscurity, suddenly had an enormous market of consumers to create content for, and developers began racing to create content for that market. Some examples include augmented reality gaming applications, utilities that place 3D objects within a real room for interior decorating planning, map utility applications that overlay the real world with turn-by-turn directions or points of interest, and apps that can translate foreign language signs simply by pointing a mobile device camera at them.