How AR and VR can be applied to marketing

Augmented reality (AR) and virtual reality (VR) technologies are making the in-store experience truly fun and innovative. They improve interactivity, level of engagement and offer great excitement.

By Dr Deepak Halan

Can you imagine a bus stop that entertains while you wait? Pepsi created such an experience for commuters in 2014 at a bus shelter in London. Augmented reality (AR) technology was used to project UFOs, giant robots, balloons and a tiger inside the shelter. The technology made it look as if those scenes were actually happening on the street.

Technology is advancing at an unprecedented rate. Increasingly, the real, physical world we occupy seems to be blending with the technological world we have created. And there is no better example of this amalgamation of two worlds than the concept of AR and virtual reality (VR).

But what is AR and VR? What purpose do these serve? And above all, how are AR and VR changing the way goods and services are marketed?

Introduction to AR
AR was introduced in 1974 when Myron Krueger, an American computer artist, built Laboratory Videoplace, combining projectors with video cameras that emitted onscreen silhouettes, to provide an interactive environment for users.

In 1992, American inventor, Lous Rosenberg, developed Virtual Fixtures for the US Air Force. It had an overlay of augmented sensory information to improve performance of direct and remotely-controlled tasks.

Seven years later, in 1999, Hirokazu Kato developed ARToolKit, an open source software library for AR applications that overlay virtual imagery on the real world.

In 2009, Esquire magazine used AR. On scanning the cover of the magazine, Robert Downey Jr came alive to introduce the issue—tilting the magazine in various directions made models in the magazine alter their clothing, and on checking the magazine after midnight, the reader found content different from what it was in the daytime.

Between 2011 and 2013, AR was successfully used for marketing by such companies as Disney, Coca-Cola and National Geographic for campaigns, and in public spaces like shopping malls and Times Square in New York. Later, car manufacturer Volkswagen used AR for service manuals to provide step-by-step repair assistance, enabling service technicians to see the repair processes on the vehicles.

In 2014, Google introduced Google Glass, the first mass-produced, wearable AR device that made it easy to get digital information simply by nodding one’s head. A few months later, Snapchat added the geo-filter feature, which allowed adding geographic locations to photos. Later, they introduced Lenses, a feature that mapped users’ faces to add motion graphics to photos and videos.

In 2016, Nintendo used AR technology for the first time for a mobile game, Pokémon GO, which caught the public eye and became highly popular. In Pokémon GO, animated creatures are overlaid in real-world locations, and the player needs to go out and catch these creatures in the wild by travelling around in the real world.

In April 2017, Mark Zuckerberg, chief executive officer, Facebook, added an AR platform to provide digital effects to still images, making users’ interaction more interesting and enjoyable. The next development was merging computer graphics with the physical environment and making them interactive in response to users’ voice, gestures and touch.
Currently, AR is being used to project graphics onto sports pitches to identify markings and distance, and promote products and brands. It is increasingly becoming an indispensable market tool for business houses to effectively engage shoppers with their product and brand story. The AR industry is a steadily growing market and slated to touch almost US$ 120 billion by 2020 with one billion people estimated to be using it.

AR is an experience that involves integration of digital information with information from the user’s real-world environment on real-time basis. This experience is usually displayed on a screen or monitor, which could be as small as a tablet or mobile, or as large as an advertising billboard. AR uses visuals, sound and even haptics (touch/movement) to create an immersive experience. Simply put, AR is an embellished or altered form of reality where content lays over the user’s real-world views, enhancing information about the world around.
While shopping, extra information about products can be obtained by just scanning the product tag with a smartphone. With AR, it becomes possible to try out clothes and accessories in the comfort of your own home, instead of physically going to the marketplace.

What is VR
The concept of VR dates back to the 1930s, when Stanley G. Weinbaum, a science fiction writer, in his book Pygmalion’s Spectacles referred to a game in which players wear goggles to watch a holographic recording of virtual stories, and experience touch and smell.

The real development started in 1968, when Ivan Sutherland, an electrical engineer at Harvard University, created a head-mounted display (HMD) system for military training applications, nicknamed The Sword of Damocles. The massive headset, being very heavy, was anchored to the ceiling and the user was strapped into the system, making the whole experience quite uncomfortable.

In 1990s, VR headsets were used in some arcade games for simulation. The first home system, Virtual Boy from Nintendo, appeared in 1995. It had stereo sound, LCD screens and head tracking. It did not become a big hit as using the console was uncomfortable and the software was not easily available.

A revolution in VR technology took place in 2014 when Facebook acquired Oculus VR system. Oculus introduced modern easy-to-use plug-and-play compatible headsets with a wide range of supported applications. Modern VR devices are available with 360° cameras and cheaper headsets, and improvements are continuously taking place to improve 3D graphics and make the devices wireless and faster. VR training tools are being widely used in military for combat situations, flight exercises and the like.

VR is an artificial, computer-generated simulation or recreation of a real-life environment or situation. It makes users experience simulated reality first-hand using vision and hearing. It transforms users into a 360-degree environment where they can move around at their own pace to explore whatever is of interest.

A headset is necessary to experience VR. Mostly, VR is used in gaming, entertainment, simulators, education and marketing industries, and is an excellent tool for practising medicine, surgery, flying, driving and so on. The user wears an HMD, makes simple head movements and looks around to interact with the virtual world.

Difference between AR and VR
Both VR and AR present users an enhanced or enriched experience. VR is a distinct concept from AR, as there is no element of real world involved, whereas AR is the blending of computer-generated elements with the real world. AR adds virtual components such as digital images and graphics to perceive a live view with the help of a camera. It may or may not interact with the elements in the live view.

When using VR, what one sees and experiences is different from what is actually around the user, since it is entirely simulated. AR is less intrusive, costs less and is easier to apply to everyday life, versus VR.

Use of AR and VR as a marketing tool
Many brands have either adopted or are thinking of adopting creative ways to promote their products.

Digital technology for advertising conventionally uses online video, pop-ups, pop-unders, displays, banners, etc. Several users found these annoying and intrusive. As a result, blockers started appearing, and this reduced the effectiveness of the advertisements. It was felt that new techniques were required to provide novel experiences to the target audience.

Use of VR and AR for marketing changed the playing field dramatically. Leading businesses are incorporating AR or VR, or both, as part of their digital marketing strategy to attract customers, promote their products, increase sales, retain customers and provide unique experiences to shoppers. Their use is growing across various industries, be it healthcare, medicine, astronomy, education, defence or consumer goods.

AR and VR technologies enable integration of digital technology with marketing strategies, transforming the shopping experience to a new level.

AR is relatively more effective as it involves lower costs and helps customers in making shopping decisions while engaged with the content. AR and VR also provide in-store experience in the comfort of home without having to physically visit the marketplace. e-Commerce platforms are increasingly using AR to enable customers to experience the product as if present in the real-life setting. Use of AR and VR in retail stores, e-commerce and advertising is the next big thing ready to disrupt marketing concepts.

Artificial reality marketing (ARM) is a multi-media promotional activity that encourages interaction between the customer and the brand, creating a greater level of emotional engagement, increasing customer loyalty, enhancing publicity and building long-term relationships. Businesses integrate these technologies into marketing strategies for providing richer and more realistic feel of their products (AR catwalks, AR showrooms), for enabling shoppers to try-on products (AR mirrors) and for raising awareness of their brand (creative and interactive AR campaigns). AR and VR technologies provide shoppers an opportunity to explore products and customise them to their liking.

For example, while shopping for a car, users can try out different colours, check out features and manoeuvre the car through a virtual test drive. Realtors use VR to design virtual tours to provide 360-degree walkthrough of the property, try out different paint colours on the walls or simply add furniture to experience how the property will actually look. ARM combines traditional advertising with mobile devices to attract customers’ attention by projecting 3D images of products onto the screen of their mobile devices, offering them a unique and personal experience.

ARM campaigns need to be aligned with the company’s objective, be utility-oriented and offer a valuable experience to customers. For example, Burberry store in London provides an interactive and exclusive experience to its customers. A hundred screens with 500 speakers enable them to interact with the store. Mirrors with cameras get transformed into screens displaying live video of the customer wearing the product, giving a glimpse of how well a product fits on the customer.

Lego uses ARM during the pre-purchase phase of marketing. In the store, a virtual model of the Lego toy can be created, which gets projected on the package when it is presented to the camera of the Lego kiosk. Once the Lego toy is built, the customer is able to bring the creation to life using Lego Storyteller app.

AR has been successfully used with brochures and catalogues, too. On mobile devices using AR app product information and features can be seen. Automaker Audi created an AR brochure offering customers interactive experience that allowed them to explore the virtual cockpit of 2016 Audi TT. Lowe created VR experience Holoroom using HTC Vive VR headset. Using it, customers could virtually test different types of tools and equipment and pick DIY skills.

AR and VR in retail
Using VR technology, retailers can create simulated experiences to capture the curiosity of shoppers and share their brand story with them. The technology helps them stand out from the competition, attract more customers and boost sales.

John Lewis, a UK-based departmental store, ran a VR campaign, Monty’s Christmas, about a young boy Sam and his best friend, Monty the Penguin, to promote their holiday campaign. The experience allowed children as well as adults to explore the story of Monty and Sam using a Google Cardboard VR headset. The campaign boosted sales and left a lasting impression, making customers eager to visit again during the next season.

Outdoor apparel company, Merrell launched VR campaign TrailScape, using an Oculus Rift headset, to launch Capra hiking boots. It was a 4D, motion-tracked, multi-sensory virtual experience that simulated extreme landscapes like wobbly bridges and landslides. Motion-capture technology put adventurers into scenarios they would not normally encounter on a hike.

Easing the buying decision: Using VR, retailers facilitate shoppers making purchase decisions even where they cannot physically interact with the products. eBay Australia launched the world’s first virtual department store. With the help of Google Cardboard headset, called Shoptical, shoppers could view 3D images of products, and examine different features, colours and materials, getting the familiar physical shopping experience. It allowed users to browse, sort and engage with items of interest by just looking at them. It generated invaluable data for eBay to analyse what catches the shoppers’ attention and how they browse in a VR space.

When product prices are high, one hesitates in making a decision to spend money. For example, booking a costly holiday trip entails considerable research to decide tour plans and make a financial commitment. What if one gets to experience various travel destinations without actually travelling there?

Thomas Cook, a travel agency, provided potential vacationers opportunity to explore a series of virtual holidays through Samsung Gear VR. They created Try Before You Fly, 360-degree VR films showcasing available holidaying opportunities. They even made travellers experience helicopter tours for such sites as Egyptian pyramids. To complement the in-store experience, Thomas Cook created an app to view the content from the comfort of home. The VR experience created immense interest and generated about US$ 15,600, a forty per cent return on investment.

Building an intelligent testing environment: Retailers combine virtual environments and shopper analytics to improve store layout, marketing displays, product packaging and customer journey. Smith Brothers, USA, developed day- and night-time cough drops. They successfully used a virtual store to test the developed product, validate packaging and measure expected performance of the product. The test enabled Smith Brothers to improve the product before finally launching it in the market.

Virtual fitting rooms: Many shoppers consider trying on clothing inconvenient and time-consuming, feel embarrassed asking for a trial room or are simply lazy to take off shoes and clothes. AR-based virtual fitting rooms are a cost-effective solution to get such reluctant shoppers try products, enhance shoppers’ convenience, improve shopping experience and promote business.

Timberland set up a virtual fitting room in Warsaw, Poland, for virtually trying-on the products without going into a dressing room, using a magic mirror in which, the shopper and objects around were augmented on a 2.03m (80-inch) screen. They could take pictures using cellphones for sharing on social media. Images were displayed on shopping windows too, to drive more foot traffic. Virtual fitting rooms are a game changer as they have changed the way traditional and digital marketing operates.

French clothing company Lacoste launched 3D AR app to trigger interactive 3D models of the new range of LCST trainer shoes. To virtually try out the trainer shoes, shoppers placed their foot on the in-store floor graphic and scanned it with a smartphone. The shoe image was superimposed around their feet without having to wait for a shop assistant or taking off their shoes. They could examine the features and swipe to view other trainers in the range. They could also share images via social media.

Italian beauty retailer Sephora launched an AR mirror that enabled shoppers to virtually put makeup on images of their faces and like in real life, mix and match different shades of cosmetics by tapping a palette on the screen.

Ikea created its Ikea Catalogue app to develop AR images of products, which appeared on mobile screen as if they were in the customer’s home. This enabled customers to virtually try out various products for size, shape, colour and positioning to see how these would actually fit into the room.

Volvo uses AR for its customers to view the inner components and features of a Volvo model using Volvo X-ray app, turning their iPads into portable x-ray scanners. Customers can see underneath the car’s exteriors, inner mechanics and components, as well as explore the car’s features through a 360-degree viewing element.

Starbucks offers customers an app for an immersive in-store AR experience in Starbucks Reserve Roastery in Shanghai. The experience includes an online marketplace, Tmall, where shoppers can purchase merchandise or experience the specially-curated coffee. Starbucks is also trying out VR technology to transport viewers to coffee farms in various countries to

19 Crimes, a wine label established by Treasury Wine Estates, launched AR app Living Wine Labels. On hovering a smartphone camera in front of the wine bottle, mug shots of notorious 18th century British criminals (convicts who committed one of the nineteen crimes punishable by exile to Australia) come alive as 3D characters who narrate their story to amuse wine drinkers.

The marketing strategy was simple, creative, entertaining and enhanced the consumer experience. As the experience was surprising and unexpected, consumers were eager to share it with family, friends and social networks. Sales of 19 Crimes increased, helping them become one of the fastest-growing brands.

Making AR marketing a success
The following aspects can guide one towards using AR effectively for marketing purposes.
Consider holistic customer experience: Do not pick AR technology just because it is the latest and fashionable. Before investing think about the holistic customer experience and the purpose AR would actually serve to enhance it. AR should be used when it is essential to showcase content as part of the environment.

Focus on simple, straightforward concept: The campaign should not be overcomplicated; rather it should be simple and straightforward, providing a simple and interesting way to bring the desired experience to shoppers. The campaign should be memorable, making shoppers wanting to share it with others and help in taking a purchase decision.

Make AR experiences Aha! moments: Shoppers should be provided necessary instructions and technical help to guide them. Those who are less tech-savvy will need extra technical help. The AR experience should trigger an Aha! moment and light up their faces.

AR experience should be useful, not just fun: The AR marketing campaign should be useful, educating and convenient. Fun and usefulness should be combined. For example, Ben & Jerry’s AR ice-cream lids, while making the brand image playful, educated shoppers about usefulness of the ingredients used.

Design the AR experience considering the environment: Take advantage of the physical environment, be aware of where and how users will be interacting with the content—out on the streets, at home or inside a shopping mall; in a rush or at leisure—how far will they be from the products they scan, will they be using their own mobile data or like to connect to a hotspot, and what would be the best moment to make them try the experience.

Test effectiveness of the experience: It is important to test the AR experience before launching, to check its usability, whether it resonates with shoppers and whether they will understand the concept as intended. Consider providing free trials and demo apps to try out the concept.

Measure profitability: At the end of the day, it is profitability that wins the race. ROI counts the most. Some of the ways that can be used to measure the success and ROI are media coverage, increased brand value, engagement time spent and number of interactions, and clicks needed to make a purchase.

The future
AR and VR technologies are making the in-store experience truly fun and innovative. They improve interactivity, level of engagement and offer great excitement. The experience cannot be equalled by online shopping and will be of great assistance to the brick-and-mortar retail stores to considerably increase the numbers of shoppers. This is more applicable to India, where physical stores are struggling against huge discounts being offered by online shopping sites.

The technologies can be explored in several ways by retailers to improve their customer base and enhance customer satisfaction. These technologies are adaptable for many uses as they integrate physical and digital worlds. Enormous opportunity is available to combine online and offline experiences into an omni-channel marketing strategy to improve sales.

While AR and VR technologies have been around for quite some time, businesses are only just learning and experiencing their true potential. There is plenty still to be explored to make AR and VR technologies work to promote businesses and brands. The technologies provide great opportunities to explore and experiment with out-of-the-box brand engagement to increase sales and help strengthen the bond with customers.

Adoption of these technologies may have been slow, but they are undoubtedly set to become highly popular and useful. Mixed reality (MR), also known as hybrid reality or extended reality (XR), is an intersection between VR and AR, and joins the finest aspects of both technologies.

The author is the associate professor at School of Management Sciences, Apeejay Stya University.

This article was first published in the March 2020 issue of Electronics For You.