The three AI technologies powering checkout-free retail

Checkout-free shopping is revolutionizing the retail space, cutting down on time wasted in line and improving customer experience while shopping. Though the idea of checkout-free technology powering your shopping trips may seem like a far-off technology, it actually utilizes three established technological components that work together to provide a smooth and frictionless shopping experience.

Each of these technologies, deep learning, sensor fusion, and computer vision, work synergistically to make futuristic-seeming technology part of our everyday lives. 

Machine Learning & Deep Learning

Machine learning, at its core, works by recognizing patterns. Whether it be from simple activities that come up in our day-to-day lives like reading and writing or more complex, technical activities like a protein structure unfolding, patterns are patterns, and they all can benefit from  machine learning technology.

The first technology at play in checkout-free retail is deep learning. This is an evolution of machine learning that can help find more complicated patterns by crunching numbers through different layers of neural networks. The deeper the network, the better able it is to recognize and model more complex patterns.

There are three key elements of deep learning - those multiple layers of neural networks, the availability of data to accurately train the neural networks and get an accurate result and high-performance compute availability like those found in GPU or specialized AI accelerators to tune these deep neural network layers.

A common problem with implementing checkout-free technology is the issue of determining who picked what from the shelves. At a busy concession stand, in the middle of the basketball game where there’s a high volume of consumers at a checkout-free store, two customers’ arms might reach for the same item and grab drinks next to each other. Deep learning, when functioning in its proper role for checkout-free shopping, has studied the patterns of shoppers and is easily able to detect which shopper was reaching for which item on the shelf.

Sensor Fusion

Sensor fusion is just as it sounds -- the fusion of multiple sensors to compile information from different sources. In nature, honeybees see through many eyes and feel through antennas, with all that information converging in their brains to relay sensations of color, motion, light, scents, and the speed of the wind flying around it to help them navigate and look for nectar to turn into honey. 

There are three different forms of sensor fusion. The first is when multiple sensors, all the same kind, are used in non-overlapping ways to increase the quantity of information without affecting the quality. The second kind of sensor fusion allows similar sensors to overlap, which can gather more precise information rather than a larger quantity. The third kind called multi-modal sensor fusion brings information from sensors of different modalities together. This allows gathering information from different modes that may be less reliable or imprecise on their own, but when put together provide a much better image of the whole.  

Though it may seem complicated, its application in retail is quite simple. Cameras placed around the stores allow customers to be identified and tracked while protecting their privacy. Information from multiple cameras and overlapping fields of vision provide a complete picture of the store and products on shelves. Some products, often those smaller than the average human hand, can be hard for cameras to properly identify. In order to combat any issues this may cause, sensors can also be placed directly on the shelves to sense when items are picked up or put back. This multimodal approach to checkout-free technology gives the clearest picture of all parts of a store for all kinds of products -- smallest to the largest. 

Computer Vision 

The third and final technology that powers checkout-free shopping is computer vision. A field of computer science, computer vision was first utilized in the 1960s to understand the way the human brain processes visual information and replicate that process with computers. Getting computers to see and process the way that humans do is incredibly difficult. After 60 years of research, the task still seems impossible due to not only the complex nature of the human brain but also the way light and objects interact with our eyes. 

Computer vision, as it functions today, involves sensor technology, physics, math and statistics, neuroscience, AI, computer graphics, and computer science. It’s been incredibly successful in applications where there is a constrained environment like in check processing, postal sorting, and vehicle license plate reading. It also has promising uses in facial recognition, filtering content on social media, and augmented reality.

For e-commerce, computer vision can be a great help in organizing online product catalog based on photos. In traditional brick-and-mortar retail, computer vision aids in tracking shoppers and their location in the stores, providing essential data like high traffic areas to help guide product placements. With multiple cameras, computer vision can give a more holistic and complete bird’s eye view of the store, products and customers. 

When considering checkout-free technology for retail, cost of deployment is an important factor, but accuracy is what matters most. If you’re 90 percent accurate there’s 10 percent missing! Being able to achieve 99 percent accuracy or more is the goal, and the combination of these three technologies provides the most forward-thinking, precise and error-free approach to retail technology. It also brings about the best results for retailers looking to increase throughput and access data. Not only that, this technology offers customers a frictionless shopping experience where they can shop quickly and easily, without having to worry about being charged correctly. 

Motilal Agrawal is the Chief Scientist and Co-Founder of Zippin.