Traditional and new-school retailers alike are using AI and robotics to automate various parts of the retail chain, from manufacturing to last-mile delivery.
Retail is under pressure to crack the AI code.
After all, corporations in every industry are scrambling to adapt and integrate artificial intelligence into their products — and retail is no exception. Between Q1’13 and Q2’19, retail AI startups looking to address retailers’ AI needs raised a total of $2.67B across 564 deals.
For traditional retail giants, this means entering the playing field with the likes of e-commerce behemoths Amazon and Alibaba, both of which are leveraging big data and powerful AI algorithms to transform the retail space. For example, Alibaba has turned more than 1M mom and pop stores across China into AI-backed smart stores that can predict surges in demand for certain goods and use heat sensor data to better analyze foot traffic, while Amazon is working on expanding its range of cashierless Amazon Go stores.
In addition to fierce competition, the need for a change in strategy is being underscored by the record rates at which many US retailers are shutting down.
Radioshack, Toys R Us, and Aerosoles all filed for bankruptcy in 2017, while 2018 brought us the bankruptcies of Sears, Mattress Firm, Brookstone, Nine West, Claire’s, and Remington Outdoor.
In just the first months of 2019, Diesel, Charlotte Russe, Payless, and Gymboree have joined their ranks.
Please click to enlarge.
But in our 2018 analysis of 1,600+ publicly traded US retailers’ earnings calls, we found that only 9 of 50+ companies had started to discuss an AI strategy. While most traditional retailers have yet to develop an AI strategy, some stores and e-commerce players have started using AI and robotics to transform the retail space. With recent advances like computer vision-based cashierless stores, an increasing number of retailers may be forced to improve their AI game in the coming years.
Using CB Insights data, we dug into the changing retail landscape. Below, we analyze how AI, machine learning, and computer vision-based technologies — including robots used for heavy lifting, navigation, and assembly tasks — are impacting all parts of the retail chain, from the manufacturing of goods all the way to their distribution.
Table of contents
- AI & robotics attack different parts of the retail chain
- Design
- Manufacturing
- Warehouse automation
- Consumer channels
- Supply chain & logistics
- Bridging online and offline retail
- Conclusion
AI & robotics attack different parts of the retail chain
E-commerce giants Amazon and Alibaba are using big data and AI to implement end-to-end solutions that focus on the overall retail experience (both online and offline). However, most retailers are focusing their AI-inspired efforts on more specific parts of the retail chain.
We looked at the different stages in bringing a product to market, from manufacturing to delivery, and how companies are using AI-enabled automation — including facial recognition, demand forecasting, and computer vision-based robots — to enhance each of these stages.
Design: Using AI to inform fashion decisions
Artificial intelligence can be a powerful tool for deciding how to design better products.
In 2018, Tommy Hilfiger announced that it was working with IBM and the Fashion Institute of Technology on a project called “Reimagine Retail,” the goal of which was to use artificial intelligence to help Tommy Hilfiger better understand customer sentiment around its products and design better patterns, silhouettes, colors, and styles.
Researchers from FIT used IBM technology to analyze 15,000 of Tommy Hilfiger’s product images, alongside 600,000 runway images and almost 100,000 images of different generic patterns and clothing fabrics.
An example of an AI-powered design from the IBM project.
“In the future of fashion with AI, designers could get insights from internal and external data sources so they can make their designs more informed and relevant. It also may give them the ability to design elements that will customize and personalize looks for certain markets or consumers.” — Steve Laughlin, General Manager of IBM Global Consumer Industries
That corpus of information was then used to develop new styles, rooted in Hilfiger’s style but incorporating trending patterns and colors, for Hilfiger’s design team.
Startups are already working on making this kind of artificial intelligence-driven development available to other companies. 3D design platforms like CLO offer brands the ability to simulate designed garments in full 3D and shorten the development process.
True Fit has raised more than $100M, including a $55M Series C in early 2018, for its fashion-based big data platform. The company helps more than 100M registered users find new clothes through AI-powered discovery.
At L’Oreal retail locations, the company allows customers to virtually try on different kinds of makeup using both mobile apps and an interactive smart mirror technology developed by Alibaba. The technology gives customers the ability to sample a range of products before they buy, and supplies L’Oreal with data they can use to better match a user’s facial features and appearance to their products.
Monolith AI is one company working on bringing this kind of AI-backed product design to a wider variety of industries, with a particular focus on automotive, aerospace, and consumer packaging.
Manufacturing: Bringing a product to market
Meeting ever-shifting consumer demands requires reducing manufacturing lead times. To do so, some retailers are turning to robots using computer vision for assembling products like apparel and footwear.
Adidas benefits from both in-house and third-party manufacturing automation
Manufacturing jobs are notoriously vulnerable to being outsourced to developing countries where labor costs are cheaper.
But dropping industrial robot costs are bringing manufacturing bases closer to the site of demand, with retailers like Adidas benefiting from this trend.
The Chinese t-shirt manufacturer Tian Yuan Garments, which produces garments for customers including Adidas, Reebok, and Armani, announced plans to construct a new textile factory in Little Rock, Arkansas in 2017. As of June 2019, more than 150 workers are employed at the factory, with plans to hire about 400 full-time workers when the plant is at peak production capacity.
The company plans to use an army of automated “Sewbots” to make up the world’s first fully automated t-shirt production line.
The machine vision-based sewing robots, originally developed by Georgia-based startup SoftWear Automation, will eventually manufacture apparel for Adidas at a per-unit cost of 33 cents.
“We will install 21 production lines. When fully operational, the system will make one T-shirt every 22 seconds. We will produce 800,000 T-shirts a day for Adidas… Around the world, even the cheapest labor market can’t compete with us.” — Tang Xinhong, chairman of Tianyuan Garments, ChinaDaily
Most of the heavy lifting will be done by the AI-driven robots, with human workers taking over jobs around robot maintenance and operation.
In fact, a 2012 DARPA contract awarded to SoftWear Automation states, “complete production facilities that produce garments with zero direct labor is the ultimate goal.”
Moving textile production facilities from China to the US means moving them closer both to the source of cotton, a crucial production resource, and to the American market, where many of the clothes will eventually be sold.
Adidas unveiled an ambitious robot-run shoe factory called Speedfactory in Germany in 2015, and opened another Speedfactory in Atlanta in mid-2018. Using its Speedfactories, Adidas can produce and sell a pair of sneakers within its target market in just days, as opposed to four months in the conventional process.
Adidas’ internal goal is to build 50% of their shoes using automation, though for now, it forecasts the total production of these two Speedfactories to amount to about 0.3% of its total worldwide sneaker production.
Between the two facilities, Adidas plans to manufacture footwear localized to each market in 5 key cities: London, New York, Paris, Los Angeles, and Shanghai. The shoes will be designed based on data collected from athletes in each city, along with data on local terrain and weather.
Rival Nike is also focusing heavily on manufacturing innovation and speed
In 2013, Nike backed manufacturing startup Grabit in a $3M Series A round. Grabit develops robots using electroadhesion technology and machine learning. These robots can arrange the flexible upper part of a shoe in just 50 — 70 seconds — a task that would take a human employee closer to 10 — 20 minutes, according to Bloomberg.
“With 30% fewer steps and up to 50% less labor, we can produce a complete pair of uppers in just 30 seconds at scale with less waste.” — Eric Sprunk, COO at Nike
Grabit has been secretive about its client base: a May 2017 press release only reveals that its material-handling robots are being shipped to a Fortune 100 “industry leading athletic shoe and apparel company.” However, Bloomberg recently confirmed that the robots are being used in a few Nike manufacturing facilities.
Nike has also applied for patents to automate shoe parts assembly and identification, underscoring its commitment to manufacturing innovation.
Another example of a retailer leveraging AI in its manufacturing is multinational makeup brand Shiseido, which recently piloted humanoid robots in its factory assembly line. The company aims to further develop its AI technology to enable its robots to perform more complex tasks.
Warehouse automation: Sorting, storing, & managing inventory
The road to automation passes through warehouses and factories where robots collaborate with humans. As more people shop for products online, there is greater pressure on order fulfillment centers to ship items on time.
Robotic automation in fulfillment centers gained momentum when Amazon acquired robotics startup Kiva Systems (now known as Amazon Robotics) in 2012. Amazon’s robots use computer vision, depth sensing, object recognition, and other AI software to move heavy items and handle packages, among other functions.
After Amazon acquired Kiva Systems, new startups emerged to fill Kiva’s shoes for the broader ecosystem.
Robots are still less-than-perfect at gripping, picking, and handling items in unstructured environments. But startups are beginning to address some of the challenges in robotic gripping and handling of delicate goods.
RightHand Robotics, for example, has raised a total of $34M from Menlo Ventures and GV, among others, to build its piece-picking robots. In the UK, Fieldwork Robotics, a startup spun off from the University of Plymouth, has developed a robot that is capable of picking 25K raspberries per day. Humans, in an eight-hour shift, can pick only about 15,000.
Infrastructure-as-a-service: Companies are profiting from selling their in-house automation solutions to other retailers in need
One of the biggest recent stories in warehouse automation news comes from Europe.
Like Amazon, UK’s online grocery supermarket Ocado (which provides services similar to FreshDirect and AmazonFresh) was early to invest in warehouse automation, highlighting machine learning as a “core competency” at the company.
“We are investing very, very heavily at the moment in innovation. We’re investing across all of these sectors, automation and robotics, data science and AI, big data and the cloud, and the Internet of Things.” — Time Steiner, Ocado CEO, Q2’17 earnings call
In 2002, Ocado opened its first customer fulfillment center, which is “equivalent to 11 football pitches in size and stands 20 meters tall.” Since then, it has opened a second and third, each time adding to its technological capabilities and warehouse capacity.
Ocado says it built most of the hardware and software for its automated warehouse in-house.
A search on the CB Insights platform for Ocado patents filed in the United States shows the types of warehouse automation technologies the company has been working on, from parcel sorters to robotic object handling to automated bag handling.
Ocado saw an opportunity and branched out its business model. In addition to its e-commerce operations, it started offering software and infrastructure-as-a-service to other retailers in the UK, Europe, and beyond.
Ocado partnered with France-based grocery giant Groupe Casino in 2017 to construct a “latest generation, state‐of‐the‐art automated warehouse” for Groupe Casino, and on the software side provide solutions like a front-end web interface and last-mile routing.
In 2018, Ocado first entered the North American market with its second warehouse automation partnership, a deal with Canadian food retailer Sobeys.
Later in 2018, Ocado announced a partnership with Kroger to build 20 customer fulfillment centers for the North American supermarket giant over 3 years, with the first set planned for Ohio, central Florida, and the mid-Atlantic region. As part of the deal, Kroger invested $247M into Ocado.
Micro-fulfillment centers: Companies are going lean to better connect with their customers
Startups like Instacart can help retailers offer fast, on-demand delivery to their customers, but it often comes at the expense of the customer’s relationship with the store. Companies save the money they would spend building out their own delivery infrastructure, but customers can become loyal to the delivery service rather than the store.
To adjust, some retailers are experimenting with building micro-fulfillment centers — lean operations for in-house delivery — within their existing locations.
These micro-fulfillment centers are between 3,000 and 10,000 square feet, vertically stacked, and designed to be installed inside supermarkets, parking garages, and basements. Retailers use robots to fetch items for packaging and delivery, and AI to organize placement and prioritize the movement of goods through the system.
Takeoff Technologies is one startup offering an end-to-end technological solution for running a micro-fulfillment center, including inventory management and online ordering. Earlier this year, Florida-based Sedano’s Supermarket worked with Takeoff Technologies to build its micro-fulfillment centers in Miami. The supermarket has 15 micro-fulfillment locations and plans to expand this service to all of its stores.
Consumer channels: selling online vs. in-store
In our 2018 analysis of 1,600+ earnings call transcripts from 50+ publicly-traded US retailers, only 9 retail companies had mentioned AI-related strategies for their websites or physical stores. (Note: our analysis excluded big tech companies like Amazon).
Today, discussions of AI strategies on earnings calls are still infrequent.
Some retailers, like Lowe’s, have focused on internal R&D, while others like Sephora and Walmart have announced partnerships with startups to try new AI-based solutions. Below, we look at a selection of companies deploying AI and robotics online as well as in physical stores.
Online solutions
One of the earliest brands to start discussing AI solutions for online operations was eBay.
The company first mentioned “machine learning” in its Q3’15 earnings calls. At the time, eBay had just begun to make it compulsory for sellers to write product descriptions, and was using machine learning to process that data to find similar products in the catalog.
Fast forward to Q2’16, and activity had ramped up: in the quarter, eBay acquired an AI company (Expertmaker), was in talks to acquire another (Salespredict, which it bought in Q3’16), and mentioned AI almost 15 times during the quarterly earnings call.
In the company’s Q4’17 call, CEO Devin Wenig spoke about AI for ad placement, personalization, visual search, and shipping recommendations for customer-to-customer (C2C) sellers.
In a March 2019 blog post, eBay’s Chief Architect Sanjeev Katariya talked about how eBay uses AI to better understand user and shopper intent, permit cross-border trade, and help sellers create product listings faster using natural language processing together with pattern matching.
After eBay, Etsy was the next retailer to mention an AI strategy. Its initial mention of machine learning in Q3’16 was in reference to its language translation tool. In the same quarter, Etsy acquired computer vision startup Blackbird Technologies. In May 2018, Etsy announced the opening of its third AI research and development center. In 2019, Etsy announced it had added $260M in gross merchandise sales over the previous two years since having search and another site functions use machine learning.
Others companies, like GAP, have mentioned AI technology but not yet discussed a robust AI strategy.
Incumbents partner with startups to personalize search and marketing
Image search startup ViSenze works with clients like Uniqlo, Myntra, and Japanese e-commerce giant Rakuten. ViSenze allows in-store customers to take a picture of something they like at a store, then upload the picture to find the exact product online.
The startup, which has offices in California and Singapore, raised a $10.5M Series B in 2016 from investors including the venture arm of Rakuten, and a $20M Series C in 2019.
Other startups focus on very specific markets. For example, China-based 9KaCha offers an online marketplace for imported wine, using computer vision for product searches.
Another startup developing AI for online search recommendations is Israel-based Twiggle, which has raised $35M.
The Alibaba-backed company is developing a semantic API that sits on top of existing e-commerce search engines, responding to very specific searches by the buyer. As of 2019, the company has search partnerships with Walmart and the e-commerce company Spring.
AI has also found applications in personalizing consumer experience.
Russian e-commerce retail giant Lamoda, for example, reportedly separates its visitors into 160 geographic segments, and recommends products based on the local weather in its banner ads. It also uses additional metrics like past purchase behavior and customers’ preferred brands and colors to drive decisions.
A case study on Lamoda (published by Dynamic Yield) claims that the company saw a significant ROI “using only one person on its team,” indicating AI is beginning to restructure the retail workforce.
The Bessemer Venture Partners-backed startup Dynamic Yield worked with brands like Sephora, Urban Outfitters, Ikea, and Stitch Fix before being acquired by McDonald’s in 2019 for $300M in the company’s largest acquisition in 20 years.
Beyond extending the personalization experience online, retailers want to understand consumer behavior across devices.
For example: are consumers more likely to order on a phone or a laptop? When do people use tablets instead of mobile devices?
This kind of information gives brands the option to not only tailor marketing messages to each user, but more specifically to each user’s device.
With its acquisition of Dynamic Yield, McDonald’s plans to roll out a drive-thru menu that can recommend items to a customer based on their order, trends, and even traffic & the weather.
One startup focusing on this area is SoftBank-backed, Taiwan-based Appier, which has clients including America luxury products manufacturer Estee Lauder, Japanese skincare line Naruko, and Unilever’s brand AXE.
Its AI platform, Axion, identifies device ownership and creates audience profiles. This allows retailers to engage with audiences across multiple platforms using the most applicable strategy.
In-store solutions
Several US stores are closing shop due to the growth of e-commerce: a market dominated by Amazon, a company with AI at the core of its operations.
But at the same time, Amazon itself is diving headfirst into the brick-and-mortar business.
The company has taken its AI-inspired approach to the physical retail world, leveraging artificial intelligence to help power in-store operations.
Amazon tracks consumers offline
Amazon opened its first computer vision-based cashierless “Amazon Go” grocery store to the public in Seattle in 2018. Customers can walk into the store, take what they want, and leave without checking out, as AI algorithms track their shopping activity.
Amazon uses a variety of technologies to monitor patrons in its Go stores, mitigating the threat of theft that has hurt previous cashierless store attempts in China. Guoxiaomei, which raised more than $64M for its “unmanned snacking shelves,” had to lay off staff and pivot its business model due mainly to the theft problem it encountered.
Amazon Go, on the other hand, relies on customers scanning QR codes to enter the store. Then, a shopper’s activity is monitored using AI-powered tracking systems. When a customer leaves the store, he leaves a digital footprint of his purchases, which are then charged to his Amazon account.
Today, there are 13 Amazon Go locations across Seattle, Chicago, San Francisco, and NYC. The company is planning to open up to 3,000 Amazon Go convenience stores by the year 2021, according to Bloomberg.
Amazon’s initial announcement of its Amazon Go store came in conjunction with a cashierless store frenzy in China.
Unmanned store startups in China raised 73 deals in 2017. In comparison, there were just 5 deals in this space in 2016. However, the frenzy tapered off after 2017. (Note: not all the deals below use AI-related technologies.)
Guangdong-based BingoBox raised $80M in Q1’18, bringing its total funding to $94M. Its unmanned stores currently rely heavily on RFID tags, but the company recently announced that it is moving towards AI-based image recognition solutions. JD.com, the second largest e-commerce platform in China, runs a collection of more than 20 cashierless convenience stores that also use RFID tags to keep track of merchandise and prevent theft.
Some US stores are beginning to test in-store robots for shelf scanning
Several retailers have begun testing autonomous robot technology for inventory management and customer interaction assistance in stores.
The Lowe’s Innovation Lab partnered with startup Fellow Robots to build retail robots OSHBot and LoweBot, which, among many tasks, can assist customers with finding specific products in store. The lab is also experimenting with AR/VR solutions for customer assistance.
Target introduced robots designed to count cash in the backrooms of their stores in 2018.
In April 2019, Walmart announced that it would roll out autonomous shelf-scanning bots (designed to identify out of stock items) in at least 300 stores through the rest of the year, with 1,500 stores automatic receiving floor cleaning robots. According to Walmart, a single robot can cut several hours of work off a human’s workload, allowing the company to allocate fewer employees to tasks like maintenance and checking stock. 900 more stores will receive robots that can hand online orders to customers who elect to pick up their items in-store.
Some beauty brands are turning to virtual reality
In addition to the in-store robotics solutions mentioned above, brands are also experimenting with facial recognition and VR technology to attract customers in stores.
Modiface, which used AI and augmented reality to power virtual try-on experiences for Sephora and other beauty brands, was acquired by L’Oréal in early 2018. Today, Modiface technology powers the AR feature in L’Oréal’s “Style My Hair” app, which lets users simulate the outcomes of different kinds of hair coloring treatments.
Another startup working on a similar technology is Perfect Corp, which raised $25M in 2017. Perfect Corp’s apps reportedly have over 500M downloads.
In the nail polish world, there’s Wanna Nails, an AR tool for trying on different colors of nail polish.
It’s not just beauty brands using this kind of technology to sell products — the D2C eyewear brand Warby Parker has built an AR app using the face-mapping tech built into the iPhone X that lets users virtually try pairs of glasses on before buying them.
Supply chain & logistics: Delivering orders to consumers
Shipping companies are using AI and IoT to better track global shipments
The global retail supply chain is getting increasingly complicated.
Sellers and consumers alike want to know where their products are, what condition their shipments are in, and what their delivery estimates are, every step of the way.
But the sheer scale and complex networks of people involved in transporting goods — from freight forwarders and freight operators to retailers and warehouse owners — makes supply chain visibility a challenge.
Startups like ClearMetal are attempting to use machine learning to improve transportation visibility. The company is developing a predictive intelligence platform that collects data from shipment carriers, as well as aggregating data points like real-time weather and currency fluctuations, to help predict shipping events, shipment times, and shipping demands.
In 2018, Microsoft Research Asia and Orient Overseas Container Line Limited (OOCL) announced they were working together to apply research in artificial intelligence into creating shipping efficiencies. The partnership has reportedly helped OOCL save $10M a year in operation costs.
In 2017, Maersk, the largest container shipping company in the world, hired 200 people focused on data science and AI in India. Maersk has previously partnered with companies like Ericsson and Maana for industrial IoT solutions.
The company wants to connect all its assets to the cloud. For example, connected vessels could provide real-time information on unexpected weather conditions. Maersk also uses IoT to get visibility on food quality in its refrigerated containers during transit.
In last-mile delivery, Amazon and Walmart are competing to disrupt the $15T logistics industry
Amazon is one of the biggest customers for traditional freight forwarders like UPS. There has been concern for a long time now that Amazon’s in-house logistics & automation efforts will turn the company into a competitor for freight giants like FedEx and UPS.
“…automation, something no traditional freight forwarding company can do even one percent as well as Amazon can, becomes the key competitive advantage over legacy freight forwarders.” — Ryan Peterson, CEO of Flexport
Amazon first described itself as a “transportation service provider” in a 2016 10-K filing. It also applied for a license in the US and China to operate as a freight forwarder.
Amazon’s next-gen drone patents show that the company is serious about developing the technology for use both in distribution centers and for deliveries.
Amazon’s in-development delivery drones use sensors and machine vision to detect conflicting air traffic and navigate the world autonomously.
Amazon demonstrated the newest model of its fully electric delivery drone at re:Mars in June 2019, with the company claiming that it would begin making thirty minute residential deliveries in certain markets within the coming months. The drone is designed to be more resilient to weather changes and deliver packages up to 5 pounds in weight within a 15 mile range.
Walmart has recently accelerated its own work on the last-mile problem. In February 2019, the company took part in a FedEx pilot testing last-mile delivery using autonomous robots built by DEKA Development & Research.
The company also filed ten patents in robotics and UAVs in 2016, as many as Amazon in 2017, and more than Amazon in 2018. These include patents for in-flight UAV rechargingand another, filed in June 2019, for “delivering merchandise using a network of unmanned aerial vehicles.”
One place Walmart has a clear advantage over Amazon when it comes to last-mile delivery is its distribution: with 5,000+ locations across the US, Walmart’s footprint far eclipses that of Amazon (150+ warehouses).
According to Walmart, more than 60% of the company’s customers live within three miles of a store. That proximity could make it easier for Walmart to quickly roll out short-distance drone delivery nationwide once it has been tested, developed, and vetted.
Bridging online and offline retail
While many retailers are focusing specifically on either online or in-store solutions, others are merging the two.
Alibaba, for example, is using artificial intelligence to better understand how online and offline consumer behavior work in tandem.
In some ways, Alibaba is ahead of Amazon in its online and offline integration using AI. It relies on technology — like smart stores, deep learning, and AR/VR — as well new business models to bridge the online and offline divide in China.
Alibaba refers to this as its “New Retail” strategy.
During Single’s Day 2018 — the country’s annual 24-hour shopping extravaganza — the e-commerce giant raked in more than $30.8B. That’s almost a 27% year-on-year increase from Single’s Day 2017, when the company sold $25.3B.
In its efforts to bridge brick-and-mortar with online commerce and improve the overall retail experience for consumers, Alibaba has made it clear that the future of its retail ambitions is omnichannel — a cross-channel approach that fuses the physical and digital shopping experience.
Conclusion
Despite the rise of AI-based solutions, only a handful of traditional brands have been effectively implementing AI strategies to drive business efficiency.
But AI is reshaping the retail workforce — from manufacturing to last-mile logistics — and players across the retail ecosystem will have to adapt to stay relevant.
Tech giants like Alibaba and Amazon continue to push the boundaries, applying AI to retail and amassing massive consumer datasets.
Smaller startups are also seeing an opportunity here and seizing it. In January 2019, California-based startup AiFi raised $11M to democratize the “cashierless store” automation solution, helping retailers achieve something similar to Amazon’s Go stores. Its technology can track as many as 500 people at once, allowing them to scale the cashierless store to businesses as large as grocery stores.
Retailers may increasingly compete with each other — and with tech companies working in other industries — for AI startups and talent, as artificial intelligence continues to spread across the retail ecosystem.
