3 Retail Trends in Data and Generative AI from the 2023 Databricks Summit

Retail is at an inflection point with data and AI. Executives no longer think about generative AI as the future, but are starting to ask their teams how they’ll deliver it to drive business improvements now.  This was never more evident than at the Databricks Data and AI Summit 2023. The annual event drew thousands of data engineers, scientists, analysts and industry leaders to ponder data and AI innovations that will no doubt change the way companies do business.

Top data leaders at Sephora, Gucci, Marks & Spencer, Mars and Sephora, shared the stage to share how the advent of generative AI has already changed the way they work— and how other retail and CPG firms can embrace new opportunities starting today. It was an event highlight for retail and CPG companies. Here’s how we unpacked the discussion.

Executives get techy

C-level executives are actively engaging in tech discussions about how generative AI can be applied in retail organizations to improve the bottom line. “Leadership used to think we were nerdy, geeky guys sitting in the basement solving problems. Now our leaders were speaking the language of technology, data and AI is coming out of the basement,” said Deepak Jose, head of ODDA data solutions at Mars Corporation.

Data and AI leaders are riding that wave, acting as a change agent, said Anna Russo, global director of data science at GUCCI. “We’ve had to change our lingo for understanding, showcasing the pain points and what an outcome could look like.”

Generative AI has also given data leaders the opportunity to explain to executives how their companies are already using AI today to manage data across large numbers of SKUs and product categories as well as supply chain and warehousing networks. Many CEOs hadn’t previously made the connection that their companies have been using AI for a while, panel members said. But the new generative AI hype has put all AI technology at the front and center of their discussions.

Data and AI is changing from a cost center to a profit center— Data and AI leaders now have a seat at the c-suite table.

Embracing generative AI

CEOs want to put generative AI in the context of the company’s business. Because retail is a heavily customer-facing industry, the opportunity is huge for generative AI to complement previously existing AI programs. For example, generative AI’s ability to personalize offerings could optimize marketing and sales activities already handled by existing AI solutions. Likewise, generative AI tools excel at data management and could support existing AI-driven pricing tools.

There is also a big opportunity to enhance customer value management by delivering very personalized marketing campaigns through a chatbot. Such applications can have human-like conversations about products in ways that can increase customer satisfaction, traffic, and brand loyalty.

Databricks highlighted a recent collaboration with Burberry, a long-standing luxury brand. Burberry’s team developed a ground-breaking technology, that deconstructs marketing imagery into features and uses AI to learn which design components drive engagement.

Now here’s the ROI part: Using a self-service app designed for their teams (not data scientists or developers), they can upload marketing images, retrieve a ranking of the content according to its predicted performance, and explore the insights in more detail. Brand teams are able to predict engagement with up to 87% accuracy at channel and market levels (e.g., what sort of images work better on Instagram versus e-commerce and emails). It’s a great blend of creativity and science: while data provides insights, the human remains in the loop providing human expertise and creativity.

Democratizing data and AI

There’s the world of data in the data warehouse that is focused on using structured data, and the world of AI that uses unstructured data to train models and predict the future. Now they are coming together. But first, companies need to get control of their structured data, breaking down data silos between departments by building a connected data foundation, Jose said. “Get your data cleaned and reconciled. Start there,” he said.

With a clean, unified customer data platform as the foundation, organizations can then prioritize the problems they want to solve and the value they need to see.  The upside is potentially hundreds of millions in profitability, Jose said. “With clean, trusted data, organizations can make it available for all functions within the entire organization and start driving ROI in a multitude of ways,” he said.

Companies can then start thinking about the decisions they need to make on a daily basis and whether employees have the information they need to make them, said Manbir Paul, vice president of engineering data insights and marketing at Sephora. “They can really start to answer the question: Can we use that data to drive value?”

New solutions for the evolving Generative AI world

As CEOs are reaching out to their data teams about AI, they’re also concerned about the risks as well as the opportunities, issues like improved governance, the cost of training models, and using AI responsibly. To address these challenges, Databricks is taking steps with some notable partnerships and acquisitions. These alliances all play a critical role in enhancing transparency, data governance and security concerns.

 One acquisition, Mosaic ML, will help address the massive cost of training LMM models with tools that allow companies to easily train and deploy LLMs and other generative AI models on their proprietary data, in their own secure environment. Databricks CEO Ali Ghodsi described it as an “incredible opportunity to democratize AI and make the Lakehouse the best place to build generative AI and LLMs.”

Clean rooms

Another technology, clean rooms, is a privacy-first, closed environment, that gives brands the ability to access data from a partner or media platform and safely match it with their own first-party data. Data clean rooms offer a solution that vastly expands access to data — and to data partners — via secure data collaboration.

Participants in a clean room have full control of their data and can decide which other participants can perform analysis on their data without exposing any sensitive data such as personally identifiable information (PII). There’s a big upside for retail and CPG firms. For example, CPG companies can see sales uplift by joining their first-party advertisement data with the point of sale (POS) transactional data of their retail partners.

LakehouseIQ

On the more technical side, Databricks is augmenting its existing Lakehouse data platform with LakehouseIQ, an engine that learns the unique nuances of a company’s business and data. It’s being designed to make natural language interfaces, like chatbots, more accurate and relevant to the needs of a specific organization.

LakehouseIQ can use a broad and deep array of internal data like notebooks, usage patterns, and org charts to understand a business’s jargon and unique data environment, and thus give chatbots the ability to provide significantly better answers.

Hitachi Solutions and Databricks

Databricks is positioning itself within the retail industry as a data pioneer, and Hitachi Solutions is a proud partner. We leverage their lakehouse technology in our Empower Data Platform solution a state-of-the-art system for moving on-premises data warehousing solutions to the cloud quickly— customers can get usable data 55 percent faster. Databricks recently awarded Hitachi Solutions with its Disruption Partner of the Year award for our innovative Empower platform. Be sure to read more about that here.

Just like the shift to the cloud from on-premises servers, the shift to generative AI is starting to take center stage, and the companies that start learning how to get there now will be better positioned in the future. Neglecting to explore the possibilities could be risky, given the pace at which data and generative AI solutions are evolving. Reach out today— we’d love to talk with you about our innovations in the data and AI space for retail and CPG firms.