Artificial intelligence (AI), undoubtedly, is helping build stronger connections for brands and create easier customer experiences. It is increasingly used to personalize product recommendations and power voice/chatbots. However, with all emerging technologies, including the most recent development of generative AI (GenAI), it’s all about the data — “garbage in, garbage out” remains as accurate as ever. Consequently, if AI models are not monitored and kept up to date, incorrect responses, biases, and hallucinations can creep in.
Retail brands that want to take advantage of AI are wrestling with keeping their products accurate and unbiased, as evidenced by the recent Talkdesk Bias & Ethical AI in Retail survey. We asked 1,000 consumers about current personalization experiences and their expectations for retailers to deploy ethical and transparent AI applications. The results of this survey should be an eye-opener for every retailer, whether they use AI or not.
Why consumers have issues with AI in retail.
Biases, stereotyping, and inaccurate personalization are some common threads of frustration among the survey respondents:
- 64% have received an AI-powered product recommendation that did not match their preferences, interests, or previous shopping behaviors. The number is even higher for Hispanic (72%) and Black (69%) respondents.
- 79% refrain from purchasing product recommendations because they are not tailored to their interests but appear to be the top-selling products that retailers are pushing for sales purposes.
- 60% avoid AI recommendations because they believe recommended products are biased or stereotypical. Hispanic (75%) and Asian (67%) consumers are most concerned about the problem.
Shoppers want AI in retail to reflect DEI principles.
Developing, assessing, and deploying responsible AI involves ensuring safety, trustworthiness, and ethics while including people from diverse ethnic, racial, and cultural backgrounds. A perceived neglect of diversity, equity, and inclusion (DEI) in AI has many consumers suspicious and concerned. Less than half (46%) of shoppers believe AI-powered recommendations include diverse shoppers, and only 39% believe retailers consider diverse communities when using or developing AI products. As a result:
- 53% believe that AI-powered facial recognition software will lead to increased instances of racial discrimination. This concern is not unfounded when you consider the example we shared earlier regarding AI image generators perpetuating dangerous race and gender stereotypes.
- 44% worry that AI will make the shopping experience less inclusive because brands aren’t deploying it responsibly, with Hispanic (50%) and Asian (52%) consumers being the most concerned.
- 45% are fearful their online shopping experience will no longer be personalized because retailers are not practicing responsible AI. For example, they worry that they’ll no longer receive personalized products inclusive of all demographics or that AI will recommend irrelevant products.
- It is important for brands to note that 31% would stop shopping with a brand if they felt their AI products did not represent or include diverse communities, with 34% of Gen Z and 35% of Millennials willing to go elsewhere
- 69% of men (compared to 53% of women) avoid AI recommendations because of bias and stereotypes.
- 71% have never purchased a recommended product because it makes them feel like a brand is tracking them — i.e., listening to their in-person conversations, tracking their browsing history, and other concerns.