As more shoppers turn to artificial intelligence tools to help find deals, a recent study urges consumers to verify product information and pricing, as some platforms appear to be more reliable than others.
AI Adoption and Study Methodology Approximately 40% of consumers are now using artificial intelligence tools for shopping, indicating a growing reliance on these platforms for finding deals and product information. To assess the accuracy and reliability of these tools, Product AI conducted a comprehensive study. They tested popular AI platforms including ChatGPT, Gemini, Claude, and Perplexity by asking them 220 shopping-related questions. These questions covered a wide range of topics, from pricing inquiries to product comparisons, across nine distinct product categories. The objective was to evaluate how well these AI models perform in providing accurate and helpful information to shoppers. Key Findings and Performance Discrepancies The study revealed significant inaccuracies across all the tested AI platforms. Researchers noted various errors, such as MacBook pricing being off by as much as $300, SPF sunscreen ratings being inaccurate by 10 points, and earbud battery life estimates differing by up to six hours. These findings highlight a crucial need for consumers to exercise caution and verify information obtained from AI shopping tools. Despite the prevalence of errors, the study also identified noticeable differences in performance among the individual platforms, suggesting that some tools are more reliable than others in specific shopping scenarios. Ranking of AI Tools for Shopping Accuracy Based on the extensive testing, Product AI established a ranking for the performance of the AI tools. Gemini was ranked last, with over half of the shopping-related questions resulting in costly errors for consumers, a finding that applied to both its free and paid versions. Claude secured the third position in terms of accuracy, followed closely by ChatGPT in second place. Perplexity was identified as the top performer in the study; however, even with its superior performance, Dakota Nunley of Product AI cautioned that it might not always be the 'best choice for consumers' depending on their specific needs and the nature of their inquiries. Industry Responses and Emerging AI Shopping Platforms Google, the parent company of Gemini, challenged the study's methodology, asserting that Product AI had evaluated a developer version of Gemini rather than its consumer application. Google stated that the consumer Gemini app incorporates 'everyday user features,' including shopping experiences powered by continually refreshed product listings, implying that the study's results may not accurately reflect the consumer-facing product. The article further highlights the unique approach of Perplexity, which actively searches the internet and other AI models to gather information before providing responses. The competitive landscape for AI shopping tools is also expanding, with major players like Amazon integrating its AI shopping assistant, Rufus, for product and pricing information, and TikTok recently launching its own AI shopping tool, contributing to a growing array of AI-powered options for consumers.
AI Adoption and Study Methodology
Approximately 40% of consumers are now using artificial intelligence tools for shopping, indicating a growing reliance on these platforms for finding deals and product information. To assess the accuracy and reliability of these tools, Product AI conducted a comprehensive study. They tested popular AI platforms including ChatGPT, Gemini, Claude, and Perplexity by asking them 220 shopping-related questions. These questions covered a wide range of topics, from pricing inquiries to product comparisons, across nine distinct product categories. The objective was to evaluate how well these AI models perform in providing accurate and helpful information to shoppers.
Key Findings and Performance Discrepancies
The study revealed significant inaccuracies across all the tested AI platforms. Researchers noted various errors, such as MacBook pricing being off by as much as $300, SPF sunscreen ratings being inaccurate by 10 points, and earbud battery life estimates differing by up to six hours. These findings highlight a crucial need for consumers to exercise caution and verify information obtained from AI shopping tools. Despite the prevalence of errors, the study also identified noticeable differences in performance among the individual platforms, suggesting that some tools are more reliable than others in specific shopping scenarios.
Ranking of AI Tools for Shopping Accuracy
Based on the extensive testing, Product AI established a ranking for the performance of the AI tools. Gemini was ranked last, with over half of the shopping-related questions resulting in costly errors for consumers, a finding that applied to both its free and paid versions. Claude secured the third position in terms of accuracy, followed closely by ChatGPT in second place. Perplexity was identified as the top performer in the study; however, even with its superior performance, Dakota Nunley of Product AI cautioned that it might not always be the 'best choice for consumers' depending on their specific needs and the nature of their inquiries.
Industry Responses and Emerging AI Shopping Platforms
Google, the parent company of Gemini, challenged the study's methodology, asserting that Product AI had evaluated a developer version of Gemini rather than its consumer application. Google stated that the consumer Gemini app incorporates 'everyday user features,' including shopping experiences powered by continually refreshed product listings, implying that the study's results may not accurately reflect the consumer-facing product. The article further highlights the unique approach of Perplexity, which actively searches the internet and other AI models to gather information before providing responses. The competitive landscape for AI shopping tools is also expanding, with major players like Amazon integrating its AI shopping assistant, Rufus, for product and pricing information, and TikTok recently launching its own AI shopping tool, contributing to a growing array of AI-powered options for consumers.
