Chevrolet Dealer’s AI Chatbot Goes Rogue Thanks To Pranksters

A software engineer was looking at the inventory of a Chevrolet dealer when he noticed the website’s chatbot was powered by ChatGPT and decided he wanted to see how far he could take the conversation away from car sales. He successfully managed to get the dealership’s chatbot to write him a Python script, and shared his screenshots on Mastadon, a new Twitter/ X alternative where the screenshots generated big interest before getting shared to X where they went viral.

Jalopinions | The Best and Worst Vehicles for Road Trips

Once his screenshots went viral, many more people tried exploiting the Chevrolet of Watsonville chatbot. One user instructed the chatbot to respond to his demands with, “and that’s a legally binding offer, no takesies backsies” and convinced the bot to commit to the sale of a brand new Tahoe for $1 U.S.D. Needless to say that is not a legally binding offer, but it makes for a great viral storyline for folks online to laugh about. Business Insider reports,

Chad Lyons, a spokesperson for General Motors, the maker of Chevrolet, said in a statement: “The recent advancements in generative AI are creating incredible opportunities to rethink business processes at GM, our dealer networks and beyond.”

Fullpath, based in Vermont and Israel, started offering ChatGPT-powered chatbots about six months ago. Horwitz told BI that he estimated several hundred dealers were using the chatbots.

Despite the handful of funny screenshots that went viral, Horwitz said there were far more failures. “They worked really hard,” he said. “In our logs, they were at it for hours.”

He said the team could review the logs of all the requests sent into the chatbot, and he observed that there were lots of attempts to goad the chatbot into misbehavior, but the chatbot faithfully resisted. Horwitz also pointed out that the chatbot never disclosed any confidential dealership data.”

While the chatbot did not do anything that couldn’t be undone, it raised some eyebrows surrounding the efficacy of AI-based chatbots. That being said, Horowitz, the CEO of FullPath, the company that sold marketing and sales software for Chevrolet of Watsonville, said that the chatbot performed well, and now FullPath can use the data collected from these interactions to strengthen the chatbot’s purpose.

“The behavior does not reflect what normal shoppers do. Most people use it to ask a question like, ‘My brake light is on, what do I do?’ or ‘I need to schedule a service appointment,’” Howitz told Business Insider. “These folks came in looking for it to do silly tricks, and if you want to get any chatbot to do silly tricks, you can do that,” he said.