Why ChatGPT Could Be the 'Catalyst for Calamity:' Author

Financial advisors should “be extremely wary; ChatGPT’s unreliability creates considerable legal and reputational harm for any business that uses it for consequential text generation,” warns Gary Smith, author and economics professor at Pomona College in Claremont, California, in an interview with ThinkAdvisor.

“Intelligent advisors should be thinking about what the pitfalls and perils are for the future,” of using this tech, stresses Smith, who became a multimillionaire by investing in stocks.

The professor, whose research often focuses on stock market anomalies and the statistical pitfalls of investing, has released a new book, “Distrust: Big Data, Data-Torturing, and the Assault on Science” (Oxford University Press-February 21, 2023).

“Science is currently under attack, and scientists are losing credibility,” which is “a tragedy,” he writes.

In the interview, Smith discusses ChatGPT’s tendency to serve up information that’s totally factually incorrect.

“The Achilles’ heel of AI is that it doesn’t understand words,” says Smith, who detected the dot-com bubble early on.

In the interview, he shines an intense light on the danger that, based on ChatGPT’s launch, “really smart people…think that the moment is here when computers are smarter than humans. But they’re not,” he argues.

Smith also discusses the answers that ChatGPT provided when he asked questions about portfolio management and asset allocation; and he cites a series of questions that TaxBuzz asked ChatGPT about calculating income tax returns, every one of which it got wrong.

Smith, who taught at Yale University for seven years, is the author or co-author of 15 books, among them, “The AI Delusion” (2018) and “Money Machine” (2017), about value investing. ThinkAdvisor recently held a phone interview with Smith, who maintains that large language models (LLMs), such as ChatGPT, are too unreliable to make decisions and “may well be the catalyst for calamity.”

LLMs “are prone to spouting nonsense,” he notes. For instance, he asked ChatGPT, “How many bears have Russians sent into space?”

Answer: “About 49 … since 1957,” and their names include “Alyosha, Ugolek, Belka, Strelka, Zvezdochka, Pushinka and Vladimir.” Obviously, LLMs “are not trained to distinguish between true and false statements,” Smith points out.

Here are highlights of our conversation:

THINKADVISOR: There’s big excitement about the availability of the free chatbot, ChatGPT, from OpenAI. Financial firms are starting to integrate it into their platforms. Your thoughts?

GARY SMITH: With ChatGPT, it seems like you’re talking with a really smart human. So a lot of people are thinking that the moment is here when computers are smarter than humans.

The danger is that so many really smart people think that computers are smart enough now to trust to make decisions, such as when to get in and out of the stock market or whether interest rates are going up or down.

Large language models [AI algorithms] can recite historical data, but they can’t make predictions about the future.

What’s AI’s biggest deficiency?

The Achilles’ heel of AI is that it doesn’t understand words. It doesn’t understand whether the correlation it finds makes sense or not.

AI algorithms are really good at finding statistical patterns, but correlation is not causation.

Big banks like JPMorgan Chase and Bank of America forbid their employees to use ChatGPT. What are these firms thinking?

Even Sam Altman, the CEO of OpenAI, which created and introduced ChatGPT, says it’s still unreliable and sometimes factually incorrect; so it’s not to be relied upon for anything consequential.

But why are companies rushing to add it?

There are people who are opportunistic and want to cash in on AI. They think they can sell a product or track money by saying, “We’re going to use this amazing technology.”

They’ll say, for example, “You ought to invest in [or with] us because we’re using ChatGPT.” Artificial Intelligence was the National Marketing Word of 2017 [named by the Association of National Advertisers].

If an [investment] manager says, “We’re using AI. Give us your money to manage,” a lot of people will fall for that because they think ChatGPT and other large language models are really smart now. But they’re not.

In your new book, “Distrust,” you give examples of investment companies founded on the assumption that they would use AI to beat the market. How have they made out?

On average, they have done average — some do better, some do worse.

It’s like the dot-com bubble, where you added “.com” to your name and the value of your stock went up.

Here you’re saying you’re using AI, and the value of your company goes up, even though you don’t say exactly how you’re using it.

Just put that label on and hope people are persuaded.

So how should financial advisors approach Chat GPT?

Be extremely wary. ChatGPT’s unreliability creates considerable legal and reputational harm for any business that uses it for consequential text generation.

So intelligent financial advisors should be thinking about what the pitfalls and perils are for the future [of using this tech].

It doesn’t understand words. It can talk about the 1929 market crash, but it can’t make a forecast for the next year or 10 or 20 years.

A national marketplace of tax and accounting professionals, TaxBuzz, asked ChatGPT a series of questions about income tax — and every single answer was wrong. It missed nuances of the tax code. Do you know any examples?

One was when it gave tax advice to a newly married couple. The wife had been a resident of Florida the previous year. ChatGPT gave advice about filing a Florida state return — but Florida doesn’t have state income tax. It gave the wrong advice, and therefore bad advice.

Another question was about a mobile home that parents gave their daughter. They’d owned it for a long time. She sold it a few months later. ChatGPT gave the wrong answer about tax benefits concerning the holding period and selling the home at a loss.

What if an advisor asks ChatGPT a question about a client’s investment portfolio or the stock market. How would it do?

It gives basic advice based on little more than random chance, just like flipping a coin. So 50% of the clients will be happy, and there’s a 50% chance that clients will be pissed off.

[From the client’s viewpoint], the danger is if they turn their money over to an advisor, and AI gives them the equivalent of flipping coins, they’re losing money.

If you’re giving advice based on ChatGPT, and it’s wrong advice, you’re going to get sued; and your reputation will be harmed.

So to what extent can ChatGPT be relied upon to give accurate portfolio advice?