Oct 16 • 4 min read
Should You Trust AI With Your Retirement?
|
In 2002, best-selling author Michael Crichton delivered a speech titled “Why Speculate?” The entire speech is worth reading, but I want to share one point that’s of particular relevance today. In it, Crichton described a theory he called the “Murray Gell-Mann Amnesia effect.” Gell-Mann was a friend of Crichton’s and a physicist, and Crichton named the idea after a conversation the two had shared. In short, it's a quiet trick our brains play on us, and it's never been more relevant than it is right now. In today's email:
As always, I'll wrap up with a few of the best retirement and investing articles I read in the past week. *** Before we dive in, did you catch this week's podcast? 👇 Wet Streets Cause RainHere’s how he Crichton explained Gell-Mann Amnesia: “You open the newspaper to an article on some subject you know well.
In Murray’s case, physics. In mine, show business.
You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backward—reversing cause and effect.
I call these the ‘wet streets cause rain’ stories. Paper’s full of them.
In any case, you read with exasperation or amusement the multiple errors in the story, and then turn the page to national or international affairs, and read as if the rest of the newspaper was somehow more accurate about Palestine than the baloney you just read.
You turn the page, and forget what you know.”
I suspect most of us have experienced this same phenomenon. We read a story about an industry, company, profession, or event we know well and immediately notice errors, missing context, or conclusions that don’t match our firsthand experience. Yet, for some reason, we turn the page or click to the next story and assume the reporting on subjects we know less about must be accurate. Crichton goes on to point out that this amnesia effect rarely appears in other areas of life. For instance, imagine having a friend who regularly gives you incorrect information. After seeing that happen enough times, you’d probably stop asking that person for advice or, at the very least, you would be highly skeptical of their answers. It appears that, for whatever reason, we give the media more credibility than it probably deserves. In any case, I share this Gell-Mann Amnesia effect with you because I think it’s becoming more relevant by the day. Not because the media has gotten worse since 2002—though maybe it has—but because of the rise of and increased reliance on artificial intelligence (AI). Should You Trust AI With Your Retirement?There was a study published last month by the UK firm Saturn titled Artificial Authority: Should You Trust AI to Deliver Financial Advice? In the study, they tested 18 of the most popular AI models (including models from ChatGPT, Claude, Gemini, and CoPilot) by asking 121 different money questions with each question being deemed easy, medium, and hard. Because these AI models can change their responses, they asked each model the same question five times. In total, these questions were run through these models more than 10,000 times, at which point the responses were evaluated against key criteria to determine a pass or fail. The results were shockingly bad. On average, across all categories of questions, the models made mistakes 57% of the time. But on questions that were deemed hard, the models’ accuracy dropped to just 12%. Even for easy questions, the accuracy was just 54%, barely better than a coin flip. Even the best-performing paid-for models failed far more often than one might expect, still failing about 40% of the time. The report summary says everything we need to know: All Claude, ChatGPT, Gemini, and CoPilot models were wrong on average most of the time. Why Retirement Questions Are Hard for AI to Get RightI’ve seen these AI errors for myself countless times now. As a credentialed financial professional, I’ve tested most of these AI tools and, while I’ve never measured the results specifically, I can say that anecdotally, I’ve found Saturn’s research to be on the mark. At least for now, AI still doesn’t seem to understand nuance, nor is it truly capable of understanding our unique financial situations even if we take the time to describe it. Not to mention, AI continues to “hallucinate” from time to time, citing sources that don't exist and providing erroneous (or simply outdated) support for its “thoughts.” Sadly, our exposure to AI doesn’t start and end with the prompt box on your AI tool of choice because Pew Research recently noted that over one-third of all web pages published after ChatGPT’s release show signs that they were written or substantially edited by AI. In other words, we may be reading AI without even realizing it’s AI. Bottom LineI’m not suggesting that everything produced by the media, AI, or online "experts" is wrong or misleading (there’s plenty of credible information out there). But until someone has earned your trust over time, I think a healthy dose of skepticism is warranted. So, as AI becomes a more common source of information and advice, I encourage you to be more mindful of what you read, watch, and ultimately act on. Take the time to question the source, verify the facts, and consider whether the advice actually applies to your situation. Until next time, stay the course. 📚 What I've Been Reading
Thank you for reading! Please reply to this email with comments, questions, and/or feedback. Stay wealthy, Taylor Schulte, CFP® |