Why ChatGPT Can't Pick Stocks: The Limits of LLMs for Market Predictions
A language model that has never seen today's prices is not the trading oracle the hype implies
Contents
<p>I asked a large language model, in a moment of weakness, which stock it would buy tomorrow. It gave me a confident, well-structured, thoroughly-reasoned answer complete with a target price and a neat paragraph on the company’s “strong fundamentals”. It was also, I am fairly sure, drawing on financial data that was over a year out of date, had no idea what the share had done that morning, and would have produced an equally confident answer if I had asked about a company that went bankrupt in 2019. That is the whole problem in one anecdote: the fluency is real, and it is precisely what makes the model dangerous as a trading tool.</p>
<p>The temptation is understandable. These models write like an analyst, reason like an analyst, and cost a fraction of one. But a text generator that sounds like a hedge fund is not a hedge fund, and the gap between the two is where people lose money. Let me walk through exactly where it breaks, because the failure modes are specific and worth knowing whatever you use the model for.</p>
<h2 id="it-doesnt-know-what-happened-this-morning">It doesn’t know what happened this morning</h2><div class="ad-unit ad-in-article" aria-label="Advertisement">
<span class="ad-label">Advertisement</span>
<ins class="adsbygoogle" style="display:block;text-align:center"
data-ad-client="ca-pub-3726833845844946"
data-ad-slot="3291553914"
data-ad-format="auto"
data-full-width-responsive="true"></ins>
<script>(adsbygoogle = window.adsbygoogle || []).push({});</script>
</div>
<p>The most basic limitation is temporal. A language model is trained on a fixed snapshot of text with a hard cut-off date. Anything after that — an earnings miss, a rate decision, a war, a CEO resigning at 9am — simply does not exist as far as the base model is concerned. Markets, meanwhile, price in new information within seconds.</p>
<p>This matters more than people realise because the model will not tell you it is out of date. Ask it about a stock and it answers in the present tense, with total confidence, from a world that may be eighteen months stale. Retrieval plugins and live-data tools patch over this to a degree by feeding current numbers into the prompt, but that is bolting a data feed onto a text engine, not the model “knowing” the market. The core weights are frozen in the past.</p>
<h2 id="it-generates-plausible-text-not-calculated-forecasts">It generates plausible text, not calculated forecasts</h2>
<p>Here is the part that trips up even technical users. A language model does not compute a forecast. It predicts the next token. When you ask for a price target, it is not running a discounted-cash-flow model; it is producing the words that most plausibly <em>follow</em> your question, based on the millions of analyst notes and Reddit threads it absorbed in training.</p>
<p>The result reads like analysis but is closer to very sophisticated autocomplete. You can watch this fail if you ask for the arithmetic explicitly:</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span><span class="lnt">3
</span><span class="lnt">4
</span><span class="lnt">5
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">Prompt: A stock trades at £42.10. If it rises 3.7% then falls 1.2%,
</span></span><span class="line"><span class="cl"> what is the final price? Show each step.
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">Model: Step 1: 3.7% of £42.10 = £1.56, new price £43.66
</span></span><span class="line"><span class="cl"> Step 2: 1.2% of £43.66 = £0.52, new price £43.14
</span></span></code></pre></td></tr></table>
</div>
</div><p>Sometimes it nails this. Sometimes it fumbles a multiplication and states the wrong number with the same serene confidence. If a system cannot be trusted to reliably chain two percentage changes without a calculator, treating its unshown “reasoning” about a company’s future as quantitative analysis is a category error. It is producing the <em>shape</em> of analysis, not the substance.</p>
<h2 id="it-has-no-model-of-sentiment-or-tail-risk">It has no model of sentiment or tail risk</h2><div class="ad-unit ad-in-article" aria-label="Advertisement">
<span class="ad-label">Advertisement</span>
<ins class="adsbygoogle" style="display:block;text-align:center"
data-ad-client="ca-pub-3726833845844946"
data-ad-slot="3291553914"
data-ad-format="auto"
data-full-width-responsive="true"></ins>
<script>(adsbygoogle = window.adsbygoogle || []).push({});</script>
</div>
<p>Markets are driven partly by numbers and partly by mood — fear, greed, herd behaviour, the sudden decision by thousands of people to sell at once. A language model can describe sentiment eloquently because it has read a great deal <em>about</em> sentiment. It cannot measure the live sentiment of a market it cannot see, and it has no stake, no fear, and no skin in the game.</p>
<p>Worse, it is structurally blind to the events that actually destroy portfolios. Rare, high-impact shocks — a 2008, a pandemic crash, a flash crash — are by definition under-represented in any training set, because they are rare. The model has read the <em>narrative</em> of these events after the fact but has no mechanism for anticipating the next one. It will happily give you a serene 12-month outlook that assumes tomorrow looks like the average of yesterday, which is exactly the assumption that goes wrong at the worst possible moment.</p>
<h2 id="the-confidence-problem-is-the-real-hazard">The confidence problem is the real hazard</h2>
<p>None of the above would matter much if the model hedged appropriately. The danger is that it does the opposite: it packages stale, uncalculated, sentiment-blind guesses in the fluent, authoritative prose of a professional. That combination — high confidence, low grounding — is the single most expensive property an advisory tool can have, because it short-circuits the scepticism you would apply to a stranger’s stock tip.</p>
<p>This is the same tipping point I have written about when <a href="/story/is-chatgpt-is-at-tipping-point-on-the-hype-scale/">the hype around ChatGPT ran ahead of what it could actually do</a>: the output <em>feels</em> like expertise, so people extend it the trust they would give an expert. In casual use that is harmless. With money on the line it is a trap.</p>
<h2 id="what-language-models-are-genuinely-good-for-in-finance">What language models are genuinely good for in finance</h2>
<p>I am not an AI doomer, and dismissing these tools entirely would be as silly as trusting them with your pension. They are excellent at the <em>language</em> half of the work, which is a large part of the work:</p>
<ul>
<li><strong>Summarising.</strong> Feed a model a 200-page annual report and ask for the risk factors, the changes in tone from last year, or the sections mentioning debt covenants. It is fast and generally faithful when the source text is in front of it.</li>
<li><strong>Explaining.</strong> “What is a convertible bond and why would a company issue one?” is exactly the kind of grounded, well-documented question these models answer beautifully.</li>
<li><strong>Drafting and scaffolding.</strong> Turning your own analysis into a readable memo, or generating boilerplate code to pull and chart data, is a genuine time-saver.</li>
<li><strong>Structured extraction.</strong> Pulling figures out of messy filings into a clean table, which you then verify, is a real productivity win.</li>
</ul>
<p>Notice the pattern: every good use has the ground truth <em>in the prompt</em> or asks the model to explain settled knowledge. The failures all involve asking it to <em>predict</em> something it cannot see. That is the line. The same reasoning applies to its creative uses too — a model can <a href="/story/harnessing-the-power-of-chatgpt-to-generate-stunning-images-with-dall-e-2/">conjure a striking image from a text prompt</a> precisely because there is no external ground truth it can be wrong about. A stock price has one, and the model does not have access to it.</p>
<p>There is a way to make the tool genuinely useful for research without falling into the prediction trap, and it is worth spelling out because it reframes the whole exercise. Use the model as a <em>reading and reasoning layer over data you supply</em>, never as the source of the data. Paste in the actual filing, the actual price history, the actual analyst commentary, and ask it to summarise, compare, or find contradictions. Now the ground truth is in the context window, the model is doing the language work it excels at, and you are doing the judgement. The instant you ask a question whose answer lives outside the prompt — “what will this do next quarter”, “is now a good entry point” — you have crossed back into fiction, however fluent the reply.</p>
<p>The tempting middle ground of a “financial LLM” fine-tuned on market data deserves a word of caution too. Fine-tuning changes the <em>style and vocabulary</em> of the output far more than it changes the model’s ability to know something it was never shown. A model trained on a decade of analyst notes will sound even more authoritative — which, given everything above, makes it more dangerous, not less, because it lends the same ungrounded guessing an even more professional voice. The confidence problem gets worse with polish, not better.</p>
<h2 id="a-note-on-ethics-and-accountability">A note on ethics and accountability</h2>
<p>Before the legal note, there is a psychological trap worth naming, because it catches careful people. A model that is wrong most of the time is easy to distrust. A model that is <em>right most of the time</em> is far more dangerous, because it trains you to lower your guard right up until the expensive occasion when it is confidently wrong. Markets punish exactly this pattern: a strategy that works in calm conditions and fails catastrophically in a crisis loses more than one that never worked at all, because you will have staked more on it by the time it breaks. The fluency that makes the model pleasant to use is the same property that lulls you into over-trusting it. Treat every correct answer as luck you cannot rely on repeating, not as evidence the tool can be trusted with the next decision.</p>
<p>There is also a legal and ethical layer worth flagging before you wire a model to a brokerage account. Acting on AI-generated tips at scale raises real questions of accountability — if the “reasoning” is a black box even to you, you cannot explain your own trades, and in a regulated context that is a problem, not a feature. Automated systems trading on model output also blur into market-manipulation and insider-information territory faster than people expect. None of this is a reason to avoid the tools; it is a reason to keep a human, who understands and can be held responsible for the decision, firmly in the loop.</p>
<h2 id="the-honest-verdict">The honest verdict</h2>
<p>If you want a research assistant that reads faster than you, drafts cleaner than you, and never gets bored summarising a filing, a large language model is a superb hire. If you want an oracle that predicts tomorrow’s close, you have hired a very articulate person who has been asleep for eighteen months, cannot do arithmetic reliably, and will never admit to either.</p>
<p>Use it for the language, do the analysis yourself, and treat any confident number it hands you as a prompt to go and check the real data — not as the answer. The models are genuinely useful. They are just useful for the opposite of what the “AI stock picker” headlines promise, and the sooner you internalise that distinction, the more money it will save you.</p>
<p>If you take one habit away, make it this: before acting on anything a model tells you about a market, ask where the ground truth for that specific claim lives. If it is in the prompt you supplied, you can trust the model to have read it faithfully. If it lives out in the world — in this morning’s price, next quarter’s earnings, tomorrow’s headline — the model does not have it, cannot have it, and is filling the gap with plausible words. That single question separates the tasks the tool is brilliant at from the ones that will quietly cost you, and it works no matter how much the models improve, because it is a limit of what a text generator <em>is</em>, not a bug that a bigger model will fix.</p>
Advertisement
Related Content
Advertisement
