Is ChatGPT at a tipping point on the hype scale?
A December-2022 hype-scale take, and what it got right and wrong
Contents
<p>ChatGPT went public on 30 November 2022 and crossed a million users in five days — faster than Instagram, faster than Spotify, faster than anything I can remember watching. By the time I sat down to write this, three weeks later, half my feed was either declaring the end of Google or declaring the whole thing a parlour trick. Both camps are wrong in the same way: they’re reacting to the demo instead of the mechanism. So let me try to do the unfashionable thing and ask what’s actually in the box, where it earns its keep, and where the hype is writing cheques the model can’t cash.</p>
<p>The short version: this is a real step change in one specific capability — generating fluent, context-aware text on demand — wrapped in a chat interface that makes that capability legible to non-specialists for the first time. The step change is genuine. The interface is the bit doing most of the hype’s heavy lifting.</p>
<h2 id="what-it-actually-is">What it actually is</h2><div class="ad-unit ad-in-article" aria-label="Advertisement">
<span class="ad-label">Advertisement</span>
<ins class="adsbygoogle" style="display:block;text-align:center"
data-ad-client="ca-pub-3726833845844946"
data-ad-slot="3291553914"
data-ad-format="auto"
data-full-width-responsive="true"></ins>
<script>(adsbygoogle = window.adsbygoogle || []).push({});</script>
</div>
<p>ChatGPT is a large language model — a descendant of OpenAI’s GPT line — fine-tuned for conversation. Underneath, it does exactly one thing: given a sequence of tokens, predict the next one, over and over. That is the entire trick. Everything that <em>feels</em> like reasoning, memory or intent is an emergent property of having done that prediction across a staggering amount of text, then having been nudged by human feedback (RLHF — reinforcement learning from human feedback) toward answers people rate as helpful.</p>
<p>Being precise about this matters, because the precision is what protects you from the hype. The model has no database it looks things up in, no notion of truth, and no awareness of whether the next token it emits is a fact or a fluent-sounding fabrication. It has learned the <em>shape</em> of correct answers extraordinarily well, which is a different and more slippery thing than knowing them.</p>
<p>The other thing the chat window hides is that “ChatGPT” is not one fixed object. There’s a base model trained on a corpus, a fine-tuning pass, a layer of RLHF, a system prompt you never see, and a moderation filter on top. When the behaviour changes overnight — and it will, repeatedly — it’s usually one of those upper layers being adjusted, not the underlying model being retrained. That matters enormously if you’re building on the API: you are renting behaviour that can shift under you without notice, which is a very different proposition from depending on a versioned library you control. Pin model versions where the provider lets you, and treat “the model got worse this week” as a real, recurring operational hazard rather than a conspiracy theory.</p>
<h2 id="where-it-genuinely-earns-its-keep">Where it genuinely earns its keep</h2>
<p>I’ve spent the last fortnight throwing real work at it rather than party tricks, and a clear pattern emerges. It’s strong wherever the value is in <em>fluency and structure</em> and weak wherever the value is in <em>being right</em>.</p>
<p>It is excellent at:</p>
<ul>
<li>Rewriting and summarising text you provide — the source of truth is in the prompt, so it has less room to invent.</li>
<li>Drafting boilerplate: regex you can immediately test, a first-pass shell script, a config file skeleton, a polite email.</li>
<li>Explaining a concept at a chosen level, then re-explaining it differently when you push back.</li>
<li>Translating between formats — prose to a table, JSON to a struct, an error message to a plausible cause.</li>
</ul>
<p>Here’s a representative bit of the work that genuinely saves me time — handing it a task where I can verify the output in seconds:</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span><span class="lnt">3
</span><span class="lnt">4
</span><span class="lnt">5
</span><span class="lnt">6
</span><span class="lnt">7
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback"><span class="line"><span class="cl">Prompt: Write a bash one-liner that finds files over 100MB
</span></span><span class="line"><span class="cl"> modified in the last 7 days, under the current directory.
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">Reply: find . -type f -size +100M -mtime -7 -printf '%s\t%p\n' \
</span></span><span class="line"><span class="cl"> | sort -rn
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">Me: (run it, read the output, done)
</span></span></code></pre></td></tr></table>
</div>
</div><p>The key word there is <em>verify</em>. The model is a tireless junior who produces a confident first draft instantly. The moment I treat its output as a draft to check rather than an answer to trust, it’s one of the most useful tools I’ve added to my workflow in years.</p>
<p>There’s a measurable productivity case underneath the vibes, too. The tasks where it shines are precisely the ones that used to involve a context-switch and ten minutes of searching: the exact <code>find</code> invocation, the cron syntax I always forget, the boilerplate for a config format I use twice a year. Each of those individually is small. Aggregated across a week, the saved friction is real — not “AI does my job”, but “AI removes the papercuts that were taxing my attention”. That is a less thrilling claim than the headlines want, and it’s also the one that’s actually true today.</p>
<h2 id="where-the-hype-outruns-the-model">Where the hype outruns the model</h2><div class="ad-unit ad-in-article" aria-label="Advertisement">
<span class="ad-label">Advertisement</span>
<ins class="adsbygoogle" style="display:block;text-align:center"
data-ad-client="ca-pub-3726833845844946"
data-ad-slot="3291553914"
data-ad-format="auto"
data-full-width-responsive="true"></ins>
<script>(adsbygoogle = window.adsbygoogle || []).push({});</script>
</div>
<p>Now the other side, because someone needs to say it before the next round of breathless think-pieces.</p>
<p><strong>It is confidently, fluently wrong, and that’s the dangerous part.</strong> A model that was visibly uncertain would be safer. This one states fabrications in the same measured tone as facts — invented citations, plausible-but-nonexistent API methods, dates that are simply made up. I’ve watched it cite a function that doesn’t exist with the same poise it cites one that does. If you can’t verify a domain yourself, you have no way to tell which you got.</p>
<p><strong>It doesn’t know what it doesn’t know.</strong> Ask it about events after its training cut-off and it won’t say “I can’t know that”; it’ll improvise. Ask about something obscure and it’ll pattern-match to the nearest thing it has seen and present that as the answer. There’s no internal “confidence” dial exposed to you, and the tone is uniformly assured whether the model is on solid ground or freewheeling — which means you cannot use <em>how</em> it says something as a signal for <em>whether</em> it’s right.</p>
<p><strong>The customer-service fantasy is running ahead of reality.</strong> The pitch doing the rounds — bolt ChatGPT onto your support queue and shed half your agents — ignores that an unconstrained model will, eventually, confidently tell a customer something false, off-policy, or actionable in a way your legal team will not enjoy. The viable version is narrow: retrieval over <em>your</em> documented answers, tight guard-rails, and a human escalation path. That’s a real product. “It just talks to customers now” is not.</p>
<p>This is exactly the failure mode I dug into in <a href="/story/the-limitations-of-chatgpt-4-for-stock-market-predictions-and-investments-introduction/">the limitations of using GPT-4 for stock-market predictions</a>: a model that produces a fluent, authoritative-sounding forecast is not the same as a model that produces a <em>correct</em> one, and conflating the two is how people lose money. The fluency is the trap.</p>
<h2 id="the-bit-nobody-in-the-demo-mentions-your-data-leaves-the-building">The bit nobody in the demo mentions: your data leaves the building</h2>
<p>Every prompt you type goes to someone else’s servers, gets processed there, and — depending on the terms in force this month — may be retained and used to improve future models. For idle questions, fine. For your employer’s unreleased source, a customer’s personal data, or anything covered by a contract or a regulation, “I pasted it into a chat box hosted by a third party” is a sentence you do not want to say to a compliance officer. The convenience is exactly what makes this easy to do without thinking, which is what makes it a genuine operational risk rather than a hypothetical one.</p>
<p>There are already two sane responses. The first is policy: decide explicitly what classes of data may and may not go to a hosted model, and make that decision before someone makes it for you by accident. The second is technical: run a model yourself. The open-weights models aren’t yet at parity with the hosted frontier for raw quality, but they’re closing the gap fast, and for a lot of the fluency-and-structure tasks above, a model running on your own hardware is more than good enough — and nothing you feed it leaves your network. I expect that “use the hosted model for trivia, the local one for anything sensitive” will be a standard split within a year, and it’s worth setting up the muscle for it now rather than after a leak.</p>
<h2 id="how-to-build-on-it-without-getting-burned">How to build on it without getting burned</h2>
<p>If you’re going to put this in production — and plenty of people will, this quarter, hype or not — a few rules have already proven themselves:</p>
<ol>
<li><strong>Constrain the input.</strong> Give the model the source material in the prompt rather than relying on what it “knows”. You shrink the surface for invention dramatically.</li>
<li><strong>Verify the output.</strong> For anything factual or executable, the answer is a draft. Run the code, check the citation, diff the summary against the source.</li>
<li><strong>Keep a human in the loop for anything that matters.</strong> Route routine, low-stakes queries to the model; escalate anything emotional, ambiguous or consequential to a person.</li>
<li><strong>Log everything.</strong> You will want the prompt-and-response history when — not if — it says something you need to explain.</li>
<li><strong>Budget for drift.</strong> Behaviour will change as the provider tweaks the upper layers. Keep a small suite of representative prompts and re-run them after every model update so you notice regressions before your users do.</li>
</ol>
<p>The same instinct extends to the image side of the family, too. I went through it for image generation in <a href="/story/harnessing-the-power-of-chatgpt-to-generate-stunning-images-with-dall-e-2/">generating images with DALL·E 2</a>, and the lesson rhymes: the model is a fast, fluent generator whose output you steer and curate, not an oracle you defer to.</p>
<h2 id="so--tipping-point-or-not">So — tipping point or not?</h2>
<p>Yes, but not the one the headlines mean. ChatGPT isn’t tipping into “AI does your job now”. It’s tipping a much narrower thing over an edge: the cost and friction of generating competent first-draft text has collapsed to roughly zero, and an interface finally exists that lets anyone feel that for themselves. That’s enough to reshape a lot of workflows on its own. It does not require the model to be intelligent, truthful, or anywhere close to replacing the judgement of the person checking its work.</p>
<p><strong>Who should care, and how?</strong> If your work involves a lot of drafting, transforming or explaining text, start using it this week — as an assistant whose output you verify, not as an authority. If you’re tempted to build a product on it, build the narrow, guard-railed, retrieval-backed version and resist the demo-shaped one. And if you’re reading the hype, apply the same filter you’d apply to any confident junior who has read everything and understood the shape of it without always grasping the substance: brilliant for a first pass, never to be shipped unchecked.</p>
<p>The technology is real. The tipping point is real. The trick over the next year will be telling the genuine capability apart from the marketing built on top of it — and the people who learn to verify rather than trust are the ones who’ll get the value without the embarrassment.</p>
Advertisement
Related Content
Advertisement
