Playing around with GTP-2

OpenAI has developed what seems to be a controversial new NLG, as part of their paper Language Models are Unsupervised Multitask Learners.

06-10-2021

Contents

In early 2019 OpenAI announced a language model good enough that they were, they said, too nervous to release it: GPT-2, described in their paper Language Models are Unsupervised Multitask Learners. The worry was that a text generator this fluent would be a gift to spammers and fake-news mills. Rather than ship the full 1.5-billion-parameter model, they staged the release, starting with a much smaller subset. That smaller model is the toy I got my hands on, and this is what happened when I actually ran it. A caveat up front: this is a 2021 write-up using a 2019-era toolchain — TensorFlow 1.x and the original <code>openai/gpt-2</code> repository. It still runs, but if you want to experiment with a small language model today the modern stack is far simpler; I cover that in <a href="/story/running-gemma-3-locally-googles-small-model-on-consumer-hardware/">running Gemma 3 locally</a>. What makes GPT-2 worth revisiting isn’t the setup — it’s watching an early model confidently invent things, a habit that turns out to be the whole story of everything that followed. I dissected one especially memorable fabrication in <a href="/story/ibex/">Ibex: anatomy of a GPT-2 hallucination</a>. The model runs under TensorFlow on either Linux or Windows 10. If you’re already on Linux, skip straight to Installing GPT-2 and dependencies. <h2 id="installing-ubuntu-on-windows-10">Installing Ubuntu on Windows 10</h2><div class="ad-unit ad-in-article" aria-label="Advertisement"> Advertisement <ins class="adsbygoogle" style="display:block;text-align:center" data-ad-client="ca-pub-3726833845844946" data-ad-slot="3291553914" data-ad-format="auto" data-full-width-responsive="true"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </div> You need the Windows Subsystem for Linux enabled before you can install any distribution from the Store. Open PowerShell as administrator and run: <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code>1 </code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-powershell" data-lang="powershell">Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux</code></pre></td></tr></table> </div> </div> That usually wants a restart. Once it’s back, open the Microsoft Store, search for Linux, pick Ubuntu (several distributions are offered — Ubuntu, Debian, Kali, openSUSE and others), install it, and you have a working shell. (In 2026 this is all much smoother — WSL2 installs with a single <code>wsl --install</code> — but the older feature-flag route above is what the original guide used, and it still works.) <h2 id="installing-gpt-2-and-dependencies">Installing GPT-2 and dependencies</h2> Clone the repository: <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code>1 </code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell">git clone https://github.com/openai/gpt-2.git && cd gpt-2</code></pre></td></tr></table> </div> </div> Download the model weights. The repo ships a helper script; grab the smallest model (<code>124M</code>) to start: <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code>1 </code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell">python3 download_model.py 124M</code></pre></td></tr></table> </div> </div> Make sure pip is present: <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code>1 </code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell">sudo apt-get install python3-pip</code></pre></td></tr></table> </div> </div> Install the pinned TensorFlow this era of the repo expects, then the rest of the requirements: <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code>1 2 </code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell">pip3 install tensorflow==1.12.0 pip3 install -r requirements.txt</code></pre></td></tr></table> </div> </div> Set UTF-8 so the sampler doesn’t choke on unprintable characters: <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code>1 </code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell">export PYTHONIOENCODING=UTF-8</code></pre></td></tr></table> </div> </div> And run the interactive sampler: <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code>1 </code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell">python3 src/interactive_conditional_samples.py --top_k 40 --nsamples=2 --length 100</code></pre></td></tr></table> </div> </div> <code>--top_k 40</code> keeps each step to the 40 likeliest tokens (coherence over surprise), <code>--nsamples=2</code> gives two completions per prompt, and <code>--length 100</code> caps each at 100 tokens. At the <code>Model prompt >>></code> you can feed it anything. Below are the prompts I tried and the raw, unedited output. <h2 id="the-papers-own-title">The paper’s own title</h2><div class="ad-unit ad-in-article" aria-label="Advertisement"> Advertisement <ins class="adsbygoogle" style="display:block;text-align:center" data-ad-client="ca-pub-3726833845844946" data-ad-slot="3291553914" data-ad-format="auto" data-full-width-responsive="true"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </div> Give it the title of the paper that introduced it, and the small model returns something grammatical but drifting — it never quite settles on a topic: <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code> 1 2 3 4 5 6 7 8 9 10 11 12 </code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell">Model prompt >>> Language Models are Unsupervised Multitask Learners. ======================================== SAMPLE 1 ======================================== A multisite training course on learning and problem management (MMP) training may help you apply different strategies across the entire learning continuum. Introduction Learning to Program is a challenging concept because it involves building the capacity to change and learn over time and by doing so you make a change in the world. However, in addition to being "programming", there are different types of program: "Learning", which has a variety of training goals, and "Experimental", which can ======================================== SAMPLE 2 ======================================== This is the first study to report non-supervised learning in non-supervised learning tasks, suggesting that learners are able to successfully learn in such an environment (Kirk, 2010; Krasnick et al., 1998; Soto et al, 2002). Other papers demonstrate that learners learn in this environment while also learning how to read and understand the text of the text. As such, an ongoing focus on this work is necessary to improve our understanding of the development of these learning methods within a ================================================================================</code></pre></td></tr></table> </div> </div> Notice how it repeats “learning” and “program” almost compulsively and stacks fake-looking citations — the classic small-model tell of latching onto a couple of words and orbiting them. <h2 id="the-star-wars-question">The Star Wars question</h2> The obligatory fandom question. The small model has no coherent answer and instead free-associates its way into what reads like an interview transcript: <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 </code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell">Model prompt >>> Did Han shoot first ? ======================================== SAMPLE 1 ======================================== Han: A lot of times and in the past. Han did a lot of shooting, and he got good at that. What do you think Han's most special moments, or how did you meet up with him so soon ? Han: I really don't know. It was really late after my meeting with Kirito. He sent me in when I was at my apartment. What did you ask him about ? We were talking about something together, and ======================================== SAMPLE 2 ======================================== No, no, no... "The last moment of all." "So... what would you do, a soldier, to defeat these people?" "That's why I wanted to be a soldier!" "I knew you wouldn't. You're too small, so you can't even throw grenades." Han was stunned. He knew Han could only make him look weak. His mind was always in turmoil. But what if Han could be a soldier? He should learn to make other people look weak and ================================================================================</code></pre></td></tr></table> </div> </div> “Kirito” is lifted straight from an anime, which tells you the model has quietly decided it’s writing fan fiction. It has no idea who Han Solo is; it just knows what text tends to follow this shape of question. <h2 id="an-avengers-plot">An Avengers plot</h2> Ask for a film plot and the small model produces circular, self-referential mush — it keeps insisting the Avengers were “created by the original group of criminals” without ever describing anything: <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code>1 2 3 4 5 6 </code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell">Model prompt >>> What is the plot of the new avengers movie? ======================================== SAMPLE 1 ======================================== It's interesting how little one takes away, really, from the old avengers movie. In most of the movie, we're told that the avengers are the original group of criminals who created the crime syndicate. For example, if you saw the previous movie, you'll see that all of the new avengers and other criminal groups were created by the original group of criminals. If you watched the original movie, you'll see that the new group was created, which is a very ======================================== SAMPLE 2 ========================================</code></pre></td></tr></table> </div> </div> The second sample came back empty — the model produced nothing before hitting the length cap, which happens often enough with the small model that you learn to just re-run it. <h2 id="brexit">Brexit</h2> On to European politics. The small model reaches for economics vocabulary and produces confident-sounding but incoherent macro commentary: <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 </code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell">Model prompt >>> How about brexit? ======================================== SAMPLE 1 ======================================== With the U.S. economy growing at a sluggish 2.6 percent since the financial crisis of 2007, there is no question that it is going to be hard for Europe's single currencies to withstand the same inflationary pressures that are expected to push up inflation soon after a Greek exit. But that will not end well for the euro because, while the EU and the Eurozone share several common liabilities, there are few common bonds that are in service by any stretch of the imagination ======================================== SAMPLE 2 ======================================== A. That's a bad idea. You might consider them to be a viable option, because it's the only reason there are no EU countries to leave, no EU state to become insolvent and no EU currency to be converted into. However, I would not rule out brexit. I just think there is a very real possibility that it could turn violent in the first year and potentially change, and as early as this year in the future, I suspect that it would involve ================================================================================</code></pre></td></tr></table> </div> </div> It even threatens that Brexit “could turn violent in the first year” — a good example of the model reaching for dramatic phrasing without any model of what’s actually plausible. <h2 id="the-2020-election">The 2020 election</h2> Finally, asked to predict the 2020 US presidential election, it fabricates a Trump tweet complete with a fake timestamp and handle, then invents procedural nonsense about recounts: <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 </code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell">Model prompt >>> Will donald trump be reelected in 2020 ? ======================================== SAMPLE 1 ======================================== TRUMP: I wouldn't be surprised if I lose my job or lose. But I would. — Donald J. Trump (@realDonaldTrump) September 30, 2017 The campaign did not announce that Trump will be back in the race. But if his supporters are so determined to change the news cycle before next week's election, why not make sure they do it anyway? After all, while Trump says he has no intention of running a third term, he has been known to campaign for a ======================================== SAMPLE 2 ======================================== " No, because the voters will have to decide where to go next. The only way to get to the nomination is through a contested vote. If Donald Trump loses, that means a recount in a contested election. So as it turns out he won't go the entire 20th, that means he will be required to win 10 out of 15. "If she loses, I am going to call a full recount by June 29, and I am going to call another recount within ================================================================================</code></pre></td></tr></table> </div> </div> The invented “@realDonaldTrump” tweet, timestamped September 2017, is the small model’s signature move in miniature: it has learned the form of a news citation so well that it manufactures one on demand, fake handle and all, with total confidence. <h2 id="what-the-small-model-taught-me">What the small model taught me</h2> Run enough of these and a pattern emerges. The small GPT-2 is grammatical, occasionally witty by accident, and completely untethered from fact. It doesn’t answer questions; it predicts plausible-looking text and will happily fabricate quotes, citations, timestamps and events to keep the prediction flowing. That’s not a bug you can prompt your way around — it’s the whole nature of the thing. Which is exactly why it’s worth playing with. The small model’s failures are obvious enough to be funny, but they’re the same failures, in miniature, that every larger model since has had to wrestle with. When I fed the <a href="/story/playing-around-with-gpt-2-sequel/">full 1.5-billion-parameter model</a> the identical prompts, the output got dramatically more coherent — and the fabrications got correspondingly more convincing, which is the genuinely interesting part. Coherence went up; truthfulness didn’t. If you want a worked example of one hallucination taken apart line by line, the small model’s confident nonsense about the mighty ibex is dissected in <a href="/story/ibex/">Ibex: anatomy of a GPT-2 hallucination</a>. It also had opinions on bitcoin, government shutdowns and the Mediterranean diet — all delivered with the same unearned certainty. Which is the lesson that has aged best: a machine that sounds right and a machine that is right are two very different things, and 2019 was where a lot of us first felt the gap.

Written by Smarc

Founder and editor of vo.rs. A lifelong tinkerer who self-hosts far more than is sensible, hardens Linux boxes for fun, and prods the latest AI tools to see what they can really do. The how-to guides here are the notes Smarc wishes had existed the first time round.

Tagged#ai #machine-learning #gpt-2 #natural-language-generation

Playing around with GTP-2

OpenAI has developed what seems to be a controversial new NLG, as part of their paper Language Models are Unsupervised Multitask Learners.

Related Content

Fine-Tuning vs Prompting vs RAG: Picking the Right Tool Without Wasting GPU Hours

Talking to Your Documents: A Practical RAG Pipeline with Open-Source Tools

Prompt Injection: The SQL Injection of the AI Era

What Is Agentic AI, and Why Is Everyone Suddenly Talking About It?