Playing around with GTP-2

Playing around with GTP-2

machine brain

OpenAI has developed what seems to be a controversial new NLG, as part of their paper Language Models are Unsupervised Multitask Learners.

Although they don't seem to be ready to release the full trained model, they've release a smaller subset of the model.:

"For now, we have only released a smaller (117M parameter) version of GPT-2." (github project readme)

This is great news if you're like me and can't wait to try out this new toy the model can be run using tensorflow on either linux or windows 10, if you're using linux please skip right ahead to Installing GPT-2 and dependencies.

Installing ubuntu on windows10

You need to enable the linux subsystem for windows before you're able to use any linux distributions from the windows store, so run a powershell as administrator and type.:

Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux

This will most likely require a restart, when thats done head into the windows store and search for linux, currently a few distributions are available - kali, suse enterprise, ubuntu, debian, ubuntu 18.04 and ubopensuse.

Install ubuntu and you're set.

Installing GPT-2 and dependencies.:

Start by cloning the gpt-2 repository 

 git clone https://github.com/openai/gpt-2.git && cd gpt-2

download the model data

 git clone https://github.com/openai/gpt-2.git && cd gpt-2

Make sure you have pip installed

sudo apt-get install python3-pip

Then install tensorflow

pip3 install tensorflow==1.12.0

Install the remaining dependencies 

pip3 install -r requirements.txt

Now you should be ready to try out the limited model, make sure to setup UTF-8 to avoid unprintable characters

export PYTHONIOENCODING=UTF-8

And now we can run the model

python3 src/interactive_conditional_samples.py --top_k 40 --nsamples=2 --length 100

When prompted for an input (Model prompt>>>) you can feed it basically anything, if we give it the name of the paper we might get.:

Model prompt >>> Language Models are Unsupervised Multitask Learners.
======================================== SAMPLE 1 ========================================
 A multisite training course on learning and problem management (MMP) training may help you apply different strategies across the entire learning continuum.

Introduction

Learning to Program is a challenging concept because it involves building the capacity to change and learn over time and by doing so you make a change in the world. However, in addition to being "programming", there are different types of program:

"Learning", which has a variety of training goals, and "Experimental", which can
======================================== SAMPLE 2 ========================================
 This is the first study to report non-supervised learning in non-supervised learning tasks, suggesting that learners are able to successfully learn in such an environment (Kirk, 2010; Krasnick et al., 1998; Soto et al, 2002). Other papers demonstrate that learners learn in this environment while also learning how to read and understand the text of the text. As such, an ongoing focus on this work is necessary to improve our understanding of the development of these learning methods within a
================================================================================

Or we could ask it a classical question.:

Model prompt >>> Did Han shoot first ?
======================================== SAMPLE 1 ========================================



Han: A lot of times and in the past. Han did a lot of shooting, and he got good at that.


What do you think Han's most special moments, or how did you meet up with him so soon ?


Han: I really don't know. It was really late after my meeting with Kirito. He sent me in when I was at my apartment.


What did you ask him about ? We were talking about something together, and
======================================== SAMPLE 2 ========================================
 No, no, no... "The last moment of all." "So... what would you do, a soldier, to defeat these people?" "That's why I wanted to be a soldier!" "I knew you wouldn't. You're too small, so you can't even throw grenades." Han was stunned. He knew Han could only make him look weak. His mind was always in turmoil. But what if Han could be a soldier? He should learn to make other people look weak and
================================================================================

Or ask it for a movie plot 

Model prompt >>> What is the plot of the new avengers movie?
======================================== SAMPLE 1 ========================================


It's interesting how little one takes away, really, from the old avengers movie. In most of the movie, we're told that the avengers are the original group of criminals who created the crime syndicate. For example, if you saw the previous movie, you'll see that all of the new avengers and other criminal groups were created by the original group of criminals. If you watched the original movie, you'll see that the new group was created, which is a very
======================================== SAMPLE 2 ========================================

About current hot politcal topics in the EU.:

Model prompt >>> How about brexit?
======================================== SAMPLE 1 ========================================


With the U.S. economy growing at a sluggish 2.6 percent since the financial crisis of 2007, there is no question that it is going to be hard for Europe's single currencies to withstand the same inflationary pressures that are expected to push up inflation soon after a Greek exit.

But that will not end well for the euro because, while the EU and the Eurozone share several common liabilities, there are few common bonds that are in service by any stretch of the imagination
======================================== SAMPLE 2 ========================================


A. That's a bad idea. You might consider them to be a viable option, because it's the only reason there are no EU countries to leave, no EU state to become insolvent and no EU currency to be converted into.

However, I would not rule out brexit. I just think there is a very real possibility that it could turn violent in the first year and potentially change, and as early as this year in the future, I suspect that it would involve
================================================================================

Or about the result of the 2020 US presidential election .:

Model prompt >>> Will donald trump be reelected in 2020 ?
======================================== SAMPLE 1 ========================================
 TRUMP: I wouldn't be surprised if I lose my job or lose. But I would. — Donald J. Trump (@realDonaldTrump) September 30, 2017

The campaign did not announce that Trump will be back in the race. But if his supporters are so determined to change the news cycle before next week's election, why not make sure they do it anyway?

After all, while Trump says he has no intention of running a third term, he has been known to campaign for a
======================================== SAMPLE 2 ========================================
 "

No, because the voters will have to decide where to go next. The only way to get to the nomination is through a contested vote. If Donald Trump loses, that means a recount in a contested election. So as it turns out he won't go the entire 20th, that means he will be required to win 10 out of 15.

"If she loses, I am going to call a full recount by June 29, and I am going to call another recount within
================================================================================

For more examples - related to more current events checkout what the small GPT-2 model has to say on the mighty Ibex, - trying to explain what bitcoin is all about, the concept of goverment shutdowns, how to eat healthy the mediterranian way and whats exactly going on with brexit