Hi there, it's been a while, hasn't it?↑§
I recently began using GPT-2 to help me write satirical articles on a another project I run called Interdimensional News. I use the 774M model rather than the maximum super-powerful 1558M/1.5B model for speed.
For this blog post, I used the 774M model.
In Interdimensional News, every article is built with the help of GPT-2, but about 60-80% of each article is still made by me.
Now that I’ve gotten the hang of things, I’m going to try using GPT-2 to create various short stories.
To add a bigger challenge, roughly half of the text must be generated by GPT-2.
Reading Key↑§
bold text = I wrote it
normal text = GPT-2 wrote it
italicized text = My commentary
Story #1: Alice and Bob escape the catacombs↑§
“Bob! I found the doorway!” Alice yelled.
Bob was overjoyed.
It had been his idea to journey into the catacombs, and if it hadn’t been for Alice’s sense of direction, they would’ve been screwed.
It looked like this would be their final day in the catacombs. It had been nearly two days since they’d entered.
“Alice,” Bob said, “I was wondering if you had any idea of where we’re going to take the rest of the day.”
Alice shrugged. “I’ll take you up on your offer, if you tell me the rest of the story.”
Commentary: Gotta say, I love this fake pretentiousness. Feels like an Inkitt bestseller.
Story #2: Alice and Bob are in a stereotypical YA romance novel↑§
“I don’t know what I want to do anymore.” Alice said quietly, still in the doorway.
“Hm?”
“I mean, I used to want to get out of here. Go somewhere. Leave.”
“You don’t anymore?”
Alice smiled and shook her head. “I don’t know, it’s all so complicated now.” She walked toward me.
“I’d go with you, wherever you wanted. I mean, it’d be nice to be out of this place and not have to deal with these people.
“I know, but it’d make me sad to leave, too,” she sighed, “it was so easy when I was alone and miserable. And it just… I don’t know.”
“Sorry for complicating things.”
“You should be,” Alice turned around and gave me a soft, lingering kiss, “but also, thank you for everything.”
I smiled at her, “thanks for everything too.”
I wrapped my arms around her back and kissed her gently. She giggled. We remained in that position for a long time.
Story #3: Alice and Bob attempt to discern their reality↑§
Alice punched Bob playfully on the shoulder.
“So, what’s next?” she asked.
“Anything as long as it doesn’t include you jumping into black holes. That was a traumatizing experience.” Bob replied.
“Oh, well, I’m sure we can work something out. I really am glad to have you here, Bob. You’re the only one who’s ever come along and made me feel like this.” Alice said as she turned back around to face him.
“Oh… that’s nice to hear,” Bob replied, “are you going to tell me what’s on your mind?”
“I’ve been thinking about our relationship. We’re not married yet, and you’re still very much in my thoughts. The fact that you’ve been so sweet to me the last couple of days really makes me happy.”
“But Alice, you know those feelings are superficial. We’re just simulations after all. We aren’t even conscious right now. This is all a dream.”
Commentary: Tried to stop GPT-2 from going into another YA romance novel.
“Yes, you’re right. We’re all simulations. You and I aren’t really here right now. It’s all a game. What’s more, I’m just a simple simulation of your own subconscious mind. You’re the only real person I have ever known.”
“What? I’m real?”
“Well, if you believe the dream you’re in, you are, aren’t you?”
Commentary: We've now reached YIIK levels of pretentiousness.
So Yeah...↑§
Overall, I’d say that writing with GPT-2 is fun. But if you let it run off too far you’ll end up with some pretty weird stuff and tiny logical inconsistencies.
This might just be a side-effect of using the 774M model instead of the 1558M/1.5B version, but NLP is extremely hard anyways. GPT-2 handled it amazingly.
Bonus Content↑§
As a bonus, here’s a sample of book reviews GPT-2 generated for an Interdimensional News post (yes, the reviewer names were created by GPT-2):
“I can’t imagine a better story for the human condition.” —JEFF BEZOS
“The sex scenes are just incredible. Alice is so much more than an object of sexual desire, she is a person and she is complicated, intelligent and passionate and in many ways just a better version of me” —STEPHEN HAWKING
“This book is a great celebration of the female libido. I don ‘t know what you’ve read in science fiction or fantasy about female libidos, but the notion of ‘vagina porn’ is a new concept in the canon of literature. It’s an intriguing and important new development, because the female libido is really the core of our human consciousness. “ —DAVID FRICKE
In Conclusion↑§
I’m really looking forward to a better version of GPT-2 or maybe a (*deep inhale*) TensorFlow.js implementation.
When I was messing around with GPT-2, it would usually give stuff that you’d expect, but sometimes it would suggest things that completely 180’d the story.
For example, I gave GPT-2 a string about two friends hugging. The friend being hugged was suggested to be feeling “happy”, or they “smiled”. However, GPT-2 also gave me a suggestion that the friend being hugged was “scared”. This was mostly due to the vagueness and shortness of the prompt, but it was interesting and happened a couple of other times again when I was too vague with my prompts.
And finally, probably the best implementation of GPT-2 would be in school essays.
Need to boost the word count of your already-finished essay? Just go to the middle of each paragraph and have GPT-2 suggest a sentence or two!
Need to write a blob of text that supports a vague idea nobody gives a shit about? Just write 10 sentences and have GPT-2 turn them into 10 paragraphs!
And if you just have GPT-2 generate a couple of words at a time, not only will you have finer control of what gets generated, but it becomes much harder to tell if the text has been generated, especially if you shove in a couple of keywords here and there to nudge GPT-2 in the right track.
It’s quite an exciting time to be alive. Even someone like me, who struggles to use Excel, can jumble together an amazing janky suite of tools that sets us up above everyone else for no good reason at all.
What an exciting time to be alive!
References↑§
- GPT-2 GitHub Repo
- Max Woolf's great blog post on using GPT-2
- (Not used for this post) An article about retraining GPT-2
- Wrapper for GPT-2