Hi there, it's been a while, hasn't it?↑§
I recently began using GPT-2 to help me write satirical articles on a another project I run called Interdimensional News. I use the 774M model rather than the maximum super-powerful 1558M/1.5B model for speed.
For this blog post, I used the 774M model.
In Interdimensional News, every article is built with the help of GPT-2, but about 60-80% of each article is still made by me.
Now that I’ve gotten the hang of things, I’m going to try using GPT-2 to create various short stories.
To add a bigger challenge, roughly half of the text must be generated by GPT-2.
bold text = I wrote it
normal text = GPT-2 wrote it
italicized text = My commentary
Story #1: Alice and Bob escape the catacombs↑§
“Bob! I found the doorway!” Alice yelled.
Bob was overjoyed.
It had been his idea to journey into the catacombs, and if it hadn’t been for Alice’s sense of direction, they would’ve been screwed.
It looked like this would be their final day in the catacombs. It had been nearly two days since they’d entered.
“Alice,” Bob said, “I was wondering if you had any idea of where we’re going to take the rest of the day.”
Alice shrugged. “I’ll take you up on your offer, if you tell me the rest of the story.”
Commentary: Gotta say, I love this fake pretentiousness. Feels like an Inkitt bestseller.
Story #2: Alice and Bob are in a stereotypical YA romance novel↑§
“I don’t know what I want to do anymore.” Alice said quietly, still in the doorway.
“I mean, I used to want to get out of here. Go somewhere. Leave.”
“You don’t anymore?”
Alice smiled and shook her head. “I don’t know, it’s all so complicated now.” She walked toward me.
“I’d go with you, wherever you wanted. I mean, it’d be nice to be out of this place and not have to deal with these people.
“I know, but it’d make me sad to leave, too,” she sighed, “it was so easy when I was alone and miserable. And it just… I don’t know.”
“Sorry for complicating things.”
“You should be,” Alice turned around and gave me a soft, lingering kiss, “but also, thank you for everything.”
I smiled at her, “thanks for everything too.”
I wrapped my arms around her back and kissed her gently. She giggled. We remained in that position for a long time.
Story #3: Alice and Bob attempt to discern their reality↑§
Alice punched Bob playfully on the shoulder.
“So, what’s next?” she asked.
“Anything as long as it doesn’t include you jumping into black holes. That was a traumatizing experience.” Bob replied.
“Oh, well, I’m sure we can work something out. I really am glad to have you here, Bob. You’re the only one who’s ever come along and made me feel like this.” Alice said as she turned back around to face him.
“Oh… that’s nice to hear,” Bob replied, “are you going to tell me what’s on your mind?”
“I’ve been thinking about our relationship. We’re not married yet, and you’re still very much in my thoughts. The fact that you’ve been so sweet to me the last couple of days really makes me happy.”
“But Alice, you know those feelings are superficial. We’re just simulations after all. We aren’t even conscious right now. This is all a dream.”
Commentary: Tried to stop GPT-2 from going into another YA romance novel.
“Yes, you’re right. We’re all simulations. You and I aren’t really here right now. It’s all a game. What’s more, I’m just a simple simulation of your own subconscious mind. You’re the only real person I have ever known.”
“What? I’m real?”
“Well, if you believe the dream you’re in, you are, aren’t you?”
Commentary: We've now reached YIIK levels of pretentiousness.
Story #4: Alice and Bob are caught in an embarrassing situation↑§
It was quite a shock when Alice’s parents came home while we were having sex.
Alice slowly pulled herself off of me, making a loud plop.
A thin strand of spit dangled between my cock and her chin before she swiped at it with the back of her hand.
“You guys are back early,” she said meekly.
“Not really,” her mom replied.
Alice was still on her hands and knees, and there was a long pause before she got up to her feet.
“Mom my, where are my clothes?”
“In the corner,” her mom replied.
Alice was still a little unsteady on her feet.
I pulled on my shirt and put on my pants, embarrassed beyond belief.
Alice had been in a very intimate position on my lap for a long time, and she’d gotten used to it.
She stumbled slightly as she made her way to her clothes.
“Um,” she stammered, “I can’t find my bra.”
She turned around and looked over at her mom who had turned her back to her and was trying to keep a straight face.
Overall, I’d say that writing with GPT-2 is fun. But if you let it run off too far you’ll end up with some pretty weird stuff and tiny logical inconsistencies.
For example, in Story 4, Alice was on her “hands and knees” according to GPT-2, but then she had also apparently been sitting on Bob’s lap for a long time as well. (My head-canon is that Alice was in the seiza position on Bob's lap doing some kinky shit.)
This might just be a side-effect of using the 774M model instead of the 1558M/1.5B version, but NLP is extremely hard anyways. GPT-2 handled it amazingly.
GPT-2 also wouldn’t stop trying to get Alice laid or married, which was an issue because I’d always have to reel it back in.
As a bonus, here’s a sample of book reviews GPT-2 generated for an Interdimensional News post (yes, the reviewer names were created by GPT-2):
“I can’t imagine a better story for the human condition.” —JEFF BEZOS
“The sex scenes are just incredible. Alice is so much more than an object of sexual desire, she is a person and she is complicated, intelligent and passionate and in many ways just a better version of me” —STEPHEN HAWKING
“This book is a great celebration of the female libido. I don ‘t know what you’ve read in science fiction or fantasy about female libidos, but the notion of ‘vagina porn’ is a new concept in the canon of literature. It’s an intriguing and important new development, because the female libido is really the core of our human consciousness. “ —DAVID FRICKE
I’m really looking forward to a better version of GPT-2 or maybe a (*deep inhale*) TensorFlow.js implementation.
When I was messing around with GPT-2, it would usually give stuff that you’d expect, but sometimes it would suggest things that completely 180’d the story.
For example, I gave GPT-2 a string about two friends hugging. The friend being hugged was suggested to be feeling “happy”, or they “smiled”. However, GPT-2 also gave me a suggestion that the friend being hugged was “scared”. This was mostly due to the vagueness and shortness of the prompt, but it was interesting and happened a couple of other times again when I was too vague with my prompts.
And finally, probably the best implementation of GPT-2 would be in school essays.
Need to boost the word count of your already-finished essay? Just go to the middle of each paragraph and have GPT-2 suggest a sentence or two!
Need to write a blob of text that supports a vague idea nobody gives a shit about? Just write 10 sentences and have GPT-2 turn them into 10 paragraphs!
And if you just have GPT-2 generate a couple of words at a time, not only will you have finer control of what gets generated, but it becomes much harder to tell if the text has been generated, especially if you shove in a couple of keywords here and there to nudge GPT-2 in the right track.
It’s quite an exciting time to be alive. Even someone like me, who struggles to use Excel, can jumble together an amazing janky suite of tools that sets us up above everyone else for no good reason at all.
What an exciting time to be alive!