I’ll grant that I have a relatively nerdy social circle — but it’s still sort of shocking just how many people I know are actually doing useful and interesting things with ChatGPT. Just a sampling:
- A friend in the shipping business, building a customer service bot that can coordinate in real-time between drivers and customers to fine-tune deliveries.
- One in the biotech software business, exploring new ways to market and explain complex feature concepts.
- Another in healthcare, using it to “double-check” diagnoses made by human doctors.
- Yet another in Microsoft Research, using it to automatically sift through Epic’s enormous EHR database dump.
- Even one asking it (with limited success) to improve on their resume.
Just to iterate what I’ve said before, I believe this thing is really real, and it behooves everyone to spend some time with it to build an intuition for what it is (and isn’t) good at. Like any technology, it’s important to have at least a basic understanding of how it works — otherwise folks that do will use it to take advantage of you. The fact that this technology appears to be sentient (hot take from Sean, see how I just dropped that in there?) doesn’t change the reality that people will use it to create phishing scams. Two of my past posts may help:
Anyways, all of this peer pressure got me thinking that I’d better do something important with ChatGPT too. And what could possibly be more important than creating more amusing content on Twitter? I know, right? Brilliant! So that’s what I did. And I figured I might as well write about how I did it because that might help some other folks stand on these impressive shoulders.
AI News Haiku
You’re definitely going to want to go visit @AINewsHaiku on Twitter (don’t forget to follow!). Three times a day, roughly just before breakfast, lunch and dinner, it randomly selects a top news story from United Press International, asks ChatGPT to write a “funny haiku” about it, and posts to Twitter. That’s it. Funny(-ish) haikus, three times a day.
The rest of this post is about how it works — so feel free to bail now if you’re not into the nerd stuff. Just don’t forget to (1) follow @AINewsHaiku, (2) tell all your friends to follow it too, and (3) retweet the really good ones. Be the trendsetter on this one. No pressure though.
The Code
I’ve reluctantly started to actually enjoy using Node for little projects like this. It’s super-easy to get going without a thousand complicated build/run steps or an IDE, and with a little discipline Javascript can be reasonably clean code. Have to be really careful about dependencies though — npm makes it really easy to pick up a billion packages, which can get problematic pretty quick. And “everything is async” is just stupid because literally nobody thinks about problems that way. But whatever, it’s fine.
There is not a lot of code, but it’s all on github. Clone the repo, create a “.env
” file, and run “node .
” to try it yourself. The .env
file should look like this (details on the values later):
OPENAI_API_TOKEN=[OpenAI Secret Key]
TWITTER_API_APP_KEY=[Twitter Consumer API Key]
TWITTER_API_APP_SECRET=[Twitter Consumer API Secret]
TWITTER_API_ACCESS_TOKEN_KEY=[Twitter Authentication Access Token]
TWITTER_API_ACCESS_TOKEN_SECRET=[Twitter Authentication Access Secret]
index.js
starts the party by calling into rss.js
which loads the UPI “Top News” RSS feed and extracts titles and links (yes RSS still exists). xml2js
is a nice little XML parser, a thankless job in these days of JSON everywhere. You’ll also note that I’m importing “node-fetch” for the fetch API; it’s built-in in Node v18 but the machine where I’m running the cron jobs is locked to Node v16 so there you go.
Talking to Chat-GPT
After picking a random title/link combo, next up is openai.js
which generates the haiku.. The OpenAI developer program isn’t free but it is really really cheap for this kind of hobby use; you can get set up at https://platform.openai.com. My three haikus a day using GPT-3.5 run somewhere on the order of $.10 per month. Of course, if you’re asking the system to write screenplays or talk for hours you could probably get into trouble. Live on the edge, and make sure to add your secret key into the .env file.
In its simplest form, using the chat API is just like talking to the models via the user interface. My prompt is “Write a funny haiku summarizing this topic: [HEADLINE]” which I send with a request that looks like this:
{
"model": "gpt-3.5-turbo",
"temperature": 0.5,
"messages": [ "role": "user", "content": PROMPT ]
}
“model
” is pretty obvious; I’m using v3.5 because it’s cheap and works great.
“temperature
” is interesting — a floating point value between 0 and 2 that dials up and down the “randomness” of responses. In response to a given prompt, a temp of 0 will return pretty much the same completion every time, while 2 will be super-chaotic. 0.5 is a nice conservative number that leaves some room for creativity; I might try dialing it up a bit more as I see how it goes. There is also a parameter “top_p
” which is similar-but-different, typical of many of the probabilistic dials that are part of these models.
I’ve sent a single element in the “messages
” parameter, but this can become quite elaborate as a way to help explain to the model what you’re trying to do. The guide for prompt design is really fascinating and probably the best thing to read to start building that intuition for the system; highly recommended.
There are a bunch of other parameters you can use that help manage your costs, or to generate multiple completions for the same prompt, that kind of thing.
The JSON you get back contains a bunch of metadata about the interaction including the costs incurred (expressed as “tokens,” a vague concept corresponding to common character sequences in words; you can play with their tokenizer here). The completion text itself is in the “choices
” array, which will be length == 1 unless you’ve asked for multiple completions.
Over time it’s going to be interesting to see just how challenging the economics of these things become. Training big models is really, really computationally-expensive. At least until we have some significant quantitative and/or qualitative change in the way its done, only big companies are really going to be in the game. So while I’m sure we’ll see pretty fierce competition between the usual suspects, there’s a big risk that the most revolutionary technology of the century is going to be owned by a very small number of players.
For now, just have fun and learn as much as you can — it’ll pay off no matter what our weirdo economic system ends up doing.
And… Tweet!
Honestly I thought this was going to be the easiest part of this little dalliance, but the chaos that is Twitter clearly extends to its API. It’s bad in pretty much every way: 2+ versions of the API that overlap a lot but not entirely; four different authentication methods that apply seemingly randomly to the various endpoints; constantly changing program/pricing structure with all kinds of bad information still in the documentation. Worst of all, the API requires signed requests which pretty much makes calling their REST endpoints without a library enormously painful. Wow.
Having tried a few libraries and trial-and-errored my way through a few approaches, the actual code in twitter.js
isn’t bad at all — but the journey to get there was just stupid. To try and save you some time:
- Sign up for free access at https://developer.twitter.com/en/portal/dashboard. They will try to direct you to “Basic” access but this is $100/month; don’t be fooled.
- You’ll get a default “Project” and “App” … scroll to the bottom of the app “Settings” and choose “Edit” under “User Authentication Settings.” Make sure you have read/write permissions selected (you won’t at first). A bunch of fields on this page are required even if you’re not going to use them — just do your best until they let you hit “Save.”
- Now under “Keys & Tokens” choose “Regenerate” for “Consumer Keys / API Key and Secret” and “Authentication Tokens / Access Token and Secret” … save these values and add them to the appropriate spots in your .env file.
This will set you up to call the v2 method to post a tweet using the OAuth v1.0a authentication model. There are surely many other ways you can get things working, but that was mine. I also chose to use the twitter-api-v2
library to manage the noise — it does a fine job trying to hide the dog’s breakfast that it wraps. At least for now. Until Elon gets into a slap-fight with Tim Berners-Lee and decides to ban the use of HTTPS.
You’re Welcome!
The point of all this (beyond the excellent haiku content which you should definitely follow) was just to get some hands-on experience with the API for ChatGPT. Mission accomplished, and I’m really quite impressed with how effective it is, especially given the speed at which they’re moving. I just have to figure out how to reliably tell the model to limit content to 250 characters, because until I do that I’m not going to be able to release @AINewsLimerick or @AINewsSonnet. The world is waiting!