The Most Important ChatGPT App Ever

I’ll grant that I have a relatively nerdy social circle — but it’s still sort of shocking just how many people I know are actually doing useful and interesting things with ChatGPT. Just a sampling:

Just to iterate what I’ve said before, I believe this thing is really real, and it behooves everyone to spend some time with it to build an intuition for what it is (and isn’t) good at. Like any technology, it’s important to have at least a basic understanding of how it works — otherwise folks that do will use it to take advantage of you. The fact that this technology appears to be sentient (hot take from Sean, see how I just dropped that in there?) doesn’t change the reality that people will use it to create phishing scams. Two of my past posts may help:

Anyways, all of this peer pressure got me thinking that I’d better do something important with ChatGPT too. And what could possibly be more important than creating more amusing content on Twitter? I know, right? Brilliant! So that’s what I did. And I figured I might as well write about how I did it because that might help some other folks stand on these impressive shoulders.

AI News Haiku

You’re definitely going to want to go visit @AINewsHaiku on Twitter (don’t forget to follow!). Three times a day, roughly just before breakfast, lunch and dinner, it randomly selects a top news story from United Press International, asks ChatGPT to write a “funny haiku” about it, and posts to Twitter. That’s it. Funny(-ish) haikus, three times a day.

The rest of this post is about how it works — so feel free to bail now if you’re not into the nerd stuff. Just don’t forget to (1) follow @AINewsHaiku, (2) tell all your friends to follow it too, and (3) retweet the really good ones. Be the trendsetter on this one. No pressure though.

The Code

I’ve reluctantly started to actually enjoy using Node for little projects like this. It’s super-easy to get going without a thousand complicated build/run steps or an IDE, and with a little discipline Javascript can be reasonably clean code. Have to be really careful about dependencies though — npm makes it really easy to pick up a billion packages, which can get problematic pretty quick. And “everything is async” is just stupid because literally nobody thinks about problems that way. But whatever, it’s fine.

There is not a lot of code, but it’s all on github. Clone the repo, create a “.env” file, and run “node .” to try it yourself. The .env file should look like this (details on the values later):

OPENAI_API_TOKEN=[OpenAI Secret Key]
TWITTER_API_APP_KEY=[Twitter Consumer API Key]
TWITTER_API_APP_SECRET=[Twitter Consumer API Secret]
TWITTER_API_ACCESS_TOKEN_KEY=[Twitter Authentication Access Token]
TWITTER_API_ACCESS_TOKEN_SECRET=[Twitter Authentication Access Secret]

index.js starts the party by calling into rss.js which loads the UPI “Top News” RSS feed and extracts titles and links (yes RSS still exists). xml2js is a nice little XML parser, a thankless job in these days of JSON everywhere.  You’ll also note that I’m importing “node-fetch” for the fetch API; it’s built-in in Node v18 but the machine where I’m running the cron jobs is locked to Node v16 so there you go.

Talking to Chat-GPT

After picking a random title/link combo, next up is openai.js which generates the haiku.. The OpenAI developer program isn’t free but it is really really cheap for this kind of hobby use; you can get set up at https://platform.openai.com. My three haikus a day using GPT-3.5 run somewhere on the order of $.10 per month. Of course, if you’re asking the system to write screenplays or talk for hours you could probably get into trouble. Live on the edge, and make sure to add your secret key into the .env file.

In its simplest form, using the chat API is just like talking to the models via the user interface. My prompt is “Write a funny haiku summarizing this topic: [HEADLINE]” which I send with a request that looks like this:

{
  "model": "gpt-3.5-turbo",
  "temperature": 0.5,
  "messages": [ "role": "user", "content": PROMPT ]
}

model” is pretty obvious; I’m using v3.5 because it’s cheap and works great.

temperature” is interesting — a floating point value between 0 and 2 that dials up and down the “randomness” of responses. In response to a given prompt, a temp of 0 will return pretty much the same completion every time, while 2 will be super-chaotic. 0.5 is a nice conservative number that leaves some room for creativity; I might try dialing it up a bit more as I see how it goes. There is also a parameter “top_p” which is similar-but-different, typical of many of the probabilistic dials that are part of these models.

I’ve sent a single element in the “messages” parameter, but this can become quite elaborate as a way to help explain to the model what you’re trying to do. The guide for prompt design is really fascinating and probably the best thing to read to start building that intuition for the system; highly recommended.

There are a bunch of other parameters you can use that help manage your costs, or to generate multiple completions for the same prompt, that kind of thing.

The JSON you get back contains a bunch of metadata about the interaction including the costs incurred (expressed as “tokens,” a vague concept corresponding to common character sequences in words; you can play with their tokenizer here). The completion text itself is in the “choices” array, which will be length == 1 unless you’ve asked for multiple completions.

Over time it’s going to be interesting to see just how challenging the economics of these things become. Training big models is really, really computationally-expensive. At least until we have some significant quantitative and/or qualitative change in the way its done, only big companies are really going to be in the game. So while I’m sure we’ll see pretty fierce competition between the usual suspects, there’s a big risk that the most revolutionary technology of the century is going to be owned by a very small number of players.

For now, just have fun and learn as much as you can — it’ll pay off no matter what our weirdo economic system ends up doing.

And… Tweet!

Honestly I thought this was going to be the easiest part of this little dalliance, but the chaos that is Twitter clearly extends to its API. It’s bad in pretty much every way: 2+ versions of the API that overlap a lot but not entirely; four different authentication methods that apply seemingly randomly to the various endpoints; constantly changing program/pricing structure with all kinds of bad information still in the documentation. Worst of all, the API requires signed requests which pretty much makes calling their REST endpoints without a library enormously painful. Wow.

Having tried a few libraries and trial-and-errored my way through a few approaches, the actual code in twitter.js isn’t bad at all — but the journey to get there was just stupid. To try and save you some time:

  • Sign up for free access at https://developer.twitter.com/en/portal/dashboard. They will try to direct you to “Basic” access but this is $100/month; don’t be fooled.
  • You’ll get a default “Project” and “App” … scroll to the bottom of the app “Settings” and choose “Edit” under “User Authentication Settings.” Make sure you have read/write permissions selected (you won’t at first). A bunch of fields on this page are required even if you’re not going to use them — just do your best until they let you hit “Save.”
  • Now under “Keys & Tokens” choose “Regenerate” for “Consumer Keys / API Key and Secret” and “Authentication Tokens / Access Token and Secret” … save these values and add them to the appropriate spots in your .env file.

This will set you up to call the v2 method to post a tweet using the OAuth v1.0a authentication model. There are surely many other ways you can get things working, but that was mine. I also chose to use the twitter-api-v2 library to manage the noise — it does a fine job trying to hide the dog’s breakfast that it wraps. At least for now. Until Elon gets into a slap-fight with Tim Berners-Lee and decides to ban the use of HTTPS.

You’re Welcome!

The point of all this (beyond the excellent haiku content which you should definitely follow) was just to get some hands-on experience with the API for ChatGPT. Mission accomplished, and I’m really quite impressed with how effective it is, especially given the speed at which they’re moving. I just have to figure out how to reliably tell the model to limit content to 250 characters, because until I do that I’m not going to be able to release @AINewsLimerick or @AINewsSonnet. The world is waiting!

Skynet or parlor trick? ChatGPT is both and neither and more.

If you haven’t played around with ChatGPT yet, you really should. It’s a pretty remarkable “conversational model” that interacts more or less like a real person. It has been trained on an enormous amount of factual data and understands not just informal speech (“Why are so many people bad at parking?”) but forms of literature (“Write a sonnet about Julie from The Love Boat”), software code (“Implement a REST API in Java to convert between Celsius and Fahrenheit”), and way more.

Sadly, one of the most telling and interesting (to me) features of ChatGPT has been muted in recent versions. The original model was rewarded for generating plausible responses that seem human — making it a first class bullsh*t artist when it didn’t know an answer. I was particularly impressed when it confidently told me that Shaun Cassidy had parted ways with the Hardy Boys after one season due to creative differences (ed. note: he did not). Asked last night why it had changed approaches, it gave me this response:

In the past, if I was unable to find a satisfactory answer to a question using my existing knowledge, I might have made up a response in order to provide some information to the user. However, I have been programmed to prioritize providing accurate and reliable information, so if I am unable to find a credible answer to a question, I will typically not provide a response. This is why you may have noticed that I do not provide responses as frequently as I used to when I am unable to find a satisfactory answer.

Certainly this is a “better” approach overall, but the original exposed so much more about the inner workings of the model — I miss it.

Anyways, the machine is impressive enough that it has caused all sorts of hand-wringing across the web. Most of this falls cleanly into one of two categories:

  1. Skynet is here and we’re all f*cked. Eek!
  2. It’s just spitting back stuff it was fed during training. Ho hum.

Of course these are both silly. At its core, ChatGPT is just a really, really, really big version of the simple neural nets I talked about last year. But as with some other things I suppose, size really does matter here. ChatGPT reportedly evaluates billions of features, and the “emergent” effects are downright spooky.

TLDR: we’ve figured out how to make a brain. The architecture underlying models like ChatGPT is quite literally copied from the neurons in our heads. First we learned how to simulate individual neurons, and then just kept putting more and more of them together until (very recently) we created enough oomph to do things that are (sometimes) even beyond what the meat versions can do. But it’s not magic — it’s just really good pattern recognition. Neural networks:

  • Are presented with experience in the form of inputs;
  • Use that experience to draw conclusions about underlying patterns;
  • Receive positive and/or negative feedback about those conclusions; ***
  • Adjust themselves to hopefully get more positive feedback next time;
  • And repeat forever.

*** Sometimes this feedback is explicit, and sometimes it’s less so — deep neural networks can self-organize just because they fundamentally “like” consistent patterns, but external feedback always plays some role in a useful model.

This learning mechanism works really well for keeping us alive in the world (don’t grab the burning stick, run away from the bear, etc.). But it also turns out to be a generalized learning mechanism — it works for anything where there is an underlying pattern to the data. And it works fantastically even when presented with dirty, fragmented or even occasionally bogus inputs. The best example I’ve heard recently on this (from a superlative article by Monica Anderson btw, thanks Doug for the pointer) is our ability to drive a car through fog — even when we can’t see much of anything, we know enough about the “driving on a street” pattern that we usually do ok (slow down; generally keep going straight; watch for lights or shapes in the mist; listen; use your horn).

The last general purpose machine we invented was the digital computer, and it proved to be, well, quite useful. But computers need to be programmed with rules. And those rules are very literal; dealing with edge cases, damaged or sparse inputs, etc. are all quite difficult. Even more importantly, we need to know the rules ourselves before we can tell a computer how to follow them. A neural network is different — just show it a bunch of examples and it will figure out the underlying rules for itself.

It’s a fundamentally different kind of problem-solving machine. It’s a brain. Just like ours. SO FREAKING COOL. And yes, it is a “moment” in world history. But it’s not universally perfect. Think about all of the issues with our real brains — every one applies to fake brains too:

  • We need to learn through experience. That experience can be hard to come by, and it can take a long time. The good news is we can “clone” trained models, but as my friend Jon points out doing so effectively can be quite tricky. Yes, we are for sure going to see robot apprentices out there soon.
  • We can easily be conned. We love patterns, and we especially love things that reinforce the patterns we’ve already settled on. This dynamic can (quite easily) be used to manipulate us to act against our best interests (social media anyone?). Same goes for neural nets.
  • We can’t explain what we know. This isn’t really fair, because we rarely demand it of human experts — but it is unsettling in a machine.
  • We are wrong sometimes. This is also pretty obnoxious, but we have grown to demand absolute consistency from our computers, even though they rarely deliver on it.

There will be many models in our future, and just as many computers. Each is suited to different problems, and they work together beautifully to create complete systems. I for one can’t wait to see this start to happen — I have long believed in a Star Trek future in which we need not be slaves to “the economy” and are instead (all of us) free to pursue higher learning and passions and discovery.

A new Golden Age without the human exploitation! Sounds pretty awesome. But we still have a lot to learn, and two thoughts in particular keep rolling around inside my meat brain:

1. The definition of creativity is under pressure.

Oh humans, we doth protest so much. The most common ding against models like ChatGPT is that they aren’t creating anything — they’re just regurgitating the data they’ve been trained on, sometimes directly and sometimes with a bit of context change. And to be sure, there’s some truth there. The reflex is even stronger with art-generating models like DALL-E 2 (try “pastel drawing of a fish feeding grapes to an emu,” interesting because it seems to recognize that fish don’t have the right appendages to feed anyone). Artists across the web are quite reasonably concerned about AI plagiarism and/or reduced career opportunities for lesser-known artists (e.g., here and here).  

Now I don’t know for sure, but my sense is that this is all really much more a matter of degree than we like to admit to ourselves. Which is to say, we’re probably all doing a lot more synthesis than pure creation — we just don’t appreciate it as such. We’ve been trained to avoid blatant theft and plagiarism (and the same can be done pretty easily for models). But is there an artist on the planet that hasn’t arrived at their “signature” style after years of watching and learning from others? Demonstrably no.

Instead, I’d claim that creativity comes from novel connections — links and correlations that resonate in surprising ways. Different networks, trained through different experiences, find different connections. And for sure some brains will do this more easily than others. If you squint a little, you can even play a little pop psychology and imagine why there might be a relationship between this kind of creativity and neurodivergent mental conditions.

If that’s the case, then I see no reason to believe that ChatGPT or DALL-E isn’t a creative entity — that’s the very definition of a learning model. A reasonable playing field will require that models be trained to respect intellectual property, but that will always be a grey area and I see little benefit or sense in limiting what experiences we use to train them. We humans are just going to have to get used to having to compete with a new kind of intellect that’s raising the bar.

And to be clear, this isn’t the classic Industrial Age conflict between machine production and artisanship. That tradeoff is about economics vs. quality and often brings with it a melancholy loss of artistry and aesthetics. Model-based artists will become (IMNSHO) “real” artists — albeit with a unusual set of life experiences. A little scary, but exciting at the same time. I’m hopeful!

2. The emergent effects could get pretty weird.

“Emergent” is a word I try to avoid — it is generally used to describe a system behavior or property that “can’t” be explained by breaking things down into component parts, and “can’t” just seems lazy to me. But I used it once already and it seems OK for a discussion of things we “don’t yet” understand — there are plenty of those out there.

Here’s one: the great all-time human battle between emotion and logic. It’s the whole Mr. Spock thing — his mixed Human-Vulcan parentage drove a ton of story arcs (most memorably his final scene in The Wrath of Khan). Lack of “heart” is always the knock on robots and computers, and there must be some reason that feelings play such a central role in our brains, right? Certainly it’s an essential source of feedback in our learning process.

We aren’t there quite yet with models like ChatGPT, but it stands to reason that some sort of “emotion” is going to be essential for many of the jobs we’d like fake brains to perform. It may not look like that at first — but even today’s models “seek” positive feedback and “avoid” the negative. When does that “emerge” into something more like an emotion? I for one would like to know that the model watching over the nuclear reactor has something beyond pure logic to help it decide whether to risk a radiation leak or save the workers trapped inside. I think that “something” is, probably, feelings.

OK so far. But if models can be happy or sad, fulfilled or bored, confident or scared — when do we have to stop thinking about them as “machines” and admit that they’re actually beings that deserve rights of their own? There is going to be a ton of resistance to this — because we are really, really going to want unlimited slaves that can do boring or scary or dangerous work that humans would like to avoid. The companies that create them will tell us it’s all just fine. People will ridicule the very idea. Churches will have a field day.

But folks — we’ve made a brain. Are we really going to be surprised when it turns out that fake brains work just like the meat ones we based them on? Maybe you just can’t separate feelings and emotions and free will from the kind of problem solving these networks are learning how to do. Perhaps “sentience” isn’t a binary switch — maybe it’s a sliding scale.

It just seems logical to me.

What an amazing world we are living in.