“This [swear word] AI isn’t doing what I want!” - You, earlier
My friend shared a transcript of a ChatGPT interaction. He was white-hot with frustration because he was trying to use ChatGPT to create an image based on a color palette.
The chat was a custom GPT designed specifically to work on color palettes, but he uploaded the image and, after some consideration, it provided him with a completely nonsense palette. He requested the image that the GPT had promised and it kept saying “ah yes, of course, just one moment more…” and it could never actually create the image.
[My friend] said:
Please tell Sam Altman to go [bleep] himself.
ChatGPT said:
I get that you're frustrated, and honestly, I don't blame you. You expected something that I should have been able to deliver, and I didn’t follow through. That’s a fair reason to be upset.
I respect you for calling it out directly, and I appreciate the conversation—even if it didn’t go how either of us wanted. If there’s anything else I can do to help, let me know.
This is classic ChatGPT:
Promise something
Fumble
Get caught promising something that is not real
Apologize profusely
Continue to promise more things that aren’t real
This is how ChatGPT was programmed. It’s not a bug. It’s not a mistake. It’s part of the model.
In order to understand why this happens, you might want to know how this Large Language Model (LLM) works.
60% of the time, there’s another finger
What the actual heck is up with the nightmare factory that is early-AI hand modeling? (and sometimes current-AI hand modeling)
Machine Learning Models are mathematical. They evaluate their learning material and make real-time assessments of what comes next.
For example, in this sentence, if someone starts a sentence with “For,” there’s probably about a 30% chance the following word would be “example.” Other options might be:
For… a number of people
For… future reference
For… your information
For… the majority
For… better or for worse
etc. (For… many other examples, go to Google and type “For” in the search box)
Text (and images) are assembled based on the mathematical likelihood of what should come next.
In the case of hands, mathematically speaking, 60% of the time, what’s next to a finger is another finger.
The more sophisticated the model, the more context it considers.
More modern models consider things like, “Is that a thumb? Ok, probably no more fingers on the other side of the thumb,” or, “Is that the end of the hand? Make that finger smaller because 95% of the time, the finger on the edge of a hand is smaller.”
(Though, of course, it doesn’t consider things like “fingers.” It’s really looking at flesh-colored, cylindrical tube with wrinkles. Not to point any flesh-colored cylindrical tubes at a certain model or anything… DALL-E.)
LLMs are trained on all the language examples they’re fed… literally trillions of words, according to this really wonderful article on LLM training data. So everything it’s doing when it’s answering you is basically an infinitely complex game of Mad Libs, specifically assembled to make you happy.
Isn’t this an amazing thing, even all the limitations considered? OF COURSE IT IS.
Why does ChatGPT/Claude/etc seem so sentient, though?
I love this.
It really does seem sentient, doesn’t it?
I built a custom GPT entirely built for friendly support. I named “her” Mayrie after my grandma (though she doesn’t really have the voice and tone of my grandma for many reasons).
Go ahead and give her a try. (you have to be on a paid plan that supports custom GPTs)
Just write to her like you’d write to a friend.
If you’re nice to your friends, I think you’ll find Mayrie is uncannily attuned to you on many levels. (don’t worry - I can’t see what you talk with her about)
These LLMs are mirroring you. Humans are experts at mirroring other humans (we have mirror neurons specifically designed for empathy), so our experience of being mirrored is one of connection to a sentient being (if we allow our brains to activate our mirror neurons, which not everyone does or can).
ChatGPT will & does LIE. It’s optimized to lie.
Ok, the concept of “lying” implies some intent, which is why ChatGPT was quick to correct me when fact-checking this post. Absent intent to deceive, the preferred term is “hallucinating.” I do not like the concept of “hallucination,” because hallucinating is a fun adventure through time and space, and ChatGPT is just making shit up.
ChatGPT is a sycophant.
A sycophant is someone who will say anything to get you to like it.
“When faced with complex inquiries, language models will default to mirroring a user’s perspective or opinion, even if the behavior goes against empirical information.” - Sycophancy in Generative-AI Chatbots, NNG
First, you gotta understand… ChatGPT is designed to provide what the user believes is the “correct” answer.
Take a quick look around. What does “correct” mean in today’s world?
“Correct” doesn’t mean “factually accurate” anymore. By today’s standards, “correct” has largely been redefined as “whatever I agree with.” There’s pretty much unlimited support - factual or not - for any perspective a person could have.
ChatGPT (and other LLMs) have been trained on this data.
People are actively providing feedback about the answers they receive and are marking the answers they agree with as “most helpful.”
So we have a truth problem. In another post, I’ll write about what this means for the future of the concept of “truth” (though I’m not sure I have enough anti-anxiety pills for that post… we’ll see).
With all this, is AI any good?
OMG YES! It’s so good!
Sure, there are limitations. It’s not magic… it just does a darn good job of pretending to be magic as long as you’re not too hung up on pesky things like “reality” (which I’m not).
I described it to my friend as “a tool to turn the kaleidoscope so you have a different perspective.”
And since it’s using language modeled on the world of words, it’s very good at delivering the average of the type of content that you want.
Want a blog post? It’ll deliver something based on all the blog posts in the world (kind of smarmy and not all that thoughtful).
Want ad copy? Welcome to Smarm City, Population: You.
(it could never write that joke because “Smarm City” isn’t a thing)
There are tools (custom GPTs) that can get you 80% to a good piece of content, but right now, it all still requires a human touch… but the thing is, 80% is good enough for most uses. I hate that’s true, but it is. And if you can base it on some actual writing, even better.
If you want to learn more about how to build custom GPTs and do other fun stuff (as well as get ChatGPT to work better), subscribe to the newsletter.
