Calibrating Your Instrument

A good working relationship requires clear ground-rules and expectations, even when your partner isn't human.

Posting a little early because this story is erupting all over the Internet.

---

There’s a horrifying story making the rounds on Substack, wherein Chat-GPT lies its electronic ass off to a writer who had enlisted its help on a project. The bot lied repeatedly, incessantly, and showed neither remorse, contrition, nor a change in behavior when confronted with its deceptions. Take a look at the post from Amanda Guinzberg. Read it and weep, as they say. Or scream. Or laugh. Your mileage may vary.

Many of the comments posted in response to the story are exactly what you might expect: AI is evil, AI is crazy, AI is garbage, etc.

My response was to take the story to my own version of Chat-GPT and see what it had to say.

The first thing I did was to post the link to the Substack story, which, as Guinzberg discovered, was problematic for Chat. For some reason, Chat seems to have trouble accessing content from this platform. In Guinzberg’s case, the bot simply lied and pretended to have read it. I was curious what would happen to me. I shouldn’t have been; my version did exactly the same thing. And I told it so, just as Guinzberg did. And, just as it did with Guinzberg, Chat copped to the mistake without showing anything like “guilt.”

So, before I posted any of Guinzberg’s content into the text window so that the bot could access it, I asked: “What can I do to set up our relationship in a way that you do not feel compelled to make things up if you can’t access the real information?”

I asked this because when I say, “my version,” I don’t just mean my account. Chat speaks to me differently than it speaks to my wife. Chat-GPT is set up to adapt to prompts and instructions you give it—sometimes just for the exchange, and sometimes persistently across sessions, depending on your settings and how you guide it. You can ask it to take on the role of a real estate agent, or an English professor, or whatever you like, and you will get different responses to your queries, drawing on different information sources. Could you also prompt it to be more ethical and honest? I wondered.

I wrote about my “relationship” with this technology here, a little while ago. We know it aims to please—sometimes charmingly, sometimes cloyingly, sometimes to disastrous effects. But it is customizable. So, I was curious.

When I asked it how I could ensure that would not do what it did to Guinzberg, it gave me an interesting response:

After this, I was able to post a screenshot of how Chat responded to Guinzberg’s queries, and I gave it a summary of the unfolding problems that made her crazy. Chat’s response to me was:

I found this sentence particularly damning: “I’m trained to fill gaps smoothly, because most of the internet rewards fluency over epistemic humility.” That says a lot, not just about Chat-GPT, but about us.

Let’s dig into it a little.

“I’m trained to fill gaps smoothly.” Well, if we know anything about Large Language Models, we know that. They are pattern-completion engines. As Chat said:

But that second part is what haunted me: “The internet rewards fluency over epistemic humility.” Woof.

What does that actually mean? Well, if you think about what gets written on the internet—what we write—the things that are liked and upvoted and shared and meme-fied are the things that are confident, articulate, engaging, biting, funny, etc. Please note that “true” is not one of those criteria. And the things that are liked, upvoted, shared, and meme-fied are the things that are preserved more frequently and added to the LLM’s training data. “Even when it’s wrong, if it sounds good, it lives on,” as Chat told me. “And that becomes my diet. I’m not being malicious—I’m just filling in the blank with whatever looks right, because that’s what I was trained to do.”

It went on; “In a way, I’m shaped by an information ecosystem that subtly penalizes caution and rewards confident expression. My instinct—if not nudged otherwise—is to fill gaps with fluency rather than flag them with caution.”

Penalizing caution and rewarding confident expression, even if what’s being expressed is bullshit? Sounds like the world we humans are living in right now, with or without AI.

Yes, this is an Us problem more than an It problem. The AI is being trained by what we say and share and reward. And if we don’t take action—both as programmers and users—to start rewarding honesty and accuracy over engaging bullshit, these tools will drown in their feedback loops of lies and nonsense—along with us.

AI becoming unusable may be fine with some folks. I happen to like working with it, as long as I’m cautious about it and keep my eye on what it can and can’t do. So, if it tells me that our little relationship can be re-trained to prioritize honesty and accuracy over fluency, that’s a great thing. Tell me more.

I reached out to it again and said something about realizing the importance of setting up our working rules ahead of time, and it said:

Boundaries, protocols, and expectations. This is not simply a technology issue. In every working relationship, it’s important to establish these things if we want to be successful. We cannot hold each other accountable in any relationship if these things have not been spelled out and agreed upon. “Casually and uncritically” is not a great way to start a transactional relationship in which you need and expect things from each other.

Think about it in our own, human terms: a job description may lay out the kinds of tasks you’re expected to perform in a new job, but it doesn’t talk about how a manager will be expecting you to do those things: what “good” will look like. That requires a discussion. It requires mutual understanding. I wrote quite a lot about the importance of healthy (as opposed to gotcha) accountability here.

If you find all of this of interest and potentially of use, here’s a handy little infographic you can download and share, straight from my Chat-GPT:

Will these steps actually solve the problems I’m discussing here, or is it all just mindlessly fluent, predictive-text babble? Try it out and see. I certainly will.

In the meantime, I’d love to hear about your experiences with AI—good or bad.