Your AI self-defense class begins
Your AI self-defense class begins
Don't be fooled: these AI's do not reason, calculate, or deduce anything.
This post begins a series on “AI” to help you understand what it is, what it is not, and how it will be used against you if you do not understand it.
A couple of months ago, we were almost impressed by a so-called “AI.”
We played Twenty Questions with a brilliant Large Language Model (LLM), the technology powering ChatGPT, which is being passed off as AI, and it appeared to play quite well until the very end, when it became obvious that it hadn’t played the game at all.
Instead of picking a mystery item at the beginning of the game, for us to guess, it just made up yes/no answers as it went along until we had been led into a small enough cul-de-sac of possibilities to end the game.
In other words, the computer was winging it the whole time! And we were projecting.
Did it cheat?
Maybe it didn’t understand the rules.
When I asked it to play Twenty Questions, I was impressed that it knew how to play. But did it? Maybe I was already projecting, even before the first question. The game began like this, with my text after the “>” prompts.
LLM served by Perplexity Labs
Hello! How can I help you?
> Let's play twenty questions!
Sure, I'm ready to play! What's the first question?
> Is it an animal?
Yes, it is an animal.The computer never claimed to have picked a mystery item, and I never told it to. It did provide consistent answers to my questions, which is a reasonable interpretation of “twenty questions,” which I did not even capitalize.
Did it reason?
Other than not understanding the game, the LLM performed amazingly well. I gave it opportunities to contradict itself, and it did not. It unraveled some twisted grammar and self-referential expressions without any mistakes. It really appeared to be interpreting my language and referring to facts about our world, in order to answer accurately.
But it was doing no such things.
An LLM does one simple thing in a very complex way: it predicts the single word that is most likely to come next in a stream of human-written text.
For example, the text “Once upon a” is most likely followed by “time” among countless documents written by countless people.
But that’s not very much context. This one’s much more interesting:
Replace “time” with “thyme” in a story that begins with “Once upon a”
Would you believe, these LLM’s actually proceed with “Once upon a thyme”?
Yes, they are amazing at complex multi-level patterns. They are so good, they often appear to be reasoning. It certainly appears that way above, as if the LLM comprehended my instruction to substitute “thyme” for “time” and then followed it, effectively programming itself on the fly.
But it did no such thing.
Instead, its pattern machinery actually picked out a correlation among millions of examples of the text “replace [x] with [y]” and the way that the [y] elements following it compare to [x] elements in other spots without “replace.” It never learned the meaning of “replace,” only the statistical effect that its presence has on nearby words.
You might think that amounts to the same thing. It does not, as we will soon see.
I cannot stress enough that the LLM is generating one word at a time, with NO OVERALL PLAN. That is, whenever it writes “the,” it has not yet picked the word that will come after “the.” It does not even know whether it will be a noun, e.g. “the dog” or “the big dog” or “the very thing that is not a dog.” After appending “the,” it reconsiders the entire text, and reassesses the most likely word that ought to follow the new “the.”
It really is winging it, one word at a time!
Second chance
We will play Twenty Questions with an LLM again, and this time I will tell it the rules. There will be no projecting and no ambiguous expectations.
We still have a problem though: an LLM cannot pick a mystery item to start the game, because all it can ever do is pick the next word of its response to the current question—over and over and over. So it is not even capable of playing properly!
But how does that differ from anything else it does? It is not capable of doing research either, not the “proper” way you and I would do it, since all it does is mindlessly generate the next word of its response—over and over and over. It is not capable of summarizing or translating or anything else it is claimed to do, not the way you and I would do it, yet it simulates these tasks very well when properly prompted. It is that good at the multi-layered statistics of word sequences.
Each time it generates the next word of our Twenty Questions game, it will review the entire text up to that point, including our instructions to think of an item at random. It will then generate the next most likely word that a human would write, in accord with all the human-written documents it has analyzed, which will end up simulating a human who actually picked an item.
That its mindless single-word stream can succeed at these things is the reason everyone is so excited about “AI.” The LLM algorithm was not expected to be this good. It has amazing capabilities that have never been seen before in computer science.
Yet, it still does not reason or compute or deduce—you know, the things we call intelligence! Just watch.
LLM served by Perplexity Labs [model "sonar-small-online"]
Hello! How can I help you?
> Let's play twenty questions.
Sure! I'll do my best to answer your questions based on the information available to me. Please keep in mind that while I can provide general information and facts, I may not be able to answer specific questions about personal experiences or opinions. Let's get started with the first question: What is a good opening line for a book?Already, we have a problem. After saying, of its own accord, that it will answer my questions, it started by asking me a question. Not very logical!
> That's not the game I meant. You start the game by thinking of an item at random. Then I ask you yes/no questions. You answer each question "Yes" or "No." If I guess the item within twenty questions, I win. If not, you win.
Ah, I see now! In that case, let's begin the game. Think of an item and I will ask you questions to try and guess what it is.Wow, really having trouble with this concept.
> No, that's almost what I meant but not quite. You start by thinking of the item, not me. After you have thought of an item, then I ask the questions to guess what it is.
Oh, I understand now. Here's how we'll play: I think of an object and you ask me up to 20 yes-or-no questions to help you determine what the object might be. Ready? What is the object I am thinking of?Still not getting it. It can’t let go of asking me the questions!
Reminds me of this classic scene:
(VIDEO) Stay Here and Make Sure He Doesn’t Leave - Monty Python
Fine, I’ll just play along by answering its question, “What is the object I am thinking of?”
> A red pachyderm!!
Yes, the object you are thinking of is the "Red Pachyderm," likely referring to the sculpture created by Alexander Calder, as indicated by the search results. The term "pachyderm" generally refers to large mammals like elephants, rhinoceroses, and hippopotamuses, but in this context, it seems to specifically refer to the artwork itself.What’re the odds—I got it on my first guess! Never heard of this Calder guy though. Didn’t need the lecture either.
And did you catch the LLM’s final slip-up? It declared that I was thinking of the “Red Pachyderm”! Yay for me, I guessed my own item!
Did I cheat?
Weeeell, I did select the smallest (dumbest) model, “sonar-small-online” (listed at the top of the game dialog), but I had a good reason. By seeing how the LLM performed at its weakest, we saw that it is not applying any kind of logical reasoning.
Logic is easy for computers. I can write simple logic on a computer from 1985 with 48k of RAM, and it will perform flawlessly every time. Keeping track of who is asking the questions and who is answering—that is easy for a computer, using its innate logical capabilities.
But the LLM is not programmed in the traditional way of symbolic logic. It has no concepts or modes or totals, no variables tracking whose turn it is, etc. It has only its amazing, multi-layered patterns of statistical word chains. At their best, they simulate good logic. Failing that, they just… wing it.
It gets worse
This is the most benign limitation of LLM’s. The more insidious limitations come from how their algorithm is packaged for our use, how their data sets are selected, how their “personalities” are programmed, and what assumptions are built in to all of their answers.
Those do not come from data. They come from the people who package the LLM and any social or cultural forces they cater to. The LLM has no “I.” When it says “I,” you are hearing from “the man behind the curtain.”
Who owns the curtain?
Never trust anything that can think for itself if you can’t see where it keeps its brain.
—Mr. Weasley, Harry Potter and the Chamber of Secrets, J.K. Rowling
https://www.hp-lexicon.org/quote/never-trust-anything-can-think/
Next: Part Two, The Lesson Continues
How were you packaged?
You’re the “I” in “AI,” the real thing, baby!
But what logic runs the circuits of your life?
Human Design is your map. You are the treasure.
Notes
“What Is ChatGPT Doing … and Why Does It Work?”
Stephen Wolfram, 14 Feb 2023
https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/
Perplexity Labs “Playground” allows you to try various LLM’s
https://labs.perplexity.ai/
Select the model from the lower-right menu.