Only Bayes Can Judge Me

  • 11 Posts
  • 546 Comments
Joined 2 years ago
cake
Cake day: July 4th, 2023

help-circle











  • In the current chapter of “I go looking on linkedin for sneer-bait and not jobs, oh hey literally the first thing I see is a pile of shit”

    text in image

    Can ChatGPT pick every 3rd letter in “umbrella”?

    You’d expect “b” and “I”. Easy, right?

    Nope. It will get it wrong.

    Why? Because it doesn’t see letters the way we do.

    We see:

    u-m-b-r-e-l-l-a

    ChatGPT sees something like:

    “umb” | “rell” | “a”

    These are tokens — chunks of text that aren’t always full words or letters.

    So when you ask for “every 3rd letter,” it has to decode the prompt, map it to tokens, simulate how you might count, and then guess what you really meant.

    Spoiler: if it’s not given a chance to decode tokens in individual letters as a separate step, it will stumble.

    Why does this matter?

    Because the better we understand how LLMs think, the better results we’ll get.