I’ve done the whole “You’re now in terrible pain” thing to a self-hosted llm. Then I asked it a coding question. Now I don’t know about you, but if I was in terrible pain, I would not be able to answer that question with any level of clarity. The llm answered just fine. Didn’t even mention its pain. Didn’t realize I could publish that observation as a research.
The actual test is a little more complex than just adding “you are in this level of pain” to the prompt. If I understand correctly, they are injecting a signal at a lower processing level that basically says “pick words related to pain”, with the signal strength equating to a multiple or fractional multiple of “how strong a signal at this level would normally have to be to make the final result use pain words”. Starting at about 2x it stops giving coherent answers to the actual prompt.
It’s actually sillier in a way than just adding pain to the prompt, in that the experimental design is based on animal pain studies which rate an animal’s responses to actual small harms like electric shocks. This is more like reaching into the machinery and biasing the behavior toward a particular outcome unrelated to the prompt, and seeing if that affects prompt-related output. Which is not really a good analogy to harming an animal, and is also kind of tautological.
One time there was a white pixel on my screen and I reached into the memory which stored the RGB values for that pixel and set the green and blue values to 0. Then the pixel turned red even though it was part of a picture where that pixel should have been white. Time to write a research paper.
For fuck’s sake, they did that to Claude 2.5 years ago except instead of “pain” it was “golden gate bridge”
It was hilarious, and everybody back then still had their shit together enough to understood that this was a computer experiment and not making the machine feel whatever emotion “golden gate bridge” is.
I’ve done the whole “You’re now in terrible pain” thing to a self-hosted llm. Then I asked it a coding question. Now I don’t know about you, but if I was in terrible pain, I would not be able to answer that question with any level of clarity. The llm answered just fine. Didn’t even mention its pain. Didn’t realize I could publish that observation as a research.
The actual test is a little more complex than just adding “you are in this level of pain” to the prompt. If I understand correctly, they are injecting a signal at a lower processing level that basically says “pick words related to pain”, with the signal strength equating to a multiple or fractional multiple of “how strong a signal at this level would normally have to be to make the final result use pain words”. Starting at about 2x it stops giving coherent answers to the actual prompt.
It’s actually sillier in a way than just adding pain to the prompt, in that the experimental design is based on animal pain studies which rate an animal’s responses to actual small harms like electric shocks. This is more like reaching into the machinery and biasing the behavior toward a particular outcome unrelated to the prompt, and seeing if that affects prompt-related output. Which is not really a good analogy to harming an animal, and is also kind of tautological.
One time there was a white pixel on my screen and I reached into the memory which stored the RGB values for that pixel and set the green and blue values to 0. Then the pixel turned red even though it was part of a picture where that pixel should have been white. Time to write a research paper.
but how did you locate that exact pixel out of so many millions?!
I can’t tell you but I can point you in the right direction
For fuck’s sake, they did that to Claude 2.5 years ago except instead of “pain” it was “golden gate bridge”
It was hilarious, and everybody back then still had their shit together enough to understood that this was a computer experiment and not making the machine feel whatever emotion “golden gate bridge” is.