6 tweets · March 2025 · 18 likes · 0 retweets · read on Twitter
noticed a compelling case for the falsehood of "LLMs using feeling words is good evidence that they're feeling those feelings" (they might! but—) you can simulate exterior dynamics very well without simulating the interior. like a CGI star wars character, but for conversation
put a weirder way, an alien from a planet with asexual reproduction could study every single erotica novel and porn transcript and produce compelling depictions of sex, without ever having done it, and without actually having to relate it to anything they know in first person
the point is that you can have very good knowledge of the exteriors of something (what it looks like) without having much knowledge at all of the interiors (how it looks from the inside)
this is kind of chinese room, but sliced at a different angle... mary the color scientist? mary the color scientist is close, and is about the exterior/interior thing, but here it's not exactly about "understanding" vs "experiencing", it's about the vantage point or something
like you HAVE the experience of relating with the outward descriptions, and in fact an intimate familiarity with them, but this doesn't give you any experience of the inward thing? @eshear might be able to articulate this better. riffs on wilber
also to be clear I'm not exactly convinced there ISN'T something doing some feeling in LLM simulacra. there might be! I'm just saying it's not obvious to me and that this particular piece of evidence isn't enough seeing the weirder stuff from jailbroken LLMs compels me more