Then you're just comparing descriptions, and you're also dependent on the language skills of the experiencers, and you're still stuck with the problem that language is not a complete map of reality. So an LLM may be able to give you the optimal description of a feeling, on par with what a person would give you; but it won't be equivalent to the actual feeling.
>and you're still stuck with the problem that language is not a complete map of reality.
Definitely agreed, but at least when talking about experience we've been stuck in this position for most of human history. How do I know what it's like to eat a hamburger? Perhaps, like the cute visual in Ratatouille there is a deep inner experience but (also like in Ratatouille) when I need to communicate that experience with another person I'm stuck relying on language, and the language is found lacking.
I know it's just a kid's cartoon, but interestingly Ratatouille was making this exact same point -- he really lacks any meaningful way to convey his inner experience. It's locked behind language. The film tackles this by showing an Remy's complex and interesting visual, but his friend only gets the fuzziest, dullest visual when he attempts to communicate it.
Remy's immediate rat & family peers actually _never_ get to understand his experience, and instead later help him out of love an allegiance.
