Comments by "Anony Mousse" (@anon_y_mousse) on "bycloud" channel.

If tokens were meaningfully split up, then that would make sense, but unfortunately they're basically just grouped by whitespace. That sounds useful, but the LLM's don't just look stuff up in a dictionary, and even if they did, you have synonyms and homonyms that interfere with meaning. If I say "I read XYZ", would you think that I'm currently reading it, or that I have previously read it.
4