Comments by "Roni Levarez" (@ronilevarez901) on "We had Image Gen copying LLM... and now the REVERSE?? [DiffusionLM]" video.

  1. Now we don't. I've also thought about it and I've never understand why people think a different architecture is needed to prevent "answer guessing" on LLMs, since we write the same way LLMs do: left to right. The only difference is thinking . We learn that, instead of guessing how many apples there are in the bag, like little children do, we have to actually count them one by one. Children do it the same way reflection LLMs are doing it: out loud and step by step. Then we learn to do it mentally, and in many cases, we end up skipping the counting and giving an answer from our memory. That's the next step LLMs have to achieve to get closer to general intelligence: a mental space to plan, imagine and visualize, before giving an answer. But of course a new SOTA architecture won't hurt either 😄
    3
  2. They generate some stuff in parallel on GPUs, but predicting the next token is always one after the other. Increasing context size is used to give the models more memory and thus more usefulness for task involving long documents, but not to generate the answer in one shot.
    1
  3. That's what I think. We don't write entirely unrelated words next to each other. That's why LLMs can learn the patterns. There are probabilities for words to be around others given a context. Creating a nice map. Why not using it?
    1
  4. Because most people prefer virtual tokens over actual money, right? Seriously, in my decade of experience no human being ever donates any amount of money, no matter how much they like the project/channel/cause. So if some statistically anomalous person will donate anything they'll do it in any available way.
    1