Comments by "clray123" (@clray123) on "David Shapiro" channel.

  1. You are not "activating" anything in the neural network of the model. Every prompt-response is a function of the same static, unchanging weights in the model. The only thing which matters is the input (and, if sampling is used, pure chance, that's why you get different answers to the same question when you try multiple times). The "muti-shot" chat session is just a zero-shot session with a longer input (some of which was generated by the model itself). No magic mumbo jumbo is required, you give the retrieval "better" input, you get "better" output.
    3
  2. Another nice example of a "barrier" in the language models is their perverse self-love of repetition. If an LLM session starts repeating itself, no matter how much you beg it to stop repeating, it will only get worse from your prompts. It will even answer your immediate instruction not to repeat a phrase with that very same phrase, followed by an apology. It's clearly an example of hitting a wall and not going around it, but rather striking your head against the wall over and over again - an exact opposite of what we would call "intelligence" and a definite roadblock for any sort of agent behavior. So yes, it sounds like a great breakthrough if you could modify the (primitive) transformer architecture to work around this sort of problem.
    2
  3. There is nothing inventive about the idea; search algorithms have been at the core of AI for decades. The difficulty (still present) is how to apply this idea to the algorithm which generates next token in transformer architecture.
    2
  4.  @dan110024  Right. The hallucinated responses tend to have the same "highly confident" probability distributions as the correct ones. The model is not just ignorant, it is also uncalibrated (ignorant about its own ignorance).
    1
  5. What is worrying about it? Do you think atom bomb researchers discussed their fine project with the general public?
    1
  6.  @viktors3182 Haha, that's great consolation that there was "government oversight". You have seen the result of this oversight. And I assure you, there is "government oversight" when it comes to any technology with military applications, and AI is clearly one of them. The new killer drones may not be discussed in congress hearing, but you can be certain that they are already being built in secret (government) projects as we speak.
    1