Comments by "Gilad Barlev" (@GSBarlev) on "Another glorious battle for AI dominance… GPT-4o vs Google I/O" video.

  1. Transformer models are, by definition, dumber than deep learning models—what's allowed them to take over the world hypetrain is that they're lobotomized of their recurrent network layers and just focus on "attention," which is way cheaper and easier to scale. So yeah, no 💩 they're hitting a plateau. This is also why Altman is saying he wants trillions of dollars in GPUs and electricity—because even with efficiency gains, today's clouds won't bring about The Singularity.
    2