Comments by "John Smith" (@JohnSmith-op7ls) on "Wes Roth" channel.

  1. Not sure why he mentioned “confusables” which are called homoglyphs, as they don’t pose any real threat as an LLM sees them for what they are and not what they look like when rendered, but this is less of a hack and more about OpenAI not doing basic string scrubbing on their training data. They really should have Justus removed non printable Unicode characters from regular training data and done specific training data so the model is aware of them and what they are. Anyone who has had to do text moderation or sanitization knows you need to handle these chars in a thought out way. I steady they got lazy and opened up some potential vulnerabilities.
    3
  2. Another month, another claim that AGI is basically here already. Altman is copying Musk in his hype tactics
    2
  3. Yeah and too all the people who are saying, “just give it a memory” or “they can make the context window bigger” have no idea what they’re talking about. A “memory” still needs to be added to the context window and trying to compress it by summarizing is problematic because you’re losing data and have no idea if it’s important for future prompts. And the larger the context, the worse the results get as you get compounding drift. You also need way more processing power because you need to calculate more and more probability chains. There are some hacks being passed off like they’re breakthroughs for simplifying the processing with larger contexts but they’re a bandage on a bullet hole and they introduce their own drawbacks. They’ll keep hacking together incremental improvements out of LLMs but basically they’ve already seen their growth spurt and it’s diminishing returns from here on out. A totally different architecture is needed to take things to a new level.
    2
  4. o3 isn’t very good, a marginal increase in some areas, mainly just a performance gain and a knee jerk PR response to the Deepscam narrative. If it was just a matter of making a bigger or better tuned model they would have done it already and fired all their dev staff and so would every other company that managed to do so before their competitors did and put them out of business. Stop believing the hype.
    2
  5. Too much clickbaiting
    2
  6. Gemini is racist
    2
  7. They always lie about capabilities. Release it and within minutes people will show countless examples of it getting things wrong. Plus it’ll be super racist like Gemini
    1