Comments by "Chrysippus" (@4.0.4) on "1 Million Tiny Experts in an AI? Fine-Grained MoE Explained" video.

  1. Current 8B beats GPT 3.5 on most metrics, we've come a long way.
    7
  2.  @redthunder6183  remember not to run 8b at q4 (default in ollama for example, but BAD, use q8)
    1
  3.  @redthunder6183  true but make sure you're using 8-bit quant, not 4-bit - it matters for those small LLMs
    1