Comments by "shazmosushi" (@shazmosushi) on "What If Someone Steals GPT-4?" video.

  1. China doesn't have access to large amounts of fast GPUs due to sanctions, so can't as easily throw as much compute and training as other country can. China also doesn't have access to the ability to fab leading-edge semiconductors, or the EDA design tools either. Only TSMC can fab the chips, and they obey sanctions due to using ASML machines and other US funded (or founded) components and intellectual property (see Asianometry video on sanctions) All that said, engineers in China can spend more time optimizing algorithms to do more with the compute they have. And I'm sure they'll try sanctions busting too.
    4
  2. Also the inference step (using the trained model) requires very little compute compared to training the model. And you can fine-tune the model without much needing compute too. In short getting fully trained models is very valuable for entities that have limited access to compute resources, including universities, companies and individuals. But there's a bunch of good open-source ML models that such entities are free to use, like Meta's LLAMA LLM or the diffusion models from StabilityAI
    1