General statistics
List of Youtube channels
Youtube commenter search
Distinguished comments
About
Mikko Rantalainen
Lex Clips
comments
Comments by "Mikko Rantalainen" (@MikkoRantalainen) on "The code for AGI will be simple | John Carmack and Lex Fridman" video.
That's a great idea. Make a program that generates a copy of itself with some small amount of mutations and run multiple copies (like millions or billions) in an environment where a successful program gets more copies. The evolution will then automatically take care of the rest. The only big question is how much computing power would you require in total? Training biggest LLM systems like GPT-4 already require computing resources where the computation part alone costs 100 million USD.
1
The most interesting question is when to start throwing more computing resources into it vs when to keep improving the algorithm. It seems that 175 billion weight system like ChatGPT is already big enough to be too hard to fully train – that is, if we could spend way more computing power and keep the same network and architecture, it would perform better. However, even trying to try that once might cost half a billion USD. Who is wealthy enough to try?
1