Comments by "Mikko Rantalainen" (@MikkoRantalainen) on "Anastasi In Tech" channel.

10:10 I think we shouldn't give too much value to claimed performance of a system that is not available for real. If Google thinks Gemini Ultra will be available sometime in next year, it should be compared to the then current GPT-4 variant.
72
There's no need to be afraid to go into highly detailed stuff. I'm pretty sure your viewer-base is not typical TikTok viewers but mostly engineers and scientists.
18
I hope the industry moves to transistors per square millimeter sooner than later. That's a measurement that doesn't allow the marketing department to get too creative. It also directly tells the usable transistor count instead of some measurement that cannot be interpolated over big areas.
17
Interesting idea. If you can figure out how to train such system, it could be used as an optimization (improve latency or energy efficiency). If I remember correctly, it has been shown mathematically that using just single non-linear function for every neuron is enough to have same computational abilitities (AI counterpart of Turing machine). However, the proof is about what's possible, not about what's easy/fast to compute.
6
Great video! I think in short term OpenAI will have success but in long term the near-open source style by Meta is going to win.
6
@rrmackay I think we're already at the point where no single human can understand the whole CPU package when we already have 100+ billion transistors. This is because such transistor counts can be understood only at a level of "sea of transistors" even though the CPU design is "done by humans" nowadays. In reality, humans only create the overall architecture and then rest is genererated by computer program. If I understood this video correctly, this was about replacing that fully(?) deterministic computer program with a version that run AI algorithms instead. If the AI only does the optimization it will not touch the logic of the chip so it's still equally "easy" to understand to other modern designs. When we get to the area where AI starts to drop steps from the actual algorithms (logic), then human understanding will suffer a lot more. For an example of this, see AlphaTensor and how it improved 4x4 matrix multiplication from 49 operations to 47 operations. Previous state-of-art algorithm called Strassen's algorithm was from year 1969! Now imagine something much more complex than 4x4 matrix multiplication and trying to verify that the variant with reduced steps is still correct.
3
Interesting video! I find it a bit weird that Abu Dhabi is going to host a new 900M USD supercomputer when the climate is far from optimal to run any high performance computing. On the other hand, they still have insane amount of money thanks to oil so maybe that's not a problem at all for them.
3
I think she's just using insanely bright lights so it's hard to not blink.
3
If AI "only" does layout optimization and the actual computational features are designed by humans, AI designed parts are not going to be harder to understand. Consider it like a chess board: AI can compute more accurately than humans why given board position is the best option but it doesn't prevent humans from understanding the position as-is. The design space the AI works in this context is combination of all allowed designs and the task is to find the optimal one. And when the possible space has complexity 10^90000, there can never be exhaustive search. To accomplish full exhaustive search of even 10^40 possible states has theoretical minimum required energy a bit more than the total energy generated by the Sun during the whole lifetime of the Sun. As a result, the current algorithms to approximate the optimal solution in search space of 10^90000 options is going to be a pretty crude approximation, no matter how you do it. It appears that AI can already do much better approximation than any human, especially if you give both the same time to accomplish the task. A well made AI designed layout would be one where no human can improve it even though the layout can be fully understood.
2
Great video! I wasn't aware that state-of-the-art lithography was currently computationally restricted but you explained the reasons for it nicely. PS. I think you should try applying de-esser to your audio processing chain. It sounds to my ears that you have too strong sibilants in the recorded audio. This typically results from combination of your speaking style, room echo and mic positioning.
2
@colinmaharaj50 You're right in general but the CPU leakage current gets higher and higher relative to full CPU power usage when the transistors get smaller. As a result, we'll probably see pretty limiting power scaling limits in the future when transistors get smaller and smaller because leakage currents start to dominate actual power needed for the computation.
2
1548 If they didn't include power needed for the light source (laser) then this comparision table tells nothing about the technology. In similar way, you could declare that you don't consider GPU input power for your AI network computations and suddenly the efficiencly looks really good! This is no different from fusion energy experiences that "produce energy" when you ignore total power used.
1
It currently seems that training the network is much harder problem than the inference and if this new chip can do that with less energy, great. However, it seems that the chip area required is pretty big so I would guess this doesn't yet make sense economically.
1
@thomasmaiden3356 Yes, for currently existing systems that's unfortunately the case. For an AGI, that's not good enough. And if you are designing and planning to build new chips for AI systems, you should be shooting for AGI level performance already.
1
@Johnithinuioian I'd define AGI as an AI system that can learn on the field and has no difference between learning and inference parts. We don't have any such systems currently. And we don't yet have a clear idea how to create one because our existing AI technologies do not even try to implement that part. But we know for sure that there are no learning and inference phases for human brain but both systems are concurrently running so if we want something equally adaptive, we need an artificial system emulating that part, too.
1
It would be interesting to know how they implemented 4-bit multiplication. One could do it as a pure table lookup (there are only 256 possible results) but it's hard to tell if it's faster to do the full multiplication calculation in hardware instead. With the huge amount of transistors the new chips have, it might be faster to always compute everything on the fly.
1
@johnshite4656 I meant the R&D that Meta does, not Facebook.
1
Great point on lowering packaging costs thanks to built-in memory!
1
May I suggest slightly higher gamma ramp for the video (the middle grey colors appear too dark in this video) and a bit more audio compression (audio filter, not data compression method) for your speech (your speech sounds a bit too quiet)?
1
I'd love the marketing team to actually demonstrate 4D, 5D and 5.5D chips. I would argue that there's very high probability that those are actually just 3D chips just like every physical chip ever made.
1
@monad_tcp Running an AI neural networks requires basically computing huge matrix operations with simple non-linear operation between matrix operations. Both operations are embarrassingly parallel workload so the only question is how much electricity that AI is going to require per system. Right now the hardware required to run huge LLMs is very expensive (a single LLM may require 4–8 Nvidia H100 cards to run) and the system requires all the hardware for every answer. Granted, each question+answer takes maybe half a second to compute but you need insanely expensive hardware for that half a second. As a result, currently available AI systems such as ChatGPT are implemented by having lots of beefy systems that can answer questions with queue in the front of the system to improve total throughput per dollar. There's nothing in the system that prevents running it fully parallel, except that getting even more hardware is too expensive to make financial sense. (Tip: try searching for "nvidia H100 price" and you'll understand why.)
1
I think investments into AI will continue in 2024 but hopefully investors use some judgement where to put the money. However, it's obvious that whatever company makes superior AI system is going to make a LOT of money. If you owned a publicly accessible AI that can produce better results than GPT-4, Microsoft and Google would be throwing money on you to offer buying your company. However, the AI must be available to public so that public and the potential buyers can evaluate it in ther own terms. (Being available to public doesn't need to mean free to use without paying anything. It's totally okay to ask for a reasonable payment, but you cannot make it overly expensive. It would be pretty straightforward to train a 500 billion weight network and run every customer with a server with multiple H100 cards but it would never be profitable. The magic is creating cheap enough AI, even if full blown AGI weren't invented during the next 5 years.)
1
@-Tony-Knowles I'm not sure if you're trolling but just press "Sign in" from top right corner, login with Google account and return to this page. The subscribe button is below the title.
1
Scaling to 100T models is going to be hard as long as accelerator cards have 10–100 GB RAM and you really wanted cards with 1–50 TB RAM.
1
It should be pretty simple to simulate suitable default sensory input. For example, simulate the feeling of sitting in a chair ina room with light grey walls and smooth lighting – a generic experience would be enough to satisfy the mammalian enough to avoid going insane. In addition, if you do the scanning process willingly, you would be already thinking about it, which makes it easier to accept the results. The interesting part is that one version of your consciousness will go directly from the scanning moment to experiencing the fully digital simulation. The simulated sensory input for the simulated brain should probably start from generic scanning experience and continue from there. The interesting part is to launch multiple copies of yourself in the simulation and give each one different objective in digital life. For example, one copy of you could work forever to try to generate sustainable fusion power, another copy of you could focus on becoming a poet. Neither copy would never require a biological body to be successful in the objective they're pursuing!
1
Great video! I would recommend monitoring the voice level more in the future videos. Some parts of the video had audio clipping (the microphone sensitivity was too high) and some other parts were too low. I understand that some of the audio wasn't captured by you but I think you should make the final audio level a bit more constant so it would be easier for ears. As for the content, great work!
1
I think AGI is going to happen within 10 years and the society at large is doing nothing to prepare for it. I think switching to UBI (universal basic income) is the only way forward and we should be already switching to it, because starting the process only after AGI is already taking over multiple areas of our life is probably going to be really painful.
1
I'm predicting that CFET chips turn out to be very hard to cool properly.
1
@peppernickelly AI system can also be cloned whenever it shows promise. Once we have even close to human level intelligence in AI, we can try teaching multiple copies of the AI for a year and then take one or two best behaving systems as the basis for next year. Considering how much promise a superhuman level AI would have, it should be easy business decision to allocate e.g. 50 highly trained pedagogy wizards and a couple of psychologists per AI to maximize the probablitity of great learning results in a year. Even with 10 paraller projects like that you would need "only" 500–1000 humans to do the work. Facebook/Meta is already paying salary for about 80000 software developers so a project with only 1000 people would be small risk for them. And note that the baseline AI doesn't need to be at the level of an average human. If it can execute at the level of a somewhat mentally disabled person, it's already good enough for a project like this. Imagine how much a human with mental disabilities could be helped if you could throw 1000 highly trained professionals to help that single human to achieve their best. For a single human, that's too expensive in practice. For a possible AGI system, that's just an investment decision!
1
The only really important spec for any new memory technology is latency. You can always improve bandwidth by having more units in parallel but you cannot fix bad latency by having more components. I truly wish we had something that's equally low latency to SRAM but would work efficiently with smallest process technologies. And the ultimate goal would be to get rid of SDRAM for good. I'd love to have the whole system RAM made out of SRAM (or any other memory technology with < 1ns latency).
1
@kazedcat The problem is obviously the cost. If you use 20 transistors per bit, building something the size of i9-13900K would result in about 128 MB of really fast RAM if the whole chip area were spent for this kind of super-fast RAM. And if you build optimized SRAM instead with 4 transistors per bit, you would still only have roughly 700 MB of RAM on that chip. You don't see many people willing to pay about $1 per 1 MB for really low latency RAM. 40 years ago that would have been insanely cheap but we need insane amounts of RAM these days.
1
Running low latency AI chips on cloud seems a bit weird because latency from your own system to cloud is always going to add extra latency. I guess you should locate your servers into the same cloud provider as the Croq service provider.
1
I think you meant destructive interference instead of destructive inference. And the same for constructive interference.
1
Great video about the new AI. The idea seems solid and it would have been interesting to test it myself.
1
9:00 "AI can help us in the process of chip design but we'll always need human engineers" I think "always" is too bold a prediction. When we invent AGI and it has acquired enough skills, it can do better work than any human engineer. In short term, such AGI will be more expensive to use than human work but in long run AGI will take over all design jobs. I'd currently estimate that this happens between years 2030 and 2050. And the society at large is failing to understand that we should be making changes to social structures already because the change will be so huge that it will be hard to implement.
1
@lil_ToT-XFZ1 I think majority of the people are conservative. They think that if something has been acceptable for previous 50 years, it must be acceptable for next 50 years, too, so there's no hurry. However, the actual timeframe for the need of major change in society thanks to AI getting into general use will be closer to 5 years instead of 50 and people are not ready for this.
1
11:38 It seems that Akida invested so heavily in spiking based model that even their stock graph follows spike with the analog falling signal.
1
It might have been worth mentioning that the computed mask must interact with the light beam and cause interference pattern that "happens" to result in correct details on the chip. And computing the required mask pattern that results in correct interference pattern is the hard part. (And the reason for doing it this way is that wave length is already too long to do this with regular masks.)
1