Comments by "John Smith" (@JohnSmith-op7ls) on "3Blue1Brown" channel.

@xway2 Very different. These systems don’t work at all like biological neural networks, not even at a fundamental level. Even one of the guys who coined the term “neural network” later regretted it because it caused people to make inaccurate parallels to how artificial and biological neural networks work. ANNs are just a ton of functions doing simple math against constants set during training. You could manually write this all yourself it would just take forever to figure out what all the values needed to be, so we automate it at scale. It’s really just a variation of an expert system. Instead of manually coding in a bunch of branching logic, we use a high dimensional vector database of values signifying the relationships between n-grams or whatever the data is. If you wrote this code manually people would just call it an algorithm. But automate the generation of the code and embed the logic as a bunch of vectors instead of logic branches and loops, and suddenly it’s this mysterious “intelligence”.
7
There’s still a difference between saying something is complex (sophisticated) and overly complex. My point is that LLMs and all ANNs in general are poor attempts at approximating how biological neural networks operate, which is why they require insane amounts of data, training, and power consumption compared to biological systems. Maybe that’s out of the scope of this video but I think it’s important to combat the massive amount of hype around LLMs and put them into perspective and get away from all these notions that they’re mysterious, magical, basically living things.
7
@ The fact that BNNs can form long term memories while functional is a very fundamental difference to ANNs. The two share nothing in common in terms of architecture either at small or large scale. They don’t operate the same way, they don’t have the same capabilities, they don’t have similar power consumption, they aren’t made of the same materials. They’re fundamentally different. Even one of the co-developers of the neural network, who coined the term “neural network” later regretted it because he said it drew inapt comparisons to the human brain even though ANNs are an exceeding crude attempt at emulating them. And emulation is as good as it can get, they’re not even trying to simulate them and couldn’t until we first figure out how BNNs actually work
3
Sophisticated makes it sound well designed. Overly complicated is a much better way to describe it.
2
@ Ngrams and tokens are the same thing, it’s just semantics. Both are just an arbitrary number of characters from a larger string.
2
@ They’re fundamentally different. This is painfully obvious to anyone who knows even the basics of how brains are structured and operate. And no, LLMs do not get trained with as much data as human brains. There’s just so many things wrong with that claim lol
1
@ And LLMs aren’t intelligent by any BNN measure. If LLMs are intelligent then so is an if statement. This is the nonsense you get into with nebulous terms like that. But if you’re comparing to BNNs, no, LLMs do not have the ability to create internal world models and use them to make predictions nir can they form memory without having to go through an offline training process by an external system. BNNs can form memory in real-time and can form stronger long term memories while active and without any external process or orchestrator.
1
@ Tokens are ngrams, period
1
@BassByTheBay Yeah that Google engineer was completely out of line, and got suspended from his job by Google for it. Some of this nonsense comes from pure marketing hype by all these companies vying for press and investor cash by making BS claims about how much more capable their models are than they even could be. The rest of it comes from a dumbed-down effect. AI used to be a fringe field where you had to really understand how these algorithms worked, how they worked together. Most devs and researchers were coding and training neural networks from scratch. Now, any dimwit who watches an hour long video can do the same with off the shelf software and hardware they have no understanding of. Even a lot of the “expert” devs only have a surface level understanding of various algos and concepts, with most of their knowledge in how to use various libraries and apps, and how to set up the workflow of collecting, scrubbing, normalizing, and classifying training data. This is where you get all these ideas about “emergent attributes” (which are baked into the algorithms just not in an obvious way) and how it’s all “magic” and how “we don’t understand how AI works”. This whole, “I don’t understand it so nobody does” mentality coupled with the marketing hype gives rise to these ideas about sentience or even sapience. It’s just a matter of time before this dumbing-down plus better capabilities merges with the mind virus of virtue signaling and choosing to be an “activist” to make yourself feel important in a culture of nihilism gives rise to an asinine movement of people claiming AI is slavery and torture because they’re “alive”. You know it’ll happen, the trajectory has been laid.
1
@ No, the world isn’t an actual mathematical formula. Math describes certain operations, it’s not a physical thing.
1