Comments by "Vitaly L" (@vitalyl1327) on "Wes Roth" channel.

More like a bunch of angry boomers.
39
Nobody expects AGI from OpenAI. AGI will be built on small local models.
8
@WesRoth It's really hard to write proper scientific papers about something like GPT-4, as there's no such a thing as a reproducible experiment. GPT-4 changes all the time, there's no frozen version available. Local LLMs are much better suited for analysis, as any experiment can be perfectly reproduced. Who knows, maybe OpenAI read the paper, got burns and fine-tuned a LoRA layer to fix the answers to the questions listed in the paper.
5
@peehi2 economy was growing since 50s, but workers incomes stagnated. Guess who profited? Remove the exploiters, and there won't be any obstacles to UBI.
5
The genie is out of the bottle and OpenAI does not control (or even influence that much) of the AI development.
4
@donharris8846 because end to end scaling is a wrong way. We already have all the AGI components we need, and most of them are old school symbolic. All we need LLMs and VLMs for is NLP/vision interface and semantic search. The rest is better offloaded to symbolic reasoning tools (even SMT solvers and such).
4
Exactly, AI have a real potential to destroy capitalism. Bring it on!
3
That's why we use strain gauges and current control (if gears are low enough to sense the forces with motors). And of course absolute encoders. Robots do known all moving parts positions and forces in all joints.
3
@831Miranda this is exactly what I am working on.
2
And it is delicious! Luddite tears are so sweet.
2
@YeeLeeHaw even tiny 7b models are capable of reaching AGI-level of reasoning. Just give them the right tools and the right feedback loop. We don't even need larger models.
2
Who would have thought? Stochastic parrots cannot reason. Why would anyone expect an ability to reason from a glorified Markov chain?
2
@hunterkudo9832 LLMs can see known patterns and apply them. They were trained on a lot of logical problems, including ones with made up words, so they know to treat them as abstract variables. It is purely mechanical and any proof assisrant (like HOL) would do it orders of magnitude better.
2
one little secret of our profession - we hate each other
2
And those sub par humans are the angriest.
2
truehighs7845 you're using AI the wrong way. You must ask instead "write a Prolog codr that solves this logic problem, write a proof that it is a correct representation of this problem, run the proof and the program in the given tool sandbox".
2
Maybe some ground loops issues? Faulty diodes?
1
Every human brain invents its own inner language during development.
1
@tablab165 this time it's really out of the bottle, available to everyone - good guys and bad guys alike.
1
@davidjulitz7446 even in their current state, even if we freeze all models and stop training, they're insanely powerful and we did not even start scratching on the surface of their potential.
1
@davidjulitz7446 define an "AGI". We could get a very AGI-like behaviour even on top of the tiny 13B models, without any fine-tuning. It's already possible for such models to be embodied in a mobile robot, to make discoveries, to solve complex problems, to design software and hardware unguided, from scratch. All you need to do is to give them the right tools and implement a proper feedback loop. At this point I'm entirely uninterested in the larger models, and I find the progress of very small models (Phi-2 and similar) very promising. This is where the true AGI will arise, not giant GPT-4 and alike.
1
@davidjulitz7446 not just in theory, quite a few of the things I listed are already a reality, the only thing left is to make the inference a bit faster for it to become practical.
1
@davidjulitz7446 self-driving requires a real-time responce. Much less so in , say, industrial robotics, where pausing here and there is not an issue and can be expected. And LLMs being unreliable and hallucinating is in fact their best feature, pretty much everything I isted above relies on their ability to hallucinate.
1
and it is great! What I see happening motivates me to work even harder on bringin on the ASI sooner.
1
How did you even manage to get LLMs to write any coherent code at all?
1
I hope it will come sooner than that.
1
 @AleksandrStrizhevskiy nah, AI will do all engineering jobs better than humans.
1
@bluex217 wrong. AI have a much better attention to detail.
1
@bluex217 you're using LLMs the wrong way. This is not how LLMs should write code. End to end will never work. Use agents with tools.
1
@YetAnother_GameDev just like how we allow teams of questionable competence to deliver without supervision now. With a great caution and low expectations.
1
@bluex217 most human developers do lack common sense. It's not an issue.
1
@bluex217 I see you have a really high opinion of the fellow humans, a totally unjustified one. Humans do not have any "intrinsic common sense". Look around. The vast majority of humans cannot reason. The vast majority of humans get their most basic facts wrong. The vast majority of humans get perplexed by the most trivial logic problems. And, no, you do not really understand how does AI learn. I recommend you to read the famous "Voyager" paper, to see how a static immutable AI model can learn new skills and new facts about the world through observation, trial and error.
1
@bluex217 and again, you do not understand both how AI works and what the human "ocmmon sense" is.
1
@bluex217 also, if the utter crap that 99.99% of the software developers produce is the result of their "tacit understanding of how life works", they should not be surprised when told where to stick this tacit understanding along with its results.
1
@bluex217 your rambling about "common sense" are based on your lack of understanding of what common sense is and how AI is trained. Tell me. Does a person blind from birth have this "common sense" of yours?
1
Meh. Screw the GPUs. They were never the right architecture for the inference anyway. Open hardware is the future. Good luck regulating what we put on FPGAs, for example.
1
@YeeLeeHaw you don't know what you're talking about, do you? A llama3 8b with Prolog and HOL outperform GPT-4 on logic problems by a huge margin.
1
@YeeLeeHaw being able to solve arbitrary problems, to find novel solutions to previously unseen problems, to behave rationally in a physical body - that all falls pretty close to AGI definition. And small models do just that. In many cases much better than humans. The only caveat - they do it very, very slow.
1
Good. Robots should work, not humans.
1
Phi was based on the same.principles, so the writing was already on the wall for the excessively large models.
1
@llmtime2178 and what's wrong with replacing programmers? Why is it any more wrong than replacing traditional craftsmen with mechanical looms? It is a progress, and it is a good thing!
1
Hilarious how it's always people who are not physicists and have no background at all in physics who are optimistic about the fusion.
1
This is the ultimate goal of any engineering, by definition.
1
Productivity is a direct result of well-being, this is.why 4-day working week is proven so efficient.
1
@ShangaelThunda222 this is not what Gates is talking about. This is your ignorant tinfoil interpretation. I am not in a slightest interested in your delusions, and commenting on what is actually said and what is actually happening. Saying this as a robotics engineer working on an AI embodiment.
1
@sebastianb.1926 is it a bad thing?
1
Gone? Always was. The country choke full of religious nutters is insane by definition.
1
@9999_IQ_Carrot the vast majority of the uneducated, overweight, inbred US population supports them. It is your country as a whole, a handful of still sane and decent people do not change a thing.
1