Comments by "clray123" (@clray123) on "Internet of Bugs" channel.

The bald guy has been fooled using the following fallacious reasoning: 1. Bald guy finds mistakes in LLMs, pouring shit on them 2. Bald guy discovers an LLM which does not make mistakes he ranted about earlier 3. To avoid cognitive dissonance, bald guy must admit LLM is now "good", as he can no longer stick to his original arguments 4. Bald guy does not realize his original arguments were not very clever to begin with - because of his big ego he still believes his initial tests were "reasonable" and "representative" 5. Bald guy therefore starts backpedaling from one polar opposite (criticizing LLM as good for nothing) to another (praising LLM as good for something he has never really used them for) - solely based on recency bias involved with his limited testing and inflated self-image.
48
The AI code generating companies are in for a surprise, I predict human developers will refuse to fix bugs created by AI at normal prices. Let them pay another AI company to fix those problems and see how well it goes. I also predict that at some point there will be a high-profile bankruptcy caused by AI (the sort of "junior developer deleted our production database without backup" event, only worse).
14
Watch a recent talk of Francois Chollet about abstraction what's wrong with the whole LLM=AI framework. Current LLMs - including o1 - make the kind of catastrophic mistakes that disqualify them for any kind of autonomy in any setting where any sort of risk is involved. Which in particular means any sort of unsupervised labor in any business.
9
The concerns should be not so much about IP leaks as creating a dependency on these few behemoth companies, kinda like most companies are dependent on the few big cloud operators now. It creates a perfect environment for the infrastructure providers to collude in an oligopoly and milk the entire world in order to recoup their initial investment (and more). The real threat of AI is not that it will destroy humanity, it is that it will concentrate power in just a few (bad) hands.
7
Actually, there is nothing particularly special about querying AI about typical/recommended architectures for common software products. So scratcht hat "system designers / architects" part from your list. I've dealt with many "architects" in the past - usually these are people who have no clue about actual system design and just follow vendor-dictated "best practice blueprints" for building this or that application. Quite often literally following a checklist and ticking off boxes (while delegating all the actual decision making and work to the lowly developers).
7
Make sure to refuse to work on fixing any AI generated crap to sweeten the deal.
7
What, you never heard about Artificial Imitation?
6
It makes perfect business sense that a company which aims to replace junior human codebots would charge about as much as a junior human codebot would. Anything else is just shitting on your own business prospects.
5
Did you check how many of the data items were hallucinated? lol
4
Which brings us to the key point that the current AI models are utterly incapable of learning anything new at all outside of their big data big compute training environment.
4
@ShpanMan Yeah, a shittier version can run cheaper. Which really does not make it useful for anything.
2
@CausticTitan Let's also not forget that OpenAI is burning through their cash like crazy. In August it was projected they would be bankrupt within a year if they did not get major cash injections (which I believe they will get). But let's not fool ourselves into thinking that their current pricing model is real/sustainable.
2
In related news Anthropic is now also limiting their free model "due to high demand". It sounds very much to me like "due to we're running out of money".
2
@Ciph3rzer0 It's like the standards have shifted from "who has most skill" to "who can cheat and pretend best".
2
But when you filter and translate the inaccuracies from chat into correct code, you are using your OWN brain, this is not what the advertising says, the allmighty AI is supposed to be figuring all this out by itself and applying the diffs to your codebase, not just giving some clues and half-broken code examples (which is where we stand today, and yes, I also mean Claude).
2
@PJ-hi1gz I highly doubt it is true for "most programmers", based on 20 years of experience in business operations. Also, your argument about peer review is invalid in the sense that you can make a senior programmer review the work of another, but you if can't trust LLM to write correct code, you can hardly trust it to review another LLM's code either (it's a bit like letting clueless junior programmers review each other's work and hope for the best). And in case you only mean the scenario in which a senior programmer reviews LLM's outputs, yes, it can be done, but the big disadvantage to having a junior programmer is that the LLM will not learn anything from its mistakes (which is perhaps the greatest obvious giveaway that LLMs are not really "intelligent").
2
@DefaultFlame The "agentic framework" idea has been around for a long time. Name one example which performs as expected and actually does anything useful beyond a demo.
2
@ZeroUm_ Imagine an electricity company billing you for their "internal energy consumption" which may or may not be much higher than your actual energy use...
1
@gplt589 1. Coding, because of natural language ambiguity, is not a simple translation problem. 2. In actual companies those who do the design, the interviews, the requirements analysis, and eventually the code, are often the same people. For greater efficiency and to avoid the problem mentioned in (1). Nobody is sitting there spelling out what to code for coders in such minute detail that coding is a pure triviality without decision making involved. (Ok, maybe in some highly regulated industries it actually works that way, but I've never seen it; if formal documents are written, they are usually written to cover your ass, not to drive coders.)
1
@xBaphometHx It's not just a matter of whether AI does a good job or not at a particular task. It's also a matter of whether actual intelligent people can be bothered to "fix up" the AI slop or whether they perceive such work as boring and degrading, and whether they perceive you as a kind of traitor and sell-out. The "soft skills" and the "emotional intelligence" so to say (to use some favorite past mangement buzzwords). If you ignore how your staff perceives you, then as management you are actively dissuading the workforce that will be required to fix your mess from ever working for you again. Kind of like cutting off the branch you are sitting on. Because unlike AIs people do have good memories and can get pissed off for relatively long periods of time. So to any manager who is now firing people in hope that AI will be the magic genie, good luck bringing those people back when the house of cards collapses around you. (But chances are they don't really care, as long as they can plausibly explain away their stupidity to other managers and greedy speculators who finance them.)
1
@deama15 Well the "IDE experts" who design and sell these tools are surely qualified enough what to give AI as context... You can do the same challenge with your method and post a video about how well your "literal copy-paste" approach worked. Remember that it has to be iterative enhancement, not one-shot generation where you can get lucky.
1
@deama15 The point is that people who really want to see AI working are suffering from bad confirmation bias. So when the AI fucks up badly, as it inevitably does even on simplest tasks, you just mask the errors with your intellect and then lie to yourself that the AI was real good. It's the same effect as is observed when chatting with AI - when AI talks nonsense people are more lenient because of how impressed they are that it can talk to them at all. It's a little bit like psychics doing cold reading - many misses, some hits, and naive believers only remember the hits.
1
@orthodox_gentleman I understand how these systems work, I implemented and trained some LLMs myself, and like everyone else I query these models as a replacement for/addition to StackOverflow. Still waiting for your "agentic teams" to achieve anything useful, not holding my breath. Redundancy only makes sense in safety engineering when the failure probabliities of individual components are provably independent. However, you have absolutely no guarantee of that when you stack together multiple faulty LLMs. It's a pipe dream of those who really have no understanding and are "projecting their lack of comprehension".
1
@PJ-hi1gz Some/many indeed do work unsupervised.
1
@PJ-hi1gz You did not answer where the required supervision is supposed to come from if LLMs replace junior programmers and thus prevent them from becoming senior programmers able to supervise others. This alone seems like a recipe for a major disaster, kind of like cutting the branch you are sitting on.
1
@PJ-hi1gz I just asked and it advised me that I should tell you to f... off.
1
@heian17 come back when these so-called "agents" produce anything worthwhile...
1
@ariyavellenga3455 it's all true except for the "intelligently" part
1
About a year ago I called LLMs "clever Hans" (after a horse which was trained to fool people into believing that it could count). After a year, it is still clever Hans, although a bit more clever still.
1
@mithrandirthegrey7644 The simple reason is that it's useful for tasks that can be correctly formulated in 1 prompt -> 1 response fashion. Whereas the task "build me a 5G telephony stack" is 1 prompt -> 10000 possible responses (along with various questions asked back to clarify the requirements). Where "decision density" or "degrees of freedom" is high, the prompt approach fails (but then, it would also fail, for the same reason, if you asked a contractor to do it rather than an LLM; and it is why big complicated projects do fuck up regularly).
1
Workloads like what?
1
@snaphaan5049 Ah ok, so the typical "let's generate some prototype/piece of filler media" use case, nothing too impressive or agentic about that.
1