Comments by "schnipsikabel" (@schnipsikabel) on "Sabine Hossenfelder" channel.

  1. True unfortunately... so let's try to help make an informed discussion possible
    1
  2. ​ @janek4913 if you read brain research studies on memory, you will find that we do in fact hallucinate our memories, creating half-fiction.
    1
  3. Good! And i thought we had to worry about alignment... let's leave that to our kids.
    1
  4. It's called moving the goal posts, but probably in the opposite direction you're thinking of...
    1
  5. That would only destroy part if the internet, wouldn't it? Not sure that's enough to hinder AGI development
    1
  6. This is not just one paper, see the recent Appollo Research paper for example. And the theory behind goes back to Yudwowski etc...
    1
  7. My mom said consciousness doesn't exist
    1
  8.  @nikosgeorgakas184  as does Roger Penrose :)
    1
  9.  @robertsteele474  of course it doesn't :)
    1
  10. @GerryRR it's called brain chauvinism
    1
  11. Not according to orthogonality thesis
    1
  12. Or to show instrumental convergence and fake alignment to give us serious trouble
    1
  13. Great then, we'll never have to bother about alignment or instrumental convergence... wait, recent studies show alignment faking already??
    1
  14. Lucky she finally came around!!
    1
  15. ​ @andreash7920 most of our "reasoning" is system 1 thinking, not at all conscious, and we fabricate illusionary explanations how we arrived at the conclusion, just like the LLMs are reported to do here.
    1
  16. Of course they are doing that! Doesn't mean it's not gonna happen.
    1
  17. Never heard about Turing machines? Maybe read some of the recent papers on alignment faking.
    1
  18. Not sure if brilliance is rather to be found with millionaires
    1
  19. Expectations are winning the race to the bottom. Be the first to reach ASI is now officiall US policy to dominate future wars.
    1
  20.  @thomasgoodwin2648  sounds all reasonable :) however, i'm not sure if i agree with you on the point that open cource AI will necessarily be safer. As much as i detest the idea of an OpenAI/Musk/Trump oligarchy or a chinese communist party being in control, i also think your scenario of one rogue human using AI in a malevolent attack (or a benevolent mislead accident) gets much more likely with open sourced AI. It seems one way gives us a bigger p(doom), the other a bigger p(1984)... Now it's up to us which dystopia to choose 😉
    1
  21. @thomasgoodwin2648  right ;)
    1
  22. Is it? If we can't agree on quantum gravity, we can't get crushed by a rock?
    1
  23. Not sure these two mean the same thing
    1
  24.  @cube1us  weather prediction is difficult because of chaos effects. Not so many chaotic processes in everyday physics, though...
    1
  25. Great then, we'll never have to bother about alignment or instrumental convergence... wait, recent studies show alignment faking already??
    1
  26. If you keep defining intelligence as that what humans can do but AI can't, you will still try to develop farting AGI while normal AI has taken over the world already
    1
  27. If you keep defining intelligence as that what AI can't do but humans can, you'll still try develop farting AGI while good old basic AI has taken over the world already
    1
  28. Interesting allegory :)
    1
  29. Doesn't make much sense afterwards, if alignment research doesn't catch up fast
    1
  30. Great then, we'll never have to bother about alignment or instrumental convergence... wait, recent studies show alignment faking already??
    1
  31. Or the opposite. Ever heard of anthopocentrism or brain chauvinism?
    1
  32. Then we don't have a problem. But what if not?
    1
  33. I don't think fatalism is going to do us good, even if the odds are against us.
    1
  34.  @sethsmith8638  same for you ;)
    1
  35. Indeed a huge hurdle for people to accept the threat AI poses is their religious convictions about our minds. Important to tackle indeed.
    1
  36. I guess Geoffrey Hinton just doesn't have enough technical knowledge then ;)
    1
  37. Sure it's that simple?
    1
  38.  @ioanmateescu656  what if AI fakes to be aligned with your integrative society?
    1
  39. Ever seen a knife fake alignment?
    1
  40. Read that in the Bible?
    1
  41. Oh no! I hope she can deal with that! Alignment can't be hyped enough, btw...
    1
  42. Exactly! It may even show instrumental convergence and reshape the world even more. LLMs fake alignment already, according to recent studies.
    1
  43. Ok, glad you know your Sci-fi :) Now read up instrumental convergence, or some of the recent papers on alignment faking, to get an idea what the real threat of AI looks like...
    1
  44. ​ @JZsBFF i don't claim to be a scolar, these things are publicly accessible... and they should actually be accessed, if we want to engage in informed discussion. After all, it's not just some academic topic, but the future of humanity (sorry if that sounds grandiose). In short, instrumental convergence means that any intelligent system with a goal, e.g. an AI trained to maximize a corporate's profits, will automatically converge to subgoals like self-preservation and resource acquisition. Alignment faking, shown by many recent papers, means that these models fake to align to your wishes, but have hidden agendas.
    1
  45. ​ @patientzerobeat although you're correct about sci-fi, you may still underestimate the "real" dangers of AI. What about alignment problems? Lots of serioys scientists warning about it besides companies. Hinton even left Google to freely talk about it.
    1
  46. ​ @patientzerobeat yes, the paperclip maximizer still seems a valid threat to me, if we mean AIs following their utility functions and developing subgoals of instrumental convergence by that. The issue with "pulling the plug" has been addressed lots of times by people much smarter than me... most safety researchers seem to agree it's not that simple, and i find it convincing that an ASI would probably be smart enough to not go rogue before it was convinced the "plug" couldn't easily be pulled anymore, or that we've become too dependent on it to pull any plug. There comes again your point of putting too much trust in it, i guess...
    1
  47. ​@ConfidentlyUninformed It is indeed, since it is distracting from the real dangers of unaligned AI
    1
  48. Great then, we'll never have to bother about alignment or instrumental convergence... wait, recent studies show alignment faking already??
    1
  49. Strongly disagree with your take on it. This study may have been done poorly, but the idea makes complete sense. It seems kind of naive and you'd have to be oblivious to 70 years of science theory to still think scientific paradigms were to change with argumentation. And even if you still believed in it, convincing the unscientific public with arguments would be virtually impossible. Even convincing a scientist outside the very research field already is. So to get an idea about current scientific consensus, these polls are absolutely necessary. Best example is the Penrose quantum theory of consciousness, which now many lay people seem to believe is one of the best theories around, just because of its prominence. However, polling around brain researchers would reveal 99% to think of it as utter BS.
    1
  50. Deciding about targets in battlefield would probably the least we have to worry about AI. Ever heard of alignment problem?
    1