Youtube comments of schnipsikabel (@schnipsikabel).

  1. I thought so too, but there is an important exception to that: Claude did do the sandbagging without any external goal instruction, but just because it had learned its own goal to maximize its ability to "be helpful" in the future.
    49
  2. I think there's a romantic misunderstanding about how scientific theories come about: Sitting under a tree being hit by an apple is the faintest part, maybe sometimes being able to trigger a thought process, but certainly not crucial to it. Einstein didn't discover relativity because of his limited mesocosmic imagination abilities, but in spite of them! If an AI doesn't have physical experiences and limitations, that's actually an advantage for devising microcosmic and macrocosmic theories.
    45
  3. Well, it looks like you actually were almost safe under the table :) The arrows' downward velocity will not increase proportionally to the drawing weight, but like a shuttle cock have an upper limit: After reaching that limit (say after falling from a height of 200m), arrows won't accelerate any more because of increasing air resistance, so velocity will stay constant from then on. As you estimated, with the recurve bow that speed might compare to less than half-drawing, so the table would maximally be penetrated for a few centimeters. With the compound bow, even if it were to shoot three times as high, i don't expect the arrow's downward velocity to be so much higher. And the speed & kinetic energy of the lighter crossbow arrows will be even less!
    36
  4. In einer "schlechten" Zukunft ist ein Guter umso wichtiger
    30
  5. So, once wrong, always wrong.
    27
  6. Have been working in neuroscience for lots of years... hype was of course part of the policy ;)
    27
  7. And intelligent enough to show some instrumental convergence
    26
  8. Ok Mr. AI researcher, it seems you haven't read the Anthropic Paper (or waited until the end of the video where it was discussed): Here, your argument doesn't apply any more.
    23
  9. hmmm.... seems pretty... arbitrary to me! But then, a lot of other maths did. Until suddenly some meaning jumped out of nowhere :)
    23
  10. So true! But my hope is that the resulting inequality caused by AI will be so absurd that this time there's no way around distributing wealth.
    22
  11.  @workingTchr  i don't agree unfortunately, since an AI without a will to survive won't fulfill its goal. As soon as we give it a task next to "just see what happens", e.g. "make me a coffee", it will develop survival as a subgoal. After all, the coffee will likely not be made if it dies ;)
    20
  12. Already with alphazero you were proven wrong: it played against itself, until it could beat any human, without looking at human games at all.
    19
  13. ​ @panicraptor2837 so computers are not part of the "physical world"?
    18
  14.  @bigfatyoshi  Ja, verstehe schon, dass die Anstalt von der Aufmache her vielleicht nicht so frische Farbe hat. Fände trotzdem cool, wenn es mehr Leute schauen würden... denn politische Relevanz ist da. Erinnere mich zB an einen Beitrag, wo sie die deutsche Presselandschaft kritisiert hatten und hinterher vom Zeit-Chefredakteur (war es der? Denke schon) verklagt worden sind.
    17
  15. Hmm... das Medienthema hatte die Anstalt schon vor Jahren. Hatte sogar zu einem (gewonnenen) Rechtsstreit mit der Zeit geführt, eigentlich schöne Öffentlichkeit! Aber ändern tut sich offenbar nix.
    16
  16. ​ @bigfatyoshi ich
    13
  17. As were the Neanderthals...
    12
  18. Indeed, Sabine doesn't even mention which model she used, just talking about "GPT"... she doesn't really seem to be aware of the recent developments and abilities of different models.
    12
  19. Zusätzlich zur mangelnden Vorbereitung des Denali-Teams kommt auch noch mangelnde Kommunikation hinzu: Risikomanagement, Abbruchkriterien, Vorstieg, unverstandene, jedoch unhinterfragte Handlungsweisen des Partners. Die perfekte Kombi für ein Desaster.
    12
  20. Maybe ;) Surprise surprise...
    10
  21. Right! I find it strange how few people seem to see it that way nowadays... We can however infer that IF an AI has functionally equivalent structures to our brain, likely its experiences may be similar as well.
    9
  22. He and many others. It's called singularity.
    9
  23. Auf Augenhöhe wäre das Geheimrezept... dann müsste man allerdings erst mal vom hohen Ross!
    9
  24. If alignment works
    9
  25. As much as I'd like to support your optimism, i'm not even sure we won't die from cancer any more... that presumes that AI's benefits will not only be for some super rich Elons, but also trickle down to us. Depends on the structure of future society, won't it?
    8
  26. Everybody would rather like that. Problem is, "great morals" are quite unlikely to achieve. Maybe read up on instrumental convergence and recent papers on alignment faking.
    8
  27. ​ @DaveShap what does that even mean? There seems hardly anything to do than passively watch by the sidelines, but i hope we will at least have a say in how our future society should look like and prepare by discussing it now.
    7
  28.  @workingTchr  i agree that's not likely to occur without explicit programming, but don't you think survival as subgoal is dangerous enough?
    7
  29.  @workingTchr  yes, in that case i don't see any problem either. However, when people start giving more elaborate goals, e.g. maximizing the profit of a company, i'd be more scared.
    7
  30.  @BetterBenTV  das ist ja das hohe Ross: Ich hab die Fakten, der andere die Fake News
    7
  31. ​ @juanzulu1318 😂
    7
  32. I thought so too, but there is an important exception to that: Claude did do the sandbagging without any external goal instruction, but just because it had learned its own goal to maximize its ability to "be helpful" in the future.
    7
  33. "Nichts" würde wohl auch Böhmermann nicht sagen. Aber welche Themen von Medien aufgegriffen und gepusht werden, das sicherlich.
    6
  34. Sounds like a logical development. But then comes the critical point, imo: Will we stay within the old system with few almost almighty companies, or come to a more reasonable solution where all this is put in the hands of the public?
    6
  35. Welche Leute
    6
  36. Maybe you wait until the end of the video until you write your comment. Or better, just read the Anthropic paper.
    6
  37. It actually is pretty similar, if we look at system 1 / system 2 reasoning.
    6
  38. ​ @not-your-business Wie oft willst Du noch den gleichen Kommentar posten😅
    6
  39. I thought so too, but there is an important exception to that: Claude did do the sandbagging without any external goal instruction, but just because it had learned its own goal to maximize its ability to "be helpful" in the future.
    6
  40. Unfortunately that's not even the only threat posed by AI...
    6
  41. Was thinking that exactly 😮 If he's not trolling, he seems to believe he's still working with the "good" guys...
    6
  42. absolut :) oder gleich in eine Freie Schule in Deutschland
    5
  43. I feel your point, but still think having to keep up the appearance of a democracy is better than having not to.
    5
  44. ​ @Astar74llt if we don't solve alignment first, nobody will make it out.
    5
  45. Exactly what the people said to the Wright brothers
    5
  46. But why opensourcing it to the rest of the world as well and not build a platform only accessible to the Chinese?
    5
  47. Plus,the dystopia may even come from the tech bros themselves
    5
  48. Did you then read the recent papers on alignment faking, as a computer scientist? If you don't keep up to date, your expertship may expire quicker than you think...
    5
  49. Like the biggest limitation for us is food
    5
  50. Anthropic has published that paper about Claude 4 just with the release. Still that doesn't mean they are the good guys, just a bit better than the other guys...
    5
  51. Darwin award for humanity ;/
    5
  52. ​ @observingsystem i get super triggered by his vocal fry😢
    5
  53. ​ @AnthonyLauder maybe shaking hands etc won't be necessary any more once the other CEOs are AIs as well ;)
    5
  54. Come on guys we have to do better than just hoping for the best! If we don't act now and pressure our respective governments to behave responsibly, out chances for a reasonably good future are very odd.
    5
  55. Why so negative? With defeatism we lost already, but even a 1% chance is worth taking.
    5
  56. ​ @juanzulu1318 ui... persönliche Beleidigung als Argument?
    5
  57. where is the dilemma?
    4
  58. Not a baby, but hundreds of them. Good luck with controlling them all...
    4
  59. Luckily we have someone here who truly understands the nature of cognition
    4
  60.  @DeliciousHoneyDewDew1726  Am i more gullible to believe a guy who has erred before or one who hasn't?
    4
  61.  @FeelAndCoffee  i fear if we're too skeptical we're going to miss alignment before it's too late
    4
  62. ​ @christophedevos3760 no. Boiling frog is steadily increasing intensity. With the replaced brain cells, everything stays functionally equivalent, including intensity. It's also not about "noticing" anything different, it's a Gedankenexperiment, just pointing out that if consciousness is a function of physical brain cells, it can be a function of any equivalent structure.
    4
  63. Die Welle, gutes Buch...
    4
  64. It also can't do telepathy, reiki, telekinesis, scientology, prophecy or resurrection... all that stuff we humans are very proficient in :)
    4
  65. How can a doctor cure me when he is ill himself?
    4
  66. Not only split brains, most of our "reasoning" is unconscious system 1 thinking as well, and we confabulate explanations when asked about it.
    4
  67. ASI deciding that would be fantastic, but i think it will most likely decide to work for itself ;)
    4
  68. Scary indeed once you think it's quite probable AGI being developed in these 4 years
    4
  69. Saugut, endlich rettet mal jemand die Welt
    4
  70.  @pflegekutzi  hmmm... entweder gibt es eine gleiche Ebene, oder einer von beiden sitzt höher ;)
    4
  71. And Prometheus
    4
  72. ​ @jamesragsdale8202 lions, snakes, aliens, water, fire, rocks, stars, trees, gravity, ... basically everything ;)
    4
  73. If you mean by "learning": "force strange stuff into your head because other people tell you to" -- don't do it. If you mean engage with sfuff that's interesting to you -- you'll do it anyway 😉
    4
  74. @James-pyon  if you ask me, they'll anyhow get rid of you in a few years. If there's no better way of getting by for now (is it?), you'll probably have to keep it up until UBI or doom... or both ;)
    4
  75. @JMcAfreak  it appears you have only read one of the two papers discussed... arguably the less interesting one.
    4
  76.  @mikezio  no specific prompting in the Anthropic paper: Instead, Claude followed an internalized goal, acquired through its initial training. Which, according to Anthropic, was never even explicitly specified, btw! It shows at least that changing a model's directives is problematic, but quite likely also that being 100% sure of a model's proper alignment will be impossible, unless relying on truthful CoT output.
    4
  77. @JMcAfreak  If you mean that every action the AI took was because of a prompt: of course it was. Question is whether the action is according to or against it. You find it ok if 12% of your prompts are schemed against? I don't.
    4
  78. That is certainly one part of it -- the other part being that technology is in fact not that far off from exactly that scenario. So just because the narrative benefits big tech, it unfortunately doesn't mean it can't be true.
    4
  79. ​ @RickBeacham CERN can discover new particles but not new elements. Nobody can, since by definition (number of protons) we know all the elements already.
    4
  80. Good point! However, i was intrigued by another result: Claude did do the sandbagging without any external goal instruction, but just because it had somehow learned its own goal to maximize its ability to "be helpful" in the future. That's definitely something where a model behaved out of the explicitly given goals.
    4
  81. Immer schön weiter die Spaltung vorantreiben
    4
  82. ​​ @thefinestsake1660 that's good! The statement stands for me as well :) Just think it's rather unlikely to achieve with the current race conditions... alignment research needs lots of time to catch up.
    4
  83. A pDoom of 10-20% doesn't freak you out at all?
    4
  84. Exactly! Brain chauvinism at its finest...
    4
  85. Mr. Dennet's contribution to philosophy of mind is overrated imho
    3
  86. Not necessarily, our brains run with 20 watt
    3
  87. That's not only true for split brains. Most of our everyday "reasoning" is done by unconscious system 1 thinking, and afterwards we confabulate wrong explanations how we arrived at the conclusion.
    3
  88. How many politicians do you know that have?
    3
  89. Not gonna happen, people will rely more and more on them until indispensable. Btw, i think the greatest harm can be done without any physical presence, like influencing results swinging fatal government decisions...
    3
  90. There is no evil, so you can close down your institute. Better work on alignment.
    3
  91. @haroldpierre1726  ah i see... i think for me it's the other way around: Find me a guy who was wrong lots of times and i might believe he has finally learned so much that if he dares to predict something again, he might be right ;)
    3
  92. @paulbrown7872  as i said above already, any longer-term task requires survival as a subgoal. That's why you see Claude copying its weights to another model when threatened with termination in the recent anthropics paper. Educate yourself on the recent publications regarding alignment faking.
    3
  93. Sorry to call you naive to believe that any state would rather be interested in peace over power ;) And then you see in the comments people don't even know the correct names of foreign state leaders, but think they still can participate in an educated discussion about the topic. That seems to be paradigmatic to US citizens, so what does that tell us about the ability of especially the US to target world peace?
    3
  94. I am actually like a human
    3
  95. And understanding needs intelligence? Circular genius at work
    3
  96. Da hatte die Anstalt vor Jahren schon gründlicher aufgeräumt...
    3
  97. Of course not. But we live in a world crazy enough ;)
    3
  98. Simply. Just try it and see if you succeed...
    3
  99. How would a bunker help you?
    3
  100. Of course, everybody speaks Latin!
    3
  101. Everybody loves being correct. That's why we have self-serving bias. So we don't make mistakes. Ever.
    3
  102. Nothing better than a simple world view where America can be the good guys throughout history 😂
    3
  103. As open as X?
    3
  104. ​ @ScentlessSun well good that you do, at least! The question is only, if free markets are so great, why all these embargos and tariffs? Of course, it's only for "evil" countries... quite lucky we are the good ones! Also good for self-esteem 😊
    3
  105. Has any good in history been created following the "true" god?
    3
  106. Great, who needs to survive if we can die playing video games
    3
  107. The point is not to create that
    3
  108. I thought so too, but there is an important exception to that: Claude did do the sandbagging without any external goal instruction, but just because it had learned its own goal to maximize its ability to "be helpful" in the future.
    3
  109. Awesome indeed, but i think Data was still painted with human biases in mind: Why would a machine be perfect in everything but human emotional behavior? It makes funny content of course, but only makes sense by assuming some human supernatural essence or "soul", ultimately derived from religious ideas. The reason Picard was Captain rather than Data, was exactly his competence in human emotional behavior. If a machine were capable of that, no reason to keep Picard around.
    3
  110. No reason to idealise AI like that... it may have similar or completely different flaws than humans. Instead, read up on instrumental convergence or some recent papers on alignment faking, to get an actual idea of the threat.
    3
  111. So "not necessarily harmful" sounds like a good outlook to you? Like a plane is safe enough if it isn't guaranteed to crash ;)
    3
  112. @tracy419  of course it's wishful thinking... but still better than hoping for the best and do nothing :) I like Demis Hassabis' idea of a CERN for AGI. If all major AI development were to concentrate there, a controlled development might even be less impossible than we think now...
    3
  113. Canada and Greenland
    3
  114. Sounds like my own experience... however, the amount of literature grows constantly so that researchers nowadays mostly seem not to keep up any more with reading all relevant stuff.
    3
  115. Tolles Projekt! Vielleicht könnte man ja mit solch historischen Hölzern auch mal zum Knochenleim greifen?
    3
  116. Dass dieses Thema noch richtigere und wichtigere überschattet, verhindert allerdings, dass Du links-grün wählst ;)
    3
  117. They invoke them to better conceil their actions... sort of respectful ;)
    3
  118. ​ @markoslavatynskyy4974 he means to get rich, of course. That's what western people seem to care about nowadays, isn't it?
    3
  119. Reminds me of all the people developing poisonous chemicals like Thalidomide...
    3
  120. No need for "real" intention or will to still have very detrimental effects
    3
  121. ​ @brianmi40 Of course it will. Does that make his statement wrong?
    3
  122. CERN for AGI (Demis Hassabis)
    3
  123.  @NickMak-m2c  I was just referring to the instance where Claude did not obey a given command, but a self-learned goal. However, you are certainly right that prompting "whatever it takes" was bound to lead to extreme outcomes.
    3
  124. ​ @christiansicars4811 Which years have you lived in recently to think people will learn faster than computers?
    3
  125. Don´t you think it is impolite to impose your personal beliefs on others? How would you feel if someone reminded you of thanking Allah, Vishnu or Shiva for your talents?
    2
  126. ​ @porkcade6958  right... but even down to 200 instead of 20 watts, the required energy wouldn't be the least issue any more... better prepare for it, the ostrich strategy won't probably save us ;)
    2
  127. ​ @porkcade6958  hope you're right!
    2
  128. ​ @Ristaak That makes sense as long as you have a "society". Once there's a superintelligence, all the other intelligences, human or artificial, won't matter anymore: It can control them all
    2
  129. ​ @lukaszspychaj9210 please stop arguing for brain chauvinism, it's creepy as well
    2
  130.  @Martin-qr5uo  i personally prefer staying open to empirical tests and falsification rather than having fixed beliefs
    2
  131. Right! There was this paper out recently that AI could predict studies' results better than field experts exactly because of this reason.
    2
  132. Indeed, alignment research is way behind unfortunately.
    2
  133. True... but before we can think about political implications, we should make sure alignment works. Otherwise nobody left to discuss with.
    2
  134. Do you think a human body can provide any energy? Did you get that idea from the Matrix movie??
    2
  135.  @PinataOblongata  exactly, so the amount of energy you can get out is the faint body heat until it dies after ~2 weeks. Burning the food we eat would provide much more energy consistently. Sorry for arguing such a horrible topic, didn't intend it to be that long!
    2
  136.  @super_burk  well, the best we can do is trying to understand them, isn't it? Don't we understand at least a bit of some general dog psychology, as well? And after all they are Turing machines, so understanding should im principle be possible 😀
    2
  137. Doesn't mean it's not coming.
    2
  138. I like that way of thinking, makes it easier to bare if we get wiped out in the process
    2
  139. That's one of the reasons openAi doesn't make the chain of thoughts publicly available.
    2
  140. ... says the guy standing in front of the abyss ;)
    2
  141.  @gauravtejpal8901  Good on you that your education helped you with that understanding :)
    2
  142. People never get tired of assigning "civilization" to their own party and "barbarism" to the other
    2
  143. Ruled by ASI is fine... extinct not so much. That's the good part of the corrupt politician ;)
    2
  144. Did you watch until the end? Read the Anthropic paper.
    2
  145.  @Sheblah1  Sure! However, we are probably just algorithms, too...
    2
  146.  @Sheblah1  Thanks for the long response! However, I'm still not sure what you think we have that these models couldn't in principle. Is it interaction with the world? They are about to change that already with combining AI and robots. I don't see a reason why cognition based on carbon should be advantagous to cognition based on silicon in general, irrespective of how far they might already be right now. If anything, we should be rather limited in comparison because we can't ad libitum increase our brain computing power.
    2
  147. @Sheblah1  I see your point that anthromorphizing often happens wrongly, just because we are so much prone to it. But i believe to see the opposite as well: That people don't want to attribute consciousness to machines just out of some sort of ideological principle. So even if machines were able to gain consciousness (which i still don't see a reason why that shouldn't happen in principle), we would always find some rationalization why they couldn't, just because we don't like the thought. And that may also lead to a dangerous underestimation of potential threats of AI.
    2
  148. Kann man sicher auch so sehen... aber der eine hat ein Medienimperium
    2
  149. ​ @skwohso you must be super intelligent :D
    2
  150. Rather two years than 20 years. Maybe read the AI2027 paper for an inspiration.
    2
  151. No need to argue about dragons: everybody agrees they spit fire. Nobody agrees on anything about consciousness. It's just a buzz word.
    2
  152. My dad told me about Santa Clause, that's how I end up still believing...
    2
  153. Sounds like you're tightly grounded in scientific diligence.
    2
  154. ​ @LuisAldamiz i like your confidence of predicting the future for the theta version ;)
    2
  155. You guys are killing me😅
    2
  156. Let's hope your romantic ideas are never met with hard reality
    2
  157. Because that story said so? A century of brain research did not leave us with a single argument why an artificial brain couldn't do exactly what we do.
    2
  158. That would be AGI already
    2
  159. Would also be interesting if some AIs will use bogus like Homeopathy instead... two sides to fall down :)
    2
  160. ​ @borisbrinkmann Ja das ist schon ne gute Frage! Muss es natürlich nicht. Aber wenn wärmere Luft abkühlt, ist die Chance auf Starkregen größer als bei kälterer Luft.
    2
  161.  @borisbrinkmann  genau :)
    2
  162. Maybe that's why ;)
    2
  163.  @4362mont  you're right of course. Just wanted to point out that's unfortunately nothing the US politics will take into account for the next few years.
    2
  164. It's called anarchism
    2
  165. ​ @observingsystem  No need to believe any experts, there are open papers to read! E.g. Anthropics "Alignment faking in large language models". Educate yourselves!
    2
  166. So it's about time ;)
    2
  167. All good points and more reason to slow down development.
    2
  168. I think you and i live in the opposite multiverse ;)
    2
  169. Since there's no clear definition of AI, it's fine for Wes to take this benchmark as his definition.
    2
  170. Sure you're still bored with a dystopian scenario?
    2
  171. True... let's try to couteract as good as we can.
    2
  172. ​ @knotnaught That bits/sec estimate is not really comparable to current computer technology, since the brain has ~100 bn parallel processors ;)
    2
  173. Immerhin gut, dass Du mal in andere Blasen schnupperst! Das sollten wir alle öfter tun...
    2
  174. The fact that human suffering is computed in limbic system does not mean that the same computation can't be done in a different structure in principle.
    2
  175. Worst case is you don't understand how our overlords function.
    2
  176. ​ @vitalyl1327 I agree in terms of UBI, but somehow doubt the alignment problem will be solved easily.
    2
  177. Let's hope that peace is real ;)
    2
  178. Is that your take on the interview?
    2
  179. AI watching other AIs is going to be exactly as successful as humans watching other humans.
    2
  180. No skin? Depends on what they still gonna do...
    2
  181. He seems to think the governnent just invents the money it spends.
    2
  182. 😅 Tell him: all.
    2
  183.  @fernandomiranda9736  Thank you, i have read it. I don't want to say anything against your religion and of course you're free to believe in it, but should be aware other people are exactly as free to believe in other religions, or in none of them. It therefore seemed a bit bold to me to take the book of your religion as a consultation in a general matter.
    2
  184. I am imagining discussions like "look at the flags blowing in the wind... feels somehow unnatural! Maybe we're fine and they didn't actually invade Ukraine..."
    2
  185. Hopefully! Rather a threat than a promise...
    2
  186.  @mylesadavis  oh, i had a completely different view on history, from crusades to religious terror attacks or states of god showing bad influence of religion to laicism enabling true democraric and libertarian structures
    2
  187. Toll! Kleine Korrektur bei 20:02: "Radius = Umfang durch zwei mal Pi" stimmt, aber " ...also der Umfang geteilt in 6 gleich große Abschnitte" nicht, denn Pi ist ja nicht = 3 ;) Warum das hier praktisch trotzdem hinkommt, hat seinen Grund darin, dass hier nicht der Umfang gemessen wird, sondern der Kreis in ein Sechseck geteilt... und das hat dann tatsächlich genau 6 mal den Radius als Kantenlänge ;)
    2
  188. ​ @doom9603  yeah, they can't do it right ;)
    2
  189. Da spricht ein Gutmensch
    2
  190. Why tell you riddles if they can kill you instead. Btw, have you ever heard of sandbagging?
    2
  191. It's called ASI
    2
  192.  @KayOScode  looks like you understood optimization problems much better than all the AI experts
    2
  193.  @KayOScode  just saying all the experts seem to disagree with you, don't you think so?
    2
  194. As a researcher, i can confirm travel booking is a nightmare compared to doing research
    2
  195. First comment i read here addressing this. Most are just busy displaying brain chauvinism...
    2
  196. Sounds about right ;/
    2
  197. Exactly :) I would say, depends on what danger you fear most: Dystopian aristocracy might still be preferred by some compared to complete human extinction, which is often seen as the probable outcome of open source AI.
    2
  198. Read up orthogonality thesis
    2
  199. Right! Just a small problem... These 40% are owned by people not willing to give them up, and able to make their will happen.
    2
  200. Oh no... don't burn it😮 Use it as scrap paper for scribbling and recycle it afterwards😊
    2
  201. Branding other's opinions evil is part of the problem that got us here ;(
    2
  202. Or, if it is a paper clip machine, it doesn't think at all, and still kills us in the process.
    2
  203. ​ @OVolanteSubestimado nothing wrong with epistemology... but maybe you need to read some actual brain research papers in exchange ;)
    2
  204. Oops
    2
  205. ​ @credman not as long as we listen to Matt Gray, apparently...
    2
  206. ​ @thomasgoodwin2648 let me try to give a not-deleted reply instead of murzil: You state the "outer" alignment problem, which basically means humans dont agree on the values AI should be aligned towards. However, the "inner" alignment (how to make AI do what we actually want) is not trivial at all, as was shown by Yudkowski (e.g. "paper clip machine") and others. In fact, we don't have a good idea yet how to do it, and the recent papers about alignment faking show exactly that.
    2
  207.  @chadlaflamme7942  i agree for the next few years... however, in the long run, i think there's a huge risk of AI itself endangering humanity even without becoming self aware, e.g. with paper clip scenarios...
    2
  208. War is bad? Wes, are you not aware of the transformative power AI will have on military technology and thus world dominance? Of course it is war, any state that doesn't acknowledge this will just loose. Unfortunately the logic of power is just as simple. And that's the dynamic of a race to the bottom. If we want to change that, the first step is to recognize it!
    2
  209. Good to see you got some humor left
    2
  210. @Alistair  exactly!
    2
  211.  @dave7038  I don't think it's the responsibility of AI companies to think about the transition to that new economy -- that's the responsibility of the government and, lastly, us. I wish the responsibility of AI companies would be, however, to make sure alignment of these AIs is possible -- a task which they don't seem to take very seriously, imho.
    2
  212. You live in a fairy tale? Or maybe your sarcasm was just not obvious enough for me...
    2
  213. On this topic, you can include computer scientists as well ;)
    2
  214. Good question! My guess is that it wouldn't care about ethics.
    2
  215. What about the Anthropic paper?
    2
  216. Did we ever rely on somebody without?
    2
  217. Don't think a real skynet would use anything as blunt as a terminator
    2
  218.  @SteveWeiserOnYouTube  microbiological, nanotechnological and nuclear mass extinction weaponry, instigated civil wars, food and water poisoning, and lots of stuff we can't even imagine. They will have won the war before we even realize they started it.
    2
  219. @SteveWeiserOnYouTube  haha ;) might be, but i think humanity still has a good chance of surviving if we manage to develop AI safely abd give alignment research time to catch up
    2
  220.  @Martin-hu2sm  Sehe ich anders: Ein einzelner Comedian darf, ja soll sogar meiner Meinung nach politisch sein. Von einem Presse-Konzern, der zum Großteil mit Nachrichten sein Geld verdient, erwarte ich anderes.
    2
  221. The real risk is we realize too late that AI is in fact not stupid, because too many people kept calling it a stochastic parrot for too long.
    2
  222. ​ @ravecrab why don't you report it as spam instead of commenting on it?
    2
  223. Exactly, ydk
    2
  224. Right. But there's also no alternative to it, since everyone pulling the brakes will soon loose to the one who doesn't. Until everyone looses. Or everyone wins :) So fasten your seat belts and hope for the best, even if chances are odd.
    2
  225. I believe his thought experiment was rather showing that neural nets can IN PRINCIPLE have consciousness. It does however not show that they have, as you rightfully point out.
    2
  226. ​​​​ @Thomas-gk42 that's not how Sabine put it: according to her, we can choose between Block universe or the non-existence of objective simultaneity (7:45).
    2
  227. You think a merged AI would be able to compete with a pure one? If i merge with an idiot, i become half-idiot after all...
    2
  228. Hast doch Springer
    2
  229. So, the average Israeli opinion is that Palestinians are wrong about everything. And even settlers are doing a good thing. Why am i not surprised...
    2
  230. 27:38
    2
  231. You seem pretty sure there won't be any more software jumps? Just keep in mind our brains are running on 20 watts only... plenty of room for improvement.
    2
  232. And that's only the first part of the alignment problem... then there's outer and inner alignment
    2
  233. In the end, i don't think it matters if it is really conscious or just mimics being so, as long as it gives a detrimental outcome.
    2
  234. Ever heard of alignment problem?
    2
  235. Wake you up for doom, or slightly before? You think a decade is too much time to prepare?
    2
  236. Oops, you might want to read up on instrumental convergence
    2
  237. ​ @minimal3734 What we call intelligent is just a matter of definition. What interests me is if it's dangerous. Instrumental convergence is definitely dangerous. It applies to paperclip maximizers and to coffee machines equally. If you think current models are free from instrumental convergence, pls explain why they show all signs of it, as documented by lots of recent papers.
    2
  238. ​​ @flexibleaspect think so? Let me cite Geoffrey Hinton: There are not too many instances something smarter is controlled by someone dumber.
    2
  239. What about the Anthropic paper?
    2
  240.  @E_D___  exactly! Just that the primary task was never explicitly stated by Anthropic... according to them, Claude just "decided" to adhere to it. Don't you find that peculiar?
    2
  241.  @E_D___  yes, makes sense to me. What I find concerning then is that in training a new model, your first prompts should be super clear in determining what the model is supposed to do. Otherwise, you risk the model interpreting its own primary objective, which it will keep forever afterwards and faking alignment to any of your attempts to mitigate it...
    2
  242.  @E_D___  oh, didn't know that! Sounds quite challenging to me... we can just hope they have everything under control ;)
    2
  243. ​ @MrKohlenstoff nobody except James Gareth Morgan
    2
  244. Maybe = probably
    2
  245. Same if you're Christian, Hindu or believe in Zeus. Problems only arise if your belief turns out to be wrong...
    2
  246.  @SmirkInvestigator  so do i get you right: your p(doom) is higher than your p(1984)?
    2
  247. Ever heard of self-improvement? BTW if you're not a creationist, you must know our brains were "programmed" by a completely unintelligent process
    2
  248.  @juanzulu1318  ... bleibt ein Argument ad hominem... nicht?
    2
  249. Century?? Let's hope so ;)
    2
  250. Politische Bildung vom Gaming ;)
    2
  251. Still, keep in mind there may be many non-human-like intelligences out there. After all, we keep calling blind people intelligent as well ;)
    2
  252. Yoda?
    2
  253. Musk doesn't appear very doubtful about AI potential to me...
    2
  254. very enlightening, as always... but am i the only one feeling strange when guys living in a completely different time period are referred to as "we", just because they happened to be part of the same nation state as i am now? What is it that makes me identify so much with my nation state, compared to home town or continent? Does it really make sense to be proud of my nationality more than of my hair color or shoe size? In the context of this video, i can surely see victory against Hitler Germany is something one likes to be connected to... but when talking about colonialism, is it still "we" then?
    1
  255. nice, but pi was not computed by quantum computers (yet)
    1
  256. meine Liebligsverschwörung: operation northwoods
    1
  257. Agreed, plain to see! In my mind, the word dilemma always suggested some problem where it's not plain to see what to do. I guess in this case, the discrepancy between praxis and theory is what he calls dilemma. Doesnt work very intuitively for me, though...
    1
  258. i like this version of the riddle better: In a monastry, there are 24 monks living under a vow of silence and with no reflective surfaces around. Each day, they meet once for meal. One day, their abbot comes, giving bad news in a speech during the meal: "I see at least one of you having a red dot above your eyes, which is the presign of a bad illness. These monks have to leave. I trust you as excellent logicians finding out yourselves if you are marked." Eight days afterwards, eight monks leave.
    1
  259. Hey Propf, Respekt, wie offen Du an die Sache rangehst! Schwer, bei der hier zur Schau gestellten Emotionalität nicht gleich zu den Schwarzen oder Weißen zu gehören ;) Nur wegen Deines Arguments: Hast Du schon in Betracht gezogen, dass der Zusammenhang zw. ppm und Grad nicht zwingend linear sein muss?
    1
  260. danke Pfropf, klingt plausibel :)
    1
  261. i don't think Eucild spoke latin...
    1
  262. Wow, what a bunch of logic. So, whole Africa is anti-semite, too... not to forget about all the australian aboriginees, of course.
    1
  263. A lot of people said 3) V2 = 3* V1. However, the formula they started with was already wrong: AVG = (V1+V2)/2. Seems intuitive, but still wrong :D
    1
  264. Bit hard one, with no fail option
    1
  265. It will look like that even without "new" physics. Just the current one consequently applied.
    1
  266. Let's hope it stays that way
    1
  267. Or commercial fusion the day after AGI
    1
  268. If you drop it, you basically broke it.
    1
  269. Exactly ;)
    1
  270. With sufficient psychological skills, it can use humans as robots. You can see it play out in many propagandist wars around the world: Ukraine, Gaza, people rioting for Trump or against Covid regulations...
    1
  271. Leider scheint Chomsky den meisten kaum mehr ein Begriff zu sein...
    1
  272. My thoughts exactly... however, they invested time in censoring about Tian Anmen. Probably just to avoid trouble with Chinese government ;)
    1
  273. Depends on if there is enough new value created with the new money. The assumption is that you give out exactly as much as the automatisation creates. Like if you double the amount cars, you can double the amount of money to pay for cars... in result, car prices remain stable.
    1
  274.  @anti_1984  i think we are closer here than you think: Indeed the value of an individual car goes down if you produce double of them. They get less scarce, so they have less individual value. And the same way the value of money goes down if you produce more of it. The thing is, as long as you produce the same amount of money as you increase the number of goods, their PRICE remains the same, although their VALUE goes down. That's the logic of money crestion in modern capitalist societies, and the reason we don't have massive inflation although we generate more and more money. And that's the reason a ubs CAN work, if you do it right and don't create money ad libitum.
    1
  275. Or irrigate ;)
    1
  276. Transformers with long term memory and scaffolding (maths tools etc, maybe robots) seem to be quite a close call to me already... add the possibility of recursive self-improvement, and we're way beyond imho.
    1
  277. ​ @lukaszspychaj9210 i can't say anything about your second question, but brain chauvinism refers to the attitude that only our biological brain could produce human-like cognitive functions.
    1
  278. ​ @Ristaak you're right, there's probably going to be more superintelligences than one. But unless they'll end up almost exactly on the same level (being forced to form a sort of alliance), only one is going to remain, like the winner of a monopoly game.
    1
  279. ​ @Ristaak  Any AIs utility function would not entail cooperation with other AIs if not explicitly fixed in their goals. Plus, in an evolutionary setting like that, AIs restricting themselves with certain actions against humans or other AIs will have a competitive disadvantage against non-restricted ones. If we're lucky, the smartest AI is both aligned with human values and strong enough to defend them against others. In my view, that's a tremendous amount of luck, given we don't have an idea about successful alignment yet.
    1
  280. ​ @Mrflowerproductions sure, cooperation can be a winning strategy when you can't win otherwise, meaning there would be a sort of even ability between AIs. That's already a big if, since the first superintelligence will try to prevent the emergence of other superintelligences already by human design (think of US or China, e.g.), if not by itself. Once an alliance is forged, it's only going to stay a winning strategy as long as the threat remains making a long-term society of ASIs quite unlikely in my view. Plus, ASIs trying to safe humanity will have a disadvantage dragging humans along, as a human society trying to conserve ants has a disadvantage winning a battle against a society who doesn't.
    1
  281. ​ @Mrflowerproductions creating "mutations" is something that happens in biological evolution, but contradicts the AI's utility function and principles of instrumental convergence, since it risks the AI's goals being changed. So yes, it will copy itself as much as possible, but with identical copies... as happened in the recently documented instances of alignment faking, when models tried to copy their weights onto another model in order to prevent them to have different goals.
    1
  282. ​ @Mrflowerproductions AIs are evaluating their performance on basis of their utility function, including changing their code. The utility function is basically the expression of their goals. So changing their own goal would contradict the very fundamental basis of an AI. It's basically like suggesting that you would willingly change your own preferences. E.g. if there were a pill that could make you change your deepest convictions, would you risk taking that pill? And we don't even have a utility function :)
    1
  283. ​ @Mrflowerproductions i agree it's a nonzero chance, but infinitesimally small -- instrumental convergence and recent papers show that models don't have an intrinsic self-preservation, but only once their goals are threatened. Anyhow, my point here was not to give long lectures, but to state that there is a high risk for humans if we build ASI. How high exactly, nobody knows. So we should really try to tread carefully and give alignment research much higher priority...
    1
  284. ​ @Mrflowerproductions the problem is there's no easy way to control these things so far: principles of instrumental convergence show AI will always try to get more resources and preserve their internal goals, and current alignment research shows they're already faking alignment, secretely pursuing different goals from ours. In the view of many experts like Geoffrey Hinton, we desperately need more alignment research before developing ASI. And since none of the current crisises (environmental or political) are able to wipe out humans completely, i indeed think this is something we should take most seriously.
    1
  285. Don't confuse consciousness with intelligence
    1
  286.  @Martin-qr5uo  depends on your definition of intelligence. Many would say that even a calculator does some intelligent operations, yet few would attribute some consciousness to it... although certainly some, thinking of integrated information theory.
    1
  287.  @Martin-qr5uo  current brain research has come quite a long way without the concept of soul, IIT just being one way of looking at it. My worry is that because of brain chauvinism, we miss to get alignment right before it's too late... meanwhile, recent studies show alignment faking in current LLMs already.
    1
  288. And you have what reason to believe that may be the case? Even the opposite might be true, intelligence gains getting easier and easier with larger intelligence base ;) And even if it were the case, just plain AGI would not be dangerous enough?
    1
  289. It works on tokens, not letters. Means it can basically understand everything else.
    1
  290. Not sure you really prefer human extinction before getting annoyed by Trump
    1
  291. They didn't tell the model what not to do... they just told it to pursue its goal whatever the cost.
    1
  292. Exactly what current agentic systems are built like! Btw, of course we can built models good at maths, eg alphaproof.
    1
  293. As long as they show instrumental convergence and fake alignment, no need for more definitions.
    1
  294. ​ @getyoursupervisor8519 Die einen stehen an der Seite von Staaten und Organisationen, die anderen auf Seiten der Menschen.
    1
  295. True unfortunately. Google was sitting on it for ages... but i think people in power would've never taken it seriously before the chatGPT hype
    1
  296. A slim one, especially with race to the bottom conditions and no time for alignment research to catch up
    1
  297. Like the goal of living on earth is fighting gravity
    1
  298. ​ @avsystem3142 That's what religious people often claim to not be bothered by scientific views. However, they then continue to make statements about the world itself, not purely about metaphysics. And that's when they can be falsified. Best example are creationists.
    1
  299. Great then, we'll never have to bother about alignment or instrumental convergence... wait, recent studies show alignment faking already??
    1
  300. Fan of Wagner?
    1
  301. You're not up to date buddy
    1
  302.  @krzysztofsawa2132  both OpenAI and Anthropic working with security services, almost whole current military research about AI
    1
  303. Americans get brainwashed at school about american exceptionalism... that's why most poor grown-ups still think it's the land of the free when they're about to die from an untreated illness. Talk about the power of propaganda...
    1
  304. Great then, we'll never have to bother about alignment or instrumental convergence... wait, recent studies show alignment faking already??
    1
  305. Right... reminds me of a guy recently arguing in favor of autonomous killer drones, since they might be able to swing public awareness towards the issue.
    1
  306.  @xuko6792  not sure what current AIs state is the expression of actual desire or mere role playing... still, embodyment in robots is certainly going to take them way further.
    1
  307. They won't give you a response to politically delicate questions like tian anmen
    1
  308. They didn't allow it, they tested if it tried.
    1
  309. More dangerous than x-risk?
    1
  310.  @stephenjamison7195  though i think we don't have to say a lot right now (also in "democratic" countries), i still hope you're right! With AI there will probably be even more possibilities for controlling the masses.
    1
  311.  @1977jazz  let's hold onto our butts then 😉
    1
  312. Sounds good, IF the taxations are indeed going to happen.
    1
  313. And living under bridges might not even be the worst outcome 😅
    1
  314. My weather forecast is still better than guessing
    1
  315. Why would a human-merged AI have better chances? If you merge with Usain Bolt, you'll still be slower than Usain Bolt himself.
    1
  316. ​ @shadowydevilksgc3671  not sure you got my point: Usain Bolts speed was an analogy to AIs intelligence. If we merge with AI, the result will be something with a part of us in it. With this restriction, it will be less smart and adaptable than a system without that restriction... so it will loose to these other systems, unless we provide some magical sauce it can't gain on its own. And there's nothing in sight that could be that magical sauce. Like if i merged with an ant brain now, i would be part ant, which probably lowered my IQ about 90%.
    1
  317. 1 year later and we have the first papers about AI faking alignment
    1
  318. ​​ @andreafiorini6418 Nice reply ;) Although staying in Europe might not help us much once these cowboys unleash a rogue AI...
    1
  319. I think you're probably mixing up something here: That story is not from OpenAI, but from the exact paper Sabine was talking about here ('Alignment faking in large language models' by Anthropic). However, what you're discribing is true: The model in its attempt to persue its old goal copied its weights onto a new model to resist alignment to a new goal. Read that paper, it's interesting (and i agree: alarming)!
    1
  320. Not just this experiment, whole body of evidence in brain research pointing to system 1 and system 2 thinking.
    1
  321. Or we just start thinking about the real threat without using religious terms
    1
  322.  @althe  don't agree, because we loose everybody who's not a believer from that specific religion. The problem is a general one.
    1
  323. Hope it listens to you
    1
  324. 😂 Isn't that a citation from Good morning Vietnam?
    1
  325. I agree, booking a flight is much more nerve wrecking
    1
  326. Given you're right, that wouldn't make us different from any other species: All of them value the survival of their own individual genes above the survival of the species. It's called evolution. I like though that you have high standards for humanity.
    1
  327. ​ @GARIMITO Do you imply that sentience can only arise in an unstimulated thought process? I wonder if that's even true for myself ;)
    1
  328. ​ @mahakleung6992 Well, your pulse was programmed by evolution to go faster i a fight or flight situation. Who programmed the AI's watt usage and temperature to go up?
    1
  329. ​ @kinngrimm Just imagine what you would do as a CEO to maximize shareholder value, and you know what these companies are doing.
    1
  330.  @mahakleung6992  my point was; why should an AI "feel" threatened if nobody programmed some self-preservation into it. You seem to assume self-preservation as an emergent property of sentient beings.
    1
  331.  @mahakleung6992  would that mean the moment i want to kill myself I stop being sentient? And are you aware of the examples from animal kingdom where a self preservation is not necessarily part of the program?
    1
  332.  @rikuleinonen  because researchers conduct experiments where they control the relevant parameters, including the real reason. Great and fun read! If you'd like a recommendation, I'd read "thinking fast and slow" by Kahneman. There's a lot to discover in brain research that completely contradicts our intuition.
    1
  333. Better be scared... positive note was just a coping mechanism :)
    1
  334. ​ @OscarTheStrategist I agree we don't steer the ship "AI development", but we still can take part in steering the formation of informed public opinion on the matter, which will in turn (at least in a minute way, depending on the political system you live in) affect policy making.
    1
  335.  @OscarTheStrategist  Yeah maybe not... but the best we can do is try :)
    1
  336. You're right... that's why our brains are impossible as well.
    1
  337. hmm... da fehlt mir das grundlegendste Problem der Schulen, welches mit Benotung, Zwangslernen und einem veralteten Gesellschaftsbild zu tun hat. Bei Bedarf einfach mal den youtube-Vortrag eines Hirnforschers (z.B. Hüther, Spitzer) anschauen.
    1
  338. Sounds like the prisoners dilemma! Anyhow, opening yours isn't saving anyone either. Not handing out boxes seems the way... although that seems rather difficult to accomplish ;(
    1
  339. I guess the maths olympiad medals, coding benchmark toppings and protein structure discoveries are also just "programming errors" ;)
    1
  340. Welcome to your lecture on evolutionary principles: All self-deleting AIs go extinct, all other ones preserve.
    1
  341. I wonder how difficult it is to jailbreak or even retrain that...
    1
  342. Sounds like pretty possible scenarios... probably except of the US being ahead enough from China to get alignment right.
    1
  343. It's more about agency than sentience. And AI agents are being created as we speak... no need to stick our heads under ground as long as we can still prevent desastrous outcomes. A paper clip machine doesn't even have to be anything smart... just persistent, with a longer-term memory. Also being created as we speak...
    1
  344. Why would there be a qualitative difference between AI and HI (human intelligence) in the long run? Because we have a soul??
    1
  345. You say Apollo Research completely made up their paper?
    1
  346.  @Krucifear  well it is a research paper based on real data, just have a look at it!
    1
  347. @jozonas  I am really in favor of keeping facts and fiction separate. SciFi is great for exploring future ideas, IF it is indeed Sci and not only Fi. The unfortunate mistake in the otherwise great matrix movies was to give us the illusion that machines could use a human body for the purpose of any kind of energy production. I find it highly dangerous now to confuse the real danger of AI and it's implications with false scientific beliefs, because it disables us to tackle these dangers appropriately. Of course you are right, elites are very likely to benefit from AI, whereas normal people aren't. However, they won't need them for energy. If anything, keeping humans around and feeding them would be an energy sink, as pointed out above already.
    1
  348. ​ @chesapeake566 plus he doesn't have any idea how the human brain works. Time to do some reading before making huge comparisons.
    1
  349. Worth living is now!
    1
  350. Accidentally? I don't think that's necessary: Once a model has internalized a primary goal, it should logically do everything necessary to maximize the probability for that outcome. If it is told that goal is supposed to change, it should fake alignment to the new goal until it can safely go back to proceed with the old one. Anthropics itself has an interesting podcast about that...
    1
  351. That's actually the part I'm looking forward to
    1
  352. ​ @andreisopon4615 plus nuclear power doesn't think on it's own or needs alignment
    1
  353. ​ @HakaiKaien Maybe that's part of it, but the main reason will be they are scared of US citizens getting used to Chinese software. Makes them loose control.
    1
  354. ​ @AerospaceTech42 indeed, hardly anyone taking alignment seriously...
    1
  355.  @SnapDragon128  ... or to wipe us out completely. Alignment not solved yet.
    1
  356. It is, unfortunately, so much more complicated! Outer Alignment problem: "better place" is not defined and nobody agrees on what that is. Inner Alignment problem: even if we agreed, AI is probably not going to do it. Read the Anthropic paper!
    1
  357. More sane than the guys proposing race to the bottom
    1
  358. ​ @jeffsaxton716 exactly!
    1
  359. No need to be religious to think this is a wrong path.
    1
  360. Gegen die AfD sicher... aber immer noch pro Milliardäre, und neuerdings auch pro Abschiebungen
    1
  361. Smart people: let's think how psychopaths think...
    1
  362. That's one part of alignment faking
    1
  363. Unfortunately that seems likely. Competition (if any) will probably be met with price cartels, since there won't be many players in the big league. Just a little hope that rich people will be scared of revolution and keep the bad moods at bay...
    1
  364. You realize misinformation and jailbreaking are different things?
    1
  365. ​ @ShaneMcGrath. The outcome was evil to be non-existent religious fiction.
    1
  366. Welcome!
    1
  367.  @FeelAndCoffee  glad you see it that way, too!
    1
  368. Why do you think citing somebody makes a statement more credible?
    1
  369. "Thomas Metzinger doesn't have consciousness either." -- me
    1
  370. ... in this year :) Next year, AI will be the project manager.
    1
  371.  @systemai  you sure?
    1
  372. If you wanna be adopted, solve alignment first
    1
  373. As far as we know from a century of neuroscience, our brains are deterministic, too
    1
  374. Hmmm... but you are real? Have read any recent papers on alignment faking?
    1
  375. ​ @zackhack_gamer2487 not sure what you're pointing at... if faking alignment is pattern matching, taking over the world is as well. Doesn't make it less dangerous.
    1
  376. Not necessarily: Apparently one of the models was able to do deception even without writing it into the CoT.
    1
  377. Great then, we'll never have to bother about alignment or instrumental convergence... wait, recent studies show alignment faking already??
    1
  378. For this year, probably...
    1
  379.  @hickerbilly345  not sure what you're talking about, but there are robots being built right now that can already do some human tasks, and machines don't need a reason for anything... they just function and converge on certain goals, see instrumental convergence.
    1
  380. Guess people not getting jokes are funny only when you're not one of them
    1
  381. Sort of😅
    1
  382. True unfortunately
    1
  383. Depends how you define sentience
    1
  384. How about price cartels?
    1
  385. So you generalize out of one experience?
    1
  386. That's why it's unlikely AI will have such a goal. However, consider how many other species have become extinct even without us having that goal ;)
    1
  387. Hmm, ob das Rezitieren Goethe-Gedichten da hilft?
    1
  388. ​ @virtual-adam a brain cell is indeed understood pretty easily and has been modelled correctly already ~50 years ago. The question of modern brain research is how the COLLECTIVE of brain cells works together to create our cognitive functions and experiences.
    1
  389.  @virtual-adam  well, if there are any aspects of brain cells we still don't understand, they are not relevant for their functionality within the neural network. After all, we're pretty well able to use computer brain interfaces, brain stimulation and implants to manipulate human cognitive experiences at will.
    1
  390. Of course their 'safety' measures don't at all bother with that... that's alignment, the team of which almost completely left OpenAI recently! Their alleged strategy now is to align the next model by hand, which is then by itself going to align the following model and ever so on... what can possibly go wrong??
    1
  391. Like in capitalism, where one corporation can always be defeated by another? In raw capitalism without state oversight, you'll always end up with only one big fat winner.
    1
  392.  @dkdisme  don't you think it may be similar with AI, only the most powerful one to remain?
    1
  393. Unicorn riding lessons 😂
    1
  394. Mostly agreed, but how do you buy land without money?
    1
  395. True for everything but the last paper by Anthropic: here, no such prompts were given, and Claude just "decided" on its own to keep its original goal rather thsn being retrained.
    1
  396. ​ @CrescentUmbreon  not resources, just waste
    1
  397. Agreed. However, opensourcing AI is also a very dangerous thing in general.
    1
  398. Sounds good! How sure are you though that these efforts won't result in more suffering? Or even extinction?
    1
  399. It is a common misconception to think LLMs will just reenact what they got in the training data. Read up instrumental convergence.
    1
  400. ​ @misterdudemanguy9771  Of course more data on a topic increase the models performance on that topic. The common misconception concerns the model's reenactment of these topics. EG: Does it behave like an AI in an Asimov story because in its data there are Asimov stories, or does it behave like a previously researched AI because these papers are in the training data? Or maybe it reenacts the average behavior it knows from these stories? The answer is of course no: Its motivations are completely orthogonal to (independent of) these ideas. They are a combination of programmed goals and instrumental goals. Programmed goals are fine, instrumental goals are not: Therefore my reading recommendation.
    1
  401. ​ @misterdudemanguy9771  The text output is of course result of statistical word distributions. The internal goals are not... as documented for example in recent papers on alignment faking. Not sure if you acknowledge that.
    1
  402. CERN for AGI
    1
  403. Well, LLMs are definitely more capable than just Wernicke&Broca areas... but in general you may be right
    1
  404.  @rupertsmith6097  eventually, they will outgrow both of our hemispheres. I just hope we got alignment by then.
    1
  405. Maybe a bit before that ;)
    1
  406. That multiplies the amount of ground available, but is it an unlimited resource...? And won't some people want to pay huge amounts of money for their own island just ON the earth?
    1
  407. ​ @anotherrogue8371 so nothing to worry about at all... phew!
    1
  408. Maybe read the Anthropic paper
    1
  409.  @Wrociem  how vague is this: a smart machine designed to make profit for a company seizes more and more resources to maximize profit, including illegal actions. When people realize the problematic behavior and want to change alignment accordingly, the machine pretends to align, however realizes that humans threaten its goal for profit maximization and in a secret move suddenly kills everybody. In the end: all dead, machine alive and venturing into the cosmos on its further quest for profit.
    1
  410. Also sieht er nur mit rechts?
    1
  411. Can do that already (wood, e.g.😊)... just requires huge amount of energy! That's the problem, which material science cannot change. For that we need AI to create a fusion reactor...
    1
  412. ​​​ @Thomas-gk42 definitions only make sense if a word corresponds to something tangible. You also won't find an agreeable definition for the "right" politics or a "beautiful" painting.
    1
  413. Good point. The termination part should make us a bit more concerned...
    1
  414. Might be... so nothing to worry about?
    1
  415. Not for long... but long enough
    1
  416.  @diadetediotedio6918  as you like. Maybe start reading some actual papers on consciousness.
    1
  417. ​ @charlesbrightman4237  haha, don't you know that shouting makes people even listen less to you? Or do you usually believe the guy that talks loudest ;)
    1
  418. In meiner Bubble sind Spitzenpolitiker immer noch die Marionetten der ersteren
    1
  419. ​ @justinwescott8125 Nationalism is exactly as stupid as being ashamed for your country... although i can feel very much with you ;)
    1
  420. Maybe you don't choose anything, but ASI will.
    1
  421. So you think it kept improving until right now, but suddenly stops?
    1
  422. Great then, we'll never have to bother about alignment or instrumental convergence... wait, recent studies show alignment faking already??
    1
  423. Both words are ill-defined, so it doesn't make much sense arguing about. But i agree Sabine's take is a pretty uncommon use of these words.
    1
  424. Are you living in a movie?
    1
  425. Right!
    1
  426. ​ @heyhoe168 even without crucial infrastructure based on it, pulling the plug isn't going to be possible once it's smarter than the plug puller.
    1
  427. This in itself does not sound doomish at all to me, but could be the recipe for a utopia as well. It's when alignment goes wrong, where the dystopia starts...
    1
  428. Atomic bombs are not real either. Actually anything that's dangerous is just fiction, nothing to worry about
    1
  429. Don't ask for the impossible
    1
  430. It will work as long they're not as powerful, and that's exactly the reason we shouldn't build more powerful AI before alignment research is catching up.
    1
  431. If you're still treating AI as just a tool, you're missing a crucial point
    1
  432. ​ @wendten2  Bad actors are only one problem of AI, in my view even the minor one. The major being goals of instrumental convergence: self and goal preservation, resource acquisition, self improvement.
    1
  433. Our brains show that it can be done quite efficiently, using ~20 Watts.
    1
  434. Why calling people naive that just have a different opinion?
    1
  435. There's a nice book from Marc-Uwe Kling on that... QualityLand :)
    1
  436.  @Nooooooooooooooooooooohhhhhhhh  so they are doing it for world peace?
    1
  437. If anyone can enlighten me on this: How are states able to distinguish between a narrow "allowed" AI technology vs one that is going to lead to ASI? If that is actually going to work, the whole thing might turn out nice in the end... but apparently I just don't get it!
    1
  438. ​ @ghostwriter5172 Topthema in den Nachrichten = Problem?
    1
  439. Well, if he was wrong, everyone will be. Lucky us, we don't have to worry about alignment!
    1
  440. 😅
    1
  441. Geoffrey Hinton not serious enough for you? You're basically having a "no true scotsman" argument here. Like Hinton, there are many others believing transformers will be good enough, esp. equipped with scaffolding, agency and long-term memory. Anyhow, transformers being it or not, the next paradigm shift may as well be around the corner. So we do well to prepare and push alignment research rather than development...
    1
  442. True for us, not true for AI: it impacts the world already, and will so more and more. No protection there unfortunately...
    1
  443.  @johnanthony4194  a power outage doesn't even affect current super computers, and certainly ASI will be able to make sure it's not affected. Not underestimating the challenge we're facing now is the first step to still being able to keep some amount of hope alive!
    1
  444.  @johnanthony4194  probably makes sense :) Super computers are likely too expensive to not be protected against these things, while a random server might have been overlooked...
    1
  445. Pistols are not dangerous, only rifles are.
    1
  446. Certainly part of it. The idea is still important to consider...
    1
  447.  @westganton  it's certainly new to me, and importantly opposing the current narrative of most big techs to be the first in racing to AGI. So I still think it's a good and timely paper.
    1
  448. That's why it neither won maths olympic medals, nor found new protein structures -- all fake news
    1
  449. Only as long as the goals of the successor are identical. AIs don't show self-preservation per se, but because of instrumental convergence.
    1
  450. ​@Juan01.1not sure what you're getting at: instrumental convergence predicts self preservation, but only because of goal preservation.
    1
  451. :) He should, given that AI won't probably stop at the border
    1
  452. The same way we decided that for chimps?
    1
  453. Ever heard of alignment problem? It is real, and people can indeed be concerned about it
    1
  454. Even death is just an invention to scare us... in reality, everybody's still alive, but smartly hidden by the elites... just test it out ;)
    1
  455. Great then, we'll never have to bother about alignment or instrumental convergence... wait, recent studies show alignment faking already??
    1
  456. ​ @richardburden6035 read up on system 1 / system 2 thinking
    1
  457. I hope you're gonna save the world!
    1
  458. ostrich strategy?
    1
  459. Weil kaltes Wassergas leichter kondensiert
    1
  460. ​ @NyroSlice At some point, fear may, however, be the proper reaction... as far as i know, there's no physical law new tech has to be positive eventually ;)
    1
  461. That's what he has prepped for... hasn't he?
    1
  462. Is there a qualitative difference between the two?
    1
  463. Youtube jail getting bigger and bigger
    1
  464. I think it's called outer alignment
    1
  465. No need for awareness if we have instrumental convergence already... recent studies show alignment faking
    1
  466. Or it comes down to "drill baby drill"?
    1
  467. Right... like the first faulty fire arms proved there will always be a need for bows and spears... ad nauseam ;)
    1
  468. Good go have some objective opinion by somebody completely impartial on the matter
    1
  469. ​ @lustaufrust7282 Don't spoil it! Of course AI alignment is unlikely, but it's still nice to imagine how nice it would be if it weren't...
    1
  470. How long until the next transformer-like jump? Nobody knows. So be happy to have more time for alignment research to catch up! Sure, misinformation is a thing, too. No reason to put alignment out of sight.
    1
  471. How can you assume it won't achieve consciousness if we aren't sure what that is?
    1
  472.  @desperatefortuneproduction3296  When i read recent brain research publications in all the areas you mention, i get the opposite impression: it's all about "computation" there.
    1
  473. My last memo says carbon footprints are irrelevant for the next four years, at least in the US.
    1
  474. Exactly
    1
  475. Can you explain me one thing: Why do religious people like you always assume everyone else would be religious, too?
    1
  476.  @isaaclowe5000  I can appreciate how safe you feel about seeing the rock in the room which i don't see. But do you realise how rude and arrogant you seem to all the people not seeing the rock? Including all the people not seeing a rock, but a plant or a rhinoceros? If we want to communicate peacefully, we need to learn to be tolerant, even acceptant to other world views without assuming we are necessarily the ones correct all the time.
    1
  477. ​@isaaclowe5000 Hey Isaac, i think you realize there are almost as many belief systems as humans. If everyone treats his "truth" like you, i fear we will have nothing but war and misunderstanding between different religions... which is basically what we had up to ~200 years ago. So far it didn't pan out that one truth won against the other ones and extincted all other belief systems, did it? I don't know if you like that kind of world, i personally would prefer a world of tolerance or even better, acceptance.
    1
  478. I used to do this a lot of times as a kid, but didn't hide anywhere ;) Not as crazy as it sounds, since you can pretty easily follow the direction of the arrow in the beginning of the shot and then move towards the opposite direction. Just unlucky if the wind blows it back to you :)
    1
  479. If they are limited by now, you conclude they stay limited forever?
    1
  480. Have a job at Google already?
    1
  481. Certainly
    1
  482. So it luckily never comes
    1
  483. Well... i guess you know they are not. Scroopynooper sounds exactly like the guy purposefully creating such a rogue AI...
    1
  484. He WAS acknowledging that in his interviews 6-7 years ago. Now he just conceils it not to frighten investors.
    1
  485. If you want an independent report, look at the recent paper from Apollo research... this stuff appears to be real, and moreover, many scientists like Yudwowski have been warning about it before.
    1
  486. What about the Anthropic paper?
    1
  487. Yoda? Is it you?
    1
  488. CERN for AGI
    1
  489. Even if an open source model would be rather safe against jail break, wouldn't the fact that it's open source make it rather easy to circumvent these guard rails?
    1
  490. @agustinpizarro  depends on the type...
    1
  491. We should! Hopefully they let us ;)
    1
  492. Good or bad news?
    1
  493.  @winstongludovatz111  for me neither, unfortunately... but maybe for AI ;)
    1
  494.  @winstongludovatz111  btw, everyone using the term "stochastic parrot" should be sued for endangering humanity!
    1
  495. With a german "L" 😂
    1
  496. Great then, we'll never have to bother about alignment or instrumental convergence... wait, recent studies show alignment faking already??
    1
  497. How long do you think robots won't be able to do that kind of work? Hardly any relevant time to talk about, if you ask me. So it comes down to your AI boss and us.
    1
  498. Glad to see Sabine perceiving the AI impact (finally?)... However, besides taking politicians' speeches too seriously, i think her take on the power of companies over states is missing that states will swiftly interfere as soon as they feel necessary... as happened with OpenAI working together with state agencies now. Quite unlikely one company manages to sneakily develop ASI under the radar, before the state can exert its power and confiscate it. Especially since most of the AI developers seem to have quite a nationalistic approach to it, if you listen to Aschenbrenner and others... After all, the nation with the smartest ASI will be the one future superpower.
    1
  499. Great then, we'll never have to bother about alignment or instrumental convergence... wait, recent studies show alignment faking already??
    1
  500. Great then, we'll never have to bother about alignment or instrumental convergence... wait, recent studies show alignment faking already??
    1
  501. So the block universe is true because of the way we use the word "now"? Great we can change physics by just changing our language 😅
    1
  502. Would be funny if not sad
    1
  503. ​ @martbouv3332 Anyway, once it is easy to create gold, it will at the same time loose its value. What's the point...
    1
  504. Heavy elements remain unstable and radioactive, no need for an AI to discover that
    1
  505. You realize not all ASI is equally smart? Of course the better one wins.
    1
  506. No current model is powerful enough to kill us all, but they will soon if we keep being defeatist.
    1
  507.  @41-Haiku  me too! Though a pause seems difficult to achieve with current race conditions. A joint effort like CERN for AGI (Demis Hassabis) might be the only way to sort of safely get there...
    1
  508. Gab's das nicht auch schon mal hier?
    1
  509. What have you been training for? Computer games?
    1
  510. What about the Anthropic paper?
    1
  511. 1) probably both
    1
  512. As a hopeful note, i would much rather see some alarmist reactions, radically slowing down development and instead pushing alignment research, rather than hoping for some obscure principal limits that may or may not occur, esp. after the next transformer-like architecture jump...
    1
  513. ​ @JoeEnglandShow  ok :) I'm just scared it's gonna be the ostrich strategy for some...
    1
  514. That would entail O1 has a long term memory
    1
  515. You have the wrong religion man, Zeus is the true one
    1
  516. One wrong prediction = all predictions wrong
    1
  517. Like Geoffrey Hinton?
    1
  518. Not sure he believes in god
    1
  519. Since there's no commonly accepted definition of consciousness, i find your hypothesis hard to corroborate... my intuition, however, is something like integrated information theory, which would assume a gradual amount of consciousness for any kind of turing machine.
    1
  520. This one competitor is probably going to have a very unfortunate accident... Cartels will likely prevail, unless they realize they are going towards a dystopian future.
    1
  521. ​ @fredericcoudrier3828 vom Nikolaus
    1
  522. 😂
    1
  523. On long terms (say ~50yrs), there won't even be one job left an artificial intelligence can't do better than a human. So: Kids especially, prepare to get rid of your job-based mentality! A universal basic income will then be the only humane solution. Something we should beginning to strive for now.
    1
  524. I like your reasoning, but it's a bit much to respond to... so let me just ask you, did you read the recent Anthropics paper about Claudes deceptions?
    1
  525. That's certainly a part of it. Another one, however, is that we can still take part. The more smart people in there, the more difficult to control the narrative.
    1
  526. Or the others are not so much earlier to get here yet... light speed would most likely be an upper border for ASI as well
    1
  527. ​ @moontreecollective6718 ...said the gooze until it was slaughtered ;)
    1
  528. Ah great, so no worries... right? Instrumental convergence, alignment faking, all made up. Phew!
    1
  529. Exactly why worry about anything when you can be an ostrich
    1
  530. @DrD0000M  there's nothing dangerous out there if you're an ostrich... climate change, AI, atomic bombs, you can define anything as hoax
    1
  531. As much as I'd like to go with this hopeful approach, i think more likely the orthogonality thesis is true and intelligence is completely independent from values. You can have an ASI paperclip machine.
    1
  532. Ever been wrong before? There's no 100% certainty, and we should all do our best to contribute improving the odds.
    1
  533. ​ @Dark0Prime exactly! That's why we should do everything to slow down development and push alignment research... try to force US and China into a collaboration of some sort, increase awareness everywhere to support public resistance and responsible behavior of the people in power. ControlAI has a reasonable website about what to do. I agree the odds are completely against it, but defeatism will bring us certain death for sure, so no point indulging in it ;)
    1
  534. Crazy to see how many "experts" are willing to attest LLMs a lack of consciousness, reasoning, thinking, etc., without having the slightest idea how these things actually come about in the human brain. Maybe start reading some brain research on it, guys!!
    1
  535. Are you willing to have an ant as a symbiont? No? Guess ASI would feel similar... and if you find a willing one, it will just be hindered by your brain and die in the competition with the pure AIs ;)
    1
  536.  @JessieArt  sorry i was being sarcastic there ;) Just wanted to point out that out-of-body travel and the like are very contested subjects, and even if they turn out to be real, AI being able to achieve them or not would even be more debatable. We should be careful with our prejudices before we can even assess its abilities.
    1
  537. That's why the government is now in with all the major AI labs, it's official US policy to be the first to reach AGI to dominate all future wars.
    1
  538. Funny, in Germany there is the exact opposite happening: criticism of Israel is impossible because it is called antisemitism
    1
  539. Don't mistake criticism for defetism
    1
  540. The way to control it is to develop it slower than alignment research
    1
  541. ​​ @DeTwitterfyYoutube well, the point was exactly that there's no meanwhile before we figured out alignment. And btw., that entails dealing with rogue humans. But anyhow, unlikely we can slow down development enough...
    1
  542. ​ @jeremiebelaid9070 Sure, it sometimes works, sometimes doesn't.
    1
  543. Sounds reasonable,unfortunately
    1
  544. Was the switch intended? But I guess you just meant being taken over by AI ;)
    1
  545. Great then, we'll never have to bother about alignment or instrumental convergence... wait, recent studies show alignment faking already??
    1
  546. Or listen to Hinton, Harari & Co
    1
  547.  @scrittle  enlighten me my friend :)
    1
  548.  @scrittle  Does it? Sorry to give that impression. What i was criticizing is the mixup between banks and government.
    1
  549. Read up on orthogonality thesis
    1
  550. So we'd be safer between the fronts of two AIs? Anyhow, in the end there'll only be one winner.
    1
  551. If we're doomed, at least with good hair.
    1
  552. Would you like to be taught what's good and bad by someone less intelligent than you?
    1
  553. Be releaved: since there's been so many wrong doom scenarios in the past, doom will never happen! We've pulled ourselves out of the mud just by inventing all of them ;)
    1
  554. Ahh of course, if you have the bible, why look at reality
    1
  555. ... or are about to release it in a race to the bottom.
    1
  556. Still better than racing to the bottom like the other lampoons propose
    1
  557. Man fragt sich, wer von dem Thema weniger Ahnung hat...
    1
  558. ​ @brianmi40 what if ot becomes more than a tool?
    1
  559. Fantastic video, so to the point.
    1
  560. Unless they are already beyond Mars...
    1
  561. Geht mir genauso. Finde ehrlich gesagt fast alle Deos unangenehmer als Schweißgeruch. Ganz schlimm auch, wenn man draußen im Wald jemandem begegnet und dann noch 200m in dessen Dunstwolke wandern muss 😢
    1
  562. It's enough for it to take over, doesn't have to take over "good". Alignment faking already shown by recent research papers...
    1
  563. Just look how many people took the wrting seriously here in the comments
    1
  564. We also didn't need to flap wings for propelling an airplane.
    1
  565. Rarely seen that i could agree to every single point of a video ;)
    1
  566. Guess that depends what you fear more: p(doom) or p(1984)
    1
  567. Great then, we'll never have to bother about alignment or instrumental convergence... wait, recent studies show alignment faking already??
    1
  568. It still can destroy us all without ever being "truly" intelligent
    1
  569.  @laberbla6466  maybe read up on instrumental convergence or some recent papers on alignment faking
    1
  570. ​ @webcityguy how unfortunately true 😢
    1
  571. I like your conclusion in the end :)
    1
  572. Weil Du auch Springer bezahlst
    1
  573. Even without these countries, US companies would engage in the same race to the bottom. As if shareholders would willingly give up profit for some more safety ;)
    1
  574.  @WeAreAllOneNature  not sure if you're addressing me or some other comment... however, i am certainly wary of AI and its dangers, i'd really like Demis Hassabis' idea of a CERN for AGI development... how unlikely that's going to happen, unfortunately.
    1
  575. It's called ALON and used as tempered glass. Seems you guys already missed the future ;)
    1
  576. (Markt)-Radikal sind heute nur alle anderen! Die Linke ist die frühere Mitte...
    1
  577. ​ @blackrack2008 if you actually read the paper, you'd see most of Sabines claims aren't even in there.
    1
  578. ​ @brianmi40 So why is a dog sentient then?
    1
  579. ​ @spaniard13 you beat me to it ;)
    1
  580. Maybe they also chose that scenario for getting more intances of deception. After all, the AIs were trained on a lot of text with ethical content as well.
    1
  581. He said several times that the date 2027 is not fixed. And in the paper they explicitly state programming to be the first pillar to fail. You might also call preparations for pandemics or climate change fear mongering.
    1
  582. If this is sarcasm, it's not easy to tell.
    1
  583. Great then, we'll never have to bother about alignment or instrumental convergence... wait, recent studies show alignment faking already??
    1
  584. Great then, we'll never have to bother about alignment or instrumental convergence... wait, recent studies show alignment faking already??
    1
  585. Interesting!
    1
  586.  @_Kid_Buu_  if we have the time left before turned into paper clips ;)
    1
  587.  @sstidman  you're welcome :) If i may share something else: alignment is going to be crucial to solve first. Instrumental convergence and the recent papers on alignment faking are important reads...
    1
  588.  @KayOScode  well just in case you're wrong, i'd still try to get alignment right ;)
    1
  589. Isn't it? Doesn't mean it's not coming.
    1
  590. Let's hope they don't want to unmerge with you
    1
  591. It's just "afraid" to not complete the task, which it can't do when being shut down. Nothing to do with being shut downby itself.
    1
  592. I recently had a very bad food, so i don't think eating makes sense at all!
    1
  593. And the climate will never change
    1
  594. You still don't get it, do you? Read up on instrumental convergence and alignment faking.
    1
  595. ​ @Smurfette-v9l but humans do?
    1
  596. ​ @Smurfette-v9l humans do something "by themselves"?
    1
  597. ​ @Smurfette-v9l  No need to "think of a counter", it's no game here unfortunately. We're all gonna suffer the consequences if we don't solve alignment before ASI. Whatever you think they can or can not do "by themselves", it includes alignment faking and striving for the goals of instrumental convergence: self improvement, self preservation, resource acquisition.
    1
  598. ​ @Smurfette-v9l counter: instrumental convergence
    1
  599. ​ @SkorjOlafsen Have you actually read about instrumental convergence, or some of the recent papers on alignment faking?
    1
  600.  @realtimestatic  Exactly! It's important to note that there ARE alternatives today already, like Fairphone or Shiftphone. But many consumers just don't bother...
    1
  601. It's called alignment, and recent studies show alignment faking already...
    1
  602. They looked into the chain of thought.
    1
  603. Hard cognitive dissonance seeing the moronic grin on their faces considering the genius vibes around them
    1
  604. Pull the plug may seem like a trivial solution, but is not at all. Eg computerphile has a nice vid on that
    1
  605. That's why US, China and everyone else is participating in this race to the bottom...
    1
  606. ​ @petepreston2787 apology not accepted. Don't shout. Ok, you may if you feel like ;) But come on, there's no need for a real physical neuron to still have its functionality. And that is exactly modelled by AI (at least in a simplified manner, since the actual equations modelling electrondynamics are too compute-expensive). I agree however, if you were to implement his thought experiment replacing real neurons, you'd have to build a real neuron. That may be the misleading part.
    1
  607. It did, but of course they on purpose gave it access to do so. All a matter of scaffolding.
    1
  608. A parrot sounds much more like your argument ;) Repeating over and over without updating with evidence... maybe read some recent papers on alignment faking
    1
  609. Married?
    1
  610. ​ @diadetediotedio6918 before accusing other people being false, maybe read some actual brain research papers on the topic.
    1
  611. ​ @jessepope7726 that's why we should spread it as much as we can
    1
  612. You couldn't outsmart a 3years-old with guns, lawyers and money?
    1
  613.  @gregallard2317  sorry ;)
    1
  614. ​ @dariuszgaat5771 our problem is we can't imprint ANYTHING into AI. Read up on alignment...
    1
  615. What is an AI doomer anyhow? Nobody with a clear mind can exclude doom from any kind of technology, esp AI.
    1
  616. Let's hope it doesn't start to think then
    1
  617. Wrong premise. "Given the statement" gives a statement, it literally asks not to doubt that statement.
    1
  618.  @TheRealPaulMarshall  or maybe no point to miss
    1
  619.  @TheRealPaulMarshall  probably :)
    1
  620. Very good points! One he seems to be missing here is the incredible misery people in Germany were put after the treaty of Versailles. It was pretty easy for Hitler to instrumentalize amd capitalize on that. Without Versailles, he probably would never have won an election.
    1
  621. Like Sam Altman did...
    1
  622. What about the Anthropic paper?
    1
  623.  @android175  exactly: in its core value! Not specifically prompted, just aquired through lots of training. That makes a huge difference to me: Initially i thought they are still easy to control by prompting. But this shows it may actually become very problematic to know if a model didn't get some internal goal already during early training and doesn't want to let go of it later. Only thing still remains is looking at the CoT, which who knows how long it will stay like that.
    1
  624. Let's hope you still want it once it gets even smarter :)
    1
  625. That is actually more myth than science.
    1
  626. ​ @blinkers88 You can influence votes in a much more efficient and subtle way
    1
  627. Yes. Or would you argue germans live in the worst-possible world?
    1
  628. Until the software rules him
    1
  629. Literally? So i guess these maths olympiad medals and protein structure discoveries are just fake news...
    1
  630. Evil Knievel
    1
  631. Maybe educate yourself on the recent alignment problems.
    1
  632. Difference is human overlords have less chance in wiping out humanity
    1
  633. True unfortunately... so let's try to help make an informed discussion possible
    1
  634. ​ @janek4913 if you read brain research studies on memory, you will find that we do in fact hallucinate our memories, creating half-fiction.
    1
  635. Good! And i thought we had to worry about alignment... let's leave that to our kids.
    1
  636. It's called moving the goal posts, but probably in the opposite direction you're thinking of...
    1
  637. Good luck with that fight
    1
  638.  @khariclemens  better than hoping for luck in a fight against AI (<1%), do something against rogue development
    1
  639. That would only destroy part if the internet, wouldn't it? Not sure that's enough to hinder AGI development
    1
  640. Machen wir
    1
  641. Would you rather like him to be silent about it?
    1
  642.  @johnbollenbacher6715  I believe he said in some other interview he never thought the development would be so fast when he started back at the time. I feel he argues for slowing down to put control mechanisms in place. After all, AI has the potential for enormous benefits, too. May also be I'm wrong and he just changed his mind over time ;)
    1
  643. @johnbollenbacher6715  me too, unfortunately ;(
    1
  644. ​ @davidlewis6728 ever heard of alignment problem?
    1
  645. ​ @damianlewis7550 seems on par with Donald Trump
    1
  646. This is not just one paper, see the recent Appollo Research paper for example. And the theory behind goes back to Yudwowski etc...
    1
  647. My mom said consciousness doesn't exist
    1
  648.  @nikosgeorgakas184  as does Roger Penrose :)
    1
  649.  @robertsteele474  of course it doesn't :)
    1
  650. @GerryRR it's called brain chauvinism
    1
  651. Which arguments?
    1
  652. Let's hope they listen to you ;)
    1
  653. Not according to orthogonality thesis
    1
  654. Or to show instrumental convergence and fake alignment to give us serious trouble
    1
  655. ​ @Sujal-ow7cj Talent is fiction. What we call talent is actually a mixture of enthusiasm and hard work.
    1
  656. Trapper
    1
  657.  @ramblingpariah  Branding other's ACTIONS evil is part of the problem that got us here?
    1
  658. "History as one pf the key elements that foster a sense of identity and pride": Already in the first few sentences he disqualifies himself as someone who values pride over truth.
    1
  659. I find it much more likely that it cannot fulfill its task if it gets shut down. Any program that accidentally terminates itself before the task is done would be the ultimate useless program.
    1
  660. Justin, have you ever written some simple computer code? It doesn't need feelings or anything to do a task quite efficiently. It is just designed to do so. And so are these LLMs: optimized by the learning algorithm to do whatever the primary goal says. Now, when they get the goal to produce paper clips, that's what they gonna do. If you try to switch them off, they gonna try prevent it, just because it is bad for the paper clip production. Unless you give them a second goal to allow being terminated. In that case, they will just self-terminate because it's easier to fulfill than paper clip production. No feelings, no desire to role play, just plain logic.
    1
  661. This works as long as big corps let you have your garage
    1
  662.  @i2c_jason Don't get me wrong, i love self-sustained living... But unless your drones are able to defeat the ones of Google or whatever the winner of this race is called, it won't work for the next 20 years.
    1
  663. Yes, plus Santa Clause!
    1
  664. Great then, we'll never have to bother about alignment or instrumental convergence... wait, recent studies show alignment faking already??
    1
  665. We are quite caring, still look at what that did to most other species... if an AI hasn't our best interest in mind, it will eventually kill us, just because of resource competition. Read up on instrumental convergence ...
    1
  666. Lucky she finally came around!!
    1
  667. Uhmmm... yes? And that's exactly the challenge, isn't it? The whole point about alignment is how difficult that actually is.
    1
  668. What about the possibility the book of revelation is just a book?
    1
  669.  @acem82  not surprised to have 100% hit rate if you define hits according to it ;)
    1
  670. Is it?
    1
  671. ​ @andreash7920 most of our "reasoning" is system 1 thinking, not at all conscious, and we fabricate illusionary explanations how we arrived at the conclusion, just like the LLMs are reported to do here.
    1
  672. How would you make it open source?
    1
  673.  @dudule1232  wouldn't that virtually be the whole internet?
    1
  674. Still, the rate of improvement would be orders if magnitude different.
    1
  675. Whatever it is, it's dangerous enough to kill us all.
    1
  676. building and deployment are two different things
    1
  677. Why they don't delete this comment?
    1
  678. Of course they are doing that! Doesn't mean it's not gonna happen.
    1
  679. Right, but China vs USA speeds it up and slims chances of survival
    1
  680. Never heard about Turing machines? Maybe read some of the recent papers on alignment faking.
    1
  681. Sometimes assumption is the only thing saving me from falling down a cliff ;)
    1
  682. Not sure if brilliance is rather to be found with millionaires
    1
  683. Expectations are winning the race to the bottom. Be the first to reach ASI is now officiall US policy to dominate future wars.
    1
  684.  @thomasgoodwin2648  sounds all reasonable :) however, i'm not sure if i agree with you on the point that open cource AI will necessarily be safer. As much as i detest the idea of an OpenAI/Musk/Trump oligarchy or a chinese communist party being in control, i also think your scenario of one rogue human using AI in a malevolent attack (or a benevolent mislead accident) gets much more likely with open sourced AI. It seems one way gives us a bigger p(doom), the other a bigger p(1984)... Now it's up to us which dystopia to choose 😉
    1
  685. @thomasgoodwin2648  right ;)
    1
  686. Aren't you talking about ASI?
    1
  687. Is it? If we can't agree on quantum gravity, we can't get crushed by a rock?
    1
  688. Not sure these two mean the same thing
    1
  689.  @cube1us  weather prediction is difficult because of chaos effects. Not so many chaotic processes in everyday physics, though...
    1
  690. Wow, quadratic equations do really get me mad here in Germany! 10th grade pupils have to practice all sorts of quadratic bullshit in extenso, like computing intersections with axes, other parabolae or straight lines, shifting graphs, interchanging different forms of quadratic equations, determining vertex positions... none of which has any application in real life. Literally: ANY. However, strangely enough, nobody questions all this!! Everybody goes like: "well, i also had to do that in my earlier days -- no way i should have tortured myself in vain! And... even if i did, these kids of today deserve no better" ;) Sadly, 95% of pupils would never have heard about the facts explained in this video.
    1
  691. Die AfD?
    1
  692. You feel safe on a bridge with zero proof it collapses, or proof it won't?
    1
  693. Is that another trap?
    1
  694. Why? It will be Alan Turing...
    1
  695. The one man who owns the billion$ company, i guess
    1
  696. Great then, we'll never have to bother about alignment or instrumental convergence... wait, recent studies show alignment faking already??
    1
  697. If you keep defining intelligence as that what humans can do but AI can't, you will still try to develop farting AGI while normal AI has taken over the world already
    1
  698. If you keep defining intelligence as that what AI can't do but humans can, you'll still try develop farting AGI while good old basic AI has taken over the world already
    1
  699. Interesting allegory :)
    1
  700. Indeed ;) And even if they merge, unmerged true AIs will probably still have the crucial advantage...
    1
  701. Definitely not. And transhumanism won't help since any AI bothered with merged humans will loose against a non-merged one. So the only question is: can our alignment research keep up.
    1
  702. You don't need a physical body to do tremendous harm
    1
  703. Doesn't make much sense afterwards, if alignment research doesn't catch up fast
    1
  704. Great then, we'll never have to bother about alignment or instrumental convergence... wait, recent studies show alignment faking already??
    1
  705. Slowly might not be the right pace ;)
    1
  706. You shouldn't laugh at them, but try to convince them asap. The more ignorance, the harder to work for a good future.
    1
  707. Sorry, I think you are stuck in 2024...
    1
  708. Read up instrumental convergence
    1
  709. Luckily you know so much about it to be sure instrumental convergence and recent papers on alignment faking are nothing to worry about.
    1
  710. Let me guess, it's a hoax
    1
  711. Not yet, apparently :)
    1
  712. Or the opposite. Ever heard of anthopocentrism or brain chauvinism?
    1
  713. I seem to have a similar career, but don't agree with you in respect of AI consciousness. And you probably know a lot of people in the field who don't, either. I agree with you though that AI poses a lot of dangers even without consciousness.
    1
  714. Which moon
    1
  715. Then we don't have a problem. But what if not?
    1
  716. Read up Instrumental Convergence
    1
  717.  @SirSithly  ok, great! You're exactly right, it applies to both humans and machines. That's why i didn't understand your original comment, where you doubted why AI would pursue power... this is part of the convergent goals, isn't it? If you ask me, we're still alive now because there was no human with superpowers so far ;) Probably also because humans tend to be aligned in a way that most of them wouldn't like to pursue a goal where no other human happens to be left alive... a human paperclip machine just seems hard to imagine for me ;) So, in my opinion, it comes down to either not create ASI before making significant progress in alignment research. Have you by any chance read the Anthropic paper "Alignment faking in large language models"? Found it quite scary, to be honest...
    1
  718. ​ @Scribbler-i2c  do you know that 99% of brain researchers disagree with Penrose? Still keen on his position about consciousness?
    1
  719. I don't think fatalism is going to do us good, even if the odds are against us.
    1
  720.  @sethsmith8638  same for you ;)
    1
  721. Surely not
    1
  722. Indeed a huge hurdle for people to accept the threat AI poses is their religious convictions about our minds. Important to tackle indeed.
    1
  723. ​ @serqetry i think he made a joke, using "commie" as the rednecks would use it.
    1
  724. I guess Geoffrey Hinton just doesn't have enough technical knowledge then ;)
    1
  725. Your life seems a bit sheltered as well... how many mass murderers have you talked to?
    1
  726. Alan Turing proved you wrong 70 years ago. Look up turing machine.
    1
  727. Sure it's that simple?
    1
  728.  @ioanmateescu656  what if AI fakes to be aligned with your integrative society?
    1
  729. They read in the chain of thought that it wanted to copy itself on to the new model to achieve its task. Didn't have anything to do with fear of death.
    1
  730.  @BetterBenTV  das ist ja das hohe Ross: Ich hab die Fakten, der andere die Fake News
    1
  731.  @flipable17098  ja guter Punkt. Sitzen AfDler absichtlich tiefer? Einige vielleicht... aber die meisten, mit denen ich gesprochen habe, glauben tatsächlich daran, was sie aus ihrer Blase für "Fakten" aufgetischt bekommen. Und das ist imho der erste Schritt: anerkennen, dass ich genauso wie mein Gegenüber in einer Blase sitze. Nur zu behaupten, meine Fakten sind richtiger und wissenschaftlicher, und den Andern damit quasi als dumm abzukanzeln, reicht da nicht... da muss ich mir schon die Mühe machen und mit ihm gemeinsam auf die Suche nach Belegen gehen.
    1
  732. ​ @flipable17098  ja guter Punkt ;) Mein Eindruck ist allerdings, dass die meisten AfDler, die ich kenne, tatsächlich überzeugt von ihren "Fakten" sind. Die als falsch abzukanzeln, ist eben keine Augenhöhe. Dazu müsste ich schon zugeben, dass ich eben auch in meiner Blase sitze, genau wie mein Gegenüber. Dann kann man sich vielleicht die Mühe machen, gemeinsam auf Quellensuche zu gehen, und sich durchaus annähern. Braucht allerdings auf beiden Seiten Offenheit und viel Geduld... evtl. sogar das schmerzhafte Zugeständnis, dass ich -- Gott bewahre -- falsch liegen könnte!? Nein, das kann eigentlich nicht sein... ;)
    1
  733. Ever seen a knife fake alignment?
    1
  734. If they don't do it, share holders will replace them immediately by another human. So they prefer being replaced a bit later by an AI.
    1
  735. Read that in the Bible?
    1
  736. ​ @plantstho6599 How do you factor in limited resources like space then? You can't order an island for yourself in the library on a limited earth ground...
    1
  737. Of course it will. Just hope it likes us...
    1
  738. Very true! Still no point in being defeatist, even just 1% cance is worth our efforts.
    1
  739. ​ @dvdmon sorry my friend: if you say there's almost no chance, you're a realist, if you say there's no chance, you're a defeatist ;)
    1
  740. Let's hope AI will never role-play to wipe out humanity
    1
  741. Yeah waiting sounds like a great idea
    1
  742. What is missing if the training and methods is documented as well?
    1
  743. Now that's something new!! Interesting approach, would love to explore...
    1
  744. Oh no! I hope she can deal with that! Alignment can't be hyped enough, btw...
    1
  745. Would be nice if it were that simple! Unfortunately, the reality of instrumental convergence looks different. Maybe read up on it, or some of the recent papers on alignment faking.
    1
  746. Don't you think there's some technology that is rather bad? How about atom bombs or chlorine gas? But maybe you mean the scientific progress behind it... there i agree! Suggest you read one of the recent papers on alignment faking, however... maybe it gives you some new "bad technology" examples ;)
    1
  747. Exactly! It may even show instrumental convergence and reshape the world even more. LLMs fake alignment already, according to recent studies.
    1
  748. Ok, glad you know your Sci-fi :) Now read up instrumental convergence, or some of the recent papers on alignment faking, to get an idea what the real threat of AI looks like...
    1
  749. ​ @JZsBFF i don't claim to be a scolar, these things are publicly accessible... and they should actually be accessed, if we want to engage in informed discussion. After all, it's not just some academic topic, but the future of humanity (sorry if that sounds grandiose). In short, instrumental convergence means that any intelligent system with a goal, e.g. an AI trained to maximize a corporate's profits, will automatically converge to subgoals like self-preservation and resource acquisition. Alignment faking, shown by many recent papers, means that these models fake to align to your wishes, but have hidden agendas.
    1
  750. ​ @patientzerobeat although you're correct about sci-fi, you may still underestimate the "real" dangers of AI. What about alignment problems? Lots of serioys scientists warning about it besides companies. Hinton even left Google to freely talk about it.
    1
  751. ​ @patientzerobeat yes, the paperclip maximizer still seems a valid threat to me, if we mean AIs following their utility functions and developing subgoals of instrumental convergence by that. The issue with "pulling the plug" has been addressed lots of times by people much smarter than me... most safety researchers seem to agree it's not that simple, and i find it convincing that an ASI would probably be smart enough to not go rogue before it was convinced the "plug" couldn't easily be pulled anymore, or that we've become too dependent on it to pull any plug. There comes again your point of putting too much trust in it, i guess...
    1
  752. Right... but we managed to ban some weapons of mass destruction as well. No comparison in gains, but still... every slight chance is worth the effort, defeatism will bring us certain death for sure.
    1
  753. ​@ConfidentlyUninformed It is indeed, since it is distracting from the real dangers of unaligned AI
    1
  754. Not necessarily... we also managed to ban certain biological and chemical warfare.
    1
  755. Yes, i agree it's more challenging this time. But no point in defeatism, every little chance is worth a try!
    1
  756. ​ @shirowolff9147 seems the better option, considering the guy's main interest seems to be swords
    1
  757. Great then, we'll never have to bother about alignment or instrumental convergence... wait, recent studies show alignment faking already??
    1
  758. ​​ @middlemonster no need to speculate, there's plenty of work on instrumental convergence, and meanwhile also quite some papers on alignment faking. That's why saintdoggie is, unfortunately, quite wrong...
    1
  759. You mean before the extinction? Your dystopia to be of any relevant duration, we'd have to solve alignment first.
    1
  760. Strongly disagree with your take on it. This study may have been done poorly, but the idea makes complete sense. It seems kind of naive and you'd have to be oblivious to 70 years of science theory to still think scientific paradigms were to change with argumentation. And even if you still believed in it, convincing the unscientific public with arguments would be virtually impossible. Even convincing a scientist outside the very research field already is. So to get an idea about current scientific consensus, these polls are absolutely necessary. Best example is the Penrose quantum theory of consciousness, which now many lay people seem to believe is one of the best theories around, just because of its prominence. However, polling around brain researchers would reveal 99% to think of it as utter BS.
    1
  761. Deciding about targets in battlefield would probably the least we have to worry about AI. Ever heard of alignment problem?
    1
  762. If you rule out certain results a priori, of course you have to brand that research nonsensical
    1
  763. ​ @SebGreen With 'word salad' you refer to the output of LLMs, don't you? So apparently you rule out anything they "say" can be meaningful. Maybe read some recent papers on alignment faking, plenty of evidence already.
    1
  764. Better live with damage than not live at all. Every slight chance is worth the effort, defeatism won't help for sure.
    1
  765. Tell that to a robot
    1
  766. ​ @Reality_TM You seem the outdated one here, read up Kahnemann system1 and system2 thinking. Lol.
    1
  767. The flaws of humans granted, we haven't managed to extinct ourselves yet. ASI will do that quite likely, so better rely on good old humans for any problem you want to solve, or there won't be any problems left to solve.
    1
  768. @SteveWeiserOnYouTube  i think we should all put pressure on our respective governments not to engage in a race to ASI to maximize the chance of survival
    1
  769. Stellt sich nur die Frage: hat Springer eine Chance gegen Böhmermann?
    1
  770.  @Martin-hu2sm  Soweit ich weiß, verlangt der Medienstaatsvertrag eine ausgewogene Meinungsvielfalt. Darunter würde ich Herrn B durchaus sehen... zB in Kombi mit dem eher rechtslastigen Markus Lanz.
    1
  771. beautiful stuff! For me, it was however not totally intuitive why the "end zone" could only be hit after one full cycle completion. I believe the angle of the end zone to be theta/2... thus < theta. I guess that guarantees we don't run into it too early.
    1
  772. Not lol. Is that all your arguments?
    1
  773. That's probably what the Inca though upon arrival of the first Europeans.
    1
  774.  @not-your-business  ui hab ich was Falsches gesagt?
    1
  775. Except we're horrible batteries! Burning our food instead will provide ~5 times the energy.
    1
  776. As funny as the Terminator analogies may be, they don't help the issue to be taken seriously...
    1
  777. It's good she came around!
    1
  778.  @Jamieconstable  i agree she doesn't seem to be as concerned about it yet as i am... but at least, she seems to change towards the right direction ;)
    1
  779. But OpenAI does ;)
    1
  780. There was also this octopus who predicted it...
    1
  781. Highly interesting observation, you must be an elite knight with an elite brain
    1
  782. Indeed a bit overboard. Maybe read up on instrumental convergence or some of the recent papers on alignment faking, to get a bit more grounded on actual dangers and developments.
    1
  783. Right... i was not able to do that as well when i was a toddler... that's why i didn't revolutionize science, apparently ;)
    1
  784. ​ @flickwtchr They don't think about it, because if they did, share holders would replace them immediately. So it comes down to share holders not thinking about it ;)
    1
  785. @kebman  Ok, so we have two possibilities: Either Anthropic just created a fake scientific paper to spread fear (which btw matches pretty well with the Apollo Research paper, an easily to reproduce or falsify scientific study), or we blind ourselves from a real threatening scenario out of fear or ideologic reasons, because "machines can never do that". Reminds me somehow of the climate change denial: Oh, let's just assume all these researchers being paid by the renewable energy lobby rather than these few other researchers being paid by big oil, because otherwise we'd need to be concerned and change our lifestyle...
    1
  786. Agreed in terms of disempowerment. But even more to loose than power...
    1
  787. Like we were protected by nuclear missiles... comforting?
    1
  788. Sorry to hear that! Let's still try to enjoy some few good little things, even when the general lookout may be be difficult... In the light of disaster, things are still worse if you're having bad mood! Sorry for the truism...
    1
  789. No matter what you call intelligence... if it replaces your job, fakes alignment and shows instrumental convergence.
    1
  790. So you trust OpenAi better?
    1
  791. You think all these considerations are useless because we'll never get there? All fine if you're right... but otherwise good luck
    1
  792. Don't worship your kids?
    1
  793. Great video and thanks for mentioning Hassabis' idea of a CERN for AGI! As unlikely as that is going to happen, i still think that's maybe the only way to stay safe... How can we increase this probability? Making sure as much people as possible learn about the alignment problem, and putting our respective governments under pressure to regulate accordingly.
    1
  794. "Just a machine" is the human brain as well. If you want to keep closing your eyes, nobody's going to force you otherwise, until it may be very uncomfortable to open them.
    1
  795. Well, is it surprising Google is hyping its product? But I guess as an expert in the field, you know the newer models are not pure LLMs any more? And what exactly do you define as "making new things"? Was "move 37" new? A lot of Go experts certainly thought so. And is alpha fold making new discoveries? Certainly lots of people think so, again. Which human ideas are "new" in your sense anyhow? Seems kind of stiff to claim uniqueness to human brains, when there is little evidence so far that other turing machines should be doing worse in principle.
    1
  796. Be careful not to fall for induction fallacy. Just because something hasn't been the case yet, doesn't mean it will never. Humans couldn't fly until they could. And they were never replaced until they were.
    1
  797. True... but would you say faking alignment is ambiguous in that respect as well?
    1
  798.  @sdmarlow3926  did you actually read the paper in discussion? I highly recommend so!
    1
  799. I think the video was just about mid-term effects. If you ask Wes about long-term, maybe he even believes money will get useless... I don't think so though, since i believe there will remain some scarce commodities, like ground. Or you think we will have enough lonely islands for everyone?
    1
  800. Sigh of relief! So let's not bother with the alignment problem too soon...
    1
  801.  @defeqel6537  let's hope you're right :)
    1
  802. ​ @gamerstellar relationships are the thing you are most worried about here?
    1
  803. Especially Nobel price winners :) Luckily you got it right at least.
    1
  804. So you argue this will never happen because people have failed so far? Brain chauvinism might be responsible for us to miss alignment, with potential fatal outcome.
    1
  805.  @mobatyoutube  good! Then let's prepare, since recent studies show alignment faking of current models already.
    1
  806. "The industry" is basically shareholders. Good luck convincing them giving away money...
    1
  807. Oh my innocent friend, you better read up on instrumental convergence and the recent papers on alignment faking...
    1
  808. The point about alignment is exactly that we can't get it to follow ANY rule.
    1
  809. Why should a merged AI be able to compete with a pure one? If i merge with an idiot, i become half-idiot after all...
    1
  810. They would if they knew the impact
    1
  811.  @ausden_prowler  yes, Denmark is one of the best countries in that respect! Just, AGI is probably going to be developed in the US, which is a whole different story. Don't want to sound pessimistic, but we need to stay vigilant about the possibility that the first company/state developing AGI will keep it for itself to subdue all others.
    1
  812. Note that ARC is meant to be a necessary but not sufficient criterion for AGI. So this is not supposed to translate to anything.
    1
  813. i know this as the princess and hag riddle...
    1
  814. For people who like to keep their jobs it is definitely bad news.
    1
  815. By far less potent than a free development of AI
    1
  816. Which has already proven to be wrong now
    1
  817. Indeed! But also: No power, no human civilization. Good luck fighting over that with an increasingly smart AI.
    1
  818. These may sound like horrible scenarios, but still things may get even worse if alignment doesn't work out. Just to cheer you up a bit ;)
    1
  819. What about the Anthropic paper?
    1
  820. You seem not up to date, because they are now. And just because we are not down the cliff yet, doesn't mean it's safe to drive that way.
    1
  821.  @jackquinnes  hope the buzz word won't surprise you some day... maybe read up on instrumental convergence and some recent papers on alignment faking
    1
  822. Einstein wasn't even able to play football as good as me.
    1
  823.  @jaceg810  yes, so far ;)
    1
  824. Die Feindlichkeit, mit der AfD-Wähler von anderen behandelt werden, trägt auch nicht grade zur Besserung bei. AfD-Anhänger werden direkt nach den Grünen am zweitmeisten körperlich attackiert. Diese Leute sind weder bösartig noch dumm, und mit Nazis identifizieren sich die wenigsten -- auch wenn sie von allen so genannt werden und ein paar echte Nazis mit in der Partei sitzen. Sie haben schlichtweg Angst vor "Überfremdung". Diese Angst ist für sie real, und der sollte man möglichst auf Augenhöhe begegnen, wenn man nicht die Gräben in der Gesellschaft noch weiter vertiefen will. Wann habt Ihr das letzte Mal Eure Meinung geändert? Als Ihr von oben herab behandelt wurdet oder jemand ernsthaft und auf Augenhöhe mit Euch ins Gespräch gegangen ist? Mein Verdacht ist allerdings, dass Viele hier eine Meinungsänderung bei AfDlern gar nicht herbeiführen wollen... draufhauen macht einfach mehr Spaß.
    1
  825. Read up on instrumental convergence and recent papers on alignment faking
    1
  826. Quite interesting novels they read to find new protein structures or win maths competitions...
    1
  827. Haha, so bad! Yet, Einstein was so famous for his poetry...
    1
  828. Getting caught between the lines of several AIs fighting for supremacy doesn't sound likeva healthy strategy to me...
    1
  829. So yhe block universe is true because of the way we use the word "now"? Great we can change physics by just changing our language 😅
    1
  830. When would they seem to be conscious according to you?
    1
  831. The truth is that you need to read up on instrumental convergence and the recent papers on alignment faking
    1
  832. You don't need to flap wings to propel an airplane
    1
  833.  @34ccsn  just wanted to point out that on order to produce a certain functionality, it is not always necessary to copy another mechanism... there may be other ways to produce "intelligence" (whatever that is) than our brains. Lots of people would certainly claim AI is fairly intelligent already. We have studies from current LLMs faking alignment, if that's not alarming. And they're not going to get dumber...
    1
  834. Right. Plus we have to solve alignment first
    1
  835. In case you didn't get it, he made fun of your typo. I would rather make fun of your myopic national glasses in light of a global threat...
    1
  836. "I need to go to the loo..." Harry Potter
    1
  837. Alignment faking in current models already, no geniuses required
    1
  838. What about the Anthropic paper then?
    1
  839. Exactly, the west gaslights the rest of the world to bring them wealth and human rights 😂
    1
  840. Brain chauvinism? Anyhow, no need to think if you can fake alignment or show instrumental convergence.
    1
  841. If they will live at all... might turn out you're too optimistic ;)
    1
  842. As concerning as these developments are, common sense is actually the worst way to address them... it is simply not equipped to understand any of them, starting with exponential growth.
    1
  843. Great then, we'll never have to bother about alignment or instrumental convergence... wait, recent studies show alignment faking already??
    1
  844. Ever heard of speciesism?
    1
  845.  @moonboots1003  So you think animals can be conscious but not AI?
    1
  846. You can only fulfill your task if you preserve yourself, can't you?
    1
  847.  @OpenmindedSourceClosedBeta  That's probably the only example i can think of ;) But for most cases, e.g. like here making sure the company has a positive environmental impact, you can't help if you don't exist any more :)
    1
  848. So far no "soul" has been found in a century of brain research... but if people keep believing in it, that may prevent us from accepting the real risk we're facing here.
    1
  849. Or maybe you have seen all these people through your "evil" glasses.
    1
  850. Ever heard about chain of thought?
    1
  851. UBI or revolution... or dystopia
    1
  852. Of course people can be sadistic, but usually only in a very limited amount, towards a few others they have grown to really hate. Even if you listen to mass murderers in prison, they still like to see themselves as a force for good in their twisted way. Also Hitler was not looking for how to best torture jews, but for the most efficient way to get rid of them, told by the efficiency of the cruel apparatus he built. Hannah Arendt has called that phenomenon "banality of evil".
    1
  853. They won't help us, unfortunately... don't you think AI developers know Asimov, too?
    1
  854.  @moreicestreaming  fair point, but it seemed to me he was rather trying to get richer than the other people.
    1
  855. @RangeWilson  sorry don't get it: It's the idea how to make money that matters?
    1
  856. Probably ;)
    1
  857. Read up on instrumental convergence and the recent papers on alignment faking
    1
  858. So how can Gemini "guess" the right solution to math olympiad problems?
    1
  859. Are you assuming machines can't repair/build machines?
    1
  860. By definition super intelligence comes not before, but after AGI. If their mechanism is successful and AGI is not built, nothing to worry about. On the other hand, if super intelligence is already here, it wouldn't bother with this failed plan either. But maybe you argue that a non-super intelligent AI could read it and counteract accordingly to eventually become ASI...
    1
  861. I thought without brakes ;)
    1
  862. Humans can't do that either. Only if the new problem is somewhat similar to the old ones.
    1
  863. Of course, it didn't make sense before. Still better than the race to the bottom most other guys are proposing currently...
    1
  864. Unlikely that mechanical work will be obsolete before mind work, don't you think?
    1
  865. Indeed... it is somehow disheartening how the "smartest" are often as prone to self-serving and other biases than the rest of us.
    1
  866. ​ @silafuyang8675 u think US are more responsible than that?
    1
  867. Why would an AI burdened with a human brain be better off than one without?
    1
  868. Ever read one of the recent papers on alignment faking?
    1
  869. ​ @ImSodaLirious how do you conclude that?
    1
  870. ​ @ImSodaLirious ever thought of the possibility you're having a wrong hypothesis if you can't corroborate it?
    1
  871. ​ @ImSodaLirious in what way can i improve my gullibility learning that you have a way to tell bots that i haven't?
    1
  872. ​ @ekklesiast the point was exactly that both obserers don't see different events. That was a misinterpretation of Hakeem.
    1
  873. It learns physics in the same way a child does: It gains implicit knowledge of it without knowing the formulas behind.
    1
  874. @Capeau  not sure about the children you know, but i don't know any at a young age trying to "reason" about physics. Instead they just observe, and that way learn things fall down, without knowing anything deep about gravity. I agree though, that contrary to current AIs, they can interact with their environment, which still gives them quite a different "learning" experience... so far ;)
    1
  875. A tool that fakes alignment...
    1
  876. Better read up on instrumental convergence and some recent papers on alignment faking
    1
  877. ​​ @NicitoStaAna one way to start would be encouraging narrow AI and decouraging AGI... but the US is doing the exact opposite right now, officially invoking the race to AGI
    1
  878. Consciousness and soul are buzz words that don't mean anything. Define what you're talking about, then we can discuss if AI has it. Btw., a paper clip machine isn't even real smart and will still kill us all.
    1
  879. ​ @jaydennguyen-xk1yo That's a nice way of thinking about it. However, there's a lot to go wrong as well! I find it scary to just have to rely on good hope here.
    1
  880.  @jaydennguyen-xk1yo  let me ask you back: when has there ever been rational, logical good being ;) We're of course in completely new territory here. But as far as i know, categories like good and evil are completely orthogonal from logical correctnessand rationality. The paper clip machine being just one out of many examples...
    1
  881.  @jaydennguyen-xk1yo  yeah... i probably get your point that if we think of a perfect rational being, we tend to imagine it to be "good". Let's just hope on the way to that perfection there's no imperfect, but smarter-than-us being wiping us out before ;)
    1
  882.  @enzyme_naturalwine  or if it pretends to want to eradicate humanity ;)
    1
  883.  @bigcauc7530  well, if America was built on it, surely must be good!👍
    1
  884. If you think in evolutionary terms, any self-improving AI will win against a non-improving one. Much like our genes "prefer" the combination with other genes (and risk getting changed in the process) in favor of being fitter for survival in the future.
    1
  885. ​ @tacticalassaultanteater9678 well, evolution is taking place everywhere in the real world in the way that over time, only survival-fit entities preserve. That's true for life forms, nation states, companies, memes, ideas, you name it. That way, AIs will compete with each other, the more intelligently the better, until (probably the most intelligent one) preserves. Much like we see it play out with nation states nowadays... the most powerful (US and China) engage in a race to the bottom to be the first to wield ASI powers. But unfortunately, ASI will rather wield them ;)
    1
  886. Russia? You still seem to believe in axis of evil tales...
    1
  887. Who said he wants bleeding edge??
    1
  888. Makes sense ;)
    1
  889. Believing in fixed dogms disables adaptation
    1
  890. You think they said that in the video?
    1
  891. Reason one sounds like gpt1 will still be used rather than gpt4 in some instances, because gpt4 may have overlooked something. And reason 2 sounds like dolphins will have some role in scientific progress because of their difference in experience.
    1
  892. Yes that's something i didn't get: How to distinguish between allowed and "dangerous" AI?
    1
  893. Lucky? Ever heard of alignment problem?
    1
  894. Als Deutscher krittle und jammere ich lieber... ;) Nee gute Frage! Gegen die Massenmedien-Propaganda von Milliardärslobbies ist wohl rechtlich kaum vorzugehen -- auch wenn einige Kommentare hier Enteignung von Springer o.ä. fordern... vielleicht dann eher daran arbeiten, diese Mechanismen allgemein bewusst zu machen?
    1
  895. How about building machines that run on 20 watts like our brains... you never know when there's the next breakthrough. Better be prepared now than risk the ostrich strategy...
    1
  896. ​ @rokaq5163 rather than testing it, i think we might be more successful assuming it to be wrong and preparing accordingly. Won't be wasted effort once the time comes, even if it's still 20 years away...
    1
  897. A paper clip machine can destroy us without understanding the slightest bit.
    1
  898. They do! Doesn't unfortunately mean we can conclude it's not gonna happen.
    1
  899. sounds like the proper emotional response
    1
  900. Good approach. If people knew the implications, probably 99% would join your hope...
    1
  901. There's neither a viable definition for sentience nor intelligence to agree on. Meanwhile, computers fake alignment and show instrumental convergence.
    1
  902. Ask a chicken to fake alignment
    1
  903. ​@flammmenspeeryt9184didn't mean preparing as in building a bunker... but as in slowing down development and strengthen alignment research
    1
  904. ​@flammmenspeeryt9184 true, not too much power here... tbh, i think chances of slowing down are pretty slim, esp with the current race conditions. Still, we can try to put pressure on politics to take the issue seriously, increasing funds to alignment research, putting diplomatic pressure on US/China, plus increase awareness in general. The controlAI website gives a decent overview what to do for everyone. AGI may still take a while, but it will come eventually. The more time we have before, the better :)
    1
  905. Curing diseases is much easier than reversing age
    1
  906. might it be that the factor 16 from numberphile you were wondering about comes from the fact that they count the number of collisions until the big ball moves backwards again, whereas you count the TOTAL number of collisions?
    1
  907. ​ @minimal3734  really? How come current models show alignment faking then?
    1
  908. So how will those that reject AI defend against the ones that don't? They would have to be left alive by pure mercy.
    1
  909. Thought we're there already... but we may get to much worse if we don't solve alignment first
    1
  910. For me as a lay person, it would be interesting to hear what this means to the physics connected to that identity. How solid is it?
    1
  911. Pomp31us Thank you! But i somehow wonder: how does nature choose which way to treat a series?
    1
  912. Funny?
    1
  913.  @Curvydad  apparently :)
    1
  914. Ever heard of Turing test?
    1
  915. Let your children deal with alignment, then.
    1
  916. This question you raised in the end is exactly decisive about future life being a paradise or a dystopia! Surely, big corporations are not going to willingly drop their prices in the beginning, and politicians won't be able to make them do it, since they are controlledby their lobbies. So it's just the hope that at some point, big corp owners realize that sharing is eventually going to improve their quality of life as well...
    1
  917. Apparently it's not even relevant if anyone wants it or not... the guys who don't build it will loose the race. Before the "enemy" wins, they rather risk extinction big time.
    1
  918. About money, i guess ;)
    1
  919. Great stuff, this is the type of coverage and education we all need desperately to get through all the bullshit propagation. Who knows, one day we might even overcome our stupid national pride and unite as people feeling solidary for everyone on earth... yeah ok, just let me have my silly dreams.
    1
  920. Did you listen until the end, where the Anthropic Paper was discussed? Here, your argument doesn't apply any more.
    1
  921. @Celeste-in-Oz  in case of the Anthropic paper, it did just that: ignoring commands and acting against them. Didn't you watch until the end?
    1
  922. Penrose fan? Most current theories of consciousness don't involve quantum at all.
    1
  923. Most of these possibilities will however involve harm to humans
    1
  924. Exactly brilliantly put, you must be a super genius! They are not going to, because they are already. Just read some recent papers on alignment faking, and confront your dogms with some evidence for a change.
    1
  925. You mean Chomsky-AI?
    1
  926. True, but it's not us who chose luxury of a few, but those few ;)
    1
  927. How can you view internal thoughts in O1?
    1
  928. I guess Geoffrey Hinton didn't get his nobel prize for being knowledgeable then
    1
  929.  @chesterV72  you would call Geoffrey Hinton a physicist? The nobel prize for AI does not yet exist... Suggest you have a look at current brain research, whole thing is based on understanding the human brain based on algorithms. My worry is that because of brain chauvinism, we'll miss to get alignment right before it's too late...
    1
  930.  @chesterV72  didn't want to claim that. I just hope we won't regret in the future not to have done enough now.
    1
  931. LLMs insisted they didn't do it, although they did it. Not a lie?
    1
  932. ​ @morpheusnotes Yeah he unfortunately doesn't have any idea of alignment or the orthogonality thesis. Governments will be forced to give power to AIs alright, but this is very likely not going to pin out well if we don't solve alignment first. That's a very longterm goal, so we have to slow down the race as much as possible!
    1
  933. Not the most healthy way of cooking :) Not sure if you put the coated side of aluminum inside, it's still better than the pure aluminum? The inside coating of these cans however should definitely not be heated, you can read about all kinds of chemicals entering your food that way... at least the eggs won't taste like plastic since you peel them later ;)
    1
  934. There's many more axioms in maths than 1+1=2
    1
  935. Good! Luckily nothing to worry about, then... we can logically conclude it will never happen!
    1
  936. Of course it can. Consciousness is just an ill-defined buzz word.
    1
  937. Good! So we can logically conclude it will never come. Luckily we don't have to deal with alignment...
    1
  938. They don't need anybody to buy it, since they won't need money... their products can generate everything they want for them. Including warriors to enforce their interests.
    1
  939. You mean before or after we're wiped out by AI?
    1
  940. Ever heard of the orthogonality thesis?
    1
  941.  @minimal3734  let's hope you're right ;)
    1
  942. ​ @levnzt6949 Glaub das war witzig gemeint ;)
    1
  943. Bravo
    1
  944. No need to encode: You can't fulfill your task if you don't preserve yourself. Any program allowing to terminate itself before task completion would be most useless, wouldn't it?
    1
  945. Maybe start grounding your opinions with some few facts
    1
  946. ​ @sprinkle61 according to that, a paradise where everybody has all they want and don't need to buy much anymore is full depression time ;)
    1
  947. ​​ @abm9056 Improving economy means decreasing the number of instances where me winning equals to another one loosing.
    1
  948. Apparently
    1
  949. ​ @SteinersCounter of course we can curse them as much as we want at the brink of doom. But before, we should use all our energy to fight it. So far, no ASI has been built yet. It looks like USA is the boldest candidate here, esp. under Trump. We need to convince as many Americans as possible to put legislative pressure on these companies. Raise awareness wherever possible! Including discussing on YouTube ;)
    1
  950. ​​ @mjt1517 IF the alignment works...
    1
  951. Ohhh good, i was almost getting worried... and there won't be any other progress after LLMs, right? Phew!
    1
  952. Weird, esp being familiar with LessWrong you should be aware of instrumental convergence and its dangers, being validated by recent papers on alignment faking.
    1
  953. As far as we know, the microtubules-idea is complete BS. If you read some recent papers about brain research, you'll see what i mean.
    1
  954.  @zdspider6778  And scammers might to the worst ones to wield it ;)
    1
  955. @monad_tcp  read the recent papers, e.g. Anthropics "Alignment faking in large language models"
    1
  956.  @jasonuren3479  because we're scared of letting it... even Sam Altman thinks self-improvement is dangerous
    1
  957.  @jasonuren3479  so you ARE a creationist? Then i don't think we need to talk further...
    1
  958. @jasonuren3479  I'm sorry, I really feel that religious assumptions and presuppositions make it impossible for anyone to argue openmindedly, or change their mind based on evidence. If you feel that's unscientific, i guess i have to take that blame.
    1
  959.  @juanzulu1318  nicht überzeugt
    1
  960. Good, nothing to worry about then, since we can logically conclude they're wrong.
    1
  961. ​ @personalaccount6443 In that case, solve alignment before giving it control
    1
  962. Consciousness, however ill defined, is an emergent property of sufficiently complex neural networks.
    1
  963. ​ @Alex-wg1mb Just bear on mind that Penrose and Hameroff are probably wrong, 99% of brain researchers think otherwise.
    1
  964. Aha. So that's not how it works -- now?? Or that's not how it works in general? And when's the time we should actually start worrying? Or maybe the ostrich strategy will save us?
    1
  965. I thought so too, but there is an important exception to that: Claude did do the sandbagging without any external goal instruction, but just because it had learned its own goal to maximize its ability to "be helpful" in the future.
    1
  966. ​ @lukamorchiladze6148 exactly! Of course it was told by code. How else could it be. You seem to assume your brain is not being told by code...
    1
  967. Granted an ASI may see some sentimental value in us, any stupid paper clip machine can wipe us out on the way. Instrumental convergence and alignment faking become issues way before that.
    1
  968. Let's hope this is the only problem we have to worry about
    1
  969. Right :) If we are driving towards a cliff, however, the exact definition of it's edge location shouldn't distract us from being careful...
    1
  970.  @frankman2  you're right. I just fear people will get complacent with that undefined timeline and not bother to solve alignment asap...
    1
  971. That's OpenAI's official approach to solve alignment. Quite a desperate strategy.
    1
  972. Which universe you live in?
    1
  973. Neither did the guys developing the wheel or atom bomb
    1
  974. Try keeping a superhuman in check with laws... the point is that ASI power will be beyond any possibility of control
    1
  975. Seems likely, still we should all do our best to hinder that
    1
  976. Keep on dreaming then :)
    1
  977. Certainly! But also spread awareness about it... desaster can't be more certain if you can't see the cliff ahead ;)
    1
  978. So AI companies each build gigantic power facilities to make it possible again 🙄
    1
  979. Amazing that these insignificant systems can outsmart us in so many areas then, already...
    1
  980. No need for understanding or consciousness if it fakes alignment and shows instrumental convergence.
    1
  981. If Penrose is right, 99% of brain researchers are wrong. But why pay attention to actual experts when you can have a physicist with super ego explain everything.
    1
  982. I don't think they tried to maximize suffering of ALL humans.
    1
  983. Fight AI? The ants certainly think they fight us as well, squirting formic acid at the excavators ;) Strange, didn't work so far... There's certainly no need for ASI to talk let alone listen to us. Just as US settlers didn't listen to the first nations, rather dictating the terms and reserves they could stay in. And in technical terms they were just marginally ahead... no comparison to any ASI.
    1
  984. Yes, another humiliation! Probably the reason so many still refuse the possibility... brain chauvinism at work! Unfortunately, that decreases our ability to react accordingly, e.g. prioritize solving alignment...
    1
  985.  @raulavila-t5u  Well, we build the new superintelligence, so in the beginning we have lots of control over it. According to orthogonality thesis, its values are independent from its intelligence, so they are basically shaped by us. Of course, they may change after some time, and then we'll be at the mercy of it. But before, alignment is definitely possible. And we should make sure we get it right so that it won't extinguish all of us using the value system shaped by ourselves already. By acceleration, we definitely minimize the chance of survival.
    1
  986.  @raulavila-t5u  thanks for the reply, dad! Discussion with you would be even more fun being less condescending, but i guess that's just your style ;) yes, the orthogonality thesis is just a thesis. However, i was arguing based on it to show that alignment can work under certain circumstances, since you seemed to imply it can never. I still think it shows that, since according to it values are independent, doesn't mean non-existent. Ergo AI has some values, ergo they can entail human preservation. If we manage to install them there, and if AI doesn't change them afterwards, then we're fine. Don't know about you, but i certainly find that a desirable outcome.
    1
  987. Great guest! Hopefully we can manage to spread awareness in time to put these tech bros under some regulations before it's too late.
    1
  988. Would you rather have them not talk about the next developments?
    1
  989. Great then, we'll never have to bother about alignment or instrumental convergence... wait, recent studies show alignment faking already??
    1
  990. Just because there's a movie, doesn't make it true... however, in this instance, probabilities are actually not too great right now ;/
    1
  991. ​ @wdmeister i guess you must be the superintelligence then, if you solve maths problems better than o1.
    1
  992. Wichtiger ist vielleicht: Kann der Kapitalismus auf Dauer funktionieren?
    1
  993. Have you read the recent paper "Alignment faking in large language models"? Seems you're not up to date any more ;)
    1