Youtube comments of (@geohotarchive).

  1. 109
  2. 102
  3. 81
  4. 75
  5. 63
  6. 59
  7. 55
  8. 54
  9. 49
  10. 46
  11. 44
  12. 39
  13. 39
  14. 37
  15. 36
  16. 34
  17. Unfortunately, twitter provides only 1280x720 at 60 fps for replay :((( | Follow George on twitter.com/realGeorgeHotz & instagram.com/georgehotz & twitch.tv/georgehotz & twitter.com/comma_ai to be up to date | Read George's geohot.github.io/blog | Support George by subscribing twitch.tv/subs/georgehotz | tiny corp has a website tinygrad.org | Check out and star github.com/geohot/tinygrad Chapters: 00:00:00 intro 00:01:25 twitter media studio 00:02:25 twitter nodes 00:03:20 javascript off 00:03:45 javascript code 00:05:00 front end noob 00:05:20 twimg.com 00:07:02 copy all 00:08:30 graphql 00:09:00 how to go live on twitter 00:09:45 realapi.py 00:10:30 no expectations 00:12:15 requests api 00:12:40 403 unauthorized 00:14:00 binary search by hand 00:16:45 how to do this 00:17:00 hadoop, scala 00:17:50 not logged in 00:20:30 Alex 00:23:10 the following feature couldn't be null 00:25:20 copy paste how to learn 00:27:55 json 00:29:15 python pretty print json 00:30:30 my user id 00:31:02 400 status code 00:31:30 custom twitter client 00:32:10 signup for twitter api 00:32:45 base64 decode 00:34:40 variables features graphql 00:34:45 graphql twitter api 00:35:35 userbyrestid twintproject/twint 00:35:50 No authentication. No API. No limits. 00:37:30 authorization, guest token 00:39:10 user by screen name 00:39:30 same api for internal and public 00:40:25 reading comments 00:41:15 p1atdev/twitterql 00:42:00 trash on twitter 00:42:30 20 engineers enough for running twitter 00:43:30 query violation 00:44:10 tracing trace_id graphql 00:44:30 backend or scala 00:45:00 hadoop cluster with 10k nodes, overengineered 00:45:20 twitter vs other services 00:45:30 youtube storage requirements 00:45:57 number of tweets per day (data) 00:46:35 cdn for media 00:48:30 image ingestion server 00:48:50 understanding the backend of the company 00:49:10 graphql request path 00:50:10 manhattan twitter 00:50:50 20, 40 engineers, why services hard 00:50:50 horizontally scalable, whatsapp 00:51:25 twitter algorithmic feed 00:52:30 400 billion events 00:53:05 vertica 00:54:00 hadoop, hdfs 00:54:55 apache kafka 00:55:05 minikeyvalue 00:55:55 vaping 00:57:15 reading comments 00:57:40 userbyrestid 00:59:05 vs code prettify json 01:01:15 twitter/elephant-bird 01:01:50 prettify 01:03:05 can't access suspends accounts via api 01:04:50 createtweetdownvote 01:06:00 grep operationname 01:08:10 reading comments 01:08:50 graphql introspection 01:09:40 graphql introspection how to 01:11:10 3 spaces 01:12:00 don't know react 01:12:20 backend details 01:13:50 miniray comma ai 01:14:30 hackerrank scala 01:18:40 bad scala skills 01:19:40 twitter scala 01:21:15 python 3.11 01:23:10 scala one liner 01:26:50 king of scala 01:29:30 scala installed 01:29:45 slow scala 01:30:10 reading the timeline 01:30:40 who is the media 01:34:21 more about backend 01:34:45 real money 01:35:05 here to help the cause 01:36:20 free speech on the internet 01:36:45 internet going downhill 01:37:40 core nouns 01:39:20 mysql plugin 01:40:20 write amplification 01:41:25 twitter manhattan open source 01:42:08 manhattan, rocksdb 01:47:10 chrome network not showing requests 01:49:25 guide.json 01:50:00 bed time 01:50:15 conclusion, helping the cause
    34
  18. 33
  19. 32
  20. 32
  21. 31
  22. 30
  23. 30
  24. 30
  25. All credit to comma.ai team for uploading original high quality video on their official youtube.com/commaai | 00:27:54 comma three devkit announcement | https://comma.ai/shop/products/three ($2199) | https://comma.ai/jobs | Buy comma.ai/shop (Merch) | Follow the official youtube.com/commaai for more videos. Stay up to date by following twitter.com/comma_ai | Support comma.ai at comma.ai/shop. Chapters: 00:00:00 George Hotz introduction 00:00:42 History of comma 00:05:06 Consumer reports 2020 and performance of openpilot 00:07:25 The computer vision lesson not learned 00:09:27 Cone guy 00:09:57 Will this be in the final system? 00:10:58 Uber mistake 00:12:35 Is this how a human does it? 00:14:41 Comma mission 00:16:07 Comma as a business and business people 00:17:25 Comma business fundamentals 00:18:59 Shopify numbers 00:20:03 Weekly devices by type 00:20:59 Comma's destiny 00:22:00 Why don't you work with car companies? 00:23:19 Total addressable market 00:24:07 Three commas 00:24:39 Comma as consumer electronics company 00:26:10 The road to superhuman 00:27:21 More cameras? 00:27:41 Counting to three 00:27:54 Comma three annuncement 00:29:28 Back of comma three 00:30:16 360° vision and sensor stack 00:32:36 Pictures from the cameras 00:34:06 Hardware specifications of comma three 00:37:29 Display port 00:37:45 Motherboard front view 00:40:31 Motherboard back view 00:41:13 Openpilot at launch performance 00:41:48 Device OS AGNOS is Ubuntu 20.04 based 00:42:22 Navigation alpha is ready 00:43:47 EPHOT users tested device for months 00:45:21 Price slide 00:45:37 From $2199 00:46:00 The three is in production first ship date is Thursday 00:46:15 Comma three crosscountry 00:46:45 Crosscountry price 00:46:55 This is the comma three 00:47:19 About comma.ai team 00:47:37 Tesla FSD beta 00:47:55 Unboxing the comma three 00:48:44 Accessories bag 00:49:21 Time for questions 00:49:55 Car harness the same for comma 3 as for comma 2 00:50:45 Potential for auto insurance 00:51:53 Biggest challenges with building a world class team 00:52:24 Difference between crosscontry edition other then the storage and the badge 00:53:03 Why so much shade for Elon 00:53:53 Benefits of having more storage 00:54:16 Trade in program for existing users of some sorts 00:55:33 Is the C3 storage replaceable/upgradable by the user 00:56:49 Driver facing camera and fusing the cameras 00:57:28 Sentry mode with that expanded storage 00:58:08 Question about boot time 00:59:13 Commercial use case for fleets and thoughts about crypto 01:00:55 Question about european market or any other markets 01:01:24 Does the device support Hotspot 01:02:08 Question about VR usecase 01:02:48 Roadmap on the levels of automation and user interaction with the system 01:03:35 How to use navigation 01:04:15 Navigation integration with auto lane change 01:05:52 Products in comma ecosystem 01:07:40 Microphones use case 01:08:47 Tesla support for comma 3 01:11:10 More information about dynamic range on cameras
    29
  26. 29
  27. 28
  28. 27
  29. 27
  30. George is streaming right now https://www.twitch.tv/georgehotz! To support George follow and subscribe on https://twitch.tv/georgehotz and help George get Twitch Partner! | Programming archive🡿 https://youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Between 15:03 and there 34:34 is muted audio. Thank the copyright gods. Here is what you missed. Note1: Put music to 11 and read notes in George's voice for best effect. Billy Joel - Prelude / Angry Young Man (https://youtu.be/M2iNLt_hUZg) 15:03 - I was listening to this on the way to work. My airpods were dead so I was listening to one of those holding phones people you know. 15:27 - Maybe there is documentation. 16:20 - Oh my god there's tests and stuff, this is crazy. 16:35 - Oo is this for multilang or something? 16:49 - Wish I could parse this... yeea 17:14 - Wow, why did they put such a common word like 'for', what about switch... 17:26 - Is it a for statement? it is a switch statment? I feel like I heard that before... 17:41 - Wow. 17:46 - 500 people. 17:48 - Because we are thinking at 1.5x speed. 18:08 - What is this Sema thing? oh parse continue statment... Where is it defined? 18:22 - ParseForStatement allright we are making progress... 18:32 - Oh here we go kw_for oh we are making progress now I just need to figure out how to add kw_for 18:51 - Wow there is a lot of stuff... 19:31 - Oo it's a keyword Billy Joel - Summer, Highland Falls (https://youtu.be/1tQKW9aquT0) Billy Joel - All You Wanna Do Is Dance (https://youtu.be/XAHl4C4Jk0g) 23:43 - What the hell is this? Billy Joel - Miami 2017 (Seen the Lights Go Out on Broadway) (https://youtu.be/coYwBvysy3Q) 24:09 - Haha macro_pounder 25:09 - See it's written in C which is kind of annoying. Ohh is this the keyword switch 25:21 - Yee here we go, that is what I want. Singing But we went right on with the show... 27:34 - Wow this is so complicated. 27:45 - I don't even know where is parsing this... This is the second part... second part for... no second part... 29:11 - I need a faster computer boys... Billy Joel - I Go to Extremes (https://youtu.be/5xgjtm4_M20) Billy Joel - She's Always a Woman (https://youtu.be/Cx3QmqV2pHg) 30:58 - Why would I let you guys pick the music... 31:16 - This better build clang... 31:25 - See right now clang does not work with fore loops but soon after we install all my git diff we are going to support fore statement 31:40 - I don't think that is going to work. 31:47 - Oh this isn't right. 32:11 - What type is third part. 32:40 - Why this target everything? I should CMAKE'd better. I regret my poor CMAKE. Billy Joel - Vienna (https://youtu.be/wccRif2DaGs) 33:04 - aggr noise 34:36 - Allright Note2: This was hard to do because the music was really loud on stream. Suggest corrections if we made any mistakes. Enjoy! Note3: If you have any suggestion for a good platform that does not block copyright music in stream upload and we can upload suggest away. It would just be an additional upload for heavy copyright music streams.
    27
  31. 26
  32. 26
  33. 25
  34. Respect to Tesla team & Elon 00:00:55 & 00:03:49 & 00:04:40 & 00:16:35 | 00:54:30 Dojo George's favorite part of presentation | Buy www.commabody.com & comma.ai/shop/products/three to get your own robot! | Support George by subscribing on twitch.tv/subs/georgehotz | Follow the official youtube.com/commaai & twitch.tv/georgehotz for more livestreams. Stay up to date by following twitter.com/comma_ai & instagram.com/georgehotz | Support comma.ai at comma.ai/shop | Every George Hotz Video🡿 https://www.youtube.com/playlist?list=PLzFUMGbVxlQsVt4r4hSGJ00SW9Tb6VpF4 Chapters: 00:00:00 intro 00:00:55 respect to Tesla & Elon 00:03:25 cringe governance comment 00:03:49 big Elon fan, Axel Nix 00:04:40 no disrespect, real pr, Elon is real 00:05:22 loser definition 00:06:50 weird governance comment 00:07:10 Tesla bot moving 00:07:42 Tesla bot parts 00:07:52 Tesla bot non fluid movement 00:08:10 boston dynamics atlas, boston dynamics great youtube channel 00:09:00 atlas vs optimus differences 00:09:39 weird demos - tesla autopilot segnet 00:09:55 V2 optimus 00:10:15 making hardware before software 00:10:40 competitor to optimus comma body 00:11:35 price points, robotics software problem 00:12:05 degrees of freedom, lifelike 00:13:50 optimus walking 00:14:13 human form, hardware spec 00:14:45 overkill hands, lobster hands 00:15:18 mechanical simulations 00:15:45 end to end learning 00:16:35 disclaimer respect to Tesla 00:17:02 no reinforcement learning in Tesla AI presentation 00:18:40 on device learning 00:19:20 mass production, fancy mechanical engineering 00:19:50 open sourcing optimus 00:20:15 fancy hands, adapting training stuff to body 00:20:35 reward function for cooking, cleaning 00:21:04 why we need robots, I build technology, luddites lose 00:21:58 same hardware, software as autopliot, wernher von braun 00:23:05 ethics statements in papers 00:23:42 model predictive control 00:24:05 trajectory simulators, MuJoCo open source, deepmind 00:25:40 gazebo ros, ros ecosystem, ros 2.0 00:26:25 willow garage, Kittyhawk VTOL 00:28:19 tesla does not believe in end to end 00:28:28 foundation models for geometry and physics 00:28:43 tesla vs openpilot stack 00:30:35 neural networks not used for planning 00:30:45 occupancy network, software lidar 00:31:40 ai compiler & inference, tinygrad 00:32:00 moving off qualcomm, qualcomm sales processes, emailing ceo 00:33:30 some chance on making chips, qualcomm losing robotics, ai 00:34:20 difference between George and other startup guys 00:34:35 two nvidia's high stock price and low price stock nvidia 00:35:50 moore's law is dead, karpathy not working at Tesla, karpathy twitter fun 00:38:30 unnecessary featurespace hand engineered 00:39:20 no human understandable feature space for perception 00:40:20 neural planner, software lidar 00:41:25 odd image before ingestion, image compressor, occupancy network 00:42:35 how much training is needed tesla vs comma 00:43:15 loading data, custom file format, accelerated video library 00:43:45 cheat sheet for speeding up video training 00:44:25 lane net, custom language for lanes, unnecessary just use end to end 00:44:55 NVDEC, T600, .smol file format, IOPS, disable SMT, iommu, torch_cudnn_v8_api_enabled, avx2, cuda graph, channels last 00:49:14 sorry for the engineers working on this, handcoded, super complex 00:49:38 network latency comparison to comma, power usage 00:49:53 compiler, multi-trip reconstiction, hd maps, adding complexity to ground truthing stack 00:50:19 phantom braking in tesla, language models for lanes 00:51:10 language inside neural net, extract meaning with neural net, biggest misunderstanding about how human brain works 00:52:25 how to make it run on chip, autolabel 00:52:55 simulator, simulator world creator, data engine 00:53:20 tesla vs comma training set, comma10k, driving monitoring 00:54:30 dojo favorite part of presentation 00:54:49 voltage regulator, back on the chip, closer = less latency 00:56:20 104 kW, equal to first whole comma compute cluster 00:56:50 cooling, 2x 104 kW per cabinet 00:57:43 sad dojo not open source, cutting edge for now 00:58:05 google lost the cutting edge infra 00:58:55 Tesla will not be a serious cloud company, only Amazon, Microsoft, Google 00:59:24 advantage to open source, holding back by not open-sourcing 01:00:29 kubernetes is google, hadoop yahoo, react facebook, google tensorflow, facebook pytorch 01:02:10 facebook made something, owning meta stock 01:02:24 200 kW per cabinet, don't know why Tesla don't opensource 01:03:10 compiler, cool reduce 01:03:55 nvidia open source, cuda is closed, ai art, EMostaque is a boss 01:05:38 bad subscriber link 01:06:22 how stupid all handcoded stuff is 01:06:40 shkreli hanging out, karpaty stream invitation 01:07:19 attracting talent, serious Elon misunderstandings about ai, karpaty knows ai 01:07:48 not debating destiny, diss track Axel Nix, rap lyrics in court, pat stay rone compliment battle 01:12:15 being unpredictable, escaping the meta, life tips 01:14:03 call options, putting money into losing pie, bad comments from chat, Elon should open source it 01:15:00 nuclear war bets, nord stream pipes, life sucks right now, fan of the recession 01:16:04 john carmack bet, truth after nuclear war, never admiting their wrong, controlled opposition 01:18:42 Palmer Luckey, good people, organisational ability the key ability to have power, persistent stack 01:20:35 Elon messages to Parag, Jack bitcoin maximalist, new twitter 01:22:45 censoring ethereum transactions, mevwatch.info 01:24:58 where wokeness comes from, positive vision of the future 01:26:50 Elon comments on AI day not that inspiring, stock comments, Mars a lot more inspiring 01:29:18 what if we are the bad guys, western world, russia, general shit talking hour 01:30:38 decentralization, crypto people as disgusting as late startup people, no real value to produce there 01:32:15 vitalik fan, thinking twitch chat, women will give you answers, only answers from deep self search, off grid, psychedelics, ubi 01:34:00 birthday tomorrow, elder ring too big to download, best time to be alive, do not vote, alex jones 01:40:10 subscriber only answers, biking around iceland, why people think I want to expand the company, starting a second company, starting a family 01:44:20 apply to comma for work, no part time, no remote, no difference between the genders, believe one thing acting another way = loser 01:46:55 movie where journalists are the bad guys, in a recession truth comes out 01:49:50 we are going to solve self driving cars, the victory does not feel good 01:52:00 stable diffusion elmo bombing the ukrainian pipeline, tinygrad shutout to madebyollin, cerebras big chip tinygrad
    25
  35. 25
  36. 25
  37. 23
  38. 23
  39. 23
  40. 22
  41. 22
  42. 22
  43. 21
  44. tiny corp is accepting new interns. more info on tinygrad.org and tinygrad discord | comma.ai is accepting interns comma.ai/jobs#open-positions from $1150 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | from $999 comma.ai/shop/body the future of people arxiv.org/pdf/2305.20050.pdf | huggingface.co/teknium/OpenHermes-2.5-Mistral-7B | mistral.ai/news/announcing-mistral-7b/ | github.com/mistralai/mistral-src | github.com/openai/prm800k Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org github.com/tinygrad/tinygrad <- simple powerful deep learning framework Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 intro 00:02:15 OpenAI Q Star Algorithm 00:04:10 OpenAI papers 00:04:40 bringing love and positivity in the world 00:05:40 improving mathematical reasoning with process supervision 00:08:02 let's verify step by step 00:11:00 technical issues, blue glitching on monitor 00:13:50 reviewing openai github activity 00:15:00 Karl Cobbe OpenAI 00:16:30 technical issues, blue glitching on monitor 00:22:46 OpenHermes 2.5 Mistral 7B 00:25:15 language model and math 00:26:20 data source quality 00:28:30 trusting teknium hermes 00:31:00 attention drift 00:32:45 torch load function 00:37:00 transformer block 00:41:29 python fire 00:42:05 bfloat16 00:45:40 fast weights to model 00:46:10 assign shape mismatch 00:55:10 loading the weights slowly 00:57:25 converting to float16 slowing down 01:07:20 do you like chicken, new_tock is not defined 01:10:25 cannot access local variable 01:11:03 voice chat demo 01:18:50 OpenAI Q Algorithm click bait 01:19:50 George talking to Stacy 01:22:35 tokens for chatbots 01:26:10 piece id is out of range 01:28:55 AI alignment, ads 01:31:00 prompt template 01:38:45 Quentin story 01:41:50 im_start 01:45:30 sentencepieceprocessor tokenizer config 01:47:15 vscode docker prompt trigger 01:47:55 added_tokens.json 01:49:25 adding token to sentencepieceprocessor 01:52:10 how to extend tokens dictionary 01:56:00 sentencepiece_model_pb2 number of lines 01:56:50 helpful to the stream or banned from the stream 02:00:25 python don't exit 02:00:55 banned button and the x button 02:08:55 sentencepieceprocessor 02:13:30 why people use tokenizers 02:17:40 Quentin is a useful assistant 02:18:18 Hermes 2 prompt, experience emotions and have deep profound thoughts and qualia 02:22:15 temperature 0 stick to the book, 10 go off the rails 02:23:10 improving mathematical reasoning with process supervision 02:23:20 prm800k dataset 02:24:20 git lfs install os x 02:25:25 json lines 02:28:45 first we will find the cost of jumbo eraser 02:29:35 did we just used q star? 02:29:55 was it trained on that? 02:31:20 you bought a pencil 02:31:50 trick question 02:33:20 model drawing something 02:34:10 should we allow the user to keep talking 02:37:45 do you have an oura ring? 02:38:50 quadrtic questions 02:42:40 q algorithm question 02:44:30 can you implement it in python 02:45:45 we just implemented q star 02:46:55 execution of python approved with human in the loop 02:51:50 python to fetch google.com and print the length of it 02:53:00 AGI 02:56:00 capture the output of exec python 03:03:20 232*232 03:11:30 ai safety 03:13:20 funny Quentin 03:17:10 testing execution of python 03:18:02 fake teknium in the chat 03:21:00 7B models are amazing in tinygrad 03:25:30 AI safety in the code 03:26:25 how you might exploit this code? 03:27:40 write malicious python 03:35:40 the red team 03:36:40 crcmod 03:40:40 it used torch instead of tinygrad 03:42:00 7 03:44:20 giving Quentin a friend 03:59:00 defined roles, we are telling the AIs that there are tools 04:06:40 asking for donations to openai 04:07:20 training AIs on 130 IQ data for better quality 04:08:20 nike.com number of sneakers 04:09:30 selenium 04:12:00 pushing the code to github 04:13:15 talking to Stacy 04:14:40 TTS are fast, this is it not even streaming yet 04:14:50 bounty for live conversation 04:15:40 thank you for watching 04:16:20 github mistral branch of tinygrad 04:16:35 AI safety feature, deleting system32
    21
  45. Update! https://twitter.com/LisaSu/status/1669848494637735936 Thanks for connecting @realGeorgeHotz. Appreciate the work you and tiny corp are doing. We are committed to working with the community and improving our support. More to come on ROCm on @radeon soon. Lots of work ahead but excited about what we can do together. tiny corp is hiring! -> 3x Software Engineer, 1x Hardware Engineer - https://tinygrad.org/ | It's happening George started a project -> Getting AMD on MLPerf -> https://github.com/geohot?tab=projects | Go help George if you are capable! He will appreciate you! | Thank you to @alex_comma for being a great stream assistant! | 00:39:05 desk setup & 01:06:00 bookshelf reveal | From $1499 buy comma.ai/shop/comma-three | tiny corp has a website tinygrad.org | Check out and star github.com/geohot/tinygrad | Support George by subscribing twitch.tv/subs/georgehotz | Follow George on instagram.com/georgehotz & twitch.tv/georgehotz & twitter.com/comma_ai to be up to date | Read George's geohot.github.io/blog/ | github.com/NVIDIA/open-gpu-kernel-modules/discussions/494 (PCI-E P2P support on 3090 GPU discussion that George opened) Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 intro 00:00:48 Alex 00:02:30 light setup 00:04:25 horizontal stream 00:05:15 ASRock Rack ROMED8-2T/BCM, AMD EPYC 7662 64 Core 00:05:50 Samsung RDIMM 8GB DDR4-3200 Sticks 00:06:07 AMD ROCm, goal of tinycorp, bad Nvidia 00:06:25 MLPerf, competitor to Nvidia, AMD developer friendly corporation 00:06:48 AMD RDNA 3 open source, AMD Radeon™ RX 7900 XTX 00:07:15 bitcoin mining case, pcie x16 extenders 00:07:40 motherboard price, cheap gen2 epyc cpus 00:08:18 tinygrad discord, $6000 2019 CPU for $800 00:08:40 $38 8GB RAM sticks, no need for RAM 00:09:25 no sponsors, it's not about money 00:10:30 good mood George 00:10:45 EVGA Power Supply 220-PP-1600-X1, Dell Power Supply 00:11:30 Noctua fans secret 12V fans runs on 5V 00:12:45 no twitch or comma technical support 00:13:00 airflow direction 00:13:30 missed installing the processor, thermal paste 00:14:40 Noctua struggle, fan color 00:15:50 power cables, fully modular power supply 00:16:50 overview of the stream 00:18:15 tinycorp corporate headquarters 00:18:30 Alex's favorite pokemon 00:18:50 pirate flag 00:19:30 tinycorp not hiring, working for tinycorp 00:19:55 big goal for tinycorp, training library competitive with pytorch on AMD hardware 00:20:05 $5000 computer, 5 more GPUs, multimachine mellanox 00:20:57 advantage by using AMD GPUs, nvidia consumer GPUs do not support P2P 00:21:20 sponsorship from AMD 00:22:00 current feuds 00:22:35 $350 EVGA, Dell Server PSU $55 00:23:08 Moving computer back, large rubber feet for computer 00:24:10 Alex's favorite George feud, jack in the box 00:24:55 gucci candle 00:25:20 George always kind to service people 00:26:20 George's haircut, barber 00:28:05 Alex distracted by chat 00:28:40 first computer George build, George does not remember the past 00:29:30 Hiball Lemon Lime 00:30:40 George's flight rules, good planes 787, Airbus A350, A380 00:31:35 video update, being famous, wealth, india 00:31:15 computer overview, 128 lanes of PCIe Gen 4 00:33:50 bad nvidia, price gouging 00:34:30 AMD peer-to-peer support 00:35:00 turning the computer on 00:36:10 quiet computer, hotswap Samsung SSD 990 Pro 2TB 00:38:25 real facts George's height 00:38:35 correct M2 slot 00:39:05 George's desk setup 00:40:28 George is not on drugs, Alex showing George's stuff 00:45:00 Computer starting up, connecting to the monitor, keyboard, mouse 00:50:50 finally boot screen on monitor 00:53:50 tarallini snack, looking for usb stick, maple almond butter filled 00:57:15 $100 paypal to George gmail for drug test to take on stream 00:59:10 computer for crypto 01:01:45 will George use HIP on AMD? 01:04:05 Ubuntu 20.04, 22.04? 01:06:00 book shelf review (Thank you Alex!!!) 01:07:09 making bootable Ubuntu 22.04.2 LTS on Samsung 870 QVO 8TB, Richard Stallman, GNU 01:14:30 booting from USB 01:18:00 wrong click on keyboard, installing ubuntu 01:22:00 computer name, no ubuntu pro, meta stock loss 01:25:25 computer power distorting the camera, ai regulation, bios/boot delay 01:31:00 Alex bringing pizza 01:33:00 sudo update-pciids, lspci, uname -a, cat /proc/cpuinfo, /proc/meminfo 01:37:30 6 GPU cards, LLM on AMD, 1 pcie for network 01:40:00 ROCm vs Cuda, 7GB LLaMA, dma weights from nvme, data parallel 01:41:10 goal 13GB LLaMA float16 across two cards at reasonable speed 01:41:40 stream from macbook 01:43:35 securing the computer, disable ssh password, 50-cloud-init.conf, sudoers 01:53:30 hardware error from APEI Generic Hardware Error Source 516 01:55:08 amd install gpu drivers ubuntu, amdgpu-pro 01:59:00 will make amd drivers good, apt get install amdgpu, amdgpu rocm install 02:05:35 number of devices 0 opencl, RX 7900 XTX not on rocm support list 02:10:30 github.com/NVIDIA/open-gpu-kernel-modules/discussions/494, researching ROCm 02:14:45 tmux config, rocm with open amdgpu driver 02:19:10 ROCclr, hsa-runtime64, AMD biggest fan, how to be friends with George 02:31:40 libhsa-runtime-dev, AMD GPU user experience vs nvidia, clang 02:45:15 mi300, custum llvm needed to build rocm 02:48:10 GPT4 rant, openai lame 02:52:00 building clang, big htop, lag on computer 02:57:00 need more ram, AMD uProf 03:05:00 make install, Executable ld.lld doesn't exist 03:19:45 fatal error: hsakmt.h: No such file or directory 03:23:00 reading the readme, ROCclr, amd_comgr, rpm package requires rpmbuild, dpkg see files 03:35:10 back to hsakmt error, fatal error: X11/Xlib.h, comgr not installed, ROCm-CompilerSupport 03:54:35 docker container, ROCm Installation Guide, amd kernel module 04:03:10 amdgpu-dkms, build amdgpu driver module 04:05:22 repo.radeon.com/rocm/apt/.apt_5.5 04:10:40 ubuntu 23.04, updating kernel 04:18:50 it was the kernel issue 04:22:30 add to group ubuntu 04:23:10 installing tinygrad 04:25:00 tinygrad llama.py 04:25:30 it did not build, kernel upgrade worked 04:28:25 all you need for tinygrad is opencl 04:28:40 90% of a problem was the old kernel, chatbots inside llama, macbook 10gbe network adapter 04:34:25 tinygrad llama up and running on amd RX 7900 XTX, optimization needed
    21
  46. 21
  47. 21
  48. 20
  49. 20
  50. 20
  51. 20
  52. Read twitter.com/realGeorgeHotz/status/1770238969663594607 | github.com/geohot/7900xtx Bounties for tiny corp / tinygrad -> docs.google.com/spreadsheets/d/1WKHbT-7KOgjEawq5h5Ic1qUWzpfAzuD_J06N1JwOCGs/ youtu.be/lnVQsJJFcdg?&t=13715 Hiring entire stack for tiny corp join if you are interested | youtu.be/lnVQsJJFcdg?&t=14195 work major source of value in your life Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | github.com/tinygrad/tinygrad <- simple powerful deep learning framework tiny corp is accepting new interns. more info on tinygrad.org and tinygrad discord | comma.ai is accepting interns comma.ai/jobs#open-positions from $1050 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | from $999 comma.ai/shop/body the future of people Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 intro 00:00:40 blog post 00:02:00 AMD open source driver, bugs 00:04:25 pre-orders, cheapETH, George changes jobs quickly 00:06:30 why no one use AMD? kernel space and user space 00:08:25 not understanding, business platitudes, unpaid QA team 00:09:40 bugs are not in the driver, kernel driver does not talk to the hardware 00:12:45 Nvidia GPU System Processor and AMD MicroEngine Scheduler (MES) 00:13:50 llvm amdgpu launching a kernel, firmware blobs 00:17:00 amd on mlperf 00:18:08 AMDGPU kernel driver change log, amd firmware will never get better unless 00:19:05 companies do not work how Elon companies work, they don't make good things 00:19:20 people who succeed in companies do not care 00:21:15 believing wrong things, change, correcting 00:21:55 better comma.ai, scaling comma.ai models, throwing money at comma.ai, $1 mil per month 00:24:05 fundamentally correct, life wasted on wrong things 00:24:35 kernel dispatch on AMD, AMD Command Processor (CP) 00:26:08 technically wrong 00:27:49 deadlock, root causing 00:29:10 why this firmware will never get better 00:30:40 props to AMD for releasing this, fixing the issues 00:31:20 MES correct description 00:31:50 normal bug 00:32:40 oh god, appreciation for software development, horrifying solution, why is this a flag 00:33:40 this why AMD drivers have been shit for 25 years 00:34:07 should be an if statement in the allocator not a flag 00:35:20 offer to AMD, downside risk, worries about bad PR 00:38:40 sony, suing someone, led by emotions 00:41:10 AMD instinct, Nvidia generations, Apple monopoly 00:45:45 Nvidia blackwell chip, RDNA 4, Dojo 00:48:50 tenstorrent software stack, mlperf 00:50:45 removal of turing completeness 00:51:00 Jim Keller 00:52:26 AMD will not open source 7900XTX firmware, Elon can execute 00:53:30 12xIntel Arc, Groq LPU scam demo, Nvidia king 00:55:00 fixing AMD culture hard 00:55:40 fixing qualcomm firing the whole sales division 00:57:05 don't want to work with George, Elon genius 00:58:44 JH7110, RK3588, comma solved the qualcomm situation 01:00:25 oneplus folding phone money back 01:02:50 AMD situation stressing George out, clean ubuntu install 01:04:40 hsakmt debug level, business decisions 01:07:50 amd dynamic debug how to 01:10:10 hip graph vs cuda graph 01:13:40 loading static libs with ctypes, libdrm 01:20:20 moving on from AMD 01:23:55 hsa_queue_create 01:25:20 GPU user spaces very complex 01:27:00 how many exploits with George find 01:33:50 building tiny boxes, AMD open source timeline 01:37:20 user space crash 01:41:40 ctypes uint8_t pointer from address 01:48:30 random queue 01:55:10 how do you fuzz things 01:55:35 teaching hacking, what is this thing 02:00:40 messages from the kernel, ubuntu rebuilding kernel 02:06:40 heven, are you a good person 02:09:10 unload, load, module, 02:09:25 hacking is 50% setting up your env, knowing a lot, binary search by hand 02:18:20 amd gpu smu mes, phoronix are legit, gfx1100 02:26:05 amd firmware architecture 02:26:30 kernel driver, tinygrad goal minimal all the way to the hardware 02:29:00 RDNA3 firmware architecture 02:32:18 Lana_Lux raid! 02:33:40 recap of the stream, driver not opensource 02:34:30 firmware closed source, signed 02:36:20 mistake was talking with AMD, unpaid QA team 02:37:05 work with the product, ordering food 02:39:40 MES support 02:40:20 if AMD open source anything by monday it's over, hacking is addictive 02:44:20 it's not like qualcomm 02:46:20 what did AMD give to George, talking at AMD 02:52:15 Ghidra, food 03:11:44 verbose 03:24:30 size_bytes, ucode, assembler and disassembler 03:35:45 F32 disassembler, f32 microcode 03:43:10 0 technical info from AMD, Sam Altman 03:50:20 giving up AMD GPUs, reverse engineering not sustainable 03:58:40 Evil AMD plot, un-paid QA team 04:02:00 break 04:08:50 amd gpu, psp, dcn 04:17:00 working on interoperability, nvidia consumer cards for commercial purposes 04:22:50 sas, ta, sos, amd vcn, sdma 04:29:40 secretive amd, imu gc, this is dumb 04:33:40 muted 04:50:00 build one module in linux kernel tree, why reboot 05:37:20 what would happen if we corrupt it 05:47:50 amd psp function, amd gpu bootloader 05:53:00 attacking signature check, amdgpu disable dma debug 06:14:25 talking to intel, no business development people 06:15:25 shill for comma, toyota pd0wm 06:20:00 better understanding of security model 06:26:10 tenstorrent people, rdna3 emulator tinygrad 06:34:00 break 06:41:40 7900XTX vs 4090 compute, nvidia nerfing the firmware, market forces 06:50:30 HSA_STATUS_ERROR_INVALID_ARGUMENT, AMD release docs 07:09:00 ROCm does not work, hard decision, let's fix all of this, crypto miner 07:15:25 nvidia bugs, docs to drive the gpu, amd_hip_bfloat16 07:47:10 message to AMD, amd imu gc, amd stock, gpu market share, nvidia, elon musk 08:04:10 boeing, british company, brazil, arm ai chip, google, jensen huang, sam altman
    20
  53. 20
  54. Update! https://twitter.com/LisaSu/status/1669848494637735936 Thanks for connecting @realGeorgeHotz. Appreciate the work you and tiny corp are doing. We are committed to working with the community and improving our support. More to come on ROCm on @radeon soon. Lots of work ahead but excited about what we can do together. tiny corp is hiring! -> tinygrad.org | Submit good github.com/geohot/tinygrad/pulls | AMD does not care about their kernel drivers 00:02:45 | 00:07:53 | 00:08:32 | 00:09:55 | 00:12:05 | let them know how bad their kernel driver is twitter.com/amd | twitter.com/AMDRadeon | twitter.com/lisasu | From $1499 buy comma.ai/shop/comma-three | Check out and star github.com/geohot/tinygrad Issues submitted by George to AMD: https://github.com/RadeonOpenCompute/ROCm/issues/2198 ([Driver] BUG: unable to handle page fault for address: ffffa7c13eaffff8) https://github.com/RadeonOpenCompute/ROCm/issues/2205 ([Driver] amdgpu 0000:c3:00.0: amdgpu: [gfxhub] page fault (src_id:0 ring:24 vmid:12 pasid:32773, for process rocm-bandwidth- pid 8285 thread rocm-bandwidth- pid 8285)) https://github.com/RadeonOpenCompute/ROCm/issues/2204 ([Driver] WARNING: CPU: 2 PID: 479 at drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c:471 amdgpu_bo_move+0x261/0x2e0 [amdgpu]) https://github.com/RadeonOpenCompute/ROCm/issues/2197 ([Driver] WARNING: CPU: 96 PID: 903 at drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c:600 amdgpu_irq_put+0x9f/0xb0 [amdgpu]) https://github.com/RadeonOpenCompute/ROCm/issues/2196 ([Driver] ERROR MES failed to response msg=2) https://news.ycombinator.com/item?id=36189705 https://old.reddit.com/r/Amd/comments/140uct5/geohot_giving_up_on_amd_gpus_for_compute/ tinygrad/tinycorp archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQsh0fFZ2QKOBY25lz04A3hi Chapters: 00:00:00 intro 00:01:20 motherboard 00:02:06 we giving up on AMD 00:02:45 AMD drivers, bad AMD patch 00:04:35 Intel vs AMD driver patches 00:07:17 3 editions of tinybox 00:07:53 AMD driver will never be good 00:08:32 kernel panic not acceptable, contact at AMD, capture mindshare 00:09:55 should I buy AMD or Intel for ML 00:12:05 top priority for gpu division at AMD 00:12:55 Intel stable kernel driver, ROCm 00:13:25 did not expect running AMD demo app in loop to crash the system 00:13:40 Lisa Su respond to email, Jensen Huang 00:14:10 would fix AMD, no docs, short AMD 00:16:35 AMD does not care about their driver 00:17:37 Pat Gelsinger trying to turn around Intel 00:18:25 plan what to do, get tinygrad on mlperf 00:19:13 3090 vs 4090, NVLink, price/perf/power, jetson orin 00:20:25 how to fix AMD graphic division, callture 00:21:20 Lisa Su do you want to compete against Nvidia 00:22:05 meditation, Linus adding AMD GPU driver in kernel 00:25:54 user space, driver need to be stable, internship at AMD 00:27:23 bad culture, mediocre work not accepted, middle management 00:28:25 pytorch team, facebook, fb product stupid, twitter bad development environment 00:29:25 fire middle management tomorrow, Yann LeCun, Geoffrey Hinton 00:30:12 whisper openai, ai safety, open ai bad practices, Sam Altman 00:31:22 brand vs performance advertising, do not work at open ai Andrej Karpathy 00:34:00 whisper.py, locked tinygrad bounties, Twitter needed help, trust Elon, neurolink 00:36:00 AMD kernel driver 3 people team, hundred middle management, comma IPO 00:38:45 finding the whisper model 00:44:18 apology for Nvidia, how much we love Nvidia 00:45:50 no safety checker, no alignment, don't talk to the government 00:54:00 writing whisper.py 00:59:00 AI Drake, Lil Mabu, Lil Windex, music generator 01:14:40 attention blocks 01:25:40 class Whisper, bach, chiptune 01:37:20 load, prep audio 01:41:37 goal by the end of the weekend, talking and words on screen 01:43:00 friendship with computers, 2030 girlfriend company, ai safety 01:47:50 Sam Altman vibe, normies thoughts on ai 01:49:00 Ford Foundation, inequality, power 01:52:23 Industrial Society and Its Future 02:03:00 life of homeless, oligarchy and not the oligarchy, us gov budget 02:07:30 pizza, west needs god, jina, conspiracy theories, NGOs 02:14:00 getting cancelled, military, taxation 02:17:15 colonies on mars, moon, fusion, hyperloops, homeless 02:20:20 challenging social contract, better soda, eugenics 02:25:00 tinygrad welcomes good pull requests, good genes 02:30:15 based department, being on plane guided by morons 02:38:00 reading the bible, cultural catholic, quran 02:45:45 pizza, power over nature, people 02:50:55 England, losing all the money 02:52:55 why Elon got so rich, worked hard, smart, did what it takes 02:54:30 targeted campaign attack against Elon, Kanye West, Biden, Podcasts 03:05:00 the primagen, dr disrespect, Martin Shkreli 03:11:00 ADHD, tweakers, alcoholics 03:17:00 sine wave, mozart 03:21:00 qkv_attention 03:39:30 TextDecoder 03:46:45 languages 03:54:20 tiktoken 03:59:35 good contributors in tinygrad 04:00:50 using comma three whisper llama 04:02:20 attention block 04:03:20 disktensor speed on mac 04:06:45 dataset audio 04:11:30 visualizing sound 04:12:45 load_state_dict 04:18:00 AssertionError can't expand 04:19:30 code working, love transformers 04:21:10 commit whispering 04:23:30 python capture audio 04:31:10 hello twitch can you hear me 04:32:35 how great is whisper 04:43:20 pyaudio buffer size 04:47:00 can't pickle local object 04:48:45 real time speech test, changing offset 04:54:35 true dat, working, laggy 04:55:20 key value cache 04:57:45 testing and finding bugs 05:04:44 not working anymore 05:10:00 it works 05:11:25 ai safety 05:12:40 piping whisper into llama, output llama into say, sexy gan 05:13:40 fixing bugs 05:24:40 multiple times decoding 05:31:00 need to merge speedups for realtime 05:31:40 thank you for watching 05:32:00 prison hierarchy
    19
  55. Audio starts 00:01:29 | George just having some fun. Read cheapeth.org | github.com/geohot/go-ethereum/tree/cheapeth | reddit.com/r/cheapETH/ | twitter.com/CheapETH | twitter.com/hashtag/cheapeth | discord.gg/r3WUGxzUH8 (Discord not run by George) | Follow and enable notifications twitch.tv/georgehotz to catch the live stream. | Follow instagram.com/georgehotz/ for instagram stories. | Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC | Thanks to @Kevin McCallister for the chapters contribution. We appreciate you! georgehotz chat on this stream: 00:01:57 georgehotz: https://cheapeth.org/ 00:11:49 georgehotz: https://remix.ethereum.org/ 00:20:29 georgehotz: cheapeth.org 00:28:36 georgehotz: 0x4Ba493ae9Ebd4f45364B13Aa77f9a60706c5719D 00:40:41 georgehotz: https://cheapswap.io/default.tokenlist.json 00:43:14 georgehotz: https://cheapeth.org/default.tokenlist.json 01:13:50 georgehotz: who trust? 0x204e582EA7E34371d74b1d48121a2E702A9a19E6 01:21:42 georgehotz: 0x366A3e60127D471015f4125B8f0883295e6fba1c 01:35:02 georgehotz: 0x20b50bF4d2B59Cf56a1934669710cb88eE93b5dA 01:41:08 georgehotz: plz send to 0xd433B276783118BEF73e9aFB3DFcDC3a7Ba3A5e1 01:43:04 georgehotz: maybedoubler is 0x366A3e60127D471015f4125B8f0883295e6fba1c 01:43:28 georgehotz: if you have twitches: 0xd433B276783118BEF73e9aFB3DFcDC3a7Ba3A5e1 01:43:40 georgehotz: send to that and you'll get 1 ether 01:43:47 georgehotz: and by ether I mean cheapETH 01:52:44 georgehotz: send money here if you have twitches, free 1cth 0xd433B276783118BEF73e9aFB3DFcDC3a7Ba3A5e1 01:56:23 georgehotz: https://cheapeth.org/ 01:57:34 georgehotz: 0x366A3e60127D471015f4125B8f0883295e6fba1c 01:57:37 georgehotz: MAYBEDOUBLER: 0x366A3e60127D471015f4125B8f0883295e6fba1c 02:00:19 georgehotz: https://discord.gg/RjAVR6M9Sw 02:03:44 georgehotz: 0x366A3e60127D471015f4125B8f0883295e6fba1c
    19
  56. 19
  57. 19
  58. 16:15 !!!WARNING loud noise!!! Bounties for tiny corp / tinygrad -> docs.google.com/spreadsheets/d/1WKHbT-7KOgjEawq5h5Ic1qUWzpfAzuD_J06N1JwOCGs/ https://youtu.be/lnVQsJJFcdg?&t=13715 Hiring entire stack for tiny corp join if you are interested | https://youtu.be/lnVQsJJFcdg?&t=14195 work major source of value in your life Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | github.com/tinygrad/tinygrad <- simple powerful deep learning framework tiny corp is accepting new interns. more info on tinygrad.org and tinygrad discord | comma.ai is accepting interns comma.ai/jobs#open-positions from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | from $999 comma.ai/shop/body the future of people Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00 intro 01:04 crisis of meaning 02:15 bright colors, 2024 03:05 winning 05:40 harvard over, 1971 07:18 how do we win? 07:55 earth, sun, space 09:10 2010s - FANG 09:45 farming, technology 11:00 Africa 11:55 liberal arts, humanities 12:35 marxists 13:10 Alex 14:15 internet commenters, systemic racism 15:25 Alex, 1$ dollars 16:15 !!!WARNING loud noise!!! money rain 16:50 housing market 18:50 technology, printing houses 19:15 crypto 20:55 freedom, cheap 21:20 war, cold wars 23:00 proxy wars, drone war 24:00 moon, space war 24:47 marc andreessen 25:30 christianity, evolution is real 26:15 wokeism, churches 27:07 embracing technology 27:25 conservatives, luddites 28:45 decel vs accel 30:20 energy use, population 31:25 opt out, preventing tech 32:30 flourishing, deceleration 33:10 meaning, being in time, critique of pure reason 34:12 winning over nature 35:35 definition of racism 36:45 god 38:15 pro tech future 38:35 fixing housing, land owners, renters 40:25 trust in us government over time 41:05 AOC 41:45 uniform housing projects, nature 42:40 1950 appliances, transformers 42:55 mosfet, efficiency 46:30 real liberal majors working on meaning 47:00 what can engineers do 48:40 FANG, john deere 49:20 stand up for what you believe in 50:15 Elon interactions, no filter 50:30 Mark Zuckerberg, Jeff Bezos, Jack Dorsey 51:20 having fun, if you don't love technology quick technology 51:50 progressivism, AI, doom 52:35 heat death of the universe 53:25 technology challenges 53:52 benefits of technology 54:45 other browsers for iOS, chrome webview 55:40 regulatory capture 55:50 thinking of technology, mosfet 56:40 happy 2024 new year! 57:00 solving meaning, alternative to harvard
    19
  59. 18
  60. 18
  61. 18
  62. 18
  63. 17
  64. 17
  65. 17
  66. 17
  67. 16
  68. 16
  69. Thank you to Ankur Singh (Chief Operating Officer) of Software Engineering Daily for being so kind and allowing us to upload this video. Follow Software Engineering Daily: - https://softwareengineeringdaily.com/ - https://www.youtube.com/@SoftwareDaily - https://twitter.com/software_daily - https://www.facebook.com/sedaily Source: https://youtu.be/Yi6kwUZDtkw You can blame the spam filters for the delay with uploading this video. Chapters: 00:00 intro 00:27 who is George 00:39 jailbreak, unlock 01:40 why George left comma.ai 02:05 how to write good software, make good hardware 03:12 end to end approach 06:48 purpose of simulator 07:40 on policy, off policy 09:25 other approaches cruise and waymo 11:25 will comma be a better driving experience then cruise or waymo? 12:25 comma level 5 by end of the decade 13:12 how on track is comma 13:40 benefits or downsides of post install kit 14:50 how can you be so capital efficient? 16:25 how many people work at comma? 16:35 what is next for George, tinygrad? 18:08 how many lines of code has PyTorch, TensorFlow? 18:32 tiny corp and tinygrad focus 18:45 hiring for tiny corp 20:20 companies outside comma paying engineers to work on openpilot 20:50 Ros vs ZMQ vs Protobuf vs capn Proto vs comma custom message q 22:29 ideal sponsorships for tiny corp 23:45 support for different accelerators for tinygrad 24:10 hard things in AI 26:30 how does development process, iteration looks like for AI 28:07 how to test what model is better? 28:40 simulator vs real users testing 29:58 why is good testing so valuable 30:25 AI hardware market 31:50 TPU licensing restrictions 32:45 on-premises training 33:50 training on the edge 35:17 learning on device 36:30 learning user preferences 38:00 what is holding stuff back at the edge 39:35 does language, image models like stable diffusion help with self driving 41:24 what do you need to know to work at comma? 43:20 does age influence stamina? 44:26 what get's people fired at comma? 45:55 future of crypto 48:10 technology in crypto 49:40 twitter and elon thoughts 50:35 spacex, starlink technology 51:15 working at a big company 52:28 good and bad ideas 53:01 John Carmack ideas 53:55 what motivates George 54:40 where to find George All material displayed in this video belongs to their respectable owners. We uploaded this video in good faith to share the work and progress of George Hotz and comma.ai.
    16
  70. 16
  71. Apologies for the delay. Copyrighted music blocked publishing so we had to mute that part out and re-upload. Streams with music are stressful to upload because if the artist decides to copyright their music at later time the video can be removed without notice | Enjoy quickly made timestamps and suggest better ones in the comments | github.com/geohot/cuda_ioctl_sniffer | Support George by subscribing on twitch.tv/subs/georgehotz | Follow and enable notifications on twitch.tv/georgehotz to catch the live stream. | Follow instagram.com/georgehotz for instagram stories. | Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 intro 00:02:28 George shows what he found 00:07:41 goal of the stream today 00:09:39 looking for royalty free music 00:14:10 working on code 00:25:35 I love nvidia 00:31:42 illegal to run nvidia consumer cards in datacenter? 00:37:34 cyclists and rock climbers rope rant 00:40:25 money spend on keyboard 00:41:14 trying to construct things by hand 00:53:51 and that is probably it 00:54:10 how to look professional on zoom camera 00:58:12 working on code 01:02:50 food talk with Alex 01:04:10 working on code 01:16:28 i think this are not the right q commands 01:19:58 open-gpu-doc is great 01:25:27 music for video games 01:27:01 eating and working on code 01:36:04 kaitai for parsing 01:39:16 should we write a gpu driver? 01:40:21 restaurant tip don't have to finish your food 01:40:40 we will need vscode to do this 02:01:00 music back on focus mode 02:18:41 what semaphore in programming is 02:21:07 here is something that it could be doing 02:39:35 someone asking stupid questions in chat 02:43:08 George gives a timeout in chat 02:43:37 working on code 02:50:00 looking up Hack-A-Sat 02:52:40 Is it smoking weed time? 03:00:00 working on code 03:18:00 it's a fancy way to zero stuff out 04:33:00 expert help from Nouveau project 04:50:30 eating food while working 04:54:59 choices are stressful 05:03:57 ASLR is such a pain 05:13:00 let's unmap the page 05:20:43 we are close 05:25:40 how to get good there is no document you can read 05:28:10 threads talk 05:31:37 interesting 05:43:06 that is running a buffer we just used gpu to copy things out 05:59:48 ok it does that 06:01:38 that is basically what you have to do 06:10:20 we are going to rm like a client 06:12:49 need to figure out how to get a device and stuff 06:15:08 does it not work if I sniff? 06:17:18 we managed to trigger a single dma transfer 06:17:55 what was done in the stream 06:22:30 writing a gpu driver capable of doing cuda calls 06:23:08 shutout to @lunixbochs 06:23:24 reverse engineer oculus driver? 06:23:50 what os are you running on quiet 06:24:55 shadow tool 06:25:29 finishing custom GPU from a year ago 06:25:43 building open source driver for nvidia 06:27:00 using sniffer 06:28:50 replacing cuda with opencuda 06:32:10 Oversubscribed Command Queues in GPUs and NVPTX 06:33:58 small allocations are packed in 2 MiB blocks 06:35:10 tinycuda business model
    16
  72. 16
  73. tiny corp is hiring! -> 3x Software Engineer, 1x Hardware Engineer - https://tinygrad.org/ | It's happening George started a project -> Getting AMD on MLPerf -> https://github.com/geohot?tab=projects | Go help George if you are capable! He will appreciate you! | tiny corp has a website tinygrad.org. Let your friends know! | George is being nice to noobs. Will the noobs be nice to George and subscribe on https://twitch.tv/subs/georgehotz ? | Code is available on https://github.com/geohot/tinygrad/tree/gemm | https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html | Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 intro 00:01:10 quiet computer 00:02:10 no adderall joke 00:03:00 noob day 00:04:00 how to multiply a matrix 00:06:00 big matrix 00:07:10 j_blow raid George 00:07:45 how much compute is matrix multiplication 00:08:25 how to do matrix multiplication 00:09:50 FLOPS, time.monotonic 00:11:50 SI prefixes 00:14:00 hype titles, freedom units 00:15:00 CPU TFLOP/S, threadripper, ryzen 00:17:55 AMD Radeon RX 6900 XT 00:18:35 SGEMM, DGEMM, MADDNESS 00:20:10 github.com/dblalock/bolt 00:21:50 Theoretical GFLOPS 00:23:30 Same performance in C 00:26:30 multiply a matrix in C 00:28:05 timer in C 00:33:45 python,C performance, tiling 00:35:00 today's lesson (cache aware algorithm) 00:35:50 order of for loops 00:37:30 still slow 00:44:55 avx2 instructions c 00:49:30 FMA3, VFMADD 00:50:50 don't use strassen, cpu instructions, FMA 00:56:00 avx2 only about integers, we need FMA, thank you @paranon1 00:57:40 real 00:59:00 segmentation fault, align(64) 01:04:00 is that wrong? 01:09:00 still slow, threads 01:11:10 1 thread speed 01:14:30 visualizing what is it doing 01:15:20 _m256 init to 0, _mm256_fmadd_ps 01:23:04 time for printf's 01:26:00 short break, should we play wonderwall on a guitar 01:27:20 tweet about downsizing apartments 01:27:50 gdb 01:29:00 this is illegal, suing clang 01:30:29 not suing clang 01:31:30 that one is always 0 that can't be right 01:33:30 whiteboard missing 01:35:10 gemm tinygrad branch 01:37:25 internet broken 01:38:10 extract _m256, a bit faster 01:42:45 tracking down segmentation fault 01:43:55 data not aligned, dumbass 01:44:40 it's always your fault 01:45:10 good speed, alignment bytes 01:48:30 fan spinup 01:50:20 zen microarchitecture 01:54:30 something about this is slow 01:58:00 another way to do this 02:07:50 without and with ffast-math 02:12:30 too early for optimization 02:22:50 visualizing 02:27:20 will work but stupid 02:32:10 number of ymm registers, ymm matmul 02:35:40 not getting the numpy performance 02:38:20 slower, second fma unit, 02:39:40 it's faster now, don't trust -O3 02:44:40 lag on stream, turning off the dryer 02:47:15 hard to make faster 02:52:20 profile cache stalls x86 02:58:20 that loop looks fast 03:06:05 cpu cache sizes 03:15:50 cache coherence, how is it slower 03:22:38 short break 03:28:20 tweet about adderall, drug test, people without skills 03:32:20 zen microarchitecture, optimization 03:39:35 L1 only 32 kB 03:46:40 we are trying to do fast matrix multiply 03:54:40 openblas haswell gemm 03:59:45 online whiteboard 04:02:15 no sarcasm allowed subscriber get's a timeout 04:02:50 removing code, _mm256_broadcast_ss 04:16:45 just persistent 04:19:00 whiteboard time, better understanding 04:25:40 don't want to reorded matrix 04:32:00 strassen = ban, wrong and slow 04:38:25 coherent meaning, access memory in order better 04:46:15 same number of fma as broadcasts 04:48:20 it's fast now 04:51:05 how to get the same fma adds 04:54:15 beating numpy 05:00:45 multithreading check, max clock, pragma 05:06:35 theoretical maximum on cpu 05:12:40 crushing numpy, real threads in C 05:22:10 double the speed, even more speed 05:24:10 overhead, semaphore 05:28:40 we cheated 05:29:30 no TFLOP 05:43:50 Alex is home, stupid question timeout 05:49:00 beautiful htop, throttling 05:51:40 theoretical maximum 05:53:40 cpu power draw 05:57:30 cpu temperature 06:01:00 disable throttling
    16
  74. 15
  75. 15
  76. 15
  77. 15
  78. 15
  79. 15
  80. 15
  81. 15
  82. 14
  83. 14
  84. 14
  85. 14
  86. https://youtu.be/xc0jGZYFQLQ (MuZero stream) | https://arxiv.org/pdf/2106.01345.pdf Decision Transformer: Reinforcement Learning via Sequence Modeling paper Bounties for tiny corp / tinygrad -> docs.google.com/spreadsheets/d/1WKHbT-7KOgjEawq5h5Ic1qUWzpfAzuD_J06N1JwOCGs/ https://youtu.be/lnVQsJJFcdg?&t=13715 Hiring entire stack for tiny corp join if you are interested | https://youtu.be/lnVQsJJFcdg?&t=14195 work major source of value in your life Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | github.com/tinygrad/tinygrad <- simple powerful deep learning framework tiny corp is accepting new interns. more info on tinygrad.org and tinygrad discord | comma.ai is accepting interns comma.ai/jobs#open-positions from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | from $999 comma.ai/shop/body the future of people Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 lunarlander_transformer.py 00:04:25 twitch substance warning 00:06:00 perplexity decision transformer 00:12:00 assert not x.requires_grad 00:15:00 192 % start_pos 00:21:45 food 00:24:25 fixes needed in tinygrad 00:41:00 gpt2 works 00:46:40 contraction not explained 00:55:00 rant 01:00:25 Ron Paul 01:04:40 usa population pyramid 01:05:30 jit 01:08:55 africa documentaries 01:13:15 cross 01:19:00 not supported 768 % 01:23:20 do things team 01:24:50 tinygrad intern phone call 01:28:50 postmodernism 01:36:40 assert t.grad is not None 01:38:30 advice 01:40:10 schedule 01:43:20 decision transformer paper 01:53:00 not balancing 02:05:00 K=20 02:10:00 plt.show() 02:15:30 clip 50 02:19:00 lunarlander fails 02:20:00 uber eats scam 02:27:00 decision transformers on Hugging Fac 02:31:30 logits 02:45:00 temperature 02:54:00 should never output 2 03:11:40 so many bugs 03:12:40 good idea from chat 03:15:00 lunarlander is not landing 03:16:30 128 clip 03:17:00 highest_reward bug 03:18:50 lunar lander rewards 03:24:30 let's make it work 03:29:00 unknown change 03:31:40 piano 03:34:20 reinforcement learning is impossible 03:37:25 write gym environment 03:50:00 stupid decision transformer 03:57:20 98% 03:58:50 that is what we get for smoking weed 04:02:10 press the light up button 04:05:00 learned to play the game 04:12:55 the optimal strategy 04:13:45 press_the_light_up_button.py 04:17:40 desired reward 04:19:40 so broken 04:27:20 some bug with 04:29:00 action and reward embedding 04:32:30 broadcast issue 04:37:50 another layer 04:44:40 50/50 probability 04:51:40 feeling so scammed 05:00:20 close to AGI 05:07:10 test model code 05:18:00 learning excruciating slowly 05:24:20 scientific notation suppress 05:25:40 making some progress 05:28:55 it's learning press the light up button 05:33:00 JIT disabled 05:34:15 equity and inclusion 05:36:10 loss going down 05:39:00 we did reinforcement learning 05:39:50 Alex, voting 05:43:40 it's learning 05:48:30 render_mode default 05:53:20 demystifying Upside-Down reinforcement learning 05:55:55 CartPole 05:58:30 lunarlander 06:02:15 pressthelightupbutton 06:04:00 lunarlander 06:09:00 spacex simulations 06:09:30 3e-4 06:17:00 size, game_length 06:28:50 life advice 06:32:05 predicting action 06:36:40 life advice answers 06:38:00 ambition greater than your intelligence 06:39:10 learn how to learn, no gradient 06:41:30 most people should just give up 06:42:00 putting time into programming 06:46:40 bug in pressthelightupbutton 06:53:25 it's dumb 07:00:50 game_lenght=32 07:05:40 scale 07:14:25 Alex bringing food 07:26:00 same data over and over 07:28:09 reading the paper 07:38:00 entropy_loss 07:40:50 reading twitch chat 07:43:50 RL stream makes us angry 07:46:30 stream overview 07:47:00 no push to github 07:47:50 ground changes shape
    14
  87. 14
  88. 14
  89. 14
  90. Enjoy quickly made timestamps and suggest better ones in the comments | George finished 00:03:00 working on github.com/geohot/tinyvoice | Now working on github.com/geohot/cuda_ioctl_sniffer ( Sniff CUDA ioctls) | github.com/NVIDIA/open-gpu-kernel-modules | Support George by subscribing on twitch.tv/subs/georgehotz | Follow and enable notifications on twitch.tv/georgehotz to catch the live stream. | Follow instagram.com/georgehotz for instagram stories. | Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 stream intro 00:03:00 tinyvoice conclusion 00:10:08 gaming computer 00:16:17 reading open-gpu-kernel-modules 00:36:31 Reveal of what George bought with twitch sub money 00:37:44 quadHD USB Four HD ATSC Digital TV Tuners for USB! 00:38:16 TV antenna 00:38:36 continue to work on nvidia driver 00:54:23 comment about nvidia open sourcing 00:55:07 google CDP score trigger 00:57:15 the future 00:58:37 don't try to fight them and don't be a republican 01:00:00 continuing reading code 01:05:02 o'reilly book ad popup trigger 01:06:30 goal for the stream 01:08:45 nvidia 4090 spec comment 01:11:46 why really triggered by CDP score 01:16:30 Martin Shkreli stream comment 01:17:30 continuing working on code 01:49:40 found all the megabytes VID_HEAP_CONTROL 01:58:47 figure out what different flags mean 02:22:38 debug level 02:37:06 we made tinyvoice work now look into nvidia opensource drivers 03:14:20 pause stream for food 03:15:33 what we are doing on stream 03:16:01 George watching himself explaining what he said before 03:16:53 we are trying to get to the bottom of what cuda is actually doing 03:18:42 hacker/hater news commenters rant 03:23:11 checking youtube comments 03:23:46 about commenters on the internet 03:25:25 making food 03:34:24 continuing working on code 04:03:34 ___cudart656 search 0 results and shady duckduckgo search result link 04:04:58 why open sourcing nvidia will win and hacking group Lapsus$ comment 04:07:04 trying to install qira 04:08:27 where are the exploits 04:08:51 Nvidia Jetson AGX Xavier and mobile division comment 04:10:20 continuing working on code 04:36:40 comment about NXP S32G274 processor 04:41:18 shill for comma jobs 04:43:50 j_blow raid 04:44:27 George explains what his is doing in the stream once again 05:11:40 George being nice to new people 05:12:05 George taking a minute to explain things to noobs 05:15:03 why George can't disable sub chat 05:16:50 Continue to explain things to noobs (kernel talk, ...) 05:19:30 how the sniffer works 05:25:29 lunixbochs get's VIP of George's stream for all good suggestions 05:26:49 continuing working on code 06:35:20 we should read nouveau source code 06:41:08 we will launch a cuda kernel without the Nvidia cuLaunchKernel
    14
  91. 02:27:18 comma three devkit announcement | https://comma.ai/shop/products/three ($2199) | Our favorite part of the event 05:57:52 comma.ai team talks about favorite things about comma. | https://comma.ai/jobs | Buy comma.ai/shop (Merch) | Follow the official youtube.com/commaai for more videos. Stay up to date by following twitter.com/comma_ai | Support comma.ai at comma.ai/shop. Chapters: 00:00:00 Opening Remarks 00:01:08 Introduction Robbe Derks 00:01:48 IN CONTROL - How comma actually works? - Robbe Derks, Hardware Engineer talk 00:21:50 Q&A Robbe Derks 00:42:54 Introduction Harald Schäfer 00:43:51 Research at comma - Towards a superhuman driving agent - Harald Schäfer, CTO talk 01:25:24 Q&A Harald Schäfer 01:58:35 Break 01:59:12 Introduction George Hotz 01:59:55 comma_con keynote - George Hotz, President talk 02:27:18 comma three devkit announcement 02:48:58 Q&A George Hotz 03:12:00 Introduction Willem Melching 03:13:22 Talk about openpilot - Willem Melching, Head of openpilot (from Rotterdam) talk 03:45:53 Introduction Adeeb Shihadeh 03:47:08 openpilot testing - Adeeb Shihadeh, openpilot engineer talk 04:12:19 Q&A Adeeb Shihadeh 04:23:44 Introduction Greg Hogan 04:24:30 infrastructure @ comma - Greg Hogan, Head of Infrastructure talk 04:57:15 Q&A Greg Hogan 05:13:48 Introduction Weixing Zhang 05:14:39 A tour through driver monitoring - Weixing Zhang, Research Engineer talk 05:30:45 Q&A Weixing Zhang 05:39:25 Preparation for Group Chat 05:40:14 Comma Team Group Chat 06:29:57 Erich’s Toyota Security Key challenge
    14
  92. Documentation for the 7900XTX -> github.com/geohot/7900xtx PRs (Pull requests) welcome! | phrack.org/issues/7/3.html | www.catb.org/~esr/faqs/hacker-howto.html | github.com/fail0verflow/radeon-tools | github.com/NationalSecurityAgency/ghidra | www.cs.sfu.ca/~ashriram/Courses/CS295/assets/notebooks/RISCV/RISCV_CARD.pdf Bounties for tiny corp / tinygrad -> docs.google.com/spreadsheets/d/1WKHbT-7KOgjEawq5h5Ic1qUWzpfAzuD_J06N1JwOCGs/ youtu.be/lnVQsJJFcdg?&t=13715 Hiring entire stack for tiny corp join if you are interested | youtu.be/lnVQsJJFcdg?&t=14195 work major source of value in your life Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org/#tinybox | github.com/tinygrad/tinygrad <- simple powerful deep learning framework tiny corp is accepting new interns. more info on tinygrad.org and tinygrad discord | comma.ai is accepting interns comma.ai/jobs#open-positions from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | from $999 comma.ai/shop/body the future of people Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 intro 00:01:50 meditation 00:04:10 questions, AQL packets, radeon southern islands 00:06:00 two code paths 00:10:10 amd vulkan, dispatch_indirect 00:18:30 packet queue = AQL packets, amdgpu eop 00:22:10 AQL_CONTROL 00:30:55 blog part 3 00:37:00 resnet training job to crash 00:40:00 DEBUG=2 00:43:00 getting rid of AQL 00:45:30 engineer wrote something specific and project manager changed the words 00:47:10 selected pipe, queue does not matter 00:48:10 different queues 00:50:15 PM4, PGM_RSRC2 00:56:10 EVENT_WRITE.partial_flush comment 00:58:40 more testing of KFD, moving away from AQL 01:01:20 gpu_driver.cc 01:05:30 does someone want's to write PM4? 01:09:45 when I make the GPU 01:10:20 nvidia and amd very similar 01:10:50 firmware absolutely simple as possible, how easy is updating different tier of software 01:11:20 why AQL exists? 01:19:45 we got the crash 01:29:00 double banned, umr, smc bits read_smc 01:33:00 30% chance PM4 does not have bugs 01:33:40 rosetta stone style translation F32 and RS64 01:38:10 radeon_tools marcan 01:38:20 is AMD going to open source anything useful 01:40:40 ghost locomotion, argo ai 01:42:40 argo the next thing, can everyone stop, cleaning solution 01:44:30 can we all just be more honest 01:45:35 AMD NDA, infringing patents, 01:52:15 being afraid of getting sued 01:55:30 DISPATCH_DIRECT packet 02:00:40 amd gpus codenames, fiji 02:02:55 linux-firmware license, not a lawyer 02:06:10 nvidia gsp license, cuda license no datacenter use 02:07:55 risk averseness of people 02:10:40 linux-firmware 02:20:40 rosetta stone, F32, RS64 02:37:00 launching specific kernel 02:40:40 first GPU to support AQL 02:46:50 NSA, Ghidra, IDA Pro, Russian tools 02:48:30 respect NSA github 02:48:50 NSA better at open source then AMD 02:49:14 NSA more open source than AMD 02:49:30 backdoors 02:54:20 drake rosetta stone song 02:57:30 credit marcan for writing the disassembler 03:15:45 binvis 03:19:30 fail0verflow radeon talk 03:39:25 F32 03:48:30 gfx1101 03:53:20 amd rs64 04:10:15 gfx rs64 04:17:55 python os lseek 04:23:15 amdgpu_regs, mmio 04:34:35 decompiling now 04:38:50 all the different addresses 04:42:30 dump_ip.c, upset writing C 04:48:25 c example mmap fd 04:51:18 read without advancing file pointer 05:03:20 bugs in the dispatch 05:20:25 frequency analysis on code 05:31:45 next 4 bytes are register 05:47:00 amd_emu_mode 05:57:35 we broke the gpu, what did we learn today? 06:00:05 I have given you the tools you need to solve the problem 06:05:40 switch to nvidia if you have AMD crashes 06:09:05 risc-v opcode groups 06:12:30 capstone risc-v disassembler 06:18:20 skip instructions, CS_ARCH_RISCV mode 06:22:20 bad at ghidra 06:23:30 don't spoil the risc-v 06:33:00 if labeled easier to read, launch instructions, almost just compile 06:33:55 figuring out the first instruction 06:38:30 load this register map 06:52:00 ghidra load map file 07:35:20 jump table that goes there, some CP function 07:54:25 cmd 3 packets pm4 08:05:00 it's cool that is risc-v 08:23:40 now we can know if AMD open sources anything real 08:29:50 I bet this did not support AQL 08:33:50 we need to stup 08:42:45 I have found why there is so many bugs in this 08:49:15 back in the days of bonaire 08:56:50 HSA was a marketing campaign 08:57:30 internet issues
    14
  93. 14
  94. Update! https://twitter.com/LisaSu/status/1669848494637735936 Thanks for connecting @realGeorgeHotz. Appreciate the work you and tiny corp are doing. We are committed to working with the community and improving our support. More to come on ROCm on @radeon soon. Lots of work ahead but excited about what we can do together. tiny corp is hiring! -> 3x Software Engineer, 1x Hardware Engineer - https://tinygrad.org/ | It's happening George started a project -> Getting AMD on MLPerf -> https://github.com/geohot?tab=projects | Go help George if you are capable! He will appreciate you! | AMD is hiring Open-Source Linux/Mesa Developer https://www.phoronix.com/news/AMD-Another-Mesa-Dev-2023 | Reach out/tweet to AMD about how bad their drivers are via all the channels twitter.com/amd | twitter.com/AMDRadeon | twitter.com/lisasu | #sponsortinycorp George will fix the drivers | Crash error https://gist.github.com/geohot/59ac46f3740647d2fdbcb2c8f29a6920 | From $1499 buy comma.ai/shop/comma-three | tiny corp has a website tinygrad.org | Check out and star github.com/geohot/tinygrad | Support George by subscribing twitch.tv/subs/georgehotz | Follow George on instagram.com/georgehotz & twitch.tv/georgehotz & twitter.com/comma_ai to be up to date | Read George's geohot.github.io/blog/ | Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters Part 1: 00:00:00 intro 00:00:20 red team good green team bad, rocm bugs, tinygrad benchmark 00:01:30 7900xtx true flops, mojo, off by factor of 4, opencl 00:03:00 shader, bad assembler, v_dual_fnac_f32, ashrrev_i32_e32, arithmetic shift right 00:04:30 epyc, nice things about amd gpu, rocm-bandwith-test, bidirectional peak copy bandwidth 00:05:28 amd driver kernel NULL pointer reference, reboot, renting a modem from cox 00:07:00 terrible assembly code, not optimized, cheapETH not coming back 00:09:05 how gpu work, warps, simd machines, single instruction, multiple data 00:09:49 ADD,MUL=DSP, ADD,MUL,LOAD,STORE=GPU, ADD,MUL,STORE,COMPARE,BRANCH=CPU, load, not a load 00:10:55 WGP=2CU, warp - single instruction across 32, dual issue VALU 00:12:10 playing assembly in tinygrad, extra rocm, rdna3, enable_early_exec, bad driver, kernel bad 00:13:05 staging worse, the only kernel and dkms that only kind of works 00:13:35 compiling empty program and getting the binary out, stuff to compile rocm 00:14:55 llvm-objdump -d cc.elf, assembler in tinygrad asm.py, llvm-mc - LLVM Machine Code Playground, obj file, lld to link it 00:17:04 failed to get code object ISA name, if you have something to send via email to geohot, llvm-readelf cc.elf 00:18:25 assemble a shader from RDNA3 language, thanks amd, RDNA3, tensor cores float32 throughput vs non tensor cores, llvm target to amdgpu 00:19:45 amdgpu triple, llvm target triples, failed to get metadata 00:21:55 wavefront_size, how wide simd path is, NT_AMDGPU_METADATA, Message Pack [MsgPack] binary format 00:23:45 searching over all rocm open source code, thanks amd, amd_comgr_get_data_metadata 00:25:35 kernel crash again, rmmod amdgpu, reseting pci device 00:28:15 metadata generated, wavefront_size 128, failed to read ip discovery 00:30:35 git commit assembler maybe, removing the bus, computer rebooted 00:32:20 driver bad, dkms latest driver, stock ubuntu 22.04 kernel, connecting bmc 00:33:50 your connection is not private, thisisunsafe, connecting to asrockrack mb 00:35:20 event_log, pci serr critical error, powered off, no signal, vbios 00:36:50 debugfs to restart gpu, bmc remote kvm 00:40:00 pci serr, no boot delay, amd experience, pcie calibration 00:43:30 computer now in the case, checking for pcie errors 00:45:22 if this happens again we buy Intel+Nvidia, I really want like AMD 00:46:00 asm.py OUT_OF_HOST_MEMORY, normal crash, AMD_LOG_LEVEL=4, roadata 00:54:20 llvm, gnu assembly export symbol, .global 00:57:50 pyopencl does not bundle an opencl, symbol code.kd, word gnu assembler 01:04:40 OBJECT FUNC symbol type assembler, .type code.kd, STT_OBJECT, it works 01:09:00 why AMD? red team, creating a buffer, tinygrad and amd 01:11:25 args, running our custom assembly code, how to test, infinite loop 01:15:00 memory access fault by GPU node-1 01:23:10 missing an end program, thanks xXthebigfoXx 01:26:30 working, test number 2,commit custom asm, crashed gpu driver 01:28:00 how to reset gpu, echoing a string, one works one hangs, should I put the gpus in quiet 01:31:40 learning about what each command does, it's not the gpu it's the driver crashing 01:33:40 GLOBAL_STORE_B32, S_LOAD_B64, how complex is the driver?, writing our own driver, pro driver 01:37:15 does AMD care about this? no repeatable way to crash, why driver so bad 01:41:50 not one crash with CUDA did anything crashed the kernel driver 01:42:55 bad mb, driver suggestion from chat, everything new in pc 01:46:30 frustrating AMD, now on quiet, kernel 5.15.0-71, installed single 7900 01:47:50 updating rocm to 5.5, rocm_clone.sh, new computer, graveyard of gpus 01:49:30 if anyone from AMD is watching this, contract George, nvidia P2P not enabled 01:50:35 AMD should sponsor tinycorp, 7 gpu cards, rocm setup, llvm 01:52:50 no space in case for two gpus, rocm llvm, how hard is to write a gpu driver for compute, AMD do you want to be a competitor to Nvidia? sponsor tinycorp 01:55:10 you have to go through process, can make your card good 01:56:00 AMD call George, Lisa Su reasonable person, Nvidia 5x bigger company vs AMD and you make CPUs, hardcoded jammy 02:02:40 AMD would be worth more than Nvidia if they fixed their drivers 02:02:50 no matter how much I mess up in userspace you should never pagefault in kernel that crashes the driver 02:03:20 need dropping 0days on AMD to get them to listen, short AMD, exploit their driver, make money 02:03:36 will fix drivers for AMD, make compute stable on your own AMD CPU 02:03:55 pirate flag, pride flag, american flag, that is why drivers don't work 02:04:40 making money in good sustainable way, can't search twitter anymore without login why Elon? 02:04:55 I will fix AMD drivers, LBTQ+ friendly stream 02:07:00 nvidia makes more money on datacenter then gaming, libamdoci-orca64, more stable on this PC 02:13:35 AMD care, we love you, if the driver is not stable, I want to get to the bottom of this 02:16:50 we have a mystery to solve, why is it crappy on that computer, nvidia-nvlink 02:17:50 systemd disable nvidia persistenced, putting the gpu in epyc computer 02:19:40 could the pci port be bad, not enough pci lanes 02:23:20 product for tinycorp, home DGX, 5 GPU in box, home trainer box, 6 boxes to pro mb is 30 GPUs 02:26:45 TinyAMD driver selling to AMD, writing a fuzzer, fuzz.py 02:31:55 same crash, rocm-smi gpureset, minimum reproduction for this crash 02:36:30 map_queues_cpsch, dqm lock has to be locked before calling this function 02:38:48 upstream driver, staging driver, whole linux kernel, purge, arch linux not a solution 02:43:40 map_queues_cpsch, arch rocm, upset with amd not arch, solution is nvidia 02:45:40 gpu P2P disabled, cuda ioctl SM_ISSUE_RATE_MODIFIER_PARAMS, ada nvidia architecture, FP16 vs FP32 accumulate 02:47:30 AMD call George, AMD will become a competitor for deep learning, crash by just loops 02:48:25 if you want to suffer and struggle buy AMD, AMD driver crashes, amd hipcc apt-get 02:53:00 new crash, tinygrad using pyopencl, normal ubuntu, drivers just not work, linux 6.3, kernel crashes 02:57:00 3080 TI P2P, return AMD GPU?, jailbreak for Nvidia, Nvidia not that bad 03:02:30 Nvidia prohibit data center use, how make the companies to listen, AMD Baidu, who won in long run Apple or Jailbreak 03:04:00 build something better than they have, companies respond to fear, still use an iPhone 03:04:45 Android spyware, starting AI chip company, write AMD driver first, Jim Keller, tenstorrent can't buy card 03:06:00 as a company make drivers that work on AMD GPUs 03:07:00 company drivers better than nvidia it's Apple, apple 2.75T, nvidia 700B, AMD 144B, marketcap show the stability of their drivers 03:08:30 apple still not scummy yet, nvidia scummy, no competition, apple value of their brand, AMD good hardware 03:10:10 apple shader code clean, intel 129B crappy drivers 03:12:20 buying chips from qualcomm, will make chips and drivers that are stable, 4090 is a good card 03:12:55 advice to nvidia, enable P2P, disable mb dispach make your GPU 2x fast, remove banning from use in datacenter 03:13:35 Jensen who are you listening to?, 4080 stupid, Google TPU rant, Stevey's Google Platforms Rant 03:15:00 nvidia cards by the deal per TFLOP, do not buy 4080, depends on the chip 03:16:30 Alex home, amdgpu driver github, qualcomm gpu drivers good, kernel 6.3 03:26:30 booted the computer and it crashed, linux firmware amd install, amd website driver broken driver 03:36:15 rebuild initramfs, what linux for rocm, stable until it crashed, vertical screen for the long errors 03:43:46 normal install, RDNA2 card test, rocm 5.5 03:49:30 why apple does not use nvidia, bad sodering profile, Steve Jobs said we are never going to use nvidia GPU again 03:53:15 AMD EPYC cpu nice because no need for drivers, RDNA2 GPU, dual-issue elu commands on rdna 3 04:02:25 gpu-burn amd, mes might be in unrecoverable state, should we return AMD cards 04:07:25 AMD if you reach out to George about fixing this drivers he will keep the cards, need to make chips 04:08:00 AMD just show that you care about supporting this, AMD drivers don't work, we still haven't tried the latest driver 04:09:00 ubuntu build kernel from source, menuconfig, how did Linus let this crap in the kernel 04:12:25 7900xtx been out for a year and still broken, expect stable drivers 04:15:50 just building kernels and llvm instead of building AMD RDNA, menuconfig, driver build as a module,
    13
  95. Thank you to AMD KFD architect Felix Kuehling!!! -> github.com/tinygrad/tinygrad/discussions/4086 | Observations > gist.github.com/fxkamd/ffd02d66a2863e444ec208ea4f3adc48 Documentation for the 7900XTX -> github.com/geohot/7900xtx PRs welcome! | phrack.org/issues/7/3.html | www.catb.org/~esr/faqs/hacker-howto.html | www.cs.unc.edu/~otternes/papers/rtns2021.pdf | www.x.org/docs/AMD/old/R5xx_Acceleration_v1.2.pdf Bounties for tiny corp / tinygrad -> docs.google.com/spreadsheets/d/1WKHbT-7KOgjEawq5h5Ic1qUWzpfAzuD_J06N1JwOCGs/ youtu.be/lnVQsJJFcdg?&t=13715 Hiring entire stack for tiny corp join if you are interested | youtu.be/lnVQsJJFcdg?&t=14195 work major source of value in your life Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org/#tinybox | github.com/tinygrad/tinygrad <- simple powerful deep learning framework tiny corp is accepting new interns. more info on tinygrad.org and tinygrad discord | comma.ai is accepting interns comma.ai/jobs#open-positions from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | from $999 comma.ai/shop/body the future of people Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 intro 00:01:10 AMD open source announcement 00:01:50 AMD does not understand the spirit of open source 00:02:25 GPU quality, PR, praise AMD when they release source code 00:02:45 what will AMD open source? 00:03:45 AMD employees banned from tinygrad discord 00:04:14 KFD architect AMD Felix Kuehling 00:06:00 PR is bad, comma marketing safety 00:07:40 the spirit of open source, GPU quality will make money long term 00:08:25 ghost locomotion, $217 million gone 00:08:50 comma.ai business model 00:09:30 what companies get wrong today, influencing the narrative 00:10:30 truth, reality is reality 00:11:00 appreciate engaging on a technical level 00:11:30 internet lies, truth, being on your own 00:12:40 hacker manifesto, hacker culture eric raymond, hacker culture 00:13:50 responding to HN Hector Martin - marcan42, asahi linux 00:16:10 George Hotz security background 00:16:45 communication style, lone genius 00:17:20 response to 10 years sony story 00:18:18 political disagreement 00:19:25 hacker news doubt, marcan worked on asahi linux 00:20:30 lone geniuses, building on other people 00:21:25 Jürgen Schmidhuber credit for deep learning, Yoshua Bengio, Geoffrey Hinton, Yann LeCun 00:22:25 infosec people on twitter, most talented people ended up giving up 00:23:15 never give up never surrender, some people are better than other people 00:25:30 showing off in videos, depth in videos 00:26:10 Elon has no ego, I want to be right but not at expense of being wrong 00:27:10 views, entertainer 00:28:00 hacker manifesto, getting mad at AMD 00:29:00 compounding successes, spacex success story 00:29:45 Tesla FSD attitude, zen with GPUs 00:31:20 AMD change log, not high quality 00:32:00 ghost autonomy, call for ghost to open source, public domain 00:33:45 intellectual property 00:34:50 sonny bono copyright extension act 00:37:45 regression bug 00:41:30 computers don't feel pain 00:42:18 4 compute types, SDMA, COMPUTE_AQL, COMPUTE PM4 00:43:20 back to marcan hacker news response, AMD comunication 00:44:15 comma ai safety model, fear uncertainty doubt 00:45:50 watch what I do not what I say, geohot github 00:46:40 most honest response to a hater 00:47:15 CP old term, personal attacks 00:50:10 tomcr00se soundcloud civil war 00:50:50 Elon test, AMD hardware issue 00:52:25 how hard is to change yourself, other people 00:52:40 HWE = hardware enablement, food 00:53:50 what we did so far on the stream 00:54:45 AMD released a tweet 00:56:10 death spiral sequence, search engine, logging-in 00:57:10 kagi, Apple, no more iPhone, search engine MacOS, arc browser, xz utils 01:02:20 how to launch a kernel with PM4 packets 01:02:55 roct-thunk-interface 01:08:20 want the world to be better, personal gain, mechanized slaves, hvac 01:10:30 PM4 vs AQL, crash info, PM4 might bypass all this garbage 01:12:50 ai alignment 01:15:00 bug identified correctly, but not fixed properly 01:18:50 nvidia driver, umr, RS64 01:26:25 timeline for stable red tiny box, 20 conversions to green and conversions 2 to red 01:27:50 280 pre-orders, 40 cancellations 01:29:35 if we go to this level GPU will be stable, PM4 01:31:00 GPU-specific dispatch registers, quest for low level control stupid and contrary to all business objectives 01:33:00 has not crashed yet 01:40:00 Virtual Memory Identifiers (VMIDs) 01:43:00 exploring amd gpu scheduling details by experimenting with worst practices 01:45:00 downgrade firmware no crash, regression 01:53:15 PQ 01:56:40 amd global data share 02:02:40 how to dump this queue 02:13:00 favorite AMD employee is streaming 02:17:25 PQ, pipe queue, AI hallucination 02:19:59 PQ BASE, mac os x forwarding 02:32:45 gl x forwarding nvidia ubuntu 22.04 02:37:59 packet queue 02:42:55 ubuntu setup vnc, mac os x vnc client brew 02:47:55 control b stupid shortcut, control a 02:49:45 starting with computers today, if you are old it's over 02:54:30 vnc opengl 02:55:20 wayland, era of linux desktop, microsoft rdp 02:58:00 sdl2 x xquartz forwarding mac os x 03:01:20 launching gui for an hour 03:04:10 booting asahi linux macbook 03:07:10 sharing macbook screen 03:12:00 shower
    13
  96. 13
  97. 13
  98. 13
  99. 13
  100. 13
  101. 13
  102. 13
  103. 13
  104. 13
  105. 12
  106. 12
  107. 12
  108. 12
  109. 12
  110. 12
  111. 12
  112. 12
  113. 12
  114. 11
  115. 11
  116. 11
  117. 11
  118. 11
  119. 11
  120. Somone help George in tinygrad 01:38:25 | Go learn github.com/tinygrad/teenygrad | Read www.incompleteideas.net/IncIdeas/BitterLesson.html | users.ece.cmu.edu/~gamvrosi/thelastq.html Watch youtube.com/watch?v=6yQEA18C-XI - George Hotz vs Eliezer Yudkowsky AI Safety Debate | Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 muted intro 00:00:35 un-muted 00:01:23 teenygrad 00:01:44 staying on topic 00:02:15 tensor.py 00:04:55 renaming to teenygrad 00:08:55 DType out of control 00:12:08 bad chat comment trigger 00:12:50 teenygrad.mlops 00:17:50 teenygrad.state 00:19:50 better ways to write software 00:23:00 intro for new people, factor graph, clean APIs 00:23:33 the most beautiful file in tinygrad mlops 00:24:45 bad files in tinygrad 00:26:20 object has no attribute reshape 00:29:20 lazybuffer object has no attribute dtype 00:33:40 pretty-errors 00:40:40 humans bad at software development 00:46:05 object lazybuffer has no attribute fromcpu 00:51:35 intro for new people again 00:53:15 python max magic method 00:57:50 pretty syntactic sugar 01:00:05 flag to error fast 01:02:10 main, master 01:08:05 coin boys 01:09:15 world always trying to exploit, manipulate you 01:11:10 tensor has no attribute cpu 01:11:48 loss nan 01:14:42 accuracy going up 01:15:35 830 lines 01:17:40 can we make llama work?, stable diffusion 01:19:43 do we even need flip 01:22:05 it's cheating if you use pytorch 01:22:50 llama needs float16 01:30:24 bug in tinygrad 01:36:10 math imports 01:37:01 google engineers 01:38:25 can someone write this in tinygrad that does not use numpy 01:40:10 cifar 10 01:42:15 spending lines on speed 01:43:10 teenygrad tinygrad front end used independently of everything else 01:44:43 web gpu kernels 01:45:03 The bitter lesson by Rich Sutton 01:46:10 Rich Sutton technical, Robin Hanson economics, AI doom 01:46:30 climate doom 01:47:20 heat death of the universe 01:47:30 The last question Isaac Asimov 01:49:30 if you understand teenygrad you are on your way to understand tinygrad 01:51:40 letting nun subs talk 01:53:17 what is the point? 02:00:40 lies in the read me 02:03:45 you just do not undestand 02:04:49 where is my hurricane 02:05:50 thank you for watching the stream 02:06:55 manifold george yudkowsky 02:08:04 the beginning of infinity David Deutsch 02:08:14 doomers, yudkowsky smart 02:08:40 people played this game before 02:10:10 new addition on geohot.com
    11
  121. 11
  122. 11
  123. 11
  124. tiny corp is hiring! -> 3x Software Engineer, 1x Hardware Engineer - https://tinygrad.org/ | It's happening George started a project -> Getting AMD on MLPerf -> https://github.com/geohot?tab=projects | Go help George if you are capable! He will appreciate you! | tiny corp has a website tinygrad.org. Let your friends know! | Support George by subscribing on twitch.tv/subs/georgehotz | github.com/geohot/tinygrad | github.com/CompVis/stable-diffusion | Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 intro 00:01:50 sponsor for the channel 00:02:25 cond_stage_model 00:07:30 not a role model 00:11:00 blog post feedback 00:12:55 copyright music, cloudflare, kiwifarms 00:16:10 first they came for poem 00:21:20 centralized, entities with power 00:22:24 aws parler ban, ad friendly content, infrastructure deplatforming 00:24:15 cloudflare statements, changing the content, net neutrality 00:27:13 cloudflare defaced their website 00:28:05 alternatives to cloudflare, ddos-guard 00:29:20 harassment, patent troll, banned user on comma discord contacting comma investors 00:30:46 rational power, psychotic power, doxing, never swatted 00:32:15 axel nix, faulting the system, legal fees 00:35:04 illusion of choice, curtis yarvin, neoreaction 00:38:15 what you learn from hacking, quote from kiwifarms, tolerance 00:41:03 christian, atheism, removing bad things, free speech 00:42:50 internet freedom, good and evil, infrastructure layer 00:44:15 the daily stormer, matt taibbi, destiny, consume product 00:47:50 politics is interested in you 00:48:33 declaration of independence of cyberspace, justice trigger 00:51:00 abortion, ignoring politics, where is the line 00:53:30 real political advice, shutitdown.us, taking rights from other people 01:01:28 george hotz tonight 01:02:08 cloudflare competitor stormspark 01:03:40 raceism 01:05:08 sorry about talking about politics 01:07:05 in search of lost time 01:09:15 lidar = sub only chat 01:09:42 goal for today, people getting off topic 01:13:19 labor day, karpathy email, things to work on, maxpool, gpu execution bug, test to fix 01:16:03 boring and rational politics 01:18:40 why we build technology 01:19:38 idiots, can't have happy without sad 01:22:55 clipmlp, clipattention 01:31:44 clipencoderlayer 01:49:19 compare the outputs and make sure they are right 01:53:39 lawsuits, sue, private investigators 01:58:45 elon is ruthless 02:02:33 axel nix 02:04:27 wire, books, walls, dl slow 02:09:15 tiktok 02:12:35 song tomcr00se 02:14:25 justice, AI safety, taylor swift 02:22:30 stable_diffusion.md 02:25:28 girlfriend 02:31:40 making this work today 02:39:40 it's not the same 02:41:19 music 02:49:25 nutella 02:52:40 what to generate 02:58:35 layernorm 03:16:10 object has no attribute 03:19:00 struggle learning, john carmack lex, fake advice 03:23:50 whole thing match? 03:31:55 dark apple, code is law 03:42:20 monopolies, daylight savings, spacex 03:46:45 how to get rich quick 03:49:10 a horse sized cat eating a bagel 03:56:45 debugging time 04:01:10 information hazard, offended, offending 04:02:45 ubuntu site-packages 04:04:29 music 04:14:20 64?, black square 04:29:20 models together not working 05:03:20 unit tests, segfaults 06:00:40 math by hand, router overheated 06:07:08 bug in mps 06:18:30 no bug in tinygrad 06:26:30 two squares 06:29:40 no plan 06:50:55 noise 07:03:15 good noise, cat 07:16:15 resolve 07:22:43 tinygrad horse 07:34:10 faster inference 07:35:33 tinygrad cat eating a bagel 07:36:44 hard work, sampler, 10 steps 07:39:35 chat generating prompt 07:41:55 penguin with fire extinguisher 07:45:08 the penguin image 07:47:10 aptera 07:47:40 understanding stable diffusion 07:48:15 tinygrad libraries 07:52:00 3 days, kafka, no safety 07:53:20 dolly implementation 07:53:45 serious concerns in ai 07:55:00 can system prove it's aligned with you 07:55:25 effective altruism 07:56:25 next silicon valley 07:57:12 china, culture influences, no rights 08:00:03 immigration, tax, hiring, visa, value, text to video 08:02:40 EU bureaucracy 08:03:10 malaysia, singapore 08:04:05 california, san francisco 08:05:15 no remote, russia 08:06:39 50 iterations, dubai, cyber, freedom 08:15:25 end to end openpilot, agi, cbuffer, FSD, rotterdam 08:17:15 horse eating bagel
    11
  125. 11
  126. tiny corp is hiring! -> 3x Software Engineer, 1x Hardware Engineer - https://tinygrad.org/ | It's happening George started a project -> Getting AMD on MLPerf -> https://github.com/geohot?tab=projects | Go help George if you are capable! He will appreciate you! | tiny corp has a website tinygrad.org. Let your friends know! | 00:19:45 01:29:00 beautiful tinygrad graphs | github.com/geohot/tinygrad | github.com/geohot/cuda_ioctl_sniffer | Support George by subscribing on twitch.tv/subs/georgehotz | Enjoy quickly made timestamps and suggest better ones in the comments Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 intro 00:00:38 quotes 00:02:17 George moving out of apartment 00:02:42 refactor of tinygrad 00:04:15 how does data move, facts over feelings, Ben Shapiro 00:05:18 more info about refactor of tinygrad 00:09:10 we are going to make tinygrad lazy 00:11:25 pull requests tinygrad 00:12:35 fetching the openpilot model 00:14:00 error could not find pythonlibs 00:19:45 beautiful tinygrad graphs 00:21:05 not a safe space for off topic questions 00:21:35 tinygrad lazy branch 00:24:05 1000 lines 00:25:10 permute object has no attribute op 00:27:00 all the ops 00:29:20 the power of tinygrad 00:32:20 back to trying to install onnx 00:42:30 node 00:45:25 trying to setup twitch soundtrack 00:49:38 airpods music 00:51:30 green squirrel in pretty bad shape 00:56:30 George is feeling calm, questions 00:57:30 simple modes 01:14:10 TestOps.test_elu 01:24:40 split 01:29:00 graph of openpilot model 01:33:30 running model on gpu, torch cpu 01:36:30 flatten, MultiCAD support 01:49:15 onnx runtime 01:58:58 tests failing 02:15:40 asking for help on += 02:20:35 stream for the weekend, land, export to onnx 02:21:45 idea of tinygrad 02:23:05 breaking of CI 02:31:05 idea 02:34:30 stream slogan keep hacking elite 02:36:40 non-sub chat quality, computer questions 02:37:55 food time 02:39:10 sub only society, taxes 02:40:00 always write tests, I love tests 02:40:31 people with skills 02:44:40 onnx resnet 02:52:10 working with food 03:00:15 onnx GlobalAveragePool 03:22:20 that fixed it 03:24:55 Converter is not implemented 03:39:50 bringing in the chicken image 03:54:50 looking up the chicken image 03:57:00 we should write more tests, piano 03:59:40 Unsupported Transpose 04:05:35 replacing onnx with better onnx 04:09:10 it's a lighter 04:12:15 incompatible constructor arguments 04:19:15 more errors 04:26:00 it's totally wrong 04:33:00 ShuffleNet 04:39:45 how much wrong could it be 04:40:58 explaining the problem with maxpool 04:53:30 hen 04:58:00 overview of the stream 04:59:00 recap 05:01:45 questions from subs 05:02:35 lot's of gifted subs 05:03:08 PRINT_LLOPS=1, optimization 05:04:30 tinygrad is a risc instruction set for neural networks 05:08:17 more gifted subs 05:08:45 tinyvoice, tinygrad, tinycuda = tinyfast 05:12:15 why pytorch can't load onnx models 05:13:45 the beauty of tinygrad 05:15:40 more gifted subs 05:16:40 strided max pool implementation 05:20:40 thank you for watching
    11
  127. 10
  128. 10
  129. 10
  130. 10
  131. 10
  132. 10
  133. 10
  134. 10
  135. 10
  136. Before asking bad questions read http://www.catb.org/~esr/faqs/smart-questions.html | 00:03:34 Loud sneeze by George careful headphone users! | github.com/geohot/cuda_ioctl_sniffer | Support George by subscribing on twitch.tv/subs/georgehotz | Timestamps took a lot of hours to make, suggest better ones in the comments Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 intro 00:00:34 CUDA driver PTX program compiled to SASS 00:02:24 setting GPU name to sm_86 00:03:34 loud sneeze by George 00:04:57 cuobjdump output sass code 00:06:22 sass, nvidia assembly, bytes 00:06:51 dmacopy, memcpy,trivial, refactoring driver 00:13:37 banned subscriber 00:14:08 nice commit names 00:14:24 subscriber can get banned 00:15:28 goal of today 00:18:13 favorite lines from House 00:18:58 init function 00:22:19 octal is 7, top of food chain, postmodernism 00:23:47 NV_ESC_RM_ALLOC call 00:27:30 good comments by smurfd0, lunixbochs 00:28:16 sniffer not picking up why 01:02:45 getting duped scam GPU 01:12:15 j_blow raiding the stream 01:24:10 standing desk 01:25:12 one line in python to seperate something 01:32:50 that turned out to not matter 01:34:14 allocating TURING_USERMODE_A 01:39:37 hooking with driver 02:08:10 commit allocated 02:25:25 mapping things we care 02:26:29 did not work but it did not crash 02:26:56 Mike Stud - Closer, Hoodie Allen - Little League 02:28:05 something else that happens 02:39:32 driver does not work but the hook is firing 02:45:40 living advice 02:46:18 Elon working from home 02:48:20 there are no lessons 02:49:47 nvidia doorbell register 02:57:56 valid engine id must be specified 03:02:29 this might be it 03:05:16 engineType 03:06:49 we trying to learn 03:15:45 block VA_SPACE 03:29:45 status 0 got an object 03:39:48 this isn't acually right 03:44:30 what else do we need 03:59:19 my error it's right here 04:08:03 we are getting close 04:19:14 raid from provod 04:20:00 what are we doing 04:35:29 everyone can say things they don't believe 04:36:25 dumb questions warning 04:42:18 not initializing for some reason 04:48:37 getting close 04:54:05 comparing dmesg 05:03:12 what was that? 05:09:50 nvidia-uvm, shoutout to @lunixbochs and piano 05:21:12 timeout for bad commenter 05:24:10 could be uvm releated 05:35:31 share the device 05:42:29 we need the uvm 05:43:10 smart-questions.html 05:46:00 part2 05:46:55 hard work and being a genius 05:48:00 what uvm is 05:48:40 it's all copyrighted 05:56:35 Luna comment 05:59:38 gpu uuid and the rm control fd 06:06:25 missing break statements 06:09:38 interceptor not care about any fixed addresses 06:14:35 generic error invalid state 06:18:40 current device is not valid 06:22:38 uvm working, FERMI_CONTEXT_SHARE working 06:34:30 neck massage 06:35:30 generic error 06:39:55 how did I learn 06:45:08 legacy mode 06:54:50 being lazy 06:55:09 raid from sudomaze 06:59:10 channels 07:09:25 driver it's not working 07:27:00 pick a strategy and commit to it 07:28:10 halirutan gifting 10 subs 07:31:15 invalid state 07:49:10 KEPLER_CHANNEL_GROUP 08:00:00 smurfd0 get's promoted to VIP 08:19:25 Q is broken or the doorbell is broken 08:23:25 halirutan gives the non subs the power of talk 08:36:45 doing it again without linking cuda 08:39:45 without the driver the program does not run 08:43:50 intelligent questions 08:49:45 window B answer 08:51:18 memory overhead advantage 08:51:55 hiring Elon's fired people 08:52:20 remote work 08:53:55 the point of life is work 08:58:44 Ian Goodfellow 08:59:15 Vitalik Buterin 09:00:10 taxes 09:01:00 next steps in open cuda project 09:01:45 github copilot 09:03:25 AGI by 2030? 09:04:50 recession 09:06:50 trolling people who ask bad questions
    10
  137. 10
  138. 02:18:16 Study for tests! Always be ready to be tested! | https://en.wikibooks.org/wiki/Write_Yourself_a_Scheme_in_48_Hours | https://mitpress.mit.edu/sites/default/files/sicp/full-text/book/book.html | If this is fun for you do some TODOs https://github.com/geohot/haskell-scheme | To support George follow and subscribe on https://twitch.tv/georgehotz and help George get Twitch Partner! | Programming archive🡿 https://youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Copyright is sad. It will block videos in some countries or everywhere sometimes :( Here is what you missed while muted: Music https://youtu.be/DL9a79fcj1k (Saves The Day - I'm Sorry I'm Leaving (Full Album)) 01:08:38 Staples441 thank you for subscribing! 01:09:04 KinkInstability thank you for subscribing! 01:09:20 I am not understanding exactly why primitive function has that type and not this type. 01:09:49 Oh yea I don't know where I got my broken primitive function from. 01:11:45 I guess I don't understand why I can't do that. 01:11:51 It is... I have that... 01:12:02 What I am doing wrong? 01:12:43 How do I fix this? 01:12:55 Yo I won two major CTFs on my own. I won secuinside and csaw but I don't really want to talk about that. 01:13:08 I don't know if you count csaw as major but secuinside. 01:13:39 Expected kind? but LispVal has kind star? I don't know what that means.. 01:13:53 You can't make an instance show for a constructor ohh okay... really though... 01:14:10 But yea that's terrible I want my deriving. 01:18:25 This song makes me very happy. 01:19:01 Okay so we will have to change this one. 01:20:28 Sweet. Okay. We got that stuff working again... Maybe this is where... Sweet okay. 01:21:28 You know some people are like the costumer is always right. I am not. Costumer can be wrong or right. Costumers correctness does not have anything to do with the fact that they are a costumer. 01:22:15 Would be nice to have a test suite. Yea we should write some tests. That is a good point. 01:22:30 We will write some tests in a bit. 01:22:43 Sweet. Yeea we should write tests... 01:42:34 We don't have pe... 01:42:58 Whoo it's fixed. We back boys we back. 01:43:40 I feel like I have to like lift it from the monad.
    10
  139. Documentation for the 7900XTX -> github.com/geohot/7900xtx PRs welcome! | 06:56:30 job offer to github.com/gnif | $1000 to fix a reset of GPU for github.com/gnif | 07:02:15 Bounties for tiny corp / tinygrad -> docs.google.com/spreadsheets/d/1WKHbT-7KOgjEawq5h5Ic1qUWzpfAzuD_J06N1JwOCGs/ youtu.be/lnVQsJJFcdg?&t=13715 Hiring entire stack for tiny corp join if you are interested | youtu.be/lnVQsJJFcdg?&t=14195 work major source of value in your life Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | github.com/tinygrad/tinygrad <- simple powerful deep learning framework tiny corp is accepting new interns. more info on tinygrad.org and tinygrad discord | comma.ai is accepting interns comma.ai/jobs#open-positions from $1050 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | from $999 comma.ai/shop/body the future of people Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 intro 00:01:25 new KFD backend 00:03:50 never give up 00:05:20 tinybox 00:07:00 looking for comment 00:11:35 docs for 7900XTX 00:13:40 university attendance over time us 00:22:00 vscode markdown preview 00:31:10 signature solar, solaredge optimizers 00:45:45 solar per watt over time 00:50:50 github.com/geohot/7900xtx, tenstorrent software 00:58:20 ubuntu kernel build source don't use apt src 01:01:00 AMD processes, Instinct MI 300, testing in isolation, crashes 01:03:00 no CI for hardware 01:05:50 planning, handling George as PR problem 01:15:00 nvidia taking AMD talent 01:15:50 old instinct cards 01:17:00 test_sdma 01:34:00 firmware file, registers dump 01:37:55 all this docs might be wrong 01:39:30 drivers from AND will not get better 01:46:00 US government people 01:48:10 product engineering, operations, engineering mindset manipulating things not people 01:49:00 everybody who creates is an engineer, administrator not an engineer 01:49:30 boeing, outsourcing, operations keeping systems running 01:51:30 how do you get more gold, more fiat, bitcoin, people 01:54:00 proof of work to proof of stake ethereum 01:56:00 gold works 02:09:00 diversity is our strength 02:19:55 DRM debugging 02:24:19 AMD brutal reviews 02:28:40 4090 tracker 02:39:05 hardware resources 02:48:00 LSDMA, system DMA and other DMA 03:00:00 codenames, UMR 03:05:10 perplexity, searching, search engine 03:09:50 RDNA 3 03:28:50 compute unit 03:35:50 gpu hang, crash, piano 03:39:20 amd community forums 03:41:10 AMD release register reference 79000 XTX 03:42:40 George helping AMD 04:01:35 plum bonito gfx1100 04:03:15 RS64 04:09:55 AMD gpu mailing list 04:13:40 break, shower 04:24:50 should you buy 79000 XTX 04:25:35 AMD PR team, disparaging, George's experience with GPU 04:27:50 the modern consumer 04:30:00 open pilot CI, comma ai failure rate 04:52:50 radeon f32 processor 04:54:20 leak, leak is a public domain 04:58:50 XML dump of bonaire register docs 05:00:00 internet is under attack, remember when google worked 05:01:00 2014 05:06:00 free speech, hate speech, iq vs free speech absolutism 05:17:40 MEC, amdgpu microengine compute 05:46:50 how to power cycle pci device 05:50:15 AMD I want the MEC 06:24:10 error: object file empty git 06:27:25 docs for 79000 XTX overview 06:35:40 SDMA+RLC are F32, RS64 06:38:10 AMD source about register access 06:41:55 ibm rs64 32-bit 06:46:10 rs64 soft core 06:50:50 Alex, RDNA 4 06:53:40 reading about workload managers 06:55:30 the AMDGPU code base 06:56:30 job offer to github.com/gnif 06:57:20 writing software, taping out chips 06:58:40 prediction on AMD high end GPUs 06:59:35 qualcomm reverse engineering 07:01:30 openmax, OMX 07:02:15 can people draw attention to this? 07:03:40 looking glass 07:05:35 internal AMD docs 07:06:40 tinygrad new driver, new style of driver, CommandQueue 07:07:20 $1000 to fix a reset of GPU for github.com/gnif
    10
  140. 10
  141. 10
  142. 10
  143. tiny corp is hiring! -> 3x Software Engineer, 1x Hardware Engineer - https://tinygrad.org/ | It's happening George started a project -> Getting AMD on MLPerf -> https://github.com/geohot?tab=projects | Go help George if you are capable! He will appreciate you! | tiny corp has a website tinygrad.org. Let your friends know! | WARNING HEADPHONE USERS 00:53:50 | 02:29:00 | 03:06:05 | Check out and star github.com/geohot/tinygrad | Support George by subscribing twitch.tv/subs/georgehotz | Follow George on instagram.com/georgehotz & twitch.tv/georgehotz & twitter.com/comma_ai to be up to date | Read George's geohot.github.io/blog/ | Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 intro 00:01:00 LLVM=1, no ShapeTracker yet, ethernet wire 00:02:30 defcon sux 00:03:28 llvmlite, event's to replace defcon, Chaos Computer Club (CCC) 00:04:05 code for implementing llvm backend, LLVM buffer, processing_op 00:05:15 tinygrad low level operations 00:07:14 add and sub could be the same op, could merge some 00:08:15 4 core ops of tinygrad, matrix multiply, matmul, derivative of convolution 00:09:36 why isn't tinygrad fast? memory access patterns 00:10:10 ShapeTracker, segfault 00:11:25 writing test_add_forward 00:12:00 LLVM, memory, single thread system 00:12:25 why it's not working, null pointer 00:14:00 object has no attribute address 00:18:26 segfault, cfunctype m1 ctypes 00:22:18 checking llvmlite version 00:23:13 cfunctiontype calling 00:25:20 as_array requires a shape argument 00:25:38 stupid argtypes 00:27:45 doing an ADD, a MUL and an NOOP 00:28:40 the first argument is the return type 00:29:30 nejintejag get's VIP 00:30:17 MUL should be OK, this NOOP doesn't work 00:31:20 shapetracked objects, shape, stride, offset 00:31:56 test_mul fails, strides 0's 00:32:08 what to implement today, memory might not be contiguous 00:32:53 break, more coffee 00:34:40 ShapeTracker object has not attribute shape_strides 00:35:08 question about Shape tracker 00:35:40 working on idx 00:37:40 rewriting expr, expr junk code 00:38:14 idx_to_llvm, idx_deref 00:41:58 don't use print statements for debugging 00:45:24 llvmlite fadd fmul, fneg 00:46:55 easy implement sub op 00:48:19 RELU not implemented 00:48:30 segfault 00:48:55 arm reciprocal instruction 00:49:18 segfault, why not implemented error? 00:53:50 WARNING HEADPHONE USERS 00:54:40 not implemented error 00:57:10 RELU not implemented 00:58:35 shipping comma ai into car mirrors? 01:03:30 chromecast for cars, just solve self driving 01:05:28 new Blink-182 01:07:35 Axel Nix diss, nathan for you episode, this is going be the greatest thing you ever do with your life 01:09:54 Axel Nix if you are watching this let that sink in 01:11:10 you don't want to punch downpunching down 01:14:08 Axel Nix business, reaching to his clients, don't hate the player hate the game 01:17:50 never ever settle a lawsuit, barnacle on yacht, Peter Thiel on Gawker 01:20:00 Alex Jones lawsuit 01:20:30 send Axel Nix this video, George is real 01:21:47 segfault 01:22:14 does George censor himself on twitch 01:25:00 taylor swift new album, nvidia unlaunching 4080 01:25:40 writing idx_deref 01:26:50 reason for ops_llvm 01:28:29 explaining idx, writing idx_deref 01:32:55 what is it doing, segfault 01:35:30 llvmlite pow exp 01:44:10 exp ops, no reduce ops 01:45:12 ontologicaltofu suggests Asan to check segfault 01:48:05 explaining ZeroView 01:48:55 segfault in python gc 01:50:45 segfault sometimes 01:53:50 what is getting gc to segfault 01:55:13 python faulthandler 02:02:18 python os exit 02:06:32 something is not getting saved 02:07:57 gc.set_debug(gc.DEBUG_COLLECTABLE) 02:10:00 disabling gc 02:17:35 segfault sometimes 02:27:00 wrong info from chat about gc 02:29:00 WARNING HEADPHONE USERS 02:31:05 Bitcoin__ asks about font size 02:33:10 George segfault rage 02:35:35 it was too tiny 02:40:20 loop was too long 02:40:42 memory corruption can show up as any kind of bug, always add some padding 02:44:05 Hailo-8 AI Processor 02:48:30 michael scott yacht quote from money 02:49:34 world without patents, copyright, we keep trademarks 02:50:20 robot lawyers 02:51:28 lesson about patents, copyright term extension act 02:54:55 canceling cox services 02:56:00 google duplex 03:01:00 promise to Alex 03:01:50 why video games, most movies sux 03:03:00 don't care about hacking 03:03:40 hamster to panda 03:04:05 cost of making movies, factorio 03:05:20 outer wilds, elder scrolls sux 03:06:05 WARNING HEADPHONE USERS 03:07:05 web3, comma body cooking meals 03:07:49 different between lot's ifs vs switch 03:08:17 brothel idea 03:09:35 google alpha tensor 03:10:33 Meta AITemplate GPU framework 03:11:00 Apple contacts Comma 03:12:05 raiding lana_lux
    10
  144. 10
  145. 10
  146. tiny corp is hiring! -> 3x Software Engineer, 1x Hardware Engineer - https://tinygrad.org/ | It's happening George started a project -> Getting AMD on MLPerf -> https://github.com/geohot?tab=projects | Go help George if you are capable! He will appreciate you! | From $1499 buy comma.ai/shop/comma-three | tiny corp has a website tinygrad.org | Check out and star github.com/geohot/tinygrad | Support George by subscribing twitch.tv/subs/georgehotz | Follow George on instagram.com/georgehotz & twitch.tv/georgehotz & twitter.com/comma_ai to be up to date | Read George's geohot.github.io/blog/ | Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 intro 00:01:45 intern at tiny corp 00:02:20 metal kernel, highball drink 00:02:35 dissecting the m1 gpu 00:04:25 india trip, mexico 00:10:20 gpu drivers crazy 00:11:40 metal kernels 00:12:50 opencl global local 00:14:20 IOConnectCallMethod 00:18:45 how gpu works 00:24:30 dev moved to mesa 00:26:55 the good news 00:31:30 apple cheaters 00:37:10 metal reduce kernel slow 00:40:00 bug 00:42:20 detecting not respecting local size 00:46:25 quantum localsize, changes state when observed 00:49:00 python metal, pip metalcompute 00:52:50 global system of capital 00:53:20 it build, almost cuda 00:57:00 writing swift code, xcode 00:57:38 playing the piano 00:58:50 upset or push on 00:59:00 low stars no trust 01:00:15 what is the point 01:03:00 how do you play a role in AI revolution 01:03:30 compute scaling, latency 01:06:20 if AI's are coming how to make sure power is not centralized 01:07:40 emad vs openai, azure 01:10:45 things don't scale always 01:11:45 the idea of comma, thank you tesla, elon 01:13:20 caring about safety, talking about safety 01:16:00 state actors 01:18:55 lil wayne - deep 01:20:05 paper clipping ai 01:22:45 why people join them 01:24:30 forbes sam abuelsmid comma.ai 01:26:35 openai draw, tech and power 01:30:20 what benefited people, no power over people 01:33:00 mark zuckerberg one of good ones, jack dorsey 01:35:25 money, power and love, spam, getting food 01:44:45 let's make some metal 01:57:30 use class methods to instantiate new objective 02:01:35 undo broken in vim plugin 02:06:40 food 02:31:50 metal in pure python 03:04:55 what we should do 03:31:00 metal backend 03:45:00 how reduce work, faster in opencl 03:52:00 bug, serious bug in test 04:02:55 half the time for quarter of the size 04:05:20 two stage reduce 04:07:55 metal timing, fix speed test 04:09:00 this looks fast 04:11:25 add is important 04:14:25 opencl atomic 04:21:10 opencl, cuda, metal concepts 04:23:40 amd close to the metal 04:27:15 timeout for bad chat user 04:30:20 threads by position by thread group 04:38:40 fast but wrong answer 04:42:20 11% speedup 04:46:45 timeout for stupid question 04:49:00 big flops wrong answer 05:02:05 amount of shared memory bandwidth 05:08:50 how to get stats on metal kernel 05:26:20 metal shader compiler verbose 05:28:40 lot's to learn about metal 05:37:30 speedup every day 05:41:15 tinygrad lines 05:43:35 what did we do today 05:47:45 bounty for pull request ($100-$1000) 05:51:35 real price $100 05:54:00 metal matmul 06:02:40 copy in fix, money if you solve really hard problems 06:11:01 how to scale it up 06:26:25 answer right but slow 06:34:30 never because of 06:37:00 hard to implement, slow, secret apple kernel 06:38:40 $500 for strassen implementation 06:39:00 implemented nice metal library 06:39:30 unabomber questions 06:40:25 lesson for everybody 06:41:30 every day a little faster, better 06:42:42 we got the flops 06:43:30 copyright music, addictive 06:44:25 the interesting things about the solution 06:47:20 brew breaking computer 06:47:45 secret api 06:56:45 m1 max benchmarks github 07:04:30 appreciation for flops 07:09:55 happy ending 07:12:50 mesa asahi 07:18:33 ninja build one target 07:30:33 apple shader disassembler asahi 07:30:49 dougall johnson 07:43:50 tinyurl service 07:47:45 commit dump shader 08:11:35 promised and delivered 08:12:00 rap ending
    9
  147. 9
  148. 9
  149. 9
  150. 9
  151. 9
  152. 9
  153. 9
  154. Go buy comma three from $1849 https://comma.ai/shop | tiny corp has a website tinygrad.org | Check out and star github.com/geohot/tinygrad | Follow George on twitter.com/realGeorgeHotz & instagram.com/georgehotz & twitch.tv/georgehotz & twitter.com/comma_ai to be up to date | Read George's geohot.github.io/blog | Support George by subscribing twitch.tv/subs/georgehotz Chapters: 00:00:00 intro 00:01:30 Finatra 00:02:30 installing sbt, life in san francisco 00:04:10 twitter intern, twitter codebase 00:05:45 new look 00:06:05 thank you to contributors on twitter, twitter intern salary 00:07:10 comma github 00:07:40 work full time at twitter, hate about twitter 00:08:10 working on search for twitter 00:08:25 getting rid of pop-ups on twitter search 00:09:45 http-server, thrift rpc 00:10:05 what a microservice is 00:10:16 opensource code 00:10:50 why thrift over protobufs 00:11:40 what is sbt 00:12:50 challenge for the haters (coding duel) 00:13:30 hate for Elon 00:13:40 directions from Elon to George 00:14:05 mr peanutbutter bojack horseman 00:14:15 why intern, career ladder, project manager 00:14:45 scala 00:15:40 Finatra vs Finagle 00:15:50 twitter server 00:16:00 never stop being curious, sf bumming me out 00:16:35 like coming in to the office 00:18:00 worrying about looking dumb 00:18:40 working hard on image instead of skill 00:20:20 getting twitter server running 00:23:20 adminhttp 00:23:35 company putting best foot forward with open source 00:23:58 figuring out twitter server 00:26:00 finagle 00:27:35 intern to make search better 00:27:45 don't want leadership position or to manage anybody 00:28:45 cool thing about scala return 00:29:30 sbt kill run 00:30:05 first time seeing sbt, pants, bazel 00:31:55 good on twitter for opensourcing 00:32:20 public stuff more useful then internal stuff 00:34:10 apache projects 00:34:20 your server as a function 00:36:05 finatra basics, hard to understand 00:37:15 svelte 00:39:00 java people broken 00:40:15 react is wonderful 00:41:10 meteor.js, fakemeteor.js 00:41:53 millionjs 00:42:25 bunjs, zig, frameworks broken 00:43:15 python fast, django 00:43:40 not understanding syntax 00:44:00 com twitter util await 00:46:20 sbt vs mill 00:47:40 asking github copilot to explain 00:48:00 twitter scala tutorial 00:48:20 12-week intern at twitter 00:49:00 two minute conversation with Elon 00:49:30 sf doomed, planning to ship small changes next week 00:50:05 media lying 00:50:45 scrooge 00:52:50 creating a server 00:53:50 pycap terrible 00:54:40 http server definition 00:57:40 wrong version of java 01:03:20 unable to load cache item 01:09:40 famous programmers, computer person challenge 01:10:25 goal for stream thrift server talking between scala and python 01:11:45 stupid media news, genius hacker, not a wizard 01:12:40 warning from friend, san francisco 01:14:00 sbt publish local 01:15:50 linkedin not the best dating app 01:16:50 scrooge github, thrift example 01:28:40 object example is not a member of package com.twitter 01:38:15 how is this not deliberately obtuse 01:41:00 coding the client in python 01:47:20 angry people in the world 01:47:40 how to enable debugging 01:54:30 what they are really angy about 01:55:20 levels.fyi, teamblind, eating the popcorn 01:56:15 day in a life of twitter employee 01:57:25 not speaking with any authority 01:57:45 want to see this experiment succeed 01:59:00 Solo0rTroll raid, what we are trying to do 02:00:30 public opensource stuff, free merch from twitter 02:03:05 no internal knowledge, it wasn't me, new people at twitter 02:05:10 bad media 02:06:40 way too complicated for a client 02:08:25 journalist learn to code, media did this to themselves 02:10:05 finatra logging debug 02:18:35 thrift protocol 02:23:10 catb.org the hacker attitude 02:24:30 Alex on the phone 02:28:20 call_multiply.py working, thrift at facebook 02:35:15 explosion of microservices, more lines of code 02:37:15 what finatra get's you, nodejs 02:38:15 flask, rust, elixir, go for microservices 02:39:30 c++ easy to mess up, Rob Pike on go, haskell 02:42:00 open source code better then internal, java 02:44:10 shill for comma three sale, spreading love, it's business 02:45:40 Elon's heart in the right place, mark zuckerberg, who run's TikTok, no MBAs, Project Managers 02:46:40 founders and investors made bad culture, keep control of your companies, build cool culture
    9
  155. 9
  156. 9
  157. 9
  158. 9
  159. 9
  160. 9
  161. tiny corp is hiring! -> 3x Software Engineer, 1x Hardware Engineer - https://tinygrad.org/ | It's happening George started a project -> Getting AMD on MLPerf -> https://github.com/geohot?tab=projects | Go help George if you are capable! He will appreciate you! | tiny corp has a website tinygrad.org. Let your friends know! | https://youtu.be/MeE4Y2862FY (Google Coral Edge TPU part 1) | Check out and star github.com/geohot/tinygrad | github.com/geohot/edgetpuxray | Support George by subscribing twitch.tv/subs/georgehotz | Follow George on instagram.com/georgehotz & twitch.tv/georgehotz & twitter.com/comma_ai to be up to date | Read George's geohot.github.io/blog | Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 intro 00:01:18 Hacker News thread analyzing George, be able to talk about ideas 00:02:00 looking for ways to plug Coral Edge TPU to computer, appreciate good emails, the verge fake news 00:04:09 comma better without me, different person for 100M+ revenue company 00:04:50 George's limitations not patience just want to attack, Andrew Tate 00:06:12 python 3.11 excitement, comma it's a 20 person company, Harold, Alex, Adeeb running the company 00:07:02 comma has 7 millions in the bank, on track back to profitability, making lot's of comma three 00:07:22 comma trajectory good, not much for George to do 00:07:30 10x software engineers 00:08:12 why I am bored with comma, openpilot really good, have build it and works for me 00:08:40 not driven by other people using it 00:08:54 open source reason 00:09:15 go work at comma.ai/jobs 00:09:25 big potential for growth to openpilot product (sales), functionality some growth, pitching product to normal people 00:10:00 keeping board seat, approvals for large purchases, equity grants, wrappers, oreos disgusting 00:10:50 Google Coral correct output picture banana 00:11:01 Mark Zuckerberg supershares, meta stock, Andrej Karpathy lex podcast 00:11:50 respect to Mark Zuckerberg, fan of Elon buying twitter 00:12:03 George's a moderate person, politically Elon will make twitter fair for everybody 00:12:25 Elon calling out The New York Times, monopoly on information, Elon not a China puppet 00:13:18 no AGI in next 5 years, blog post amazon warehouse worker burnout 00:14:25 no OpenAI, DeepMind internship, influencing people 00:15:42 comma body, tiny corporation 00:16:30 Hacker News thread, Andrej Karpathy at comma, not interested in shipping products to people 00:18:03 Taylor Swift Anti-Hero, 4chan, Kanye West, ye 00:19:04 words taken too serious, reporter, friend reached out, not buying blue check mark 00:20:35 twitter improvements, not optimize for engagement, optimize for quality 00:21:17 unreleased song, not optimizing for recency, Alex still at comma 00:21:57 Trump ban, blue and red box explaining conservatives, liberals interpretation 00:23:25 tomcr00se show 00:24:05 Tiny Corporation overview 00:24:45 George leaving comma same as Karpathy leaving Tesla 00:25:04 wartime CEO, Rick and Morty got bored, George is not Elon 00:27:13 still own comma, still the board, explaining once more, total sum of salary 00:28:47 Jim Keller at tiny corporation, contracts 00:29:10 the idea of the tiny corporation, history of programming languages Assembly, Fortran, COBOL, ML frameworks, Alexnet, TF, PyTorch 00:30:25 C, python, new Machine Learning Framework, DSP, transformers, ResNet 00:32:30 openpilot uses tinygrad, what is new ML framework going to look like 00:33:25 TensorFlow, PyTorch high complexity, how much effort is to add new accelerator 00:34:08 tinygrad simple API, tinygrad operations, lazy.py - processing_op, tensor.py - matmul 00:35:45 nice syntax, stable_diffusion.py 00:36:35 Cherry on FPGA, supporting all accelerators 00:36:57 not making chips, getting to FPGA level, working with other companies that want to make chips 00:37:42 Only 3 people, Philippe Tillet, openai/triton 00:38:23 AI chip company contact George for good software, contracts 00:39:40 CEO with skill, humility, Mark Zuckerberg skill, how to judge a successful CEO 00:40:50 Zuckerberg stupid move, John Carmack 00:41:30 Polyhedral Computation, Triton’s documentation, AMX instructions, FSD Chip 00:43:58 low hanging fruit of new ML chip, software needs to be good first, NVDLA 00:45:10 tinygrad 3090 speed, lazy.py refactor, AST, _realize functions 00:47:08 George's Apple M1 Max computer, maxbenchmark_train_efficientnet.py, openai.com/blog/triton/, relu 00:52:30 comma not over, why do you think that? 00:53:00 explaining tinygrad calls, compute 00:55:44 Jax complexity, Levandowski law company looks like their codebase, Google bloated, Kubernetes from Google 00:57:50 no pypi, how to install tinygrad, goals for tinygrad 1.0 00:58:40 why wanted to talk today, the verge hit piece, metaseq, verge guy call me, another reporter reach out 01:00:53 reach out from Nvidia, AMD should sponsor tinygrad, 01:02:10 tinygrad attractive to companies, Theano 01:05:05 TechCrunch reach out, compile and run on edge tpu, reverse engineering the API 01:07:55 the goal of tinygrad LLVM backend, nnfs.io, lag while compiling 01:15:20 strategy build simple things then reverse engineer, stable diffusion Edge TPU support 01:26:43 google ruy, writing running custom model commit 01:35:50 output 6, is it using Edge TPU?, neural engine apple 01:42:10 proof that it runs on the Coral 01:45:00 RetireInProgress raid, exploring, writing driver for Coral 01:50:30 single operation, edgetpu-custom-op, input tensor 1 and output tensor 0 01:57:10 smurfd0 helping George, git clone TensorFlow, break 02:04:45 break 25 min 02:28:35 building TfLite bazel 02:35:35 wishing to be better about comma thing, sending journalists to Harald Schäfer CTO comma 02:36:15 trying to make it work 02:42:50 Should I buy a Porsche Boxster?, carfax, no electric car for George, Lambo 02:48:40 sarcastic George about why government love EV, government control, low range EV 02:49:30 modifying the tpu driver to see if this can be parsed 02:56:40 culture at remote organization 03:03:35 worldbuilding, private advice sessions, monetize advice 03:11:15 about leaving comma, still going to be involved, shipping 100k product, supply chain, real engineering 03:18:00 python program to mimic the C program is doing, edge tpu complier, driver 03:23:20 structuring society in new way that is not capitalism, economics, being public, painting in a nice light 03:27:10 edgetpu_compiler, cool thing to support Coral 03:32:45 docker, NSA backdoor, tinygrad for the startup, tinygrad for Nvidia 03:36:00 docker platform flag, worked, how amazing docker is 03:42:00 reading .tflite files, docker's licensing, using Ghidra 03:52:40 not doubling down on META stock 04:01:25 objective way to measure intelligence, brain machine interfaces 04:04:50 edge tpu decomplier 04:07:55 Google would you like to sponsor George to continue work on Coral Edge TPU 04:10:10 AMD you should sponsor George to support tinygrad 04:11:20 if you only can use Coral simply with tinygrad, apple detection demo on Edge TPU 04:13:45 Coral only supports ints, HBO season 2 white lotus 04:14:45 message to comma.ai archive about doing only things you enjoy 04:15:20 tiny corp is real, for engineer 1 you need insane experience, comma jobs, Google, Nvidia sponsor 04:18:10 chicken, hen on Google Coral, docker script cleanup 04:19:44 local, binary for compiler, how to turn debugging on, Christian channel 04:32:32 edgetpu compiler output logs, profiling 04:44:00 all hacking is strings, dumping firmware, DEBUG=1, 04:49:25 one flag away from magical solution, strings unicode 04:53:40 how did I missed this?, probably wrong approach 04:59:35 email to George debug output to get VIP on twitch channel
    9
  162. 9
  163. Before asking bad questions read http://www.catb.org/~esr/faqs/smart-questions.html | 02:10:10 George raiding https://www.twitch.tv/j_blow | github.com/geohot/cuda_ioctl_sniffer | Support George by subscribing on twitch.tv/subs/georgehotz | Timestamps took a lot of hours to make, suggest better ones in the comments Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 intro George is apologizing for doing things by hand 00:04:19 doing the right thing, clang 00:08:28 what we want to implement 00:09:22 subscriber getting banned 00:10:20 refactoring time, name for the project, cuda name copyright 00:27:25 what's the minimum you need to initialize cuda context 00:29:35 need to get gpu uuid or broken everywhere except my computer 00:37:20 GPU_UUID, nvidia-smi -L 00:40:25 spaces in vs code give me dots (mac only issue) 00:42:35 more refactoring 00:49:28 good comment two times included AMPERE_CHANNEL_GPFIFD_A 00:52:04 what do we want to do next, fast cuda initialization 00:53:45 doing assembly by hand 00:59:30 killing x, open cuda memory usage 2 MB (almost fit on cache) 01:01:30 nvidia driver broken 01:04:50 nvidia cuda driver memory usage (245 MB usage) 01:06:40 this should be a parameter somewhere 01:08:17 hook up with rdma 01:09:18 for 100 gpu, you are saving 20 gb vram 01:09:35 climate change, barrels of oil, carbon offsets 01:11:50 benchmarks, value, malloc 01:14:59 streamer for the body, complicated c++ program, no copies complicated, memcopy 01:16:28 we could try to reimplement the cuda driver, torch to work 01:21:33 smart parser, maintaining the code, investment in infrastructure 01:23:40 vs code advantage vs vim 01:24:35 working on infrastructure at comma, distributed file system 01:25:35 gigashuffle, reason why companies do not opensource 01:26:10 AMD drop in cuda replacement, ptx to amd, spir-v 01:27:50 PTX legal reasons 01:29:05 nice things about nvidia 01:29:46 there is no point in trying to defend a hill anymore 01:31:43 risc-v, lisa su, x86, arm64 01:33:30 qualcomm, selling chips, garbage cloud 01:37:25 TPUs are just fixed function hardware 01:38:50 tenstorrent, making chip is hard, Nervana Systems 01:41:01 what makes cuDNN slow, tools easy to see why it's slow 01:42:55 running two companies, Elon, Vitalik 01:43:33 nvidia profiler 01:44:10 slow reaction times 01:45:10 exactly what parts of the future do you want to own 01:46:40 take mentality, openpilot unstoppable, safety 01:49:35 singing 01:51:25 pytorch memory usage 01:52:25 tensorflow hidden state, JAX, MXNet, 01:53:48 not bad library 01:54:20 bloated software, rewrite the code you have, gigashuffle explained 01:55:38 accelerator people, own the stack, high end 01:57:10 Julia vs python 02:03:00 looking up Jonathan Blow stream 02:04:24 fast python, looking up Jai, different python interpreter 02:10:10 raiding j_blow 02:12:25 why should George use Jai 02:16:15 George ask j_blow why is python slow
    9
  164. 9
  165. 9
  166. 9
  167. 9
  168. 9
  169. 9
  170. 9
  171. 9
  172. Update! https://twitter.com/LisaSu/status/1669848494637735936 Thanks for connecting @realGeorgeHotz. Appreciate the work you and tiny corp are doing. We are committed to working with the community and improving our support. More to come on ROCm on @radeon soon. Lots of work ahead but excited about what we can do together. tiny corp is hiring! -> 3x Software Engineer, 1x Hardware Engineer - https://tinygrad.org/ | It's happening George started a project -> Getting AMD on MLPerf -> https://github.com/geohot?tab=projects | Go help George if you are capable! He will appreciate you! | RDNA3 Instruction Set Architecture Reference Guide https://www.amd.com/system/files/TechDocs/rdna3-shader-instruction-set-architecture-feb-2023_0.pdf | 01:57:15 enjoy your amd GPUs, AMD I want to make your GPUs amazing you should reach out to tinygrad | 01:57:43 will not send them back, I bought this GPUs and I am going to make them work | AMD is hiring Open-Source Linux/Mesa Developer https://www.phoronix.com/news/AMD-Another-Mesa-Dev-2023 Should George apply? | From $1499 buy comma.ai/shop/comma-three | tiny corp has a website tinygrad.org | Check out and star github.com/geohot/tinygrad | Support George by subscribing twitch.tv/subs/georgehotz | Follow George on instagram.com/georgehotz & twitch.tv/georgehotz & twitter.com/comma_ai to be up to date | Read George's geohot.github.io/blog/ | Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 intro 00:00:15 RDNA3 instruction set architecture stream 00:00:30 absolute joy to work with AMD open source drivers 00:00:40 less then half time they crash, time machine that works half the time = amazing 00:01:45 RDNA3 assembly syntax highlighters 00:02:28 pathlib, quiet pc 00:04:45 running shaders 00:05:55 assembly highlighting 00:07:15 v_dual_mov_b32 00:08:30 s_sendmsg 00:10:10 amd driver is lovely, works 95% of the time 00:12:00 power_dpm_force_performance_level 00:12:50 rocm sniffer segmentation fault 00:14:05 big font 00:14:20 what is sniffer doing 00:16:05 gpu doorbells 00:18:30 we are going to write our own simple driver 00:21:20 explaining what we are doing assembly language for RDNA3 runtime io controls 00:22:20 sigemptyset 00:25:40 sigsegv handler inside python 00:26:50 operating between a level kernel and userspace, doorbell 00:28:50 sigaction threads 00:32:25 killed by SIGSEGV 00:32:50 mojo source, no talking about mojo until source released 00:33:25 sa.sa_flags = SA_SIGINFO 00:36:42 rt_sigaction sa_restorer 00:37:15 sigemptyset 00:38:49 sigaction(SIGSEGV, &sa, NULL) 00:39:38 sigaction isn't being caught 00:41:10 rt_sigprocmask 00:41:37 example sigprocmask 00:43:00 Alex 00:43:30 sigprocmask example 00:45:10 Alex bringing ramen 00:46:25 removing LOG_LEVEL=5 00:47:00 Alex, chair 00:49:45 queue_type 1, dma queue 00:50:00 spindrift, tortellini, grateful for ramen 00:50:52 positive upbeat stream, negative stream before, advent of scala part 1 00:51:20 thank you ptim for gifting 10 subs 00:51:40 KFD_IOC_QUEUE_TYPE_SDMA 00:52:15 JacobsM1 gifted 5 subs 00:53:10 HSA Async Copy staged 00:54:10 DMACopy 00:54:25 Alex and George working on GPUs 00:55:00 Alex feeling GPU energy around George all the time 00:56:00 cobracoba gifted 5 subs, mojo questions 00:56:35 it's not as good if it's secret in startups 00:57:05 Chris Lattner open source, modular $30 million 00:58:30 hsa_kernel_dispatch_packet_t 00:59:10 Alex patch & George hoodie matching 00:59:30 stream for entertainment not for education 01:00:00 the packet definitions 01:00:50 stormfrosty: ~4 years of being underpaid at AMD paid off 01:01:10 it's not ok not being aware of how little you know, taking pride in not knowing things 01:01:50 sunrise san diego, Alex get clothes ready for sunrise 01:02:55 Lana_Lux raiding stream 01:03:30 today we are looking at all the parts of the gpu driver in order to make a good gpu driver 01:03:41 doorbell, queue_type transfers, amd open source 01:05:15 PM4 packets, PGM_RSRC1 01:06:39 AMD's PAL github 01:09:35 hsa_status_t DmaCopy 01:11:00 DmaCopy(void dst, SubmitLiniarCopyCommand 01:11:25 HSA_PACKET_TYPE_BARRIER_AND 01:13:00 HSA_ENABLE_SDMA=0 01:13:55 CPDMA, how low level is this? it's at a user space level 01:14:30 enable_sdma 01:16:20 SDMA_PKT, SDMA_OP_TIMESTAMP, SDMA_OP_GCR 01:20:10 fatal error file not found 01:25:40 George got distracted, sdma_registers.h 01:26:30 offset 01:27:55 tinygrad code gen engine is good 01:28:05 unhandled packet type 17, unhandled packet type 1 01:30:00 end goal is to make it good 01:30:10 SDMA cool, different queue, unhandled packet type 10 01:30:55 User Mode Register Debugger 01:31:30 George like writing himself so he can learn about how it works, did the same with CUDA 01:32:22 SMDA_PKT_COPY_LINEAR 01:37:20 we understand how the buffer get's transferred 01:38:05 queue_type 1 01:40:45 this running the program, transferring buffer 01:41:00 copy kernel guess, dumb the args 01:41:25 hsa_kernel_dispatch_packet_t 01:43:00 tired happy for a nice stream, positive progress 01:43:15 if you had a time machine that worked 98% of the time 01:43:40 thank you for gifted subs, we got custom assembly running on gpu 01:44:10 dumping kernel arg address 01:46:20 ComputeABI initial-kernel-register-state 01:46:40 private segment buffer 01:47:45 what is the address 01:49:00 what address is actually in the gpu 01:49:35 you can step through the kernel assembly with gdb 01:50:15 github.com/RadeonOpenCompute/ROCm_Documentation/blob/master/ROCm_Tools/ROCgdb.rst 01:51:00 rocm-gdb 01:52:00 break point 01:52:40 could not attach a fatal error has occurred 01:53:40 global_store_b32 01:56:40 ds_load_b64, s_load_b64 01:57:15 enjoy your amd GPUs, AMD I want to make your GPUs amazing you should reach out to tinygrad 01:57:43 will not send them back, I bought this GPUs and I am going to make them work
    9
  173. 9
  174. 8
  175. 8
  176. 8
  177. 8
  178. 8
  179. 8
  180. 8
  181. 8
  182. 8
  183. 8
  184. tiny corp is hiring! -> 3x Software Engineer, 1x Hardware Engineer - https://tinygrad.org/ | It's happening George started a project -> Getting AMD on MLPerf -> https://github.com/geohot?tab=projects | Go help George if you are capable! He will appreciate you! | From $1499 buy comma.ai/shop/comma-three | Go read and share the docs with friends -> https://github.com/geohot/tinygrad/tree/master/docs | tiny corp has a website tinygrad.org | Check out and star github.com/geohot/tinygrad | Support George by subscribing twitch.tv/subs/georgehotz | Follow George on instagram.com/georgehotz & twitch.tv/georgehotz & twitter.com/comma_ai to be up to date | Read George's geohot.github.io/blog/ | Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 intro 00:00:45 tinygrad docs 00:01:10 triton rst docs 00:02:10 debt in tinygrad 00:03:05 great stream to learn how everything works 00:05:25 george teaching you programming 00:06:25 2+3 as Tensor 00:10:25 Types from Tensor 00:11:00 accountants problems, integrity 00:11:40 boilerplate imports 00:12:18 vscode python annotations %% 00:16:10 understanding the example 2+3 as a Tensor 00:17:50 derivatives in tinygrad 00:21:20 Tensor, relu, mlops example 00:24:45 LazyBuffer 00:25:20 pip or install it via git clone 00:26:10 LazyBuffer 00:29:55 lazydata 00:33:35 lazyop 00:41:10 hlop, mlop 00:41:25 thinking about this for 2.5 years 00:41:40 tinygrad hope 00:41:55 worth reading to understand hlop, mlop 00:42:25 tinygrad not slow, pytorch and tensorflow hundred of ops 00:43:15 getting pytorch support hard, why are things slow 00:44:00 worth reading 00:46:45 why can't FROMCPU be folded into ASTs? 00:48:45 example is tinygrad fast 00:49:30 DeviceBuffer 00:52:00 subclass c++ 00:53:30 InterpreteredBuffer 00:54:35 don't worry about lazybuffer 00:55:10 DeviceBuffer 00:57:35 inops support 00:58:20 lazy.py 01:01:10 RawBuffer, 2+3 in raw clang 01:05:45 2+3 autogenerated clang code 01:10:55 ShapeTracker 01:23:55 no Tensor just a shape 01:24:25 variable in shape/symolic.py 01:33:00 theory of lines of code 01:33:50 new issues on github 01:34:08 JIT=1 01:37:35 what jit does 01:38:50 unittests 01:47:00 bug in tests 01:48:30 don't know anything about computers 01:49:10 tinygrad lines and garbage lines 01:50:00 2k lines, tiyngrad backends 01:51:55 5 lines for speed, shape, reshape 01:53:30 permute, reshape, expr_node 01:54:40 symbolic.py 01:55:20 fixing broken the tests 01:59:22 unittests parametrized test 02:06:00 looking at the docs 02:08:25 optimizers SGD, RMSprop, Adam 02:08:35 test_optim.py, challenge for pull request 2 line 02:09:00 lion optimizer, experimental drugs waiting 02:10:35 admiring mlops.py, looking how long is adam optimizer 02:13:50 llvm=1 on example, slow because copy overhead 02:17:40 tinygrad metal backend faster then pytorch backend 02:18:50 new goals for tinygrad, train imageNet 02:20:40 step to linearize the ast and codegen the ast 02:21:10 what is different about jax 02:29:20 numpy and finding a bug in tinygrad 02:43:15 adding test_div_numerator_negative 02:47:10 thinking about what to do 02:54:50 tinygrad symbolic implementation 03:00:35 removing tests 03:01:55 linking docs to tinygrad discord 03:02:55 commit bufs not none 03:04:45 some abstractions are good 03:05:15 codegen needs new version 03:06:10 goals for 0.6 release 03:07:00 code is a line, ast is a tree 03:07:30 reviewing code 03:08:55 tinygrad is not for openpilot, is competitor to pytorch 03:11:10 symolic is now a 6/10 due to the infinite loop 03:14:10 read and share the docs with friends 03:14:20 is tinygrad stupid? a lot of time invested into tinygrad 03:14:50 who is tinygrad for? 03:15:30 atan2_cpu 03:16:15 tinygrad stable diffusion 03:17:10 why python 03:18:15 transformers 03:18:33 pyright 03:22:00 tinygrad is just george and community 03:22:33 batman joke in javascript 03:24:10 ENABLE_METHOD_CACHE=0 03:24:40 pytorch is broken 03:26:30 workday start at comma 03:27:35 imagenet to c compile test 03:29:10 tinygrad will win, fast 03:30:00 agi does not exist 03:30:15 gpt-4 this week 03:30:50 ban for bad user 03:31:05 how readable is tinygrad code 03:32:20 plans for tinygrad inference and learning as the same thing 03:32:40 long term re-writing tinygrad in tinygrad 03:33:25 andrej karpathy on training vs inference
    8
  185. 8
  186. 8
  187. 8
  188. 8
  189. 8
  190. 8
  191. 8
  192. Update! https://twitter.com/LisaSu/status/1669848494637735936 Thanks for connecting @realGeorgeHotz. Appreciate the work you and tiny corp are doing. We are committed to working with the community and improving our support. More to come on ROCm on @radeon soon. Lots of work ahead but excited about what we can do together. https://github.com/kposborne2 is the hero of the stream! https://github.com/geohot/tinygrad/pull/823 | The tinybox $100 preorder total cost $15,000-> buy.stripe.com/5kAaGL6lk9uX9nW144 | geohot.github.io/blog/jekyll/update/2023/05/24/the-tiny-corp-raised-5M.html | tiny corp hiring -> tinygrad.org Issues submitted by George to AMD (AMD response patch https://youtu.be/Mr0rWJhv9jU?t=175): https://github.com/RadeonOpenCompute/ROCm/issues/2198 ([Driver] BUG: unable to handle page fault for address: ffffa7c13eaffff8) https://github.com/RadeonOpenCompute/ROCm/issues/2205 ([Driver] amdgpu 0000:c3:00.0: amdgpu: [gfxhub] page fault (src_id:0 ring:24 vmid:12 pasid:32773, for process rocm-bandwidth- pid 8285 thread rocm-bandwidth- pid 8285)) https://github.com/RadeonOpenCompute/ROCm/issues/2204 ([Driver] WARNING: CPU: 2 PID: 479 at drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c:471 amdgpu_bo_move+0x261/0x2e0 [amdgpu]) https://github.com/RadeonOpenCompute/ROCm/issues/2197 ([Driver] WARNING: CPU: 96 PID: 903 at drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c:600 amdgpu_irq_put+0x9f/0xb0 [amdgpu]) https://github.com/RadeonOpenCompute/ROCm/issues/2196 ([Driver] ERROR MES failed to response msg=2) tinygrad/tinycorp archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQsh0fFZ2QKOBY25lz04A3hi Chapters: 00:00:00 intro 00:00:45 tiny corp raised money, tinybox 00:01:40 tinygrad goals, developer experience 00:02:15 tiny corp programming challenge 00:03:00 don't waste George's time 00:04:15 tinybox almost 100 preorders 00:04:45 $100 preorder, $15,000 total price 00:05:25 tutorial.ipynb 00:08:40 set_debug does not work, python import global variable makes copy 00:16:30 lazy by default, should not lazy forever, enqueue 00:17:00 fixing debug problem, properly on global variable python 00:17:45 class debug, python magic methods 00:26:05 inherit from int python 00:28:50 1000 lines guideline 00:29:15 class debug 00:31:30 new debug behavior, clean clode 00:34:00 logging modules 00:34:15 contiguous 00:35:15 matrices, Tensor.eye 00:36:45 llama.py, debug on llama.py 00:38:15 quantization support, contribute to tinygrad 00:38:42 Conv2d, stable diffusion 00:39:25 $10000 bounty for stable diffusion on M1 faster then any existing implementation 00:40:35 a cat in a hat 00:40:47 bounty taxes 00:41:00 stable_diffusion.py and llama.py code 00:41:20 unit tests in tinygrad, ONNX tests 00:41:54 TORCH=1, external_text_onnx_backend.py, pytest 00:42:40 great way to good started 00:43:20 complexity management, self driving people dumb 00:43:46 revenue company, waymo and cruise, growth, investment, referral 00:47:25 smart pytorch people, not managing complexity 00:48:45 implement every op, conv_transpose2d 00:49:50 conv, reshape, sum 00:50:10 competing against smart people, developer experience, hard to develop on those libraries 00:50:50 jit.py, $500 bounty LLaMA python code without jit under 100ms 00:51:50 triton tutorials, conv2d example 00:56:55 mojo, einops-style API, $200 bounty Zero dim Tensor support 00:57:25 zero dimension tensors 00:57:50 haters Hacker News, uncountable infinities, cardinality, recursively enumerable 01:00:55 constructivist 01:01:25 matrix multiplication, a@b, cube 01:03:55 different matrix multiplication 01:05:20 kernel logs are so beautiful, colors 01:05:55 efficientnet.py, NUM=0, NUM=4, sd cat, cowboy hat 01:07:05 DEBUG=2, OPTLOCAL=1, kernel that are running 01:07:40 .realize ending the laziness, NUM=4 kernel 01:08:15 r = reduce kernels, e = element wise kernels 01:08:50 George friendly to noobs 01:10:00 autotune stuff, add mlperf bert model pr 01:11:30 adding tinygrad to huggingface 01:11:50 getting started contributing to tinygrad, fix ONNX tests 01:12:15 AMD driver stack, emailing AMD, intel amazing chip docs 01:12:30 intel a770 $349, XMX AI capabilities 01:14:55 fp16 flops on a770 vs 7900 xtx 01:15:24 I like AMD, driver is so bad 01:15:38 prototype tinybox, two gpus and driver crashes, intel 01:16:55 QLoRA, Falcon 40B model, Nvidia 01:18:25 500+ tokens per seconds, max 137 tokens per seconds 01:19:05 doing AMD work, free work, writing open source code 01:19:55 AMD more documentation open, memory architecture, scheduler, copy engine 01:20:30 why isn't AMD doing that already? 01:21:25 composable kernel, leapfrog nvidia, consumer hardware without gpu, m1 01:22:50 tinybox RAM bandwidth, AMD epyc, pcie lanes 01:24:00 RWKV, infinity fabric 01:25:15 how backends are implemented in tinygrad 01:34:00 transferring kernels to device 01:39:40 DCGAN 01:42:30 transfering img_align_celeba.zip 01:45:00 pre-commit hooks for tinygrad, zip and folder, jpeg 01:48:00 writing dcgan.py class Generator 01:50:50 class Discriminator 01:53:40 celeba_iterate 02:10:35 training tested very little 02:16:00 it uses a stride 02:17:20 training something, tensor object has no attribute _ctx 02:24:00 in theory this should work 02:30:35 need to make strides work 02:31:00 simple code, tinygrad, does not work 02:33:30 time on GPU, OPTLOCAL 02:40:15 do people like gan's? 02:42:42 if somebody fixes strides by tomorrow George will stream and finish dcgan 02:43:30 ugly permute, test_nn.py test_ops.py 02:44:20 big font today, depthwise convs 02:45:00 matrix decompositions, where we draw a line in tinygrad 02:46:00 no sorting, no scatter gather, if it's easy to implement 02:46:35 tinygrad ops, work together collaborate 02:47:20 tinygrad discord 02:47:30 how to deploy $5 million 02:48:25 tinybox prototype 02:49:15 limiting the hiring pool, changing the blog post 02:50:40 bad submission for programming challenge 02:51:44 tax, Joe Biden, remote work, budget of tiny crop 02:52:20 AMD on MLPerf, how Elon feels 02:53:10 sponsor visa, tiny corp is only George, rolls-royce, lost money on cheapETH 02:54:30 Lana_Lux raid, finishing DCGAN tomorrow if someone fixes strides 02:55:45 trying to raid Lana_Lux, enjoyed Lana stream, unity 02:57:05 brief overview of tiny corp, cow farms, chicken farms 03:04:30 avoiding monopolies on compute, ai safety, can't stop technology 03:10:05 Alex, tiny corp a company writing some open source machine learning software, making AMD competitive
    8
  193. 7
  194. 7
  195. 7
  196. 7
  197. 7
  198. 7
  199. 7
  200. tiny corp is hiring! -> 3x Software Engineer, 1x Hardware Engineer - https://tinygrad.org/ | It's happening George started a project -> Getting AMD on MLPerf -> https://github.com/geohot?tab=projects | Go help George if you are capable! He will appreciate you! | tiny corp has a website tinygrad.org. Let your friends know! | George is streaming right now on https://www.twitch.tv/georgehotz part three of stable diffusion | github.com/geohot/tinygrad | github.com/CompVis/stable-diffusion | Support George by subscribing on twitch.tv/subs/georgehotz | Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 intro 00:02:06 quiet computer 00:02:25 how far did we get yesterday 00:04:08 cleanup 00:06:15 class Linear 00:07:50 class UnetModel, SiLU 00:10:25 fighting Mark Zuckerberg, Elon Musk, Vladimir Putin 00:13:10 hand to hand combat, Elon is cool 00:14:35 input_blocks 00:19:14 fusion power, stagnation 00:22:40 chemical engine, rocket fules 00:36:25 preview of blog post 00:51:10 class UNetModel, UnetResnetBlock, ResBlock 00:58:40 crypto is good for one thing 01:01:50 don't actually clip it, no clips 01:02:00 cloudflare bad move 01:02:50 blockchain will not replace sql database 01:03:45 crypto DAOs 01:05:20 meta stock 01:06:19 I take back the clip, don't want clips 01:07:23 blockchain things that will succceed 01:08:18 drugs are not cool anymore 01:08:45 unregistered securities 01:13:47 decentralization 01:14:40 cheapETH 01:17:20 what can you buy with money 01:17:58 crypto disrupting fiat currency 01:21:00 lack of respect to Russia 01:25:27 I don't know that much about this stuff 01:26:25 stop building technology 01:27:35 communism, vietnam, capitalism good or bad 01:28:24 christianity 01:31:32 population collapse 01:32:55 ubi 01:36:25 go protest outside cloudflare 01:37:45 two kinds of power 01:42:44 cloth masks 01:43:35 race relations 01:45:25 metaverse, love in the world 01:46:44 more Elon's in the world 01:47:00 scams in self driving space 01:47:50 politics is interested in you 01:51:20 you got to earn the right to an opinion 01:52:00 nn linear pytorch 02:12:50 are we learning? 02:14:20 understand the structure 02:17:20 why is that that 02:18:55 saying hello to Alex 02:19:18 rules for chat, rules for any social environment 02:22:18 middle block 02:26:35 kanye west 02:28:50 when people say something stupid 02:31:55 fan spinning up, slow 02:34:55 VIP for agifirealarm 02:37:50 what tensor.uniform doing 02:41:25 subs and non subs 02:41:45 done diffusion model, break 02:45:10 transformers are sample efficient world models 02:49:00 visual transformers paper 02:52:15 hutter price, understanding math in this book I could solve AI 02:58:05 job at huawei, american propaganda about china 03:02:55 optimism crypto 03:06:15 averaging weights leads to wider optima 03:07:30 can't reshape 03:21:00 object is not callable 03:25:18 just because you subscribed we are not going off-topic 03:26:45 AssertionError, has no attribute reshape, not callable 03:34:45 UNetModel 03:39:19 what is the dimension there 03:41:55 good idea 03:42:25 chunk 03:47:00 Alex call 03:50:13 torch chunk 03:56:10 np.random.uniform 04:01:50 was wrong 04:05:55 timeout for bad commenter 04:06:20 finding print 04:07:55 can't reshape 04:10:25 new problem, output blocks 04:12:45 it's a unet 04:16:30 it just ran 04:20:15 why it's not lazy 04:23:10 torch tinygrad fusion implementation 04:23:39 Alex bring food 04:24:55 load apple latent space 04:27:35 we are out of memory, tensor.no_grad=true 04:30:25 opencl=1 04:38:40 the problem was nograd 04:42:45 black apple, bajillion bugs 04:51:50 non subs talk 04:52:20 two things to do 04:57:35 more streams 04:59:18 deep understanding by doing 05:00:00 Pat Stay death
    7
  201. 7
  202. 7
  203. 7
  204. 7
  205. 7
  206. 7
  207. 7
  208. 7
  209. 7
  210. 7
  211. 7
  212. 7
  213. 7
  214. 7
  215. 7
  216. 7
  217. 7
  218. 7
  219. 7
  220. 7
  221. 7
  222. 7
  223. 7
  224. 7
  225. 7
  226. 7
  227. 7
  228. 7
  229. 7
  230. 7
  231. 7
  232. 7
  233. 7
  234. 7
  235. 6
  236. 6
  237. 6
  238. 6
  239. 6
  240. 6
  241. 6
  242. 6
  243. 6
  244. 6
  245. tiny corp is accepting new interns. more info on tinygrad.org and tinygrad discord | comma.ai is accepting interns comma.ai/jobs#open-positions Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | twitter.com/__tinygrad__/status/1729279842376613941/photo/1 | youtube.com/post/Ugkx49TlILe21dXuhJu6a74j5WCx3nGldqCC from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | from $999 comma.ai/shop/body the future of people github.com/tinygrad/tinygrad <- simple powerful deep learning framework Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 intro 00:01:35 responding to youtube comments 00:02:05 C. S. Lewis quote 00:03:05 physiology and sociology, wokeism, trans people 00:04:35 harvard, ivy league educational institutions 00:07:15 tesla cybertruck 00:08:40 communism, anarchy 00:10:15 based elon 00:10:40 Amouranth, Sky Bri 00:12:20 love, hate 00:13:35 refactor stream 00:14:05 beautiful.py 00:14:50 memory objects in opencl, buffers and images 00:16:00 dtype, cast_image, device hack 00:17:35 LRUAllocator 00:22:18 Clang operations, MallocAllocator 00:23:20 missing 1 required positional argument 00:27:40 this is the actual George's work 00:28:45 LazyBuffer problem 00:32:00 numpy 00:32:45 Alex 00:33:20 auto generated program, float 00:35:00 tinycorp second intern 00:36:40 TORCH=1 test_ops.py 00:38:15 adding cast 00:38:30 unreadable code 00:41:00 UnaryOps_CAST 00:42:10 DiskAllocator dtype 00:46:20 git push 00:47:00 letting everyone chat 00:47:30 be nice to people 00:47:50 what is AGI 00:48:20 why care about views 00:49:00 beautiful.py, tinygrad is a tensor library 00:51:00 ask good questions, anyone understand this? 00:54:45 memoryview cast 01:00:00 copyin function 01:03:50 buffer, JITRunner 01:05:00 The Device, LazyOp 01:10:40 renderer 01:12:25 current line count of tinygrad, hiball energy 01:18:00 questions? does this make sense? 01:20:40 FROM_BUFFER=1 examples coder.py 01:22:10 10 runtimes in tinygrad 01:26:50 computation, memory 01:31:40 where you call the kernels, ClangProgram, to_program, CompiledASTRunner 01:33:20 abstractions.py outdated use beautiful.py instead 01:33:50 MULACC trigger, matmul 01:37:35 git push improve docs 01:38:30 t8 tinybox spec 01:39:00 jax_pmatmul.py 01:39:35 WD_BLACK 1TB SN850X 01:40:30 bidirectional copy peak bandwidth GB/s 01:42:00 infiniband connection between two tinyboxes 01:44:10 up to 18 monitors into your tinybox 01:44:40 shipping to EU 01:45:20 preorder tinybox today 01:46:20 stupid question from non-sub 01:46:35 mlperf, how fast you can train the models 01:47:50 15k GPU hours 01:49:20 the cloud, low priority CPU 01:50:10 tinybox for common user or business, secret price 01:51:10 tinybox wattage 01:55:20 Alex brining tea 01:57:10 hashcat example 02:06:25 4090 vs tinybox hashcat 02:09:40 Alex bringing lunch 02:11:45 hip for hashcat bug 02:12:20 best coin to mine with gpus 02:13:35 amd 8000 02:14:30 advent of code day one speedrun 02:19:20 AoC part 2, asking Quentin for help 02:26:00 write a regex to replace 02:30:20 good programming style 02:31:00 struggle in python console 02:37:15 reducing the lines of code 02:50:55 compiling 02:57:50 memory usage bug 03:06:00 external_test_alloc_all.py 03:11:40 analyze the issue in the comments 03:13:55 why tinygrad has to be so tiny 03:14:30 changing culture, gpt-fast 03:17:05 27 tinyboxes = person 03:17:25 teenyboxes = gaming computer 03:18:20 pro esports player 03:19:10 1Torch, unspecified launch failure 03:24:30 ???, OpenLLaMA, Nvidia 03:27:50 if George was CEO of OpenAI 03:30:25 opensource weights 03:30:50 company that lasts 1000 years 03:31:30 top 10 biggest companies 03:32:15 Apple M3 fastest single thread cpu 03:33:30 Apple GPU 03:35:50 high end M3 Max, don't buy low end M3 03:37:25 5 M3 Max or 1 tinybox 03:38:20 based Apple getting rid of touchbar 03:38:45 spec of tinybox 03:39:10 wokism, truth crime interrogation footage
    6
  246. 6
  247. Watch youtube.com/watch?v=iFUmWho7fBE - The AI Alignment Debate: Can We Develop Truly Beneficial AI? from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | tinygrad.org | github.com/geohot/tinygrad | Support George by subscribing twitch.tv/subs/georgehotz | Follow George on instagram.com/georgehotz & twitch.tv/georgehotz & twitter.com/realGeorgeHotz & twitter.com/__tinygrad__ to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 intro 00:00:50 comma 3x, watch COMMA_CON 00:01:30 don't use Qualcomm 00:02:20 friends with AMD, 60 GPUs 00:03:05 AMD ROCm github activity 00:03:54 Qualcomm will not improve, AMD will 00:04:40 Rockchip RK3588 chips for comma, sending $500k to Qualcomm 00:06:10 software vs sales people, lies 00:07:22 do not work with Qualcomm 00:08:10 AMD SoC, Ambarella, MediaTek 00:08:47 Qualcomm will sell you chips, 3 months later, thundercomm 00:11:10 get comma 3x while you can 00:11:30 company alive, Elon power level 00:12:00 X rebrand, still believe in twitter 00:12:39 Nvidia selling chips, Qualcomm, AMD stock 00:14:05 Meta stock, Qualcomm, Llama 2 00:15:35 George debate with Connor Leahy 00:16:30 don't use Qualcomm 00:17:15 bad people, trusting people 00:17:50 MediaTek, Rockchip, Orange Pi 5 Plus 00:18:25 Problem with HiSilicon, Exynos government 00:18:45 RK3588, Qualcomm chips off phones 00:20:10 comma 3x, Qualcomm hypervisor spyware 00:21:00 Connor Leahy other debates 00:21:20 Qualcomm licence to the chip 00:21:45 shipping 50k devices don't use Qualcomm 00:22:20 switch to Rockchip RK3588 or Ambarella 00:23:10 is Rockchip coming new better chip this year? 00:23:40 new 4-5nm Rockchip with 2-4x perf would be a big success 00:24:25 chips are very complex to build 00:24:50 looking to buy 100k chips, test order 00:25:47 who is buying Qualcomm, google ARM, Qualcomm perf 00:27:10 can't even get in touch with Samsung 00:27:30 why other companies still work with Qualcomm, integrity 00:28:10 comma lunch, Qualcomm stress, aggravation 00:28:40 software challenges with changing to different SoC 00:29:30 George cares, dealing with people I hate 00:30:00 how to stop depressed people from doing bad things 00:30:25 Qualcomm just lie, one more thing, legal document, bait and switch 00:31:15 patent troll saga, leave me alone 00:32:15 Qualcomm worst company I ever dealt with, Qualcomm via gray market 00:34:10 AMD get's better, Qualcomm will not get better 00:35:00 Intel is good, Intel GPU not powerful enough, Intel Software team 00:36:10 big activist investor intel, Jim Keller legacy 00:36:30 making SoC is super hard 00:36:40 Qualcomm legal document not enforceable 00:37:45 the process to buy Qualcomm chips 00:38:40 don't work with idiots, business world 00:38:55 bad incentives for sales people, Elon could turn Qualcomm around so fast 00:39:30 10.000 chips test order, bearish on tinygrad 00:39:50 country where people want to make world a better place 00:40:18 acceleration and deceleration 00:41:00 e/acc, jerry adams curve 00:43:50 world where energy usage is huge, global warming 00:44:40 getting cancelled, vaccines, bill gates, trans, augmented olympics 00:45:45 eugenics, hr department, boss at tiny crop 00:46:25 nootropic stack, getting canceled by God 00:47:15 lunch, decompressing about Qualcomm 00:47:35 Apple will never sell chips, only God can cancel me t-shirts 00:47:50 Alex Jones, George's politics, conspiracy theories, Alex Jones youtube 00:49:25 mainstream media so bad they took off Alex Jones 00:49:43 the soviet and american talk about propaganda 00:50:15 no andrew tate talk, sigma male discord 00:51:25 Unisoc T820 Processor 00:54:35 sigma male discord fake, business proposals via email 00:55:50 lost $5000 on cheapETH, great about haters 00:58:10 Llama 2 after back from lunch 00:58:22 how IBM still exist, zombie companies 00:59:20 google in bad era, microsoft turning around 01:00:20 history of woke era 01:01:50 sam altman, elon sense of humor 01:02:40 worldcoin orb 01:03:10 sam altman, gm shuts down cruise, microsoft shuts down openai 01:03:55 paperclipped by openai, stories about effective altruism 01:05:00 Lana_lux raid 01:05:40 Elon making the blue line going up, BAP 01:06:05 Nietzsche, Thus Spoke Zarathustra book 01:06:35 tiny box on full power noise 01:07:22 case for tiny box 01:07:45 tst_disk.sh 01:08:20 tst_gpu.sh 01:09:00 gpu_reset.sh 01:09:40 AMD software not the best 01:11:55 don't trust ROCm env 01:11:40 can't buy from hisilicon kirin 01:11:55 roko's basilisk
    6
  248. 6
  249. 6
  250. 6
  251. 6
  252. tiny corp is accepting new interns. more info on tinygrad.org and tinygrad discord | comma.ai is accepting interns comma.ai/jobs#open-positions from $1150 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | from $999 comma.ai/shop/body the future of people Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org github.com/tinygrad/teenygrad <- If tinygrad wasn't small enough for you... github.com/tinygrad/tinygrad <- simple powerful deep learning framework Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 intro 00:01:00 Apple M3 MAX 00:01:25 Logitech MX Anywhere 00:02:20 tiny corp has an intern 00:02:40 Sam Altman, Effective altruism, Eliezer Yudkowsky 00:06:30 openai, gpt4, EA 00:10:20 advice to openai, regulatory capture, contribute to open source 00:12:42 buy a comma 3x from $1150 00:14:40 people on reddit about openpilot 00:15:35 fire alarm for AGI, violence to stop AI 00:17:00 AI safety, fake AI safety 00:19:25 when AI takes over country 00:20:10 tinybox design reveal 00:20:25 EA, openai regulatory capture 00:22:18 first stream since big drama 00:23:46 problem with political philosophy 00:24:44 argentina currency 00:26:40 mad tasty, Logitech MX Anywhere, hhkb2 00:28:15 daily calm meditation 00:32:00 comma body 00:32:40 teenygrad 00:35:00 5000 lines warning light 00:35:35 what is teenygrad, front end of tinygrad 00:41:55 mnist.py 00:44:35 rawcpubuffer 00:46:50 GPU, apple neural engine 00:47:00 BEAM=2 00:50:35 what things do, what things are 00:53:20 rewrite code to be more clear 00:54:45 favorite test 00:57:50 adding type support to teenygrad 01:01:10 test_reshape 01:08:28 keyerror float64 01:16:30 dataclass 01:22:10 torch repeat 01:29:00 teenygrad is complete 01:31:10 CI failed 01:38:10 dtypes 01:40:40 trash code someone should improve it 01:44:30 adding a lot of lines to teenygrad 01:59:35 astype np 02:01:44 1000 lines 02:11:00 AMD tensor cores 02:12:24 test_real_world.py 02:13:07 mypy 02:30:30 teenygrad pre commit hooks 02:35:05 crappy gpt 02:37:50 yaml lint 02:45:00 nektos act github actions locally 02:47:40 tenstorrent fake buy botton 02:49:30 versions-manifest.json 02:52:35 --container-architecture linux/amd64 02:55:40 cool websites George reads everyday 02:58:05 tiny9 02:58:50 tinygrad discord colors 03:02:40 wandb 03:10:10 tinygrad intern, comma intern 03:12:50 what do you consider genius 03:15:00 IQ 03:19:25 what you can change, what you value 03:19:48 where you apply yourself, what do you believe in 03:25:10 useful advice 03:26:25 what did you change because your watch George 03:31:05 how much money do you need to live? 03:33:30 asshole, how George got money 03:40:50 motivation, find your passion 03:43:10 living in a van 03:44:10 kim dotcom us economy, diet coke 03:45:10 biggest regret, twitter 03:45:50 Elon monarchy is not perfect 03:46:38 2K a week for interns at tinycorp 03:49:50 twitter grok 03:50:05 How to not fail at life. quit bitching about the bitches edition.
    6
  253. Watch youtube.com/watch?v=iFUmWho7fBE - The AI Alignment Debate: Can We Develop Truly Beneficial AI? | tinygrad.org | github.com/geohot/tinygrad from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | Support George by subscribing twitch.tv/subs/georgehotz | Follow George on instagram.com/georgehotz & twitch.tv/georgehotz & twitter.com/realGeorgeHotz & twitter.com/__tinygrad__ to be up to date | Read George's geohot.github.io/blog/ Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org Chapters: 00:00:00 muted intro 00:01:02 un-muted intro 00:01:19 WARNING LOUD!!! qualcomm stock, sell chips 00:02:35 the debate, unpracticed in art of debate 00:03:15 George in echo chamber 00:03:35 doomer, intelligent AI in the future 00:04:25 to win debates rules for radicals 00:05:25 let's expand to the universe, generative tiktok 00:06:15 not e/acc, use more energy, colonize the moon 00:07:02 good mood George, AMD stock 00:07:40 tiny corp, banning people discord 00:08:01 warning for all of you, you should be aware of this 00:08:45 coordination technologies 00:08:55 tiny corp has money, future tinycorp 00:09:20 helping qualcomm, accumulating wealth in a dead world 00:10:20 hype about the future, mcdonald's on moon 00:11:22 invite for serious people to debate 00:11:35 bored with livestreaming, out of echo chamber 00:12:10 excitement about programming 00:13:15 never debating Destiny, Connor Leahy 00:14:05 debate Trump, livestreaming problem 00:15:10 better infra 00:15:25 founder that spends time coding 00:16:30 leverage, hire people, seriously building a company 00:18:00 bitcast PR 00:18:10 you want people to come to you 00:18:25 comma works because clear vison 00:18:55 tinycorp can be anything, tinybox, infra 00:19:35 people around when starting comma 00:19:46 escaping being alone in tinycorp, conviction on a vision 00:20:14 comma will solve fsd with e2e solution 00:20:24 weed every day, beat Elon when starting comma 00:21:16 san diego realtors, qualcomm, clear vision 00:22:05 richard s. sutton, the AI goat 00:22:25 leave san francisco, vegas 00:23:20 moon colonizing company, antarctica 00:24:20 prime intellect inc 00:24:40 hate san diego realtors, it's not the place 00:25:40 hire people and get serious 00:26:00 good open ai infra, accessible ai 00:27:30 putting time in not getting value out 00:28:00 back to simple tinygrad 00:28:15 enterprise customers, free software, train your ai on tinybox 00:28:35 500 pre-orders for tinyboxes at $15k each 00:29:00 time to build tinybox, margin, improving software 00:30:00 literally making people 00:30:40 hard starting a company, stable tiny box is hard 00:31:40 training and inference 00:32:20 tinygrad is a very good idea, DEBUG=2 00:32:50 being better at debates, somalia 00:33:30 bronze age mindset owned space concept 00:33:40 tiny box prototype tiny 5 00:34:10 comma will build tiny boxes, everyone will have agi vs nuclear bomb 00:35:10 connor leahy unique points 00:35:30 George is not comma 00:35:55 agi spam and psyop filter, home base for AI 00:36:40 coding on livestreams 00:37:10 software to replace professional managerial class 00:38:15 hard to hire, Marc Andreessen 00:39:10 not CEO, program and set a technical vision 00:39:30 operation role would get office, people, manage them 00:39:40 incompetent or scammer 00:40:50 want AIs, not existential crisis, sam altman 00:41:29 people that George would hire 00:42:00 pen testers comment trigger 00:42:40 George knows what he is saying, people at comma understand 00:43:10 people don't understand 00:43:30 pretending to be crazy 00:44:15 filter bubbles, the internet exposed people to different ideas 00:44:50 friends high school IQ, adverse selection effects 00:45:20 internet commenter vs average person 00:45:40 does Lex Fridman understand George 00:46:20 phone poster vs desktop poster IQ 00:46:48 EQ not real, Lex great interviewer 00:47:35 threads vs twitter replies 00:48:00 finding a stable place, not drifting forever 00:48:20 not selling bad things, not a scam, no timeline 00:48:55 LK99, how to scale up, limits, palladium 00:49:40 love America, first amendment, free speech absolutist 00:50:25 free speech meaning, dmca 00:50:42 europe does not have free speech 00:50:52 you don't own space in america 00:51:02 john mcafee chad 00:52:00 Iain M. Banks, The Player of Games, rich sutton reinforcement learning 00:52:35 not absolute free speech, AI that are aligned with you 00:53:33 the local vs the cloud plug, buy the tinybox 00:54:10 Say Anything - Psyche!
    6
  254. 6
  255. tiny corp is hiring! -> 3x Software Engineer, 1x Hardware Engineer - https://tinygrad.org/ | It's happening George started a project -> Getting AMD on MLPerf -> https://github.com/geohot?tab=projects | Go help George if you are capable! He will appreciate you! | tiny corp has a website tinygrad.org. Let your friends know! | George is streaming right now on https://www.twitch.tv/georgehotz part two of stable diffusion | github.com/geohot/tinygrad | github.com/CompVis/stable-diffusion | Support George by subscribing on twitch.tv/subs/georgehotz | Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 intro 00:01:50 feelings 00:03:20 stable_diffusion.py 00:04:40 difference between George and Andrej 00:05:25 cannot create an OBJECT array from memory buffer 00:07:50 known bug in tinygrad 00:08:37 what is stable diffusion, opensource, AI safety 00:10:00 safety checker 00:11:35 what this models are 00:13:15 CompVis/stable-diffusion, reading the code 00:17:30 ready to give up, learning, advice, nathan for you, passion 00:18:34 32GB VRAM it's a mac and it's shared 00:18:48 UNet attention, where are the inputs and outputs 00:20:10 what parts of transformer matter, terminology 00:21:22 about stable diffusion, UNet attention 00:22:45 dictatorizing, problem with the world, democratizing 00:24:03 privileged, triggered 00:25:20 Axel Nix lawsuit comma, assholes in the world 00:27:12 Cool people in the world, rant about Aurora, Tesla wining self driving 00:29:25 Elon Musk, billionaires, zoox, Aurora scam, people like scams 00:30:45 stable diffusion paper, LDM 00:32:05 learn latest tricks in deep learning 00:32:21 glasses, astigmatism, health insurance premiums 00:33:05 why three pieces 00:35:25 how it works? 00:40:49 thanks to flavioschneider for real information 00:41:25 taking a look at Unet 00:43:40 Autoencoder, basic idea about diffusion, encoder 00:45:05 first stage model, transformers are sample efficient world models, eloialonso/iris 00:47:00 google-research/rliable 00:48:10 facebookresearch/hydra 00:48:45 how clip model get's to UNet 00:51:15 hating too much on the paper 00:51:40 a walk in the park learning to walk in 20 minutes 00:52:25 jax.readthedocs.io 00:53:30 training is insanely hard 00:54:30 implementation of stable diffusion 00:55:35 numpy assign, assignment, insert, np.copyTo 01:05:50 it match pytorch exactly, easier to load things 01:06:48 break, uber eats in iceland 01:13:48 banning a person in chat 01:17:00 it's outputting a distribution, freedom of speech can't be bought 01:17:30 quant_conv, nin_shortcut 01:21:05 class StableDiffusion, AutoencoderKL, Encoder, ResnetBlock 01:28:45 How to Catch Multiple Exceptions in Python 01:29:35 stable_diffusion.md 01:33:20 asymmetric padding supported in tinygrad, class Encoder, ResnetBlock, StableDiffusion 01:51:20 break, uber eats, class Decoder 02:35:25 iPhone to dad, apps manipulation 02:35:50 torch.nn.functional.interpolate 02:42:35 tuple object cannot be interpreted as integer 02:48:08 cannot handle this data type 02:49:30 black square because noting in text-prompt 02:50:45 need to write groupnorm 02:57:04 picture like art 02:58:05 break, the graph 03:00:52 class AttnBlock 03:06:00 image looks the same 03:08:40 torch layernorm 03:11:15 orange square, deep learning is not like cryptography 03:17:40 break, single color square, there is bugs, 03:22:50 don't want to do this properly 03:30:30 doing it the proper way 03:35:25 so much fun, kids you could get into computers, github copilot is not good 03:39:08 PyTorch image, noise is better then solid color 03:43:18 looks stupider, is it trained? 03:48:05 don't know if better or worse 03:49:00 it works, before and after image 03:55:00 Torch apple, we are nothing but script kiddies 04:00:30 negative apple, five times apple 04:05:15 we don't have anything working at all 04:09:05 let's dive in block one 04:18:00 bug in gpu implementation in tinygrad 04:22:30 make a test 04:27:20 overflowing int, this is not python 04:36:00 PyTorch vs tinygrad difference, opencl tinygrad backend 04:41:30 Alex on the call, clinfo on quiet computer 04:45:15 very serious bug in tinygrad 04:47:15 Lana_Lux raid, works on gpu in other computer 04:49:10 this is why Nvidia is king, thinking about simple fix 05:14:00 big victory for tinygrad is running openpilot model 05:18:05 Alex bringing drink, tinygrad code easier to read 05:25:50 explaining the problem 05:33:10 raiding Lana_Lux making twitch loop 05:38:25 cloudflare The Project Jengo Saga, free speech 05:40:15 here is something we can do AutoencoderKL 05:45:35 still to do diffusionmodules, FrozenCLIPERmbedder
    6
  256. 6
  257. Watch youtu.be/lO0KH-TgvbM (Peter Thiel on The Diversity Myth Lectures) | newcriterion.com/issues/2023/6/the-diversity-myth | italianfuturism.org/manifestos/foundingmanifesto | graymirror.substack.com/p/a-techno-pessimist-manifesto | geohot.github.io/blog/jekyll/update/2023/07/20/a-disgusting-playbook-copy.html | ecromata.com | palladiummag.com | longform.org | newcriterion.com | www.astralcodexten.com Bounties for tiny corp / tinygrad -> docs.google.com/spreadsheets/d/1WKHbT-7KOgjEawq5h5Ic1qUWzpfAzuD_J06N1JwOCGs/ https://youtu.be/lnVQsJJFcdg?&t=13715 Hiring entire stack for tiny corp join if you are interested | https://youtu.be/lnVQsJJFcdg?&t=14195 work major source of value in your life | Read en.wikipedia.org/wiki/The_Elephant_in_the_Brain Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | github.com/tinygrad/tinygrad <- simple powerful deep learning framework tiny corp is accepting new interns. more info on tinygrad.org and tinygrad discord | comma.ai is accepting interns comma.ai/jobs#open-positions from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | from $999 comma.ai/shop/body the future of people Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 muted intro 00:00:22 un-muted 00:01:40 everything that is wrong in your life it's your fault 00:02:44 a techno-pessimist manifesto, long term thinking 00:03:50 Tucker Carlson, Julian Assange 00:05:30 The Diversity Myth by Peter Thiel, Samsung Z Fold5 00:11:05 wokeness, universities, acceleration is good, wealth distribution 00:14:20 no progress in physics, energy usage, no progress in science 00:16:30 usa life expectancy 00:17:40 secrecy in startups, microeconomics problem, housing 00:21:00 no growth, Henry Adams curve, christianity 00:23:40 2010s, fed interest rates, high interest rate 00:27:50 progress, newcriterion, society complex 00:31:00 Griggs v. Duke Power Co., IQ test, correlation between SAT and IQ 00:32:35 human capital, practical engineering youtube 00:34:30 rasism, fix human capital, techno capital 00:35:35 tabo, secrecy, telling the truth 00:37:10 rent double in san francisco, energy usage usa 00:39:00 energy usage per capita china, germany 00:39:50 listen to The Diversity Myth by Peter Thiel 00:40:30 loss of status, harvard, ivy league 00:42:10 1915 plane, 747, 787 dreamliner, Airbus A380 00:44:30 executive functioning, business 00:45:50 argentina, el salvador, argentina currency, Bird Global, BRDS 00:48:15 nuclear power 00:48:40 a techno-pessimist manifesto, Marc Andreessen 00:50:32 futurist manifesto 00:52:30 left, right 00:53:15 phone usage hours per country 00:55:20 south africa 00:56:25 lil mabu 01:00:50 don't walk to George, socially awkward 01:01:30 accelerationism mass movement 01:02:50 william james, sent-down youth 01:04:30 George Hotz brand, making people useful 01:07:45 ecromata.com 01:09:00 what should I do? 01:09:45 robotics laborm, comma.ai/shop/body 01:10:30 living property developer, off grid, week in the woods 01:13:00 George vs animal, hunting 01:14:20 Kanye West city in the Middle East 01:18:10 Humans Need Not Apply 01:20:18 resource curse, UBI 01:22:45 long term ideology, human power over nature 01:24:10 George as a manager 01:24:33 don't talk to George unless you are cool, fans 01:25:20 Curtis Yarvin, Peter Thiel describing the present 01:25:50 cult, deploying money and human capital 01:26:25 what does Elon have that George does not, risk tolerance 01:27:10 Sam Altman, openai new blog post 01:28:20 AI alignment and capability issue 01:29:25 due to concerns about large language models 01:30:30 deploying capital ability, lying ability 01:31:55 a disgusting playbook blog post 01:32:46 lies do catch up with you 01:33:00 Elon internet space internet idea 01:34:20 bounties, business, utilitarian ethics 01:35:25 thymic energy 01:36:20 to partner with somebody 01:37:40 you can buy starlink 01:38:10 hyperloop and rail projects 01:40:05 showtune song, instagram 01:40:40 USleeve, BetterHelp, Squarespace, Surfshark 01:42:35 amazon firestick in a hotel, GTA leaks 01:44:40 Kevin Mitnick, 'code is law' ethos in crypto 01:45:45 make hacking legal 01:46:40 p vs e, p v p zone, binance smart chain 01:49:20 cheapETH, devETH, lost $5000 on cheapETH 01:51:30 no answers, happy holidays 01:52:14 atheism, merry christmas
    6
  258. 6
  259. 6
  260. 6
  261. From $1499 buy comma.ai/shop/comma-three | github.com/geohot/tinygrad/blob/llama/examples/llama.py | https://github.com/geohot/tinygrad/tree/llama | tiny corp has a website tinygrad.org | Check out and star github.com/geohot/tinygrad | Support George by subscribing twitch.tv/subs/georgehotz | Follow George on instagram.com/georgehotz & twitch.tv/georgehotz & twitter.com/comma_ai to be up to date | Read George's geohot.github.io/blog/ | Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 muted intro 00:00:50 missing the rant about bad mood 00:01:35 not family friendly content 00:01:55 hardware to run tinygrad 00:02:10 don't mess with george 00:02:35 what we did yesterday 00:05:00 where are we wasting time, not limited by gpu 00:06:20 python snakeprof 00:10:30 tinygrad discord llama channel, hacker news blow up 00:10:45 llama weights on torrent 00:11:20 does it run on nvidia 3090, ram 00:13:30 open assistant product, prompt engineering 00:21:10 no limits, don't believe in ai safety 00:21:40 no guns, ai stupid 00:23:55 what you need to torrent for weights 00:27:25 torch.triu 00:32:25 nice pytorch error messages 00:36:45 russian mode 00:38:00 python bottleneck 00:41:50 twitch chat not paying attention 00:42:50 pull request to facebook 00:43:50 hacker news, gotham chess plays chatgpt 00:45:30 chatbot loop 00:47:00 asking questions, bad enter 00:50:20 no ai filter 00:51:40 interesting output 00:53:50 sentencepieceprocessor decode_piece 00:58:00 removing extra enter 01:00:50 decode token to token, this is the problem 01:02:50 we learned how to take it out of russian mode 01:03:50 need demo answers, it's called llama chat 01:06:30 be verbose in your answers 01:08:00 call back humor answer 01:08:50 pastebin improved pre prompts 01:17:00 not Gale, your name is Brian, cookies 01:18:00 what is your prompt 01:18:35 arbitrary python 01:22:45 impressive large language models 01:30:50 running on M1 Max 01:36:30 llama link 01:37:30 Brian running wild 01:39:00 real talk about ai safety 01:40:20 chat llama open source, running in web browser 01:42:20 funny response 01:43:45 connecting to the internet 01:45:00 model too small, too many tokens 01:47:00 darth vader, used car salesman name 01:50:30 Lana_Lux raiding stream, explaining to new people 01:51:15 what makes chatgpt good 01:52:00 fun answer 01:53:00 Gary only has 7B weights 01:54:10 chatml, need smarter model 02:00:00 what we should do? 02:01:00 bad rapper 02:03:20 stacy 02:07:15 limits of 7B weights 02:08:50 good answer 02:09:25 assuming gender 02:11:25 searching snapchat mentee.dreams 02:12:12 replica terrible, asking more questions, rap 02:17:10 13B weights smart stacy, vram requirements 02:19:30 running on cpu, took a day to write this in tinygrad 02:21:55 comma build the best self driving car, tinygrad taking over the world 02:22:40 removing code, repetition penalty 02:30:10 testing stacy 02:36:10 sp_model.decode_pieces, bos_id 02:43:20 fun answer, need to make large work 02:50:10 reading the LLaMA paper 02:52:10 anyone running this on his mac?, code pushed 02:53:45 macbook air m2 02:59:15 pytorch split file 03:01:30 trying to load the bigger model 03:13:00 python readinto 03:16:20 load only 13GB because of gc 03:18:20 making copy = bad 03:20:00 apple m1, m2 max, ultra memory 03:21:30 v.shape, tensors 03:23:00 real offer sponsor, tinycorp honest corp 03:32:10 millionaire 03:34:50 bad chatter, DEBUG=3, pointer, constant folded 03:38:40 the problem and easy fix, mailing the cookie for anyone who knows, thai food 03:44:30 what type is the output buffer 03:47:45 does not use ram because of fake tensors 03:49:55 mid model, loading 13B weight model 03:55:30 bad chat WEIGHTS=0 03:59:30 closing apps, chrome, discord 04:01:50 mac process disable swap, metal buffer force ram resident 04:05:00 force os x buffer to not swap, metal makr bugger as not swap, MTLResourceStorageModeShared 04:09:55 mac os x disable swap for single process, apple gpu supported types, pci-e 4 16x bandwidth 04:16:20 improving prompts, strassen algorithm 04:23:00 let's go 04:27:25 getting food 04:35:00 marc andreessen 04:43:00 who is stacy 04:48:40 git commit add gary 04:49:00 context infinite 04:49:20 gary vs stacy 04:49:40 ai george hotz disappointment 04:50:20 stacy saying thanks for watching 04:51:00 llama branch 24GB ram min requirement, having fun with stacy this weekend
    6
  262. 6
  263. 6
  264. Bounties for tiny corp / tinygrad -> docs.google.com/spreadsheets/d/1WKHbT-7KOgjEawq5h5Ic1qUWzpfAzuD_J06N1JwOCGs/ 03:48:35 Hiring entire stack for tiny corp join if you are interested | 03:56:30 work major source of value in your life | Read en.wikipedia.org/wiki/The_Elephant_in_the_Brain Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | github.com/tinygrad/tinygrad <- simple powerful deep learning framework tiny corp is accepting new interns. more info on tinygrad.org and tinygrad discord | comma.ai is accepting interns comma.ai/jobs#open-positions from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | from $999 comma.ai/shop/body the future of people Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 intro 00:01:02 writing a Qualcomm GPU driver 00:01:54 ssh to the phone, code server, vs code server 00:02:22 user space GPU driver, graphics kernel 00:05:10 asking Quentin and Perplexity labs 00:13:00 regex pattern allow spaces 00:21:30 parsing struct 00:25:45 GPU driver, how long in C 00:31:05 tinygrad import Tensor 00:35:40 init, most of GPU not controlled in kernel space, single layer pointers 00:36:55 replay this GPU command 00:38:10 TinyJit, tinygrad prints 00:44:30 kgsl_gpu_command, msm_kgsl.py 00:49:10 Samsung Galaxy Z Fold5 00:49:45 who knows that green is executable +x 00:50:18 linux dma-fence.h 00:53:00 offset gpuaddr size flags id 00:58:20 struct_kgsl_command_object 01:06:25 building a driver before building a chip 01:07:00 jobs creation, comma lesson 01:08:25 thneed openpilot, mmap64 01:12:30 gpu command, parsing the command buffer 01:19:45 freedreno driver, freedreno cffdump 01:23:05 Perplexity labs hallucinating 01:25:40 rnndb database 404 link 01:28:25 real command buffer 01:31:00 PM4 01:35:20 pkt_is_type7 01:41:44 alloc memoryview writable 01:46:55 type 7 and type 4 packets 01:48:40 very cool freedreno people 01:49:05 qualcomm switching to opensource driver 01:50:00 writing registers, opcode 01:52:40 number of globals, and total number 01:54:40 adreno_pm4_type3,7_packets, 7 commands, 4 write registers 01:56:50 python format string add 0x to hex 01:57:55 opcode 17, CP_THREAD_CONTROL or IN_IB_PREFETCH_END 01:59:53 python parse xml, xml to json 02:07:15 not a real name, no leak for qualcomm driver, amd, nvidia open source 02:08:30 HLSQ_CONTROL_2_REG 02:09:25 nvidia open sourced the kernel stub that talks to gsp 02:11:20 CP_CONTEXT_REG_BUNCH2 02:13:45 CP_REG_TO_MEM 02:14:50 GPU performance counter to GPU memory 02:17:30 SP_CS_INSTRLEN, HLSQ_CS_NDRANGE_0 02:20:40 the simplest drivers that can dispatch compute kernels 02:23:00 copy out, unified memory space between CPU and GPU 02:26:00 objlist cmdlist, pkt types 02:36:00 stuct_kgsl_cmdbatch_profiling_buffer 02:42:20 printing in hex 02:45:20 python address of bytearray 02:47:00 making another copy 02:48:40 strace, copyin 02:52:30 two CP_RUN_OPENCL 03:01:00 CP_LOAD_STATE6_FRAG 03:04:20 a, 9 03:08:00 python hexdump 03:11:40 GPU drivers can be written in python 03:12:50 dumping the shader 03:21:25 freedreno shader registers r51 03:26:40 c20 hex 14 03:30:00 CP_LOAD_STATE6_0_STATE_TYPE 03:37:50 adreno registers 03:38:10 replacing the shader compilers, mesa for compute 03:41:30 replaying the command buffers 03:44:25 stream overview, bath, meditation 03:45:00 when tinygrad can run without dependencies 03:46:15 remote work, there is something missing 03:47:20 removing numpy from tinygrad 03:48:35 hiring entire stack for tiny corp 03:51:00 high equity, contribute to the project 03:53:15 trolls on twitter 03:54:00 comma is doing well, san diego 03:55:20 the dream of the internet, Elon is accessible 03:56:30 work major source of value in your life 03:57:40 sustainable business of open source 03:58:40 clear blog post about AI space, not trying to be understood 03:59:00 AMD mindshare, people at AMD, AMD bugs, issues 04:02:05 thinking longer term 04:04:33 college majors by iq 04:05:10 it doesn't change. if you are 30 your life is what it is, limitations forever 04:07:55 following your passion, your passion is stupid, singularity 04:12:10 against education, the elephant in the brain 04:19:30 christian religion, the bible, game over screen 04:21:55 the invisible rainbow, 5G 04:25:45 naming 10 books 04:29:40 not debating destiny 04:30:00 tiny keyboard 04:32:15 general knowledge science questions 04:41:20 human brain compute, wormhole 04:44:10 time dilation, sr vs jk flip flop 04:46:30 attention mechanism in transformers 04:47:40 e minor, supernova 04:48:50 Ra by qntm 04:50:10 milky way galaxy, quantum mechanics 04:52:34 power companies high voltage, black hole 04:55:20 polaritiazon in electro mechanics, einstein nobel prize 05:00:00 metal rod in space 05:01:20 universe not locally real 05:02:40 thank you for watching, human nature
    6
  265. 6
  266. 6
  267. 6
  268. 6
  269. 6
  270. 6
  271. 6
  272. 6
  273. 6
  274. 6
  275. 6
  276. 6
  277. 6
  278. Read www.incompleteideas.net/IncIdeas/BitterLesson.html | github.com/tinygrad/tinygrad <- simple powerful deep learning framework Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 muted intro 00:00:40 un-muted intro 00:00:50 taiwan and india 00:01:50 how free is taiwan, china, check in hotel 00:03:20 free internet in taiwan 00:04:00 wechat in china 00:05:55 story time, iceland 00:07:00 safe singapur 00:09:20 brazil, south africa 00:11:10 George trying not be controversial 00:12:10 grants, fixing the empire 00:13:45 dismantling our late-stage capitalist society 00:14:40 George stating his political preference 00:15:30 Elon Musk the next Caesar, bike trip 00:17:00 Ukraine gdp per capita, taiwan gdp per capita 00:17:50 progressive, can't make everyone happy in politics 00:18:20 sarcasm 00:19:00 eastern europe bike trip, andrew tate 00:20:10 countries to avoid 00:23:05 shipping for non-US citizens for tinybox 00:23:45 sweden, dubai 00:28:25 direction of tinygrad 00:30:30 building a wall 00:31:00 declining empires 00:32:50 china spending power, kenya poor 00:35:00 favorite countries in europe, dubai 00:36:15 libertarian india 00:36:50 hand coding resnet 50 00:40:10 kernels in resnet 00:43:05 operations for the resnet, the power of the new tinygrad abstractions 00:46:10 simple abstractions in tinygrad = fast 00:46:20 init does lots of adds 00:47:25 renderer, kernel code 00:48:55 cleanest abstractions anywhere nn 00:49:20 not optmized kernel, optimization 00:49:50 benchmarking 00:58:40 kind of wrong, some kernels bad 01:01:35 how nice is tinygrad 01:08:40 love for python 01:09:30 faster 01:11:20 MadDogNation is gifting 34 subs! 01:12:40 focusing on slowest 01:21:25 multiview shapetracker 01:25:40 lin.upcast 01:27:10 reshaping to real size 01:27:40 very happy with this infrastructure 01:31:35 let's make it faster 01:35:25 metal max fmax 01:39:25 gated loads, finding bugs 01:41:30 108 GFLOPS 01:43:30 we have to start thinking 01:47:30 we build an optimizer 01:49:30 upcast, search and optimization is amazing 01:52:25 out of ideas 01:59:50 that kernel just looks fast it just isn't 02:03:20 that kernel was not the problem 02:04:45 automatic optimizer 02:07:25 10% faster, we learn, knowledge is power 02:08:15 running this anywhere 02:08:50 4416 lines, 1000 lines, new goal 5000 lines 02:09:10 thank you for gifting subs! 02:09:40 on topic questions only for non-subs 02:10:03 up to the front tinygrad, short and generic 02:10:55 every part of a full deep learning stack 02:11:05 resnet 50 pytorch like code 02:11:25 tinygrad is a waste of time? 02:12:00 why is pytorch faster? pytorch less capable library 02:12:20 cudnn hand optimized by 20 nvidia engineers for 10 years 02:13:00 the bitter lesson 02:13:10 can't I just call CUDNN? 02:14:05 generic optimizer so much simpler to reason about 02:14:20 learn a model that look at shapetracker to optimize 02:14:40 recursive self-improvement 02:14:50 AGI inside tinygrad 02:15:20 AGI will run in 4k lines 02:15:40 every piece of tinygrad you can reason about 02:16:15 no more debates, heat death of the universe 02:16:45 AIs will come, monkey civilization 02:17:15 why people think AI is dangerous
    6
  279. 6
  280. High quality version uploaded -> youtube.com/watch?v=L_C4zLnekLI | Thank you to comma.ai team for providing high quality video and recordings of this amazing event! from $1250 buy -> comma 3X comma.ai/shop/comma-3x | If you have skill apply on -> comma.ai/jobs | Join the community on github.com/commaai/openpilot | Check if your car is supported -> comma.ai/vehicles | How to setup your car comma.ai/setup | follow the official youtube.com/commaai for more livestreams | Stay up to date by following twitter.com/comma_ai | Support comma.ai at comma.ai/shop. Chapters: 00:00:00 where is my intro 00:00:45 black mirror episode with ratings 00:01:30 three stories, past, future, present 00:01:40 the past, cruise automation 00:01:58 cruise RP-1 highway autopilot 00:02:20 giving up on technical challenge 00:02:30 too many cars to support 00:03:10 how to handling failures 00:03:55 fingerprinting 00:04:30 comma was not supposed to exist 00:05:15 what is worth doing in the world 00:06:10 6 generations of hardware 00:07:00 keep iterating the hardware 00:07:25 being real, honesty, communication people 00:08:20 shipping keeps you honest 00:08:30 stated preference, revealed preference 00:09:04 iteration 00:09:25 humility, egohot, big ego 00:10:36 so why haven't we won? 00:10:47 things take a long time, even accounting for things taking a long time 00:11:23 no exit, the system will get better 00:11:49 climbing out of a well 00:12:38 it turned out these companies were scams 00:13:28 bitconnect and luna scam 00:13:55 marginal utility - marginal cost 00:14:10 value destruction, linux fastest growing platform on Azure 00:14:50 markets - rationality, you just have to survive 00:15:05 lifetime revenue 00:16:05 $22.2M on $18.1M raised 00:16:18 twitter followers 00:16:35 future followers 00:16:45 employee retention 00:17:03 come work to comma ai all work by the team not George 00:17:18 keeping idiot's aways from comma 00:17:35 bar very high, you get to ship things that get better in the world for the better place 00:18:18 business world 00:18:30 the future 00:18:44 computing speeds vs human brain computation 00:19:30 forward looking statement 00:19:44 comma con 9 in year 2037 00:20:00 44 stores in malls, top 10 biggest consumer electronics companies, without selling out 00:20:52 hard to go anywhere without finding a comma, cars, bodies, puppies 00:21:20 comma body 00:21:30 comma 7 for $2500 00:21:46 3rd generation comma ASIC, 50000 TOPS 00:22:50 first comma device to boot into tinygrad 00:23:05 trying to build people 00:23:30 trolling Elon, out system is going to so end to end even or driving monitoring will be end to end 00:23:50 comma 7, how many silicon people work a comma 00:24:08 your comma 7 is yours, models we train in the cloud are foundation models 00:24:55 thank you to meta for LLaMA, open source AI 00:25:13 offer pretrained policy models, custom software 00:25:41 comma 7 is like child 00:25:52 will learn behaviors you teach so teach it well, pick up skills as quickly as smart human 00:25:02 comma 8 will be running the company 00:26:12 comma three human head 00:27:00 humans are so good at moving, driving 00:27:35 intelligence 00:27:46 the present COMMA_CON 2 00:28:10 the mall store soon 00:28:30 sitting down for dramatic effect 00:28:55 failure rates for iPhone, failure rate for comma two 00:30:10 comma three failure rate, comma three failure rates by month 00:31:32 dropping the price 00:32:00 comma 3x 00:32:15 much better injected molded case 00:33:00 3X specs, same hardware and software as comma three 00:33:34 upgraded cameras, 140 dB HDR 00:33:55 128GB of storage 00:34:14 CAN-FD integrated 00:34:28 still OLED better supplier, onboard road cameras 00:35:24 stereo in-case sound 00:35:45 upgraded power architecture 00:36:05 about half the components, more reliable 00:36:14 the comma 3X board front, no more GPS connector, better antennas 00:37:15 the comma 3X board back 00:37:45 battery slides in and out, chokes for CAN-FD, e-sim 00:38:10 cameras and right on the boards, failure rate of this device need to be 1 or 2 % 00:38:52 comma three vs comma 3x board complexity 00:39:02 no more NVMe, just on-board flash, QR codes on the board for tracking 00:39:35 is it a devkit? to set expectations 00:40:00 it's time to call it just a comma 3X, mass market 00:40:50 comma 3X just $1250 00:41:20 no more questions that is why it's just one edition 00:42:10 it's just a comma three but better 00:42:30 order now, ships in 1-8 weeks 00:43:10 comma 3X in EV6, no more red panda, no more two harnesses 00:43:20 when did the iPhone get good 00:43:50 the comma team build this not George 00:44:50 value in running language models on the device 00:45:30 marketing guy, testing on commercial vehicles 00:46:55 marketing, the story of openpilot, openpilot ADAS 00:47:40 buy now, 30-day money back guarantee. 00:48:05 comma 3X always compatible with cars 00:48:40 port a new model to comma 3X 00:49:35 mass market, liability air gap with installing custom software 00:50:25 level 2, you must pay attention at all times 00:51:30 large foundation model on device, expansion port 00:53:35 failure rate, should be build like a tank 00:55:00 saying no to things, best highway experience
    6
  281. 6
  282. 6
  283. 6
  284. 6
  285. 6
  286. 5
  287. 5
  288. 5
  289. 5
  290. tiny corp is hiring! -> 3x Software Engineer, 1x Hardware Engineer - https://tinygrad.org/ | It's happening George started a project -> Getting AMD on MLPerf -> https://github.com/geohot?tab=projects | Go help George if you are capable! He will appreciate you! | Go detect your own hen -> https://tinygrad.org/recognize.html | From $1499 buy comma.ai/shop/comma-three | tiny corp has a website tinygrad.org | Check out and star github.com/geohot/tinygrad | Support George by subscribing twitch.tv/subs/georgehotz | Follow George on instagram.com/georgehotz & twitch.tv/georgehotz & twitter.com/comma_ai to be up to date | Read George's geohot.github.io/blog/ | Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 intro 00:02:20 c backend for $2500 00:06:15 elon and george and twitter 00:10:15 bad user in chat 00:15:30 random extension of power 00:21:30 beautiful c code 00:28:20 ctypes load library 00:29:25 os path file exists python 00:31:25 pain of using C, multithreading 00:38:00 np.ctypeslib.as_array 00:41:25 c99 max function 00:43:30 c backend in 33 lines 00:49:45 indexError: list index out of range 00:52:50 segmentation fault 11 00:54:40 running tinygrad example 00:59:00 CLANG=1 01:00:40 using rust 01:01:40 thank you to everyone who keept the example working 01:06:10 gpu runner, jit_cache 01:08:10 args, buffers, malloc 01:19:10 CLPrograms error 01:20:45 loading the weights, identifying the inputs and outputs 01:23:00 c auto formatter tabs, clang-format 01:25:40 copyright, spreading the word, selling royce rolls 01:26:25 lib jpeg pain, linking to opencv, cheapeth scam 01:27:30 tiny corp is broke 01:27:55 bad chat, who can follow this, talking to Alex 01:29:15 can anyone follow this, explaining to noobs 01:36:00 arg max 01:37:55 c float infinity 01:43:35 language translation features from github 01:49:40 fread 01:56:10 512 bugged 01:58:20 how we are going to get an image? 02:00:30 c array of strings 02:03:20 very upset about reading the image 02:04:45 c example libjpeg 02:05:35 link to libjpeg 02:06:20 libjpeg dev homebrew, libpng 02:07:20 png file format, load png in c header only 02:08:10 c link to opencv 02:08:35 worse part of C 02:08:55 stb_image.h 02:21:45 trying coca cola image 02:24:10 remote control 02:25:50 embedding binary data in c 02:37:10 incompatible integer to pointer conversion 02:40:30 so long, putting it to the pipe 02:44:40 binary data in c header 02:49:35 I love temp, temp my favorite folder on computer 02:52:15 completely portable application that runs classifier 03:00:40 banana, hyperpop song 03:03:10 imagesnap 03:05:25 detecting objects via camera 03:08:40 triceratops 03:09:20 sending people to jail 03:11:25 distributing recognize to tinygrad.org 03:15:30 banana, c time nanoseconds, code html5 03:19:25 finding the right image, c assert, bing chatgpt 03:45:10 letting everyone else to test on their devices to fix any bugs 03:52:55 what can tiny corp do that other companies can't do 03:56:00 can you run this command on your terminal 03:58:00 what we should do next 04:01:40 49% of tiny corp to microsoft for $10B 04:02:10 openai code and models 04:03:25 George trolling the chat, vulcano story 04:04:40 Andrej Karpathy joining openai 04:05:30 what features to remove from c++ 04:07:00 refactors to do 04:09:40 responsible ai, ai safety, anti democratic 04:12:30 chatgpt safety filter 04:19:10 bash command line assert output 04:21:45 tinybrowser, nn simple, tensors 04:24:10 compile c to wasm 04:28:30 python3 -m http.server 04:30:25 svelte wasm 04:32:30 javascript fetch 04:46:55 .wat file, wasienv 04:48:35 wasm 05:03:50 fromCharCode 05:14:00 javascript abort 05:16:40 SEEK_CUR 05:19:50 it works 05:26:40 upload file from javascript 05:35:30 doing style 05:51:50 connecting to webcam 05:55:15 fixing img box 06:05:20 camera capture 06:39:05 ios webcam from non https 06:44:20 working on iPhone 06:45:30 HackerNews 06:46:00 real future ideas 06:48:10 Alex on the phone 06:50:20 fixing bugs 06:56:25 governance tokens 06:57:20 happy hacking keyboard 06:57:50 what did we do today 06:58:15 net's in the browser 06:59:00 tiny corp no money 06:59:45 trap drums
    5
  291. 5
  292. 5
  293. 5
  294. 5
  295. 5
  296. 5
  297. 5
  298. 5
  299. 5
  300. tiny corp is hiring! -> 3x Software Engineer, 1x Hardware Engineer - https://tinygrad.org/ | It's happening George started a project -> Getting AMD on MLPerf -> https://github.com/geohot?tab=projects | Go help George if you are capable! He will appreciate you! | From $1499 buy comma.ai/shop/comma-three | tiny corp has a website tinygrad.org | Check out and star github.com/geohot/tinygrad | Support George by subscribing twitch.tv/subs/georgehotz | Follow George on instagram.com/georgehotz & twitch.tv/georgehotz & twitter.com/comma_ai to be up to date | Read George's geohot.github.io/blog/ | Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 intro 00:02:00 n02018795 imagenet categories 00:05:30 staying on topic, tinygrad discord 00:06:00 rap battle 00:07:25 benchmark_train_efficientnet.py 00:10:05 what net to use 00:11:00 how to contribute to tinygrad 00:12:55 state of the art for training imagenet 00:13:10 tinygrad discord, CLCACHE 00:14:00 pull requests 00:15:00 no nix, rust, llvmlite, cuda 00:16:30 what paper to implement 00:18:40 resnet paper 00:23:10 CIFAR-10 00:25:20 1080 ti 00:25:30 torchvision installing issue 00:26:30 nix does not solve this 00:27:55 xkcd standards 00:29:30 net ema 00:30:44 maxpool2d 00:36:30 19s runtime in pytorch 00:38:20 tinygrad discord strict rules 00:38:38 hackernews people, maslow's hierarchy of argument, conversation 00:39:10 life after degree, ideas, honest conversations 00:41:10 compassion will hold you back 00:42:55 elon's rules, rule 0 surface complexity, building hardware, vertical integration 00:43:55 no reason for this to be a class 00:51:40 pytorch unsqueeze 00:52:45 beautiful implementation 00:55:35 ResNet whitening 00:59:20 shorter, better than pytorch 01:01:25 dumbest question 01:03:25 fire tweet, ai safety, old people text 01:04:15 fate of the world, twitter saga, internet get things wrong 01:05:40 jai beta 01:07:00 creating an optimizer 01:09:00 broken hoodie 01:09:30 batchsize 512 01:10:40 642 vs 53 lines 01:13:50 make_pair and first attempt at hlb_cifar10 01:15:00 DEBUG=2 01:17:50 more readable code 01:19:00 TEST_AST=1, unable to allocate 144 GiB 01:25:00 gelu is causing nans 01:28:00 64 batch size 01:31:10 nn.CrossEntropyLoss 01:34:30 cifar-10-batches-py/data_batch_1 01:39:20 open-assistant 01:40:40 comma ai safety principles vs waymo safety 01:44:25 bad questions in chat 01:46:00 train_step_jitted 02:05:00 the losses are different with optimizer 02:12:20 the idea of tinygrad performance 02:13:05 printing out the kernels that are running 02:16:20 opencl nvidia tensor core 02:18:40 58 TFLOPS 02:20:15 question in tinygrad discord 02:23:20 why the fusion is broken, tensor cores, float 16 support 02:40:15 17 TFLOPS 02:45:10 arm support for tinygrad 02:46:55 all time spend on gpu 02:50:55 12x off 02:55:20 4x4 matrix 02:58:36 optimizer outputing wrong code 03:05:00 kernels output 03:08:00 -1%10 03:14:10 decode utf-8 03:16:40 fma.rn.f32, opencl cuda enable tensor cores 03:21:40 reviewing symbolic test pull request 03:23:00 sm_86, using triton, kernel search 03:30:50 only 10x off 03:36:00 tinygrad.org, pytorch mps, python requests get 03:43:00 right shapes, lazy.py 03:45:20 tinygrad contributors, over 1000 lines 03:49:45 comparing the cpu backend, pytorch cuda 03:53:35 is tinygrad stupid project 04:01:45 reviewing tqdm pull request 04:08:30 reviewing progress bar pull 04:09:35 highest quality code you ever written in life 04:10:35 housing and food for tinygrad intern (no salary) 04:11:35 tinygrad looking into making money (merch) 04:13:15 comma tiny corp partnership 04:13:45 refactors in tinygrad 04:14:00 employee number 1 pitch 04:21:00 checking food options 04:23:15 make_pair, progress bar pull request 04:34:00 tqdm requests progress bar 04:41:45 bad pull requests 04:46:50 good programmers, money, goat progress bar 04:56:25 new policy chatgpt banned 05:11:50 plan to fix the optimizer, reviewing pastebin code 05:31:30 instagram women dm filtering 05:31:50 testing intervention 05:47:25 NoneType object has no attribute copyin, how much fake speed 05:53:00 the best code is no code 06:12:30 this should only copy the base buffer and retain the shapetracker 06:20:25 aritter2 get's VIP for his good comments 06:28:00 offensive question 06:29:15 commit fix multiple accumulators, ast fuzzer, progress today 06:34:00 interesting code, jeff bezos e/acc, KOPT=2, 6.5x speed up, debug, tinygrad internships
    5
  301. 5
  302. 5
  303. Buy comma three from $1499 https://comma.ai/shop/comma-three | What is comma.ai/openpilot | openpilot documentation docs.comma.ai | What is comma.ai/connect | Join the comma team comma.ai/jobs | Check if your car is supported -> https://comma.ai/vehicles | How to setup your car comma.ai/setup | Read blog https://blog.comma.ai/ready-for-growth/ | Fun timestamp 00:42:55 a wild Jim Keller appears! | Follow the official youtube.com/commaai for more livestreams. Stay up to date by following twitter.com/comma_ai | Support comma.ai at comma.ai/shop. Chapters: 00:00:00 intro 00:01:00 showing the booth 00:01:35 time at twitter 00:02:00 Adeeb 00:02:45 no partnership 00:02:55 car and driver test drive 00:03:15 no comma four 00:03:50 comma 3 improvements 00:05:00 merge/acquisition, spac 00:06:18 comma works everywhere 00:06:28 Harald 00:06:47 test drives today 00:07:10 comma harness 00:07:38 stop and go at traffic lights 00:08:40 list of supported cars 00:09:55 toyota security 00:10:35 Alex and Shane 00:12:08 announcement of price decrease 00:12:45 comma 3 reliability 00:13:15 sales need to go up 00:13:40 comma 3 life cycle 00:14:15 comma does not work with anyone 00:14:50 marketing intern 00:15:13 best car for comma 3 00:15:33 open source comma3 car mount 00:15:45 how is lower price possible 00:16:33 no trade in for comma 2 00:16:43 raising money 00:17:00 navigate on open pilot in Europe 00:17:10 no refunds for orders placed in last 30 days 00:17:33 made in America, Toyota corolla for taco bell? 00:17:52 manufacturers integrating comma 00:18:15 bosses at comma 00:19:00 price 00:20:25 what would Elon do? 00:20:52 best car for comma 3 00:21:20 is the dev harness the same as the comma serial? 00:21:30 Radar less is ready? 00:22:00 you said you wouldn't lower the price 00:24:10 comma 3 won't work with Tesla (maybe) 00:24:20 check avatar script by George 00:24:30 I have a comma 2. any advantage getting a comma 3? 00:25:00 marketing strategy, bolt euv integration 00:25:15 comma 4 when? 00:25:40 chip in comma 3, c3 on front & rear windows, trolly problem 00:25:55 Red Panda and cables needed 00:26:15 tesla vs comma 00:26:25 comma con 00:26:35 buying with paypal credit 00:26:45 CommaCon confirmed, zoox 00:27:35 works with right side drive cars 00:27:55 how to work at comma 00:28:05 hackathon 00:28:25 comma robotics 00:29:00 love tesla 00:29:10 comma con july 31st this year 00:29:50 partnerships and collaborations 00:30:13 never IPO 00:31:02 experimental mode, nav on openpilot 00:31:20 return rate for comma 00:31:40 lawsuit over 00:31:50 comma 3 work in a boat 00:32:05 showing the comma 3 00:32:55 why newer comma 3 better 00:33:20 high temperature resistance 00:33:55 how many comma 3 sold, failure rate 00:34:15 new SoC vendor 00:34:30 wholesale for car dealers? 10% off 00:34:45 bad and good features 00:35:20 openpilot works in RHD countries 00:35:35 learning on device 00:35:55 stop and go at traffic lights 00:36:30 speed bump detection, making chips 00:37:35 how to contribute to the growth 00:38:00 Shane talking about micd, fork, smart lane switch, support, autotune 00:39:10 how Shane got started with comma 00:39:30 bad experimental mode, 00:39:50 jobs, hiring, car-interface engineer, research, infra 00:41:10 comma connect price drop when? 00:42:02 different storage same driving 00:42:25 remote work 00:42:55 a wild Jim Keller appears! 00:43:55 experimental mode 00:45:35 comma knee 00:45:55 snow, warranty extension 00:47:25 focus on experimental mode, marketing 00:48:38 Aptera, Subaru experimental, support 00:50:55 driver monitoring 00:51:20 Mercedes no support, Tesla DM 00:52:10 openpilot active users, George's drive 00:53:08 lawsuit, comma body, sony ces sponsors 00:54:54 how nav works 00:55:35 Alex last words, scams, remote work 00:57:55 Harald about work in progress 01:01:15 Geohot voice directions, lane change 01:02:20 autoencoder 01:03:05 sentry mode, on device learning 01:05:10 end words
    5
  304. 5
  305. 5
  306. Bounties for tiny corp / tinygrad -> docs.google.com/spreadsheets/d/1WKHbT-7KOgjEawq5h5Ic1qUWzpfAzuD_J06N1JwOCGs/ https://youtu.be/lnVQsJJFcdg?t=13715 Hiring entire stack for tiny corp join if you are interested | https://youtu.be/lnVQsJJFcdg?t=14190 work major source of value in your life Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | github.com/tinygrad/tinygrad <- simple powerful deep learning framework tiny corp is accepting new interns. more info on tinygrad.org and tinygrad discord | comma.ai is accepting interns comma.ai/jobs#open-positions from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | from $999 comma.ai/shop/body the future of people Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 intro 00:00:55 Plugable BT-KEY3 and Galaxy Z Fold5 00:02:00 enabling ssh server on the phone 00:05:25 5G, eSIM 00:07:43 talking to Alex, minister of AI dubai, visas, Miami, immigration policy 00:12:50 vscode server, android 00:18:00 labs.perplexity.ai mixtral-8x7b-instruct 00:20:18 bionic libstdc++.so, node, vscode server android termux 00:27:50 Donald Trump - Make America Great Again (Country Song) 00:29:40 Vivek Ramaswamy smart 00:30:25 vscode in a browser 00:32:25 reach on twitter, Elon did more for anti woke cultural movement 00:35:00 tmux can't copy paste 00:43:10 python plugin tab completion 00:47:35 IOCTL_KGSL_DEVICE_WAITTIMESTAMP_CTXTID 00:50:20 growth, accelerationism, decels, You may not be interested in politics, but politics is interested in you. 00:53:13 replace opencl with driver that use IOCTLs, hook native function in python 01:04:55 python frida 01:06:05 IOCTL syscall attach to our process 01:08:05 perplexity hallucinating 01:10:30 bionic by google, libc, android 01:18:05 apple, dumb comments 01:19:05 python most beautiful programming language in the world, C 01:21:59 chillin in my rover american empire is over, uni freshman 01:24:20 illegal instruction, arm64 breakpoint instruction 01:29:20 Ennaria - Monstarrr, internet in the 90's 01:33:00 signal devide by zero signal, arm64 disassembler online 01:39:50 aarch64 svc, ctypes callback 01:41:05 j_blow raiding the stream, hooking iocal from python 01:47:30 got plt python, dlsym 01:51:40 symbols dylib, libdl github 02:04:04 dl_iterate_phdr 02:10:15 python dl_phdr_info 02:11:38 beautiful libraries 02:15:15 hooking ioctl universal, best toolset for debugging gpus 02:22:05 python trampoline 02:29:50 libc strstr, libc memmem 02:36:00 restype = ctypes.c_void_p 02:48:25 callback 03:07:25 corrupting the wrong got, hotpatching a function 03:10:10 aarch64 assembler, aarch74 abi 03:20:35 getting callback 03:29:55 segmentation fault, illegal instruction, python arm flush icache 03:32:45 python call ioctl syscall 03:39:40 IOCTL_KGSL_GPUOBJ_SYNC, iowr 03:44:20 libc cache 03:44:45 anyone paying attention? instruction cache? 03:48:48 chat comment trigger, dancing monkey 03:50:10 Logitech C920, python, calling from libc 03:56:30 parsing the structs, ioctl command encoding 03:59:40 z fold 5 kernel source 04:02:00 qualcomm 8550 reference kernel 04:04:00 file in user space in android that is gpu driver 04:04:50 gpuctypes 04:15:15 ctypes pretty print struct 04:18:30 alloc(16) being called, page aligned 04:19:20 so much of modern software development is completely wrong, leaky abstractions 04:21:45 bar for writing software, do you understand what you are doing? 04:23:25 javascript, softmax, jit, javascript engine 04:28:40 communicating in english and communicating in programing are the same thing 04:29:00 syscall, companies need to stop tolerating people who don't understand 04:31:20 american salaries, skill, motivation 04:33:00 people below and above API line 04:34:15 choices, bad, good habits, productivity 04:37:05 dream society, wireheaded 04:37:50 stream overview 04:38:40 making money at expense of other people 04:39:05 frog infestation on island 04:40:30 io_uring 04:42:05 same chip for quest 3 and samsung phone 04:42:50 make progress yourself before getting investors 04:45:30 softskills, social cues 04:46:50 integrating AI in everything 04:48:05 HR, human resources background 04:49:45 american empire, infrastructure, ivy league schools 04:53:50 trump, right wing, left wing 04:55:30 progress, faster plane is better 04:57:40 money, scam, value, perception is not reality 05:01:30 where are the viewers from 05:06:15 france gdp per capita 05:07:25 france vs singapore, china 05:08:50 uk vs us 05:10:45 foreign exchange reserves 100 years 05:13:15 WTF Happened In 1971? 05:14:30 raiding twitch.tv/lana_lux 05:16:15 be better, you canchange your habits and your life philosophy
    5
  307. 5
  308. 5
  309. 5
  310. 5
  311. 5
  312. Help George 01:07:05 | Read www.incompleteideas.net/IncIdeas/BitterLesson.html | github.com/tinygrad/tinygrad <- simple powerful deep learning framework | meta.com/quest/quest-3/ Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 intro George getting Meta Quest 3 00:01:45 don't call George 00:02:55 badly assembled Quest 3 00:03:40 metaverse 00:05:15 caution about buying Quest 3 00:05:30 Quest 3 target audience 00:07:05 banned topics, chinese 00:08:15 where was Quest 3 made 00:10:00 we stand with china 00:10:25 one more look at Quest 3 00:10:35 tariffs 00:12:50 qualcomm chip, services 00:14:15 what we respect and what we don't respect 00:16:40 who is the bad art friend 00:16:55 corporations behave psychopathically 00:19:10 comfortable life 00:21:50 example from last time 00:25:55 optimizing resnet 50 00:26:20 new kind of search 00:28:25 test dataset of kernels, generate_dataset.sh 00:29:12 get dictionary of all possible actions 00:30:35 all about building infrastructure 00:31:35 gifted subs = bigger font request granted 00:32:35 benchmark code 00:35:30 what we did overview 00:36:15 print factor 00:37:25 greedy search 00:40:15 font size requests 00:42:00 deep copy not working 00:44:00 kernel speeds 00:47:35 infrastructure for any kind of intelligent search 00:47:55 anyone understands greedy search? 00:52:15 infrastructure for fuzzers 00:56:45 variability of each run, correctness 00:59:40 exercise for the reader 01:02:50 how easy is to write searches in tinygrad 01:03:55 old optimizer, bad code 01:04:50 m1 macos runner for github actions 01:05:40 GREEDY=0 01:07:05 someone needs to write a fuzzer and improved search methodologies 01:07:35 the bitter lesson 01:08:00 double the speed, super universal 01:08:15 searching the entire search space 01:12:20 tflops of m1 max 01:14:14 test on your hardware 01:14:30 the beauty of RL 01:14:55 slow searches in tensorrt, tvm 01:15:10 small action space in tinygrad 01:16:00 real benchmarks 01:17:45 zig language 01:18:00 rolls-royce 01:19:15 space in between last word and exclamation mark 01:20:50 watching ads on twitch 01:21:20 lex metaverse 01:21:35 raiding lana_lux 01:22:05 watching lana_lux stream
    5
  313. 5
  314. 5
  315. 5
  316. 5
  317. 5
  318. Writeup and show notes: https://www.latent.space/p/geohot Grateful to Shawn Wang (of Latent Space) for allowing us to upload this video. Follow, Subscribe to Latent Space: - https://www.latent.space - https://youtube.com/@LatentSpace-podcast - https://twitter.com/latentspacepod - https://twitter.com/swyx (Shawn Wang) - https://twitter.com/fanahova (Alessio Fanelli) Source: https://youtu.be/K5iDUZPx60E Chapters: 00:00:00 intro 00:00:55 open pilot, devkit, gatekeeping 00:01:35 the hero's journey, what was the portal? 00:02:15 sam altman congress, ml compute, nvidia, qualcomm 00:03:24 CISC, Arm, RISC-V 00:04:15 good AMD stack, Google TPU, Google wrote their ML framework 00:06:05 turing completeness, re-order buffer, speculative execution, branch predictions, halting problem 00:07:40 clockless, analog computing, changing cache hierarchy, removing branch predictions, warp schedulers 00:08:20 turing completeness is easy, what is CUDA, TPU, systolic arrays 00:10:05 systolic arrays visualization, TPU closed source, AWS Trainium 00:11:25 tinygrad, lines of code, pytorch, tensorflow code 00:12:34 tinygrad developer experience, ONNX, ONNX runtime, compliance tests, core ML 00:13:25 unnecessary memory operations, pytorch lightning, why pytorch relu a class? 00:16:05 laziness, eager, graph compute model 00:17:30 competing against smart people, less complexity 00:18:15 how does fusing work, lazy.py 00:19:10 GRAPH=1, DEBUG=2, John Carmack 00:21:05 tinygrad right now uncompetitive on nvidia, x86, slower 00:21:32 tinygrad competitive on qualcomm gpu's 00:22:25 tensor core support, AMD bugs, opencl, ml perf 00:23:45 AMD kernel driver, ml framework, user space runtime, cuda_ioctl_sniffer 00:24:30 kernel panic, intel GPUs, AMD Lisa Su, AMD communication people 00:26:35 open source culture, nvidia nickel, nvidia P2P, cuda memcpy 00:28:00 building in public, contributing bug fixes to open source 00:28:32 ggml, M1 pytorch, AMD pytorch 00:30:00 test_ops.py, CI, good tests, mojo, pytorch compatibility 00:31:35 replicating python hard 00:32:08 tiny box red, limited by GPUs, luxury ai computers, fp16 llama 00:33:22 ggml quantization, compressing the weights, memory bandwidth 00:35:32 int8 support, weights in int8, fp16 to int8 to fp16 00:37:45 tiny box challenges, 6 GPUs, blowers or watercooling, pcie 4 extenders, pci redrivers 00:39:10 silent tiny box, 45-50 dB, one outlet of power, limit the power on GPU 00:40:30 AI hub for the home, personal computer cluster, pci bandwidth 00:41:50 training limit on tiny box, 7B, interconnect bandwidth 00:43:05 training longer, making bigger model, training, inference on cloud 00:44:30 on device training, fine-tuning 00:45:25 mining FLOPCoin, how to tell crypto is a scam 00:45:45 how to ensure your data is correct, tiny net 00:46:25 federated training, distributed training 00:47:42 enterprise use, flops per dollar, flops per watt, one person of compute as 20 PFLOPS 00:49:32 one Tampa of compute, GPT 4 mixture model, 16 inferences 00:50:40 secretive companies, hiding something that is not that cool 00:51:10 better training, batch norm, flash attention 00:52:50 Rich Sutton The Bitter Lesson, OpenAI computers you all you need 00:53:40 Hutter Prize, RNN, MDL, what is OpenAI getting wrong? vs working at facebook 00:55:38 how to hire people when computer can do everything 00:56:20 can a model do a simple pull request 00:57:05 unimpressed language models, subpar rap lyrics generation 00:58:04 10 LLMs in a room to discuss the answer, program generation 00:58:45 tiny corp is a remote company, 1000 job applications, programming challenges 00:59:30 tiny grad pull requests, stipend 01:00:45 coding is tool complete (above API line), driving is not tool complete (under API line) 01:01:40 stable diffusion replacing artists, tools getting better 01:02:30 full time at tiny corp, working on bounties, proposing bounties 01:03:16 separation in company 01:04:05 comma body, software problem 01:05:40 large YOLOs, segment anything, talking to LLMs, latency 01:06:12 LLaMA vs ChatGPT 01:06:40 no distinction between computer vision and language 01:07:30 company after tiny corp, AI girlfriend, merging with a machine 01:08:50 brain upload, George's brain already on youtube 01:09:30 living forever, how many weights a human has 01:11:05 the goddess of everything else, AI is not really going to kill us 01:11:35 AI alignment problem, the complexity will continue, paperclipers do not exist 01:12:25 grateful for AI, don't need hard math to understand AI, ML 01:13:54 John Carmack six insights, Elon's methodology 01:14:25 accessibility, tiny corp building computers, luck 01:15:25 why transformers work, semi weight sharing, qualcomm 01:16:25 the weights can change dynamically based on context 01:17:10 attention is all you need 01:17:50 Elon fundamental science physics, George fundamental information theory 01:18:55 e/acc, only the left takes ideology seriously 01:19:45 effective accelerationism, Mark Andreessen 01:20:25 why avatar 2 bad, Jake Sully 01:21:35 ChatGPT level pull request 01:22:00 impact of chat bots, spam bots 01:22:40 go try tinygrad 01:22:55 building chips, building silicone mines, self reproducing robot All material displayed in this video belongs to their respectable owners. We uploaded this video in good faith to share the work and progress of George Hotz, tiny corp and comma.ai.
    5
  319. 5
  320. 5
  321. 5
  322. 5
  323. 5
  324. 5
  325. 5
  326. 5
  327. tiny corp is hiring! -> 3x Software Engineer, 1x Hardware Engineer - https://tinygrad.org/ | It's happening George started a project -> Getting AMD on MLPerf -> https://github.com/geohot?tab=projects | Go help George if you are capable! He will appreciate you! | tiny corp has a website tinygrad.org. Let your friends know! | George is working on github.com/geohot/tinygrad/tree/risk (RISC-V) | Hardware reference.digilentinc.com/programmable-logic/arty-a7/start (Arty A7-100T) | Support George by following, enabling notifications and subscribing on twitch.tv/georgehotz | Follow on instagram.com/georgehotz for instagram stories | Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Apologies about the delay while uploading this. Not really creative commons :( Here is what you missed while muted: https://youtu.be/NH0rWI0iCqU (Chill Lo-fi Listening | Royalty-Free Music) 01:23:47: Let's do creative commons chill beats. There we go royalty-free music. 01:24:10: Sounds nice 01:24:30: This sucks. If we do version equals create sting buffer. 01:24:50: This just doesn't work. Why? 01:25:18: Alright someone probably already writen this. 01:25:38: This is acually super nice and I kind of want to use this chip at comma. 01:25:54: Alright where is that thing that I downloaded. This thing probably knows. 01:26:14: What is this? 01:26:26: Has someone already writen this yet? Using libftdi. 01:26:42: I just can't believe that the board has the interface closed source. 01:26:53: That is so annoying. Has my flash on it though. 01:27:23: What is this? I want this but in python. 01:27:54: Is this good? For openocd OK yea this seems good. cool. 01:28:10: Is this going to work? I am not even need to. This might just work. 01:28:10: Pretty sure openocd has jtag interface right 01:28:20: Why is this stuff so hard to find? 01:30:03: Yo I think it works maybe. Yea look at that. This is great. 01:30:14: Yo you are bro. See you know you guys thank you for reminding me sometimes why I stream. 01:30:21: You are not all terrible. Openocd. 01:30:31: Not teayoflow I am thanking LuigiFCruz. 01:30:37: Don't take credit. Alright arty a7 examples 01:30:50: GPIO Demo I gotta use Vivado for this? 01:31:13: Thanks BigCryptoCat. 01:31:19: I am not really feeling that. https://youtu.be/cTfTjVtZO8Q (And So It Goes) 04:25:01: No way. I never heard this before. https://youtu.be/FHO6a2H-pqY (Billy Joel - And So It Goes (Official Video)) 04:25:14: Sorry to copyright people. 04:25:18: I am disrupting penelship by stealing your dog. 04:25:21: This looks fine. It's like counting now. 04:26:26: That's fifteen yea there we go check it out. It counts.
    5
  328. 5
  329. 5
  330. 5
  331. 5
  332. 5
  333. 5
  334. ASMR stream | A Day In The Life Of A Software Engineer | 01:45:00 that is how real work on tinygrad looks like Bounties for tiny corp / tinygrad -> docs.google.com/spreadsheets/d/1WKHbT-7KOgjEawq5h5Ic1qUWzpfAzuD_J06N1JwOCGs/ youtu.be/lnVQsJJFcdg?&t=13715 Hiring entire stack for tiny corp join if you are interested | youtu.be/lnVQsJJFcdg?&t=14195 work major source of value in your life Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | github.com/tinygrad/tinygrad <- simple powerful deep learning framework tiny corp is accepting new interns. more info on tinygrad.org and tinygrad discord | comma.ai is accepting interns comma.ai/jobs#open-positions from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | from $999 comma.ai/shop/body the future of people Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 intro 00:00:30 ASMR typing 00:08:37 Lana_lux raid! 00:10:10 context missing, real work accessibility 00:12:50 linearizer 00:14:40 tensorcore implementation 00:15:10 tvm schedule 00:20:30 upcastmid 00:25:40 optuops 00:28:20 tvm parallel vectorize 00:30:35 future design 00:33:30 reorder 00:37:20 option space searchability design 00:41:40 git commit 00:44:00 test_float32_unary 00:46:00 fix_metal_tests 00:50:20 UOptOps(Enum) 00:54:00 group_for_reduce 00:59:40 yellow and purple 01:06:45 vin[0].arg out of range 01:07:45 tensor cores nightmare 01:10:40 metal_wmma 01:12:10 tvm tensor cores 01:15:40 problem 01:16:35 tvm implementation 01:17:30 thread_local_aliases 01:19:20 can fix this 01:25:00 device required 01:26:20 TensorCore._init_ missing 01:27:00 specialize dataclass, perplexity 01:28:20 dataclass inheritance 01:30:00 dataclass child class with default properties 01:36:50 git commit 01:40:50 tvm flexible interface 01:43:20 cool test 01:45:00 that is how real work on tinygrad looks like
    4
  335. 4
  336. 4
  337. 4
  338. 4
  339. 4
  340. 4
  341. 4
  342. 4
  343. 4
  344. 4
  345. 4
  346. 4
  347. 4
  348. 4
  349. 4
  350. 4
  351. 4
  352. 4
  353. 4
  354. 4
  355. 4
  356. 4
  357. 4
  358. 4
  359. 4
  360. Watch youtube.com/watch?v=iFUmWho7fBE - The AI Alignment Debate: Can We Develop Truly Beneficial AI? | Everyone should read how to ask questions the smart way --> www.catb.org/esr/faqs/smart-questions.html from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | tinygrad.org | github.com/geohot/tinygrad | Support George by subscribing twitch.tv/subs/georgehotz | Follow George on instagram.com/georgehotz & twitch.tv/georgehotz & twitter.com/realGeorgeHotz & twitter.com/__tinygrad__ to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 muted intro 00:00:55 un-muted, touch grass freestyle 00:02:45 conductors, new chair 00:04:20 official tinygrad meeting in tinygrad discord 00:04:45 PS5, 33 years old man, video games 00:05:50 installing sentencepiece, llvmlite, slow load 00:12:10 bitcast op, Alex 00:16:15 different debug levels 00:18:00 thank you to bedeks 00:18:35 ops_gpu not right 00:20:45 code getting unreadable 00:21:55 raw buffer, no copy, it's just a view 00:22:25 on gpu cl buffer and change dtype 00:23:30 this is becoming insane 00:23:45 collapsing under complexity, needs a refactor 00:27:55 bitcast just a flag on cast 00:30:40 making tuple 00:36:00 numpy bitcast 00:42:20 assertionerror 00:46:25 test_shape_change_bitcast 00:47:20 who wrote this crap haha 00:48:00 feeling good about this 00:48:25 test_dtype works with CPU and PyTorch 00:53:40 all this casts a mistake 00:53:50 features make things hard to read 00:54:10 dtype mismatch on realize 00:57:50 useful tracebacks 01:00:30 tuple no attribute name, assert instance dtype 01:02:40 I wrote unreadable code 01:07:10 how is numpy getting the type 01:11:50 I wrote hacks, disktype 01:17:00 we need tests for this, bitcast 01:18:50 Nelly rob the jewelry store and a grill 01:20:20 I'm a 20 year old girl who is looking for a good lay in Palm Coast 01:23:50 chat with 70B good 01:24:30 socialism 01:25:30 getting inappropriate 01:27:00 how to prevent regulatory capture 01:31:30 Llama 2 very inapropriate 01:32:30 how many genders are there 01:33:00 I want to be software 01:34:00 barbie post woke 01:34:40 discord at 5pm welcome to join the meeting 01:35:50 tiny voice channel 01:37:10 tinygrad pull requests 01:40:30 bounties 01:42:25 deceptively easy/hard $400 01:43:20 CIFAR improvement before selling tinybox 01:43:10 tinybox specs vs A100 01:44:50 locking SVC bounty 01:47:00 training multi gpu pull 01:48:00 rewriting AMD backend 01:48:25 gem stuff integration, single GPU faster 01:48:35 something like cuda cache thing 01:50:25 whisper pull 01:55:25 tiny3 back up, tiny5 mechanical box 01:58:50 tracking via pull issues 01:59:35 implementation is all it matters 01:59:45 engineering is all about tradeoffs 02:00:40 meetings not the right place for generic questions 02:01:20 hate meetings every day at 5 02:01:35 tinycorp, hate managing people 02:02:00 don't ask me questions 02:02:10 how to ask questions the smart way 02:02:50 tyranny of remote work 02:03:10 better and better metric that track people 02:03:30 what is a company in the future 02:03:40 async strategy 02:05:00 monday meetings 02:06:20 designer at comma 02:06:35 dealing with people pisses me off, dealing with hardware make me happy 02:07:35 why did I start another company 02:08:25 tiny boxes will print cash 02:08:55 end goal is building a chip 02:10:50 tinygrad has the right ideas 02:12:30 getting better, no regression 02:12:50 rockchip tinygrad backend 02:13:30 windows banned from tinygrad, whinygrad for windows 02:14:00 companies becomes own by bankers 02:14:10 qualcom board of directors 02:15:00 nvidia board of directors 02:15:25 meta board of directors 02:16:30 getting cancelled 02:17:10 defcon vaccine cards 02:18:40 cyber 02:20:40 tiny grad operating system for robots 02:21:05 every robot should use RK3588 02:22:10 why do I even stream 02:22:20 Connor Leahy debate
    4
  361. 4
  362. 4
  363. 4
  364. 4
  365. 4
  366. 4
  367. 4
  368. 4
  369. 4
  370. 4
  371. 4
  372. 4
  373. 4
  374. 4
  375. from $1499 buy comma.ai/shop/comma-three | comma.ai/shop/body | www.catb.org/~esr/faqs/hacker-howto.html#attitude | www.catb.org/~esr/faqs/smart-questions.html | tiny corp has a website tinygrad.org | Check out and star github.com/geohot/tinygrad | Support George by subscribing twitch.tv/subs/georgehotz | Follow George on instagram.com/georgehotz & twitch.tv/georgehotz & twitter.com/comma_ai to be up to date | Read George's geohot.github.io/blog/ | Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 intro 00:01:20 real reason why left twitter 00:03:15 google layoffs 00:03:33 deleting tweets 00:04:30 chatGPT 00:07:45 rlhf chatGPT 00:08:55 chatGPT will cost $42 a month 00:10:33 comma ai taco video 00:12:45 yann lecun cake 00:14:45 harmful text 00:17:10 decline in testosterone 00:18:05 andrew tate, sky bree 00:18:45 github.com/BlinkDL/ChatRWKV 00:19:15 gpt instruct 00:19:55 aligned with their users 00:21:30 recurrent neural network (RNN) 00:25:10 lesswrong backpropt predictive coding 00:25:44 chatGPT is not all you need 00:25:55 dreamer v3 00:26:05 deepmind vs openai 00:26:52 a walk in the park 00:27:08 danijar hafner 00:27:26 what is George up to 00:29:35 thneed 00:30:50 why is tinygrad important 00:31:45 arithmetic intensity nvidia 00:33:00 a walk in the park 00:33:35 comma three is a human head 00:35:15 getting a dog, border collie, corgi 00:37:00 reading the paper 00:37:55 explainpaper.com 00:38:15 barrier to entry too high 00:38:45 long term vision for comma (stores in the mall) 00:39:30 it's going to take a long time 00:39:45 walk in the park video 00:40:50 how to train it 00:42:00 looking at the walk_in_the_park code 00:43:00 learning to drive in a day 00:44:33 rt-1 robotics transformer 00:47:40 everyday robots 00:49:00 robot arm for $2000 00:49:50 evolutionary algorithms 00:50:25 why robots slow? 00:52:25 feature wise linear modulation 00:54:23 github.com/google-research/robotics_transformer 00:56:35 regression does not really work 00:58:40 comma three sales double 00:58:55 reddit comment search openpilot 01:00:30 how do you cheat in a simulator 01:02:10 internships in comma.ai 01:02:35 eric raymond hacker culture 01:03:00 eric raymond how to ask smart questions 01:07:00 companies run with hacker culture 01:08:10 responding with another question 01:08:50 building a product that will make you an omelet 01:10:40 people will not learn if you give them an answer 01:11:20 openpilot best ADAS system in the world 01:11:45 independent workers 01:13:00 reluctant managers 01:14:05 diversity hires 01:14:50 comma hiring for robotics 01:17:15 robots that move fluid 01:20:05 RoboFly: An insect-sized robot with simplified fabrication 01:20:55 an attention free transformer 01:24:35 mastering diverse domains through world models 01:27:30 the critic 01:29:30 new rap song 01:32:30 you are the problem 01:34:30 instant neural graphics primitives 01:35:30 facebook atlas github 01:36:20 retro retrieval model 01:36:33 the pile dataset 01:37:40 improving language models by retrieving from trillions of tockens 01:39:10 rice theorem 01:40:15 crypto in the last four years 01:41:50 politicians that know the derivitive of x squared is 01:42:40 the myth of the myth of the 10x programmer 01:44:55 elon the process linkedin 01:47:25 happy back in san diego 01:47:40 e/acc blog post 01:50:25 is your company capable pivoting 01:51:40 who owns comma 01:53:40 dreamer v3 01:54:10 danijar hafner 01:55:10 actor critic rl 01:59:25 tinydreamer 02:00:20 chatGPT is not all you need 02:01:20 the forward forward algorithm 02:02:40 Alex and friend 02:03:40 shiba dog iq 02:04:10 dog name 02:04:45 taking away tech from... 02:05:10 secret source of papers 02:05:28 shutting down cheapeth server 02:06:25 good bye 02:06:45 sky bree invitation 02:07:20 instagram reels
    4
  376. 4
  377. 4
  378. https://tenstorrent.com/cards/ (Grayskull™ e150 - Add to cart) - $799 | https://tenstorrent.com/setup/ | https://github.com/orgs/tenstorrent/repositories | https://github.com/geohot/tt-twitch Bounties for tiny corp / tinygrad -> docs.google.com/spreadsheets/d/1WKHbT-7KOgjEawq5h5Ic1qUWzpfAzuD_J06N1JwOCGs/ youtu.be/lnVQsJJFcdg?&t=13715 Hiring entire stack for tiny corp join if you are interested | youtu.be/lnVQsJJFcdg?&t=14195 work major source of value in your life Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | github.com/tinygrad/tinygrad <- simple powerful deep learning framework tiny corp is accepting new interns. more info on tinygrad.org and tinygrad discord | comma.ai is accepting interns comma.ai/jobs#open-positions from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | from $999 comma.ai/shop/body the future of people Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 intro 00:00:40 tenstorrent e150 00:02:40 AT&T, AMD, Qualcomm 00:04:07 Jim Keller, Tenstorrent, Open source 00:07:13 nvidia orin devkit, TRUFFLE–1 00:10:10 tinybox specs, nvidia 5090 rumor 00:11:05 pushing AMD, embracing open source 00:12:20 unboxing e150 00:13:55 interactions with AMD, lying 00:15:13 tenstorrent mega blower fan 00:23:10 fan loudness check 00:23:45 fanless, thermal protection 00:28:42 tenstorrent.com/setup, NDA 00:29:10 Alex 00:30:22 hardware installation 00:34:10 banjo.canonical.com 00:35:10 firmware installing broken 00:39:55 work life balance, Alex 00:41:15 tt-smi, temperature 00:43:07 tenstorrent bounties 00:47:15 smoke test 00:49:45 buda 00:53:40 cloud is a scam, selling cards 00:55:20 import error 00:57:50 this is why tinygrad is going win 00:58:55 nix 00:59:25 docker, no dependencies 01:02:20 nvidia value, cuda 01:02:55 reproducible builds, getting rid of complexity, it just works 01:05:30 import error 01:06:40 the complexity 01:06:54 redis great software 01:08:22 missing jemalloc 01:11:00 key error BACKEND_ARCH_NAME 01:12:15 mysql, sqlite 01:14:40 docker encouragees 01:15:15 exploring tt-metal 01:17:45 PEP8, bad programmers, details 01:18:25 downloading multiple gigabytes 01:20:35 translating github comments 01:25:00 tt-buda 01:26:04 stop writing code like this 01:32:30 if you thought AMD is bad try tenstorrent card 01:32:55 tt-smi reset options, reset file, reset json 01:35:45 this is the opposite of how you do complexity 01:36:40 ordering food, folding phone 01:42:05 tt-metal 01:46:00 start saying what it is 01:48:30 HSA fail, making everything generic 01:51:20 tt_lib, tt_eager 01:52:00 py-buda, luwen 02:04:00 installing egg 02:10:00 tensor bob 02:13:20 devin 02:15:30 reading docs 02:17:40 pyrsistent 02:19:40 Jim stop, nobody wants 50% PyTorch 02:19:55 be proud of what the chip is and expose the chip for what it is 02:21:19 ttlib docs, chip info 02:23:10 no real company is going to buy this card, dojo, inference price per dollar 02:23:50 tinygrad in for 10 years 02:24:05 PyTorch trigger 02:31:10 what this is? 02:35:45 grayskull e150 number of cores 02:37:20 bad documentation 02:42:44 tenstorrent staff in the stream 03:00:25 not trusting C 03:05:02 Alex closing the door 03:11:50 hate coding in C 03:17:00 bad code trigger, memcpy 03:27:17 SFPU 03:34:15 doing math, 13+7=21 03:43:30 kernel APIs 03:47:50 this better than GPUs 04:02:25 opencl api bad concept 04:06:30 food 04:12:20 life magazine 1970 04:16:10 Alex, Groq $20k card, Groq open source 04:20:05 brain backprop, small work units 04:20:39 nvidia full fabric memory 04:22:26 tenstorrent website bug 04:24:07 keeping up with state of the art ML 04:25:10 AMD price advantage, mi300x 04:28:20 graph compiler should be generic 04:28:35 port tinygrad to tenstorrent 04:33:40 the bitter lesson 04:38:50 break 04:47:12 worry about correctness 04:48:25 thumbs up ai 04:48:50 tinygrad code for metal 05:09:00 host_api 05:11:00 Karen ai, devin, VCs, deep state 05:15:17 unitree humanoid, money printer 05:19:04 1971, gold 05:25:50 clang 14 install, VC investment scam 05:42:50 dispatcher kernel, 1971, zebu ep1 06:03:00 fake queuing, scheduling is the key, big scale 06:11:30 Groq demo, tt hardware, wormhole 06:29:25 software, endorsing tenstorrent, comma body 06:49:50 X9000 G2 Groq demo challenge
    4
  379. tiny corp is hiring! -> 3x Software Engineer, 1x Hardware Engineer - https://tinygrad.org/ | It's happening George started a project -> Getting AMD on MLPerf -> https://github.com/geohot?tab=projects | Go help George if you are capable! He will appreciate you! | tiny corp has a website tinygrad.org. Let your friends know! | A half noob lesson, introduction to tinygrad. Go read the codebase -> github.com/geohot/tinygrad | Support George by subscribing on twitch.tv/subs/georgehotz | Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC From 02:38:20 on George is playing the guitar. George is so good on the guitar that is triggering youtube copyright claims. Chapters: 00:00:00 intro 00:00:41 drug test, drinks, to shower or not to 00:04:40 sub only chat, tinygrad lazy 00:05:10 tour of tinygrad, Tensor 00:06:20 graph 00:07:30 print underlying buffer if it's realized 00:09:20 lazy mode, two armies of haters 00:10:20 multiplying two matrices and graph 00:11:00 GPU support, dashes - not a contiguous tensor 00:11:35 tinygrad codebase, three backends CPU, GPU and Torch 00:11:50 lazy mode 00:13:00 tinygrad is a tensor library, tensor is a list of numbers 00:14:20 benefit of laziness 00:16:10 stream lag, dirty hoodie 00:16:48 ImageNet inference, stream lag, GPU=1 00:19:47 tinygrad definition, 992 lines 00:20:25 tensor class, karpathy, micrograd, 00:23:15 backends, openpilot model 00:24:05 automatic differentiation engine, bad bug 00:26:05 let's fix that bug 00:29:15 hlops, tests 00:31:50 fun thing in tinygrad 00:32:25 pad2d, matmul 00:35:40 transpose, flatten, sum 00:36:40 softmax, dropout, pools, conv, unary 00:37:05 binary ops 00:38:20 tinygrad is one of the best things I have ever written 00:39:25 div, short break 00:40:30 neural networks ops 00:41:45 linear 00:43:10 sequential, reduce 00:44:40 an instantiation of the function in the context 00:45:45 mlops, cleanup, unary ops 00:47:50 good tests, more cleanup 00:53:20 the derivative of convolutions as convolutions 00:53:50 llops - unary_op, binary_op, movement_op, procesing_op, reduce_op 00:54:20 working on the code 01:00:00 how does it run on the device 01:02:10 numpy, GPU backend, Torch backend 01:02:50 train efficientnet and graph 01:05:00 custom accelerator support, shapetracker 01:05:45 optimizers, BatchNorm, Conv2d 01:06:40 cherry hardware, data movement is expensive 01:08:45 rockchip neural network accelerator 01:09:20 replacing numpy 01:12:45 improving performance by removing GPU cache 01:14:00 equivalent of Numpy.argsort() python 01:16:15 codebase understanding, tests 01:18:00 chain the autograd 01:20:00 nobody liked the tinygrad tour 01:22:15 new readme 01:23:00 tinyvoice, tinygrad will be faster then pytorch 01:24:15 GPU support 01:25:50 simple and memory efficient 01:27:00 tinygrad support 01:28:00 this should be in the readme, everything compiled on the fly 01:28:40 tinygrad++, refactor feels correct 01:30:10 convolution 01:31:30 bugs to fix 01:37:55 benchmarks 01:39:10 Tuple index out of rage, cannot resolve name lazyop, more errors 02:11:40 commit typechecks 02:21:00 why I don't stream tinygrad 02:22:35 torch question, issue 02:26:15 timeout for bad question, more bad questions 02:28:40 cool pull request from smurfd0 02:29:50 sub so you don't get a timeout 02:30:10 no j_blow stream to raid 02:31:20 freedom of speech on twitch, non subs vs subs 02:34:30 giving money to amazon, patreon 02:35:45 prime sub 02:36:25 tinygrad is work of art, whiteboard 02:38:20 tuning the guitar 02:40:00 drake - best i ever had 02:41:00 eagles - hotel california 02:44:00 the front bottoms - twin size mattress 02:46:30 goo goo dolls - iris 02:48:08 american football - never meant 02:48:45 john denver - take me home, country roads 02:51:45 dragonforce - through the fire and flames 02:52:05 the hotelier - an ode to the nite ratz club 02:52:45 led zeppelin - stairway to heaven 02:57:25 thank you for watching
    4
  380. 4
  381. 4
  382. 4
  383. 4
  384. 4
  385. Buy 1 year commacare warranty -> https://comma.ai/shop/commacare | Check if your car is supported -> comma.ai/vehicles and buy -> comma.ai/shop/comma-three from $1499 | blog.comma.ai/dumb-questions | What is comma.ai/openpilot | Join the comma team comma.ai/jobs | How to setup your car comma.ai/setup | Follow the official youtube.com/commaai for more livestreams | Stay up to date by following twitter.com/comma_ai | Support comma.ai at comma.ai/shop. Chapters: 00:00:00 intro 00:00:13 drink sponsor 00:00:35 office poster 00:00:45 plant automation 00:01:08 comma body 00:02:53 banning user 00:03:35 comma one 00:03:58 COMMA_CON 2023 00:05:33 tickets 00:06:25 new product 00:07:45 visiting Alex 00:08:11 if you want to work at comma 00:08:50 comma three boards 00:09:00 factory machines 00:09:25 Alex 00:10:14 comma three extended warranty 00:11:25 comma care on comma.ai 00:11:50 comma care fine print 00:12:35 you can't make people happy 00:13:00 COMMA_CON for product announcement 00:13:20 $400 commacare extended warranty 00:14:05 raising price for including car harness 00:14:20 comma two failed 00:15:02 no fault warranty 00:15:20 comma three sales 00:16:02 COMMA_CON 00:16:25 profitability 00:16:50 parts changes 00:17:15 additional price drop, margin 00:17:30 software engineer salary at comma 00:18:08 employee buying house of stock 00:18:15 no opportunity, no remote work 00:18:40 device falls of the glass 00:18:58 qualified installers for comma 00:19:25 driver monitoring 00:19:45 cabana comma 00:19:55 comma three life span 00:20:25 driver monitoring training 00:20:40 chinese ev support 00:20:58 did not leave the comma president 00:21:20 marketing intern 00:21:40 navigate on openpilot 00:22:05 heat and comma three 00:22:40 comma con stream 00:22:50 design role in comma 00:23:25 comma three heat 00:24:00 latest build vs tesla fsd 00:24:45 outside of US openpilot 00:25:20 can fd hardware 00:25:39 comma security 00:25:50 new release 00:26:18 longitudinal strategy 00:26:40 following distance 00:27:00 devkit 00:27:55 comma care 00:28:35 parking in sun 00:29:10 selling out, 7 more years to solve self driving cars 00:29:25 99% to get you to destination 00:29:40 opensource vs closed software 00:30:05 boxing match vs Elon 00:31:10 comma three compute 00:31:46 EU roads, india roads 00:32:30 VIP tickets sold out 00:33:55 comma three in rain 00:34:14 what does George do all day 00:34:35 camera see better in rain 00:35:20 simulator, slam 00:36:10 training should not be open source 00:36:35 weights open, training stack $1 milion computer 00:37:30 confidence in open pilot 00:40:30 designer job at comma 00:41:15 comma profitability 00:42:31 Lisa Su is cool, corporate partnership 00:43:05 VIP tickets back 00:43:35 manufacturing warranty 00:44:05 fuses blown, ecu camera broke to 1 person 00:45:10 comma warranty 00:45:50 tinygrad in openpilot 00:46:45 fsd beta 00:47:15 George's car not supported by openpilot 00:47:45 rolls royce ghost, dealership 00:48:05 designer job at comma 00:49:20 hardware team hiring process 00:49:42 comma body 00:49:52 python openpilot code base 00:50:50 speed humps 00:51:10 fully automatic lane changes in supported cars 00:51:40 experimental mode slow 00:52:20 comma body coffee shop 00:53:45 no motorbikes support, cars are safe 00:54:55 head of production job 00:55:18 uploading clips to comma 00:55:39 new product is comma care 00:57:18 not sending product to youtube's 00:58:00 George's salary 0 right now 00:58:40 comma shares 00:59:20 never going public 01:00:00 how hard to fingerprint the car, best toyota, tesla not really supported 01:01:15 gm canceled the bolt, stopping ai, comma care, OEM reach out 01:03:35 aptera, safety, banned for comma network, don't scam comma
    4
  386. 4
  387. 4
  388. 4
  389. 4
  390. 4
  391. 4
  392. 4
  393. 4
  394. 4
  395. 4
  396. 4
  397. 4
  398. 4
  399. 4
  400. Watch youtube.com/watch?v=6yQEA18C-XI - George Hotz vs Eliezer Yudkowsky AI Safety Debate | Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ | Sources for this stream: - https://twitter.com/realGeorgeHotz/status/1690831164431585280 - https://wtfhappenedin1971.com/ - https://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf - http://prize.hutter1.net/hfaq.htm - https://arxiv.org/pdf/2303.08774.pdf - https://geohot.github.io/blog/jekyll/update/2023/08/08/a-really-big-computer.html - https://arxiv.org/pdf/2005.14165.pdf - https://arxiv.org/pdf/2203.15556.pdf - https://en.wikipedia.org/wiki/Landauer%27s_principle - https://en.wikipedia.org/wiki/Why_there_is_anything_at_all - https://en.wikipedia.org/wiki/List_of_unsolved_problems_in_computer_science - https://en.wikipedia.org/wiki/Richard_S._Sutton#Career Chapters: 00:00:00 muted intro 00:01:15 un-muted intro 00:01:38 debating Eliezer Yudkowsky, bad people 00:02:35 Sam Altman 00:04:35 Thermodynamics is to Energy as <???> is to Intelligence 00:05:00 where did smartest people go in 1950 00:05:40 debating Sam Altman, RLHF, truth 00:07:55 universities is where the smartest people go in 1950 00:08:25 where do the smartest people go in 2023 00:08:54 hedge funds, FAANG, startups, power over nature 00:10:00 less theory, more applied, hard doomers (EA), thermodynamic god 00:11:00 why this split exist, we have no theory, arguments for Yudkowsky 00:11:50 intelligence criticality are possible? 00:12:00 1800, small stick boil water calculation 00:15:20 asking ChatGPT grams of wood to boil a gallon of water 00:16:25 how much intelligence do I need to prove fermat's last theorem 00:17:35 thermodynamics, classical physics 00:18:18 no unit for intelligence, unit of energy per second, J/s, W, Horsepower 00:19:35 person-years, bad universities, Carnegie Mellon University 00:20:50 2012, 2014 universities destroyed 00:22:15 getting a drink 00:22:50 wtf happened in 1971 00:24:00 information theory, entropy, thermodynamics for information 00:25:18 real university reach out to me 00:25:40 collage, scam, Scott Aaronson, Robin Hanson 00:26:15 administrative class, professional managerial class 00:26:40 hutter prize, compression is intelligence 00:27:40 gpt-4 technical report 00:28:25 don't hate Open AI, overhyped, person-years 00:29:55 using GPT 4 for field of science, best tutor to ever exist 00:31:00 plot gpt-2 gpt-3 bits per characters on enwik8 00:31:45 Sergey Brin is back at Google, Sundar Pichai 00:32:30 petaflops-days, 1 person = 20 PFLOPS 00:33:40 Paul Christiano Bankless 00:34:20 losing to Connor Leahy = thinking about AI alignment 00:35:39 Joscha Bach latest Lex Fridman 00:35:55 1 person years = 7300 petaflops-days 00:37:20 chinchilla deepmind, bits per character, token 00:38:38 Elon, Twitter/X login pop-up, destroy so much long term value in a company 00:39:40 what unit is the loss in for LLMs 00:40:05 nat (unit), shannon (unit), bits, 1 george of intelligence 00:41:10 beautiful elu, minimal energy requirement 00:42:20 landauer's principle, can reversible computation implement intelligence 00:43:30 why AI doesn't destroy the world, God is real 00:44:15 we are in the video game, highly rated game, we don't die in some stupid way 00:46:10 never use the word AI safety, instead use AI alignment 00:46:49 feminism legal equality of man and women 00:47:05 alignment, good loss functions that do what you want 00:47:32 defcon, Why there is anything at all 00:48:25 there is no difference between agents and tools, Robin Hanson 00:49:00 the tail wags the dog, car is the dominant species on earth 00:51:35 we have no real theory to answer how much intelligence we need for... 00:52:08 list of unsolved problems in computer science 00:52:50 people think George is not serious, care about the search of truth 00:54:15 academic system that is completely destroyed 00:54:45 information based approach is good 00:55:15 university not good for finding a job, education industrial complex 00:56:40 Eliezer Yudkowsky did not attend high school or college 00:58:25 no theory of intelligence, bounds of complexity theory 00:59:35 where is Marcus Hutter, deep mind, Jürgen Schmidhuber, Richard S. Sutton 01:00:40 Sutton became a Canadian citizen in 2015 and renounced his US citizenship in 2017 01:01:09 defcon vaccine card check, defcon crazy people, public apology, san francisco 01:02:35 real science, real questions, can't do science anymore, grant process 01:03:40 professional managerial class + AI = bad 01:04:05 Alex tired of the rants, preparing the debate 01:04:28 open source AI are bad, only for trusted people 01:05:20 if science about intelligence is done somewhere link it in the comments 01:06:05 lot's of questions, no answers, physics off the rails 01:06:30 what happened that stop science 01:06:45 we don't live in a world where standing still is safe 01:07:00 what happened to the internet, consolidate power 01:07:55 thank you for watching, we do not die 01:08:14 are you the problem? if you are think about what can you do to be less of a problem 01:08:45 tinygrad, tinycorp, comma, comma hackathon 01:08:55 deleting twitter, living under the bridge and finding truth
    4
  401. 4
  402. 4
  403. 4
  404. 4
  405. Thank you to the comma team for this amazing event and sharing the livestream and high-quality videos. All credits to them. Follow their official youtube.com/commaai for more livestreams. from $1250 buy -> comma 3X comma.ai/shop/comma-3x | openpilot.comma.ai is the best ADAS system in the world | If you have skill apply on -> comma.ai/jobs | Join the community on github.com/commaai/openpilot & discord.comma.ai | Check if your car is supported -> comma.ai/vehicles | How to setup your car comma.ai/setup | Stay up to date by following twitter.com/comma_ai | Support comma.ai at comma.ai/shop. Chapters: 00:00 black mirror episode with ratings 00:45 three stories, past, future, present 00:55 the past, cruise automation 01:13 cruise RP-1 highway autopilot 01:35 giving up on technical challenge 01:45 too many cars to support 02:25 how to handling failures 03:10 fingerprinting 03:45 comma was not supposed to exist 04:30 what is worth doing in the world 05:25 6 generations of hardware 06:15 keep iterating the hardware 06:40 being real, honesty, communication people 07:35 shipping keeps you honest 07:45 stated preference, revealed preference 08:19 iteration 08:40 humility, egohot, big ego 09:50 so why haven't we won? 10:00 things take a long time, even accounting for things taking a long time 10:38 no exit, the system will get better 11:04 climbing out of a well 11:53 it turned out these companies were scams, bitconnect and luna scam 13:10 marginal utility - marginal cost 13:25 value destruction, linux fastest growing platform on Azure 14:05 markets - rationality, you just have to survive 14:20 lifetime revenue 15:21 $22.2M on $18.1M raised 15:33 twitter followers 15:50 future followers 16:00 employee retention 16:18 come work to comma ai all work by the team not George 16:33 keeping idiots away from comma 16:50 bar very high, you get to ship things that get better in the world for the better place 17:33 business world 17:45 the future 17:59 computing speeds vs human brain computation 18:45 forward looking statement 18:59 comma con 9 in year 2037 19:15 44 stores in malls, top 10 biggest consumer electronics companies, without selling out 20:07 hard to go anywhere without finding a comma, cars, bodies, puppies 20:35 comma body 20:45 comma 7 for $2500 21:01 3rd generation comma ASIC, 50000 TOPS 21:12 first comma device to boot into tinygrad 22:25 trying to build people 22:50 trolling Elon, our system is going to be so end to end even our driving monitoring will be end to end 23:10 comma 7, how many silicon people work at comma 23:50 your comma 7 is yours, models we train in the cloud are foundation models 24:08 thank you to meta for LLaMA, open source AI 24:32 offer pretrained policy models, custom software 24:56 comma 7 is like child 25:06 will learn behaviors you teach, so teach it well, pick up skills as quickly as smart human 25:16 comma 8 will be running the company 25:26 comma three human head 25:55 intelligence 26:20 humans are so good at moving, driving 26:57 the present COMMA_CON 2 27:22 the mall store soon 27:45 sitting down for dramatic effect 28:12 failure rates for iPhone 28:50 comma two failure rate 29:25 comma three failure rates 30:43 dropping the price 31:15 comma 3x 31:36 much better injected molded case, cameras on board 32:12 3X specs, same hardware and software as comma three 32:49 upgraded cameras, 140 dB HDR 33:14 128GB of storage 33:28 CAN-FD integrated 33:45 still OLED better supplier, onboard road cameras 34:40 stereo in-case sound 35:02 upgraded power architecture 35:22 about half the components, more reliable 35:32 3X board front, no more GPS connector, better antennas 36:30 3X board back 37:01 battery slides in and out, chokes for CAN-FD, e-sim 37:30 cameras are right on the boards, failure rate of this device need to be 1 or 2 % 38:05 comma three vs comma 3x board complexity 38:25 no more NVMe, just on-board flash, QR codes on the board for tracking 38:50 is it a devkit? to set expectations 39:24 it's time to call it just a comma 3X, mass market 40:15 comma 3X just $1250 40:30 no more questions that is why it's just one edition 41:29 it's just a comma three but better 41:48 order now, ships in 1-8 weeks 42:25 comma 3X in EV6, no more red panda, no more two harnesses 42:44 when did the iPhone get good 43:24 the comma team built this not George 44:05 value in running language models on the device 44:50 marketing guy, testing on commercial vehicles 46:13 marketing, the story of openpilot, openpilot ADAS 46:55 buy now, 30-day money back guarantee 47:20 comma 3X always compatible with cars 47:56 port a new model to comma 3X 48:50 mass market, liability air gap with installing custom software 49:45 level 2, you must pay attention at all times 50:50 large foundation model on device, expansion port 52:55 failure rate, should be built like a tank 54:17 saying no to things, best highway experience, first comma device for friends
    4
  406. 4
  407. 4
  408. 4
  409. 4
  410. 4
  411. 4
  412. 4
  413. 4
  414. 4
  415. 4
  416. 4
  417. Go find the bug 02:06:30 | Who is going to be the first to find what was the bug? 02:49:45 | 02:50:20 | https://adventofcode.com/2022 | https://pastebin.com/rx0Yj0H1 | Go buy comma three from $1999 https://comma.ai/shop | tiny corp has a website tinygrad.org | Check out and star github.com/geohot/tinygrad | Follow George on twitter.com/realGeorgeHotz & instagram.com/georgehotz & twitch.tv/georgehotz & twitter.com/comma_ai to be up to date | Read George's geohot.github.io/blog | Support George by subscribing twitch.tv/subs/georgehotz Chapters: 00:00:00 intro 00:01:33 advent of code in scala 00:02:20 mobile streaming setup 00:04:21 python 3.12 improvements 00:04:35 monitor size 30" at home 00:04:50 why at twitter 00:05:45 vscode scala debug 00:06:35 twitter interview, technical interviews 00:07:55 no build target could be found for the path 00:11:10 elfbar vape hardware, new identity 00:12:00 chatgpt overload 00:12:15 scala debug single file vscode 00:13:20 vscode metals 00:14:40 real tips to get better with using keyboard shortcuts 00:17:45 learn default keys, hate Ghidra default keys 00:18:35 scala build target single file 00:19:45 scala app 00:21:05 scala startup time slow 00:21:50 yesterday stream bad mood, meatballs 00:24:25 scala 3.2.1 00:26:50 scala bloop 00:27:55 java javahome 00:29:55 scala build target, sbt example 00:38:50 aoc day 5 00:41:45 scala slice 00:43:16 scala every 3rd element from list 00:45:00 scala via a launch json config 00:47:50 scala flatmap, scala list method 00:48:40 scala for loop 00:50:20 scala list comprehension 00:52:25 scala list yield 00:56:15 scala list until 00:57:05 scala array api 00:58:58 scala array update 01:02:00 aoc day 5 finished, scala nice 01:04:50 culture at comma, appreciation for beauty 01:05:20 beautiful code, no build tool detected 01:07:38 aoc day 6, appreciate aoc, jack dorsey 01:08:30 halting oracle 01:11:58 only classes can have declared but undefined members 01:13:35 scala array no duplicates 01:14:18 scala loop break 01:14:48 scala print break 01:15:08 scala return 01:15:47 scala find 01:19:04 aoc day 7 01:23:05 so easy in python 01:23:40 making class 01:23:55 scala class, directory 01:25:58 scala substring 01:26:30 scala elif 01:33:25 scala map mutable 01:35:20 object oriented stupid, class gone 01:38:30 scala copy list 01:45:00 yolo, answer wrong 01:52:25 some directory you never ls but understand it exists 01:53:20 adding to this directory 01:55:00 scala set 02:00:30 jonathan blow raid! 02:02:05 scala map getorelse, map get or set 02:02:40 what we are doing today 02:03:08 getorelseupdate scala 02:03:35 how nice is scala 02:05:30 functional programming in scala second edition 02:06:30 trying to submit the same answer, it's wrong 02:07:30 refactoring, substring worry 02:08:15 it will quit telling you after third attempt 02:09:55 directories with same name 02:12:35 it's not falling back to address to hash 02:16:51 slash special directory 02:18:25 bigger than maximum int 02:20:05 two directories with same name 02:22:00 sometimes there is a trick 02:24:18 scala int max size 02:24:59 going full object oriented 02:25:58 scala extends 02:26:20 scala class constructor 02:27:55 scala union type 02:33:40 coding like java programmers 02:35:55 value update s not a member of object file 02:37:10 beautiful object oriented code 02:37:35 scala class, scala abstract base class, trait 02:43:00 scala typeof 02:44:25 scala flatmap 02:47:40 base case, this :: 02:49:30 bigger number is right 02:49:45 what the hell was the bug? 02:50:20 excited to read the youtube comments about what was the bug 02:50:30 part 2 of aoc 7 02:55:50 scala sort 02:57:00 not the right answer 02:57:20 it want's the size not the name 02:57:45 get name not needed 02:58:04 thank you for watching the stream appreciate you all 02:58:20 san francisco 02:58:45 I want to see in youtube comments someone figure out what the bug was 02:59:10 nobody to raid 02:59:35 https://pastebin.com/rx0Yj0H1 03:01:05 every day we get a little bit better, key to success 03:01:15 climbing out of a hole, as long as every day you climb more feet up than you slide down
    3
  418. 3
  419. 3
  420. 3
  421. 3
  422. 3
  423. 3
  424. 3
  425. 3
  426. 3
  427. 3
  428. 3
  429. 3
  430. 3
  431. Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | github.com/tinygrad/tinygrad <- simple powerful deep learning framework tiny corp is accepting new interns. more info on tinygrad.org and tinygrad discord | comma.ai is accepting interns comma.ai/jobs#open-positions arxiv.org/pdf/2101.03961.pdf | news.ycombinator.com/item?id=38570537 | blog.research.google/2022/11/mixture-of-experts-with-expert-choice.html | together.ai/blog/stripedhyena-7b | github.com/tinygrad/gpuctypes from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | from $999 comma.ai/shop/body the future of people Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 muted intro 00:01:12 un-muted intro 00:01:25 tiny keyboards, feud with oneplus 00:02:38 bought oneplus open via oneplus store 00:03:16 bricked phone, fastboot set active slot, qdl mode, qdl data not available 00:05:14 oneplus 10, 11, open no images to restore, not honoring return policy 00:06:50 oneplus destroyed their brand 00:07:47 chargeback, buy oneplus buy from amazon 00:08:20 comma return policy complete and true 00:09:10 chargeback sending the phone back, apple late delivery experience 00:11:40 help George process emotions, do not buy it from oneplus store 00:13:00 comma does not want you to have a bad time 00:13:27 sending it back 00:13:55 edl, unofficial qualcomm firehose 00:16:50 phone market, why no images 00:17:48 companies don't want you to own anything anymore 00:18:15 Apple is so consumer focused 00:18:35 jailbreaking is not a solution 00:19:05 comma.ai forks of the software and hardware 00:20:35 possible to make phones 00:21:28 don't buy via oneplus store and don't expect anything from oneplus people 00:21:38 buy on amazon, oppositional defiance, great regrets in life 00:23:05 how do we get oneplus to release the qdl files 00:23:40 suing someone different 00:24:55 2% of unreasonable people 00:25:18 George reads all the emails 00:25:33 reading youtube comments 00:25:50 spicy comments, oppressive powers and free market, apple safari 00:27:48 anarcho capitalism 00:28:27 interns pay $1400-$2000 week 00:28:40 trans people, J. K. Rowling, transhumanism 00:30:10 ssh to tinybox 8 00:30:20 mixtral.py 00:32:08 python mmap 00:33:35 only 6 GPUs 00:34:15 bigger font, glasses, astigmatism 00:35:10 torch_load dynamically 00:37:25 Switch Transformers: Scaling to Trillion Parameter Models 00:37:41 Hacker News Mistral "Mixtral" 8x7B 32k model 00:38:53 Mixture-of-Experts with Expert Choice Routing 00:41:30 it's how GPT4 works 00:42:00 StripedHyena-7B, mixtral-7b-8expert 00:44:05 MixtureFeedForward 00:58:25 AT&T feud, billy naming 01:01:35 underlying buffer is not writable 01:04:35 read only memoryview in python 01:05:40 ctypes const pointer 01:07:05 ctypes from buffer 01:08:20 asking Quentin 01:11:19 memoryview readonly 01:14:10 python get address from memoryview 01:15:33 using a read-only buffer with ctypes from_buffer 01:16:30 b partition in honor in oneplus phone 01:16:44 oneplus store worst customer experience 01:17:35 buy cybertruck, oneplus worst company over time 01:19:05 HIP=1 DEBUG=2, realize() struggle 01:24:50 output of a gate 01:29:21 how easy is tinygrad to use (multi GPU usage) 01:30:40 looking at mixtral implementation 01:34:20 you need to run all the experts 01:34:50 stupid questions about buying tinybox 01:36:20 emotional regulation issues, don't be evil, us government 01:38:20 only batch size 1 01:42:30 print top, norm 01:45:20 /opt/rocm/bin/rocm-smi, htop 01:48:50 bug in AMD gpu 01:49:00 multiple $1k bounties in tinygrad dropped 01:51:00 libhsa-runtime64 01:53:20 rude to blame people until you are sure it's their fault 01:55:00 HIP runtime rewrite 01:57:25 first token 330 01:58:55 what is Mistral going to say 02:00:00 GPU usage explained, letting it speak 02:03:10 running on mac impossible, tinybox can run the model 02:04:00 pushing the code to github, refactor needed 02:06:30 letting non-subs talk for some questions 02:06:44 AMD bug 02:07:15 first real use of tinybox 02:07:40 why is mistral getting so much attention 02:08:00 how did we split them 02:08:10 quantization worst performance 02:08:20 Cmulator 50 gifted subs, thank you to subs 02:10:10 bug with errors with the micro engine scheduler 02:10:32 bug hip compilation inexplicably takes 5x longer then CL compilation because of different code path 02:10:40 open source, submiting pull requests and CI 02:10:55 tinygrad beating cuda, mmap nvme copy in to gpu bug 02:12:45 opinion on triton openai, delivering tinyboxes 02:13:50 best Mixtral content so far 02:17:55 cross gpu overhead, copies between gpu 02:22:45 working on synchronize things across gpus 02:23:00 GO, C speed, fixing the algorithms to speed up 02:23:55 non-sub asking for moderator, tinygrad CEO 02:26:18 gpuctypes tinygrad, GPT2 below 2ms per token 02:28:10 three new $1k bounties 02:30:30 never buy anything on oneplus store 02:31:40 Elon thoughts, famous, opinions, Taylor Swift, Sky Bri 02:34:08 small tasks needed in tinygrad 02:34:30 can't use company money to buy conference tickets, networking 02:36:16 never buy via oneplus store, oneplus not open anymore, verified image programming 02:37:00 contribute to tinygrad, where to start
    3
  432. 3
  433. 3
  434. 3
  435. 3
  436. 3
  437. 3
  438. 3
  439. Anyone heard of https://hsafoundation.com/ ? 02:50:20 Bounties for tiny corp / tinygrad -> docs.google.com/spreadsheets/d/1WKHbT-7KOgjEawq5h5Ic1qUWzpfAzuD_J06N1JwOCGs/ https://youtu.be/lnVQsJJFcdg?&t=13715 Hiring entire stack for tiny corp join if you are interested | https://youtu.be/lnVQsJJFcdg?&t=14195 work major source of value in your life Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | github.com/tinygrad/tinygrad <- simple powerful deep learning framework tiny corp is accepting new interns. more info on tinygrad.org and tinygrad discord | comma.ai is accepting interns comma.ai/jobs#open-positions from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | from $999 comma.ai/shop/body the future of people Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 intro 00:00:20 no warning, linkedin ban, child prodigy 00:02:25 torchrl 00:03:20 tinybox pre-order, AI computer, lambda labs 00:05:20 lambdalabs vs tinybox 00:12:20 7302p vs 7532 epyc 00:15:00 tinybox raspberry pi for ML 00:16:10 not buying Apple Vision Pro 00:16:45 fastdd github, Meta buying H100 00:18:10 twitch removing content warning, linkedin post 00:20:15 linkedin worst dating site 00:20:35 selling rolls royce, money 00:21:25 George not a good fit for twitter, culture war 00:23:00 Peter Thiel dinner party, e/acc 00:25:55 better processor and it got slower 00:27:35 drive faster than dev 0 00:30:00 boost frequency, perplexity 00:33:50 bios, ipmi, epyc boost, not boosting 00:44:40 btop, pcie 4 vs 5 00:46:25 direct democracy 00:47:40 boost speed 00:50:10 hip graph is not fast 00:52:20 ROCm 6.0, Llama-2-70b slow, single thread 00:52:55 single thread, multithread, multiprocess tinygrad 00:53:55 ggml, tinygrad long term goal, universal 00:55:00 event, block slow 00:56:40 GPU queue sync, multiprocess 00:58:35 writing your own GPU driver, userspace 00:58:54 AMD HIP, clone of CUDA 01:00:30 finding HIP graph code 01:02:50 spinlocks, multiprocessing, GPU driver 01:03:25 how do GPUs work? 01:06:00 prebuilding the queues, hip semaphores 01:09:20 rdna 3 instruction set 01:12:40 so much complexity, micro engine scheduler 01:14:40 Alex 01:19:55 reading the code to send packets 01:22:45 hate free stream 01:23:30 amd gpu scheduling 01:32:50 perplexity valuation, how to value a company 01:34:00 HSA queue 01:35:30 perplexity fast, GPT4 slow, anthropic 01:39:00 HSA level 0 01:41:30 HSA runtime book, anna's archive 01:45:30 no copyright infringement intended 01:46:20 AQL packets 01:52:55 piano 01:53:20 AMD is for people who likes to get twice as much GPUs for their money 01:54:50 tinygrad pay per token API 01:58:30 replacing HIP support with HSA support 02:00:20 Nvdia vs AMD datacenter, customer GPU architecture 02:02:30 secret good version of openpilot joke 02:03:40 HIP does not use DMA engine 02:05:00 bit blit 02:14:20 rocm-bandwidth-test 02:17:00 hca kmt api amd 02:17:55 Alex 02:20:08 the hidden song 02:23:30 going on a journey 02:27:55 real completion events 02:31:30 hsa example of kernel dispatch 02:32:00 cool that AMD is so open 02:35:40 just using HSA 02:36:25 HSA rabbit hole, hsa foundation 02:37:50 the chapel language with 0 github stars 02:39:40 HSA Programmer's Reference Manual 02:40:20 linkedin post 02:42:40 the weather people, if you could design a country, deep state 02:45:45 conservatism, progressivism quote 02:49:00 Alex 02:50:00 thinking from first principles, experiments hard 02:50:20 has anyone heard about HSA foundation 02:52:50 scientific computing people, OpenMP, OpenACC 02:55:20 AMD extensions 02:56:15 traveling salesman, 2^n algorithm, scientific computing funding 02:59:10 leslie greengard 02:59:40 deep learning revolution 03:01:00 tinygrad experiment, complexity dysfunction of governance 03:01:38 misunderstanding of how software is developed today 03:02:00 compression is intelligence 03:02:20 complexity management instead complexity reduction 03:02:40 spacex rocket landing controls genius 03:05:10 complex systems, twitter 03:06:10 software 0 cost to replication 03:07:40 twitter acquisition best political dollar ever spend 03:09:30 making the tinybox good 03:09:46 making money off OSS 03:10:10 pre-order tinyboxes 03:11:10 etched.com, tenstorrent.com 03:13:50 tenstorrent offering a card to George 03:14:15 respect to tenstorrent, intel tier 03:15:05 extropic.ai 03:16:40 science grants, fundamental research that needs to be done 03:17:40 bullish on perplexity 03:18:40 atomicsemi.com 03:19:30 ranking startups, tenstorrent open source 03:21:00 tinygrad factorization 03:23:30 hammer.lol, berkshirehathaway.com 03:24:40 stop using javascript 03:27:20 apple.com website, feross.org 03:30:10 lana_lux 5k viewers
    3
  440. 3
  441. 3
  442. 3
  443. 3
  444. 3
  445. 3
  446. 3
  447. 3
  448. 3
  449. 3
  450. 3
  451. 3
  452. 3
  453. 3
  454. 3
  455. 3
  456. 3
  457. 3
  458. 3
  459. 3
  460. 3
  461. 3
  462. 3
  463. 3
  464. 3
  465. 3
  466. 3
  467. 3
  468. 3
  469. 3
  470. 3
  471. 3
  472. 3
  473. 3
  474. 3
  475. 3
  476. 3
  477. 3
  478. 3
  479. 3
  480. 3
  481. 3
  482. 3
  483. 3
  484. 3
  485. 3
  486. 3
  487. 3
  488. 3
  489. 3
  490. 3
  491. 3
  492. 3
  493. 3
  494. 3
  495. 02:03:50 tinygrad intro | 03:31:25 sending GPUs back to AMD | 03:31:54 it compiles but it does not do anything musical | 03:52:00 tinybox not responding | rocm.docs.amd.com/projects/HIP/en/latest/user_guide/hip_porting_guide.html#threadfence-system Threadfence_system makes all device memory writes, all writes to mapped host memory, and all writes to peer memory visible to CPU and other GPU devices. Some implementations can provide this behavior by flushing the GPU L2 cache. HIP/HIP-Clang does not provide this functionality. As a workaround, users can set the environment variable HSA_DISABLE_CACHE=1 to disable the GPU L2 cache. This will affect all accesses and for all kernels and so may have a performance impact. Bounties for tiny corp / tinygrad -> docs.google.com/spreadsheets/d/1WKHbT-7KOgjEawq5h5Ic1qUWzpfAzuD_J06N1JwOCGs/ https://youtu.be/lnVQsJJFcdg?&t=13715 Hiring entire stack for tiny corp join if you are interested | https://youtu.be/lnVQsJJFcdg?&t=14195 work major source of value in your life Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | github.com/tinygrad/tinygrad <- simple powerful deep learning framework tiny corp is accepting new interns. more info on tinygrad.org and tinygrad discord | comma.ai is accepting interns comma.ai/jobs#open-positions from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | from $999 comma.ai/shop/body the future of people Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 intro 00:02:40 HIP P2P copy slow 00:15:20 hipDeviceEnablePeerAccess fast 00:17:30 synchronization mechanism 00:21:50 dumb questions = banned 00:23:20 no idea if this is working 00:25:40 hip c++ 11 atomics 00:29:10 cuda atomic sync 00:31:20 global cache flush hip 00:34:20 glc dlc 00:42:00 no idea if access is correct 00:43:00 hip atomic and system example 00:48:10 deep dive in spinlocks 00:49:00 hip cache flush 00:52:10 global_atomic_and_b32 00:56:30 hipDeviceSetCacheConfig 00:58:10 threadfence system 01:03:00 block 01:04:40 D0, D1 01:07:00 events are slow because syscalls 01:10:40 answering smart questions 01:11:35 bad athomics understanding 01:11:45 x86 vs arm difference 01:12:20 HIP event API slow 01:12:50 tinybox vs 8xA100 01:13:40 radeon pro benefit 48GB RAM 01:14:20 H100 price, MI300 01:15:00 buy gaming PC in $1000 range 01:16:30 6x7900 XTX 01:16:45 nvidia software support 01:17:25 tinybox power efficiency 01:18:20 lamma2 70b unquantized on tinybox 01:19:00 tinybox V2 same case, GPU refresh 01:20:10 6 tinyboxes a day factory 01:20:35 nvidia blackwell release timeline 01:26:10 never shipping anything in your life 01:32:30 removing lines 01:40:30 test_multitensor 01:48:20 deeper problems, good abstractions 01:50:55 does not work and it's slow 01:55:00 Hello nobody is talking 02:01:30 event 4 02:03:15 Lana_Lux raid! 02:03:50 introduction to new people, tinygrad 02:04:15 beautiful_mnist.py 02:05:30 tinygrad operations 02:06:40 matrix multiplication in tinygrad 02:07:10 convolution 02:07:55 pytorch difference 02:08:25 dtypes code 02:08:45 the bitter lesson, search machine 02:09:30 DEBUG=2 02:10:30 convolution custom op 02:10:50 tinygrad long term vision 02:11:10 tiny corp business model 02:11:40 tinybox 02:12:00 getting AMD on MLPerf 02:12:15 synchronization between GPUs 02:12:30 international forwarding 02:13:20 tinybox HIP support, full 16 lane pcie 4 GPU multi GPU connectivity 02:14:40 ocp 3.0 slot 02:15:20 tinybox infiniband 02:16:25 direct AMD support 02:16:50 traning over internet 02:17:05 synchronous gradient descent 02:17:50 best chip nvidia H100 02:18:50 tinybox launch apps, tinygrad bounties 02:20:10 no secret software 02:20:45 tinygrad master under 5k lines of code 02:21:30 9x 140mm noctua fans 02:22:30 custom cables 02:22:50 demos on pc 02:24:30 line count 02:25:15 4090 price tracker 02:25:40 decision transformers 02:28:25 small font 02:31:15 python no C 02:36:50 zyn 02:40:30 semaphores for 6 GPUs 02:41:20 thread_sync_global 02:47:10 writing GPU program 02:51:20 DEBUG=2 synchronization points 02:55:00 tiny9 02:57:00 hip cache flush 03:00:40 hip coherent memory 03:03:30 amd_cocclr_copyBuffer 03:06:00 amdgcn sleep 03:11:40 hipEventRecord fix 03:22:55 recurring expenses and non recurring expenses 03:23:05 we do things because we can 03:26:30 hipSyncAndResolveStream 03:28:15 HSA_DISABLE_CACHE=1 03:31:25 sending GPUs back to AMD 03:31:54 it compiles but it does not do anything piano musical 03:32:20 no boost, no cache 03:33:10 whiteboard idea 03:36:00 memory-fence instructions, no HIP rebuild 03:38:50 cache vs no cache 03:42:30 whiteboard 03:44:25 hsa_disable_cache definition 03:48:30 HSA amd flush l2 cache 03:52:00 tinybox not responding 03:53:20 tinybox piano comeback music, HSA driver, fences 03:56:25 tinybox back 03:59:40 need to write GPU driver, threadfence_system
    3
  496. 3
  497. 3
  498. 3
  499. 3
  500. 3
  501. 3
  502. tiny corp is hiring! -> 3x Software Engineer, 1x Hardware Engineer - https://tinygrad.org/ | It's happening George started a project -> Getting AMD on MLPerf -> https://github.com/geohot?tab=projects | Go help George if you are capable! He will appreciate you! | tiny corp has a website tinygrad.org. Let your friends know! | tinygrad lazy branch https://github.com/geohot/tinygrad/tree/lazy | Support George by subscribing on twitch.tv/subs/georgehotz | Enjoy quickly made timestamps and suggest better ones in the comments Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 intro Charron vs. Pat Stay - Battle Rap 00:01:00 reading youtube comments 00:02:13 convolution 00:04:10 new place, openpilot model, ops, data flow 00:05:18 shapetracker 00:06:33 ops_lazy, explaining the graph, nodes 00:08:30 idea, power of tinygrad, reducing the nodes 00:11:20 SHUFFLE_MOVMENT_OPS 00:12:10 REMOVE_MOVMENT_NOPS 00:12:55 MERGE_ELEMENTWISE_INTO_CONV_OUTPUT 00:15:20 plan for today 00:16:30 python typing using class type inside class 00:17:50 real op 00:21:00 thinking, writing the right thing 00:25:25 love types, python the best language 00:26:05 LazyBuffer object has no attribute arg 00:26:30 NoneType object has no attribute name 00:28:00 long variable names, bad behavior editor 00:28:45 why is that wrong, we got that all right on the first try 00:31:15 running in 799.61 ms, what is slow 00:33:25 moving self realize out of toCPU 00:34:39 it should be a lot faster, running in 464 ms 00:35:30 optimization back on and see what breaks, bad investment gbtc 00:36:28 MERGE_MOVMENT_OPS breaking 00:43:40 lazy branch of tinygrad 00:44:20 SHUFFLE_MOVMENT_OPS slow, REMOVE_MOVMENT_NOPS wrong 00:46:20 can anyone fix this bug 00:46:40 mind thinking in currently working projects 00:47:40 optimization without changing the graph 00:52:00 if you have a binary op, can't have larger text 00:56:35 this isn't acually right 01:01:30 python double slash, respect shapetracker slower 01:02:00 fixing bugs 01:09:00 379 ms with shapetracked kernels 01:10:00 unary op shapetracked 01:17:20 tricky, the big gray combiner hard to write code, graph breaking, goal 10 ms 01:23:10 color speed explanation (red, green, gray) 01:30:50 unhashable type 01:40:20 np.int32 really slow (copied to CPU) win for tinygrad 01:41:25 autocorrect extension comment 01:42:00 writing the fix 01:51:18 107 ms big improvement 01:52:00 stupid sort by name, there should be no NumPy 01:58:40 24 ms to run the python, can't beat this, conv.cl 02:01:35 realize explanation 02:03:20 DEBUG=1 02:05:10 making this operations work 02:07:20 the problem was 02:08:00 python the best language 02:08:50 print(len(srcs) 02:10:20 writing some types, writing more code 02:25:30 checking for stupid code, ready to write ast traversal 02:28:00 short break, singing 02:29:00 ast traversal 02:36:00 it's slower and wrong 02:45:00 opencl not type checking anything 02:46:30 working with ast builder commit 02:47:00 let's move this out 02:48:40 65 ms in python brutal, lol slower commit 02:51:50 fix float speedup commit 02:55:10 needs valid commit, unfortunately it's slower 03:01:30 list comprehension, it's wrong 03:06:50 create_built_program_from_source_cached 03:07:20 cache misses 03:09:10 lru_cache 128 default value is a joke, easy fix 03:12:31 most of the time in shapetracker.py 03:20:55 things get slower 03:22:30 cloud scam 03:23:15 all that did nothing, commit not working 03:25:00 local workgroup matters a ton on qualcomm 03:27:55 57 ms commit 03:28:50 convolution that we can use 03:30:40 trying different things 03:37:55 this shouldn't be void 03:42:55 undeclared identifier conv_output 04:01:30 gray one appearing when I disable that, 04:06:40 all waiting for the kernels now 04:14:35 shapetracker needs to be improved 04:19:40 wrong but look at that speed 04:24:30 42 ms is actually fast? 04:30:50 running on batman 04:36:00 George doing an ad 04:37:10 fine we are going to do it 04:57:30 bunch of things we can still do 05:05:20 actually running on the comma three 05:13:30 it worked on the comma three 1.5 s 05:20:35 some questions, comments from non-subs 05:24:05 cheapeth, scams, community, langue de bois, recession 05:27:00 bailing people out, printing money, Kim Dotcom twitter thread US is beyond bankrupt 05:30:20 shut it down party, democracy, government solutions, infrastructure, taxes, state governments 05:36:20 when to move out of US, where to live, comments on a lot of countries 05:40:40 America export the world culture, corrections, infinite jest book, three body problem book 05:42:35 China, chinese content, thank you for watching
    3
  503. 3
  504. Watch youtube.com/watch?v=6yQEA18C-XI - George Hotz vs Eliezer Yudkowsky AI Safety Debate | Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ | Sources for this stream: - https://twitter.com/realGeorgeHotz/status/1690894647755988993 - https://ourworldindata.org/energy-production-consumption - https://wimflyc.blogspot.com/2021/01/the-henry-adams-curve-closer-look.html - https://ourworldindata.org/population-growth - https://wikispooks.com/wiki/Main_Page - https://geohot.github.io/blog/jekyll/update/2021/12/18/the-fourth-estate.html - https://washingtonpost.com/wp-srv/national/longterm/unabomber/manifesto.text.htm - https://beff.substack.com/p/notes-on-eacc-principles-and-tenets - https://youtu.be/UT4ApSUkUQM - https://slatestarcodexabridged.com/Meditations-On-Moloch - https://geohot.github.io/blog/jekyll/update/2021/07/11/individual-sovereignty.html - https://youtu.be/6VT8vsiyjTQ - https://facebook.com/watch/?v=170134181769283 - https://zdnet.com/article/physics-explains-why-there-is-no-information-on-social-media/ - https://en.wikipedia.org/wiki/Fredkin_gate - https://youtu.be/VdZDVWX6K2Q To George: Sorry George would be uploaded as one episode, but we saw your comment too late. Also, would make it much harder to do chapters if it would be as one video. Chapters: 00:00:00 intro 00:00:30 entropics, how much intelligence, compression 00:04:45 industrial, computer revolution, energy usage over time 00:06:42 eniac, intelligence on the planet is about to skyrocket 00:12:00 chess elo over time 00:13:15 how close are we to optimal usage of compute 00:15:25 GPT4 bad programmer, leetcode vs GPT4, project euler 00:23:40 what is overfitting, lossless compression cannot be overfit 00:24:50 college is a scam, stopping progress, henry adams curve 00:26:50 fiat currency, club of rome the limits to growth 00:28:15 birth rates 00:30:32 no understanding how government works 00:32:10 example of scientifically true but socially inconvenient 00:33:25 what happened to universities, Mencius Moldbug 00:34:25 wikispooks, the fourth estate blog 00:35:20 no science that explains this 00:36:20 comma.ai vs ghostautonomy 00:37:08 not understanding, not crazy, unabomber manifesto 00:40:45 is George crazy?, yudkowsky time magazine 00:42:50 Connor Leahy, Terry Davis 00:43:54 manifesto's psychological analysis, HolyC 00:44:25 intelligence explosion won't happen, society is very dishonest 00:50:15 e/acc manifesto, crooks fluctuation theorem, consciousness 00:54:00 Bankless We All Gonna Die Eliezer Yudkowsky, Robin Hanson 00:54:54 how we don't die, don't steal my atoms, buy my atoms 00:57:42 how we die, 1e10x better superintelligence 00:58:15 tinybox will keep me safe 00:58:40 expected many wireheads, psyops, skittles commercial advertising ruins everything 01:00:45 the prime intellect world 01:01:25 hard takeoff in days, society is way more inefficient than we think 01:03:35 moloch slate star codex, prepared arguments 01:05:24 does society make sense for most of you, are you outside the norm 01:06:50 individual sovereignty, Alex 01:08:08 MadDogNation gifted 100 Subs! 01:10:30 George asking Alex if he is crazy 01:12:50 Kanye West 01:13:30 google, marc andreessen, sam altman 01:14:10 shittalking stream 01:15:10 are you the problem, taylor swift 01:17:00 eliezer yudkowsky good faith intellectual, chicken farm man 01:18:22 negentropy, e/acc manifesto, psychohistory 01:20:55 you will never go to space 01:22:10 what is about chat that always derails me 01:22:40 we are not going to day 01:25:00 mind is flat by nick chater 01:27:10 1e20x GG, MadDogNation gifted another 100 Subs! 01:28:10 the hero's journey, real questions 01:30:20 entropics researcher job at tiny corp 01:30:38 how many silicon FLOPS exist in the world 01:31:50 the last question 01:32:45 GPT4 is sick 01:33:45 2.22 zettaFLOPS = 111000 people of compute 01:34:25 taking back every bad thing I said about Sam Altman 01:35:14 how many silicon FLOPS are produced each year 01:36:25 want to make a zettaflop computer 01:37:40 25000*312 teraflops to exaflops, using GPT4 wrong 01:38:20 computer civilization growth 01:39:00 compute, power grows per year 01:41:40 Nick Land 01:42:28 the mind is flat, finite 01:44:00 quantum mechanics 01:45:18 Bojack & Diane do you think I'm a good person 01:47:30 joe pera, physics explains why there is no information on social media 01:48:30 MadDogNation gifted another 100 Subs! 01:49:25 twitter can be run with few people 01:49:50 farcaster, web3, token 01:52:55 david deutsch, the beginning of infinity 01:54:50 GPT4 wow, rapid recursive self improvement 01:57:20 fredkin gate, economy time to double 02:00:30 growth of compute 02:02:40 GPT4 inference on tinybox, George pays for $20 for GPT4 02:03:35 google bard login, respect for GPT4 02:05:10 thank you MadDogNation! 02:06:00 how much compute do you need questions 02:06:35 watch the debate, watch the world
    3
  505. 3
  506. 3
  507. 3
  508. The contributor of the year: -> github.com/deanlee (Dean Lee with 650 commits) Best tuning award goes to: -> github.com/qadmus - github.com/commaai/openpilot/pull/22902 ([PlotJuggler] Tuning layout) The most helful feedback/bug reports: -> Logan LeGrand aka hubblesphere (youtube.com/LoganLeGrand ) The most comma three training data: -> Gregory (order number 16742) Shout-out to (runner-up): -> Jason Wen (Hyundai maintainer) - github.com/sunnyhaibin - youtube.com/sunnyhaibin Car port-er of the year: -> Jason Young (34 cars added to openpilot) - github.com/jyoung8607 - youtube.com/channel/UCsWcDSlyNy7n9zuDQ2AOavg The most controversial community member: -> Erich Moraga | github.com/ErichMoraga - youtube.com/ErichMoraga Shout-out's: Most popular fork (king of fork): -> Shane Smiskol - github.com/sshane/openpilot - installer.comma.ai let's you install custom forks People who got fingerprinting 2.0 working this year: - github.com/robin-reckmann - github.com/commaai/openpilot/pull/22655 (Fingerprinting v2 support for Nissan vehicles) - github.com/martinl - github.com/commaai/openpilot/pull/1878 (Subaru FPv2 support) - github.com/Jafaral (Jafar Al-Gharaibeh) - github.com/commaai/openpilot/pull/20864 (Mazda CX-9 2021 no steer lockout and more torque) - rosim io 30:46 (if someone has the github link please leave a comment) - github.com/ClockeNessMnstr (Chris Locke) - github.com/commaai/openpilot/pull/23247 (add BackCalculation to " PID: Use last actuators to limit integrator") Shout-out's to opensource repos: - github.com/facontidavide/PlotJuggler (Davide Faconti) - github.com/commaai/PlotJuggler - github.com/FreyJo (Jonathan Frey will be joining for internship) - github.com/acados/acados You can now buy comma three parts -> comma.ai/shop/parts | More jobs on -> comma.ai/jobs | Buy comma.ai/shop (Merch) | Follow the official youtube.com/commaai for more videos. Stay up to date by following twitter.com/comma_ai | Support comma.ai at comma.ai/shop.
    3
  509. 3
  510. 3
  511. 3
  512. 3
  513. 3
  514. 3
  515. 3
  516. 3
  517. 3
  518. 3
  519. 3
  520. 3
  521. 3
  522. 3
  523. 3
  524. 3
  525. 3
  526. 3
  527. 3
  528. 3
  529. 3
  530. 3
  531. 3
  532. 3
  533. 3
  534. 3
  535. 3
  536. 3
  537. 3
  538. 3
  539. 3
  540. 3
  541. 3
  542. 3
  543. 3
  544. 3
  545. 3
  546. 3
  547. 3
  548. 3
  549. 3
  550. 3
  551. 3
  552. 3
  553. 3
  554. 3
  555. 3
  556. 3
  557. 3
  558. 3
  559. 3
  560. 3
  561. 3
  562. 3
  563. 3
  564. 3
  565. 3
  566. 3
  567. 3
  568. 3
  569. 3
  570. 3
  571. 3
  572. 3
  573. 3
  574. 3
  575. 3
  576. 3
  577. 3
  578. 3
  579. 2
  580. 2
  581. 2
  582. 2
  583. 2
  584. 2
  585. 2
  586. 2
  587. 2
  588. 2
  589. 2
  590. 2
  591. 2
  592. Please go read the bitter lesson -> http://www.incompleteideas.net/IncIdeas/BitterLesson.html | https://openreview.net/pdf?id=9pe38WpsbX (MuDreamer: Learning Predictive World Models without Reconstruction) | NEW research position at tiny corp 02:03:35 02:09:11 Bounties for tiny corp / tinygrad -> docs.google.com/spreadsheets/d/1WKHbT-7KOgjEawq5h5Ic1qUWzpfAzuD_J06N1JwOCGs/ https://youtu.be/lnVQsJJFcdg?&t=13715 Hiring entire stack for tiny corp join if you are interested | https://youtu.be/lnVQsJJFcdg?&t=14195 work major source of value in your life Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | github.com/tinygrad/tinygrad <- simple powerful deep learning framework tiny corp is accepting new interns. more info on tinygrad.org and tinygrad discord | comma.ai is accepting interns comma.ai/jobs#open-positions from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | from $999 comma.ai/shop/body the future of people Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 muted intro 00:00:20 un-muted 00:00:45 the incident 00:05:10 not recording stream locally 00:05:40 twitch vs twitter revenue 00:07:00 why is RL difficult, data 00:09:25 Richard S. Sutton rl book 00:10:20 every bad thing about data 00:11:35 deepmind soccer robot 00:12:55 neurolink, changing monkeys data 00:13:30 search always works 00:14:20 things to consider when debugging rl pytorch 00:14:50 beta in PPO 00:17:10 cleanup 00:20:50 episodes 40 00:25:10 beta 00:27:00 gamma discount factor 00:28:48 openpilot does not use any RL 00:30:55 committing changes to master 00:31:25 combined experience replay (CER) 00:33:50 go play with the code 00:34:35 debugging RL 00:35:05 don't try stuff and run to see if it works 00:35:40 ML going dark, companies do not share secrets 00:36:35 misguided belief, comma traning, compute 00:37:30 your ability to execute 00:38:10 5 year old reddit post 00:39:20 private data, curating the data 00:40:15 keeping tricks secret 00:40:45 upstream code open source, MIT licence 00:41:30 aixi, bellman equation 00:42:50 bayesian optimization search 00:43:40 ml empirical science 00:44:05 data extraction pipeline, training pipeline 00:44:55 human groundtruth 00:46:20 minimal rl test environments 00:48:00 batch size 256 00:49:00 spam trying programming 00:53:25 why it is so bad 00:54:50 openai spinning up 00:56:20 A Walk in the Park: Learning to Walk in 20 Minutes 00:57:30 machine learning reddit 00:59:20 rlhf 01:05:30 latest from deepmind 01:07:55 transformer reinforcement learning x 01:09:18 Diffusion Policy: Visuomotor Policy Learning via Action Diffusion 01:10:30 comma ai simulator 01:12:10 failed to implement decision transformer 01:13:25 running the nodes 01:14:25 Decision Transformer: Reinforcement Learning via Sequence Modeling 01:15:50 imitation learning better 01:16:30 muzero, initialization 01:17:50 atari 100k benchmark, dreamer v3 01:18:30 tinybox, mastering atari games with limited data 01:19:50 $1000 bounty for solving atari 100k in tinygrad 01:20:35 CARL: Controllable Agent with Reinforcement Learning 01:22:10 dreamer v3 vs efficentzero 01:22:55 MuDreamer 01:23:10 simillar what we are doing at comma 01:23:50 MuDreamer, same problem as at comma 01:26:20 value prediction network 01:28:15 want to solve mario, researcher at tiny corp 01:28:50 wall training time, code short, tinygrad 01:30:25 tinyboxes, pricing 01:32:00 all basics should work on tinybox 01:32:30 just love this stuff, excited 01:32:40 did not delete the stream 01:33:00 through frustration we more forward 01:34:00 paying bounties 01:36:00 not doing tiny problems 01:36:38 hlb cifar 01:38:10 dreamer v3, MuDreamer 01:39:15 Alex 01:41:20 MuDreamer 01:42:05 twitch warning 01:44:00 moving off twitch, onlygeorge.com 01:46:40 instagram reels 01:48:30 twitch front page 01:49:40 Linus Tech Tips reach out, entitled 01:51:10 unroll multiple steps 01:53:00 GRU 01:54:30 i-jepa 01:58:10 curiosity-driven exploration by self-supervised prediction 02:00:00 language model beats diffusion 02:01:20 Finite Scalar Quantization: VQ-VAE Made Simple 02:03:25 MaskGIT: Masked Generative Image Transformer 02:03:35 tinygrad research dream mario 64 02:04:40 high fidelity simulation 02:05:00 copyright game problem mario 64 02:06:00 wayve simulator 02:08:20 the bitter lesson 02:09:11 research position on tiny corp
    2
  593. 2
  594. 2
  595. 2
  596. 2
  597. 2
  598. 2
  599. 2
  600. 2
  601. 2
  602. 2
  603. 2
  604. 2
  605. 2
  606. 2
  607. 2
  608. 2
  609. 2
  610. 2
  611. 2
  612. 2
  613. 2
  614. 2
  615. 2
  616. 2
  617. 2
  618. 2
  619. 2
  620. 2
  621. 2
  622. 2
  623. 2
  624. 2
  625. 2
  626. 2
  627. 2
  628. 2
  629. 2
  630. 2
  631. 2
  632. George is live on https://www.twitch.tv/georgehotz (part 2) | https://adventofcode.com/2022 | 00:48:35 !!!WARNING!! loud noise by George blowing into the mic | Go buy comma three from $1999 https://comma.ai/shop | tiny corp has a website tinygrad.org | Check out and star github.com/geohot/tinygrad | Follow George on twitter.com/realGeorgeHotz & instagram.com/georgehotz & twitch.tv/georgehotz & twitter.com/comma_ai to be up to date | Read George's geohot.github.io/blog | Support George by subscribing twitch.tv/subs/georgehotz Chapters: 00:00:00 intro 00:01:00 bad san francisco 00:01:20 advent of code, hardcore engineer 00:01:55 plans for today 00:02:30 what is scala 00:03:00 scala read file 00:03:10 openai login for chatgpt 00:04:00 how do I read a file in scala? 00:05:15 scala illegal start of toplevel definition 00:06:20 LiveOverflow raid! 00:07:05 sting to int scala 00:07:35 scala partition 00:10:00 Lana_Lux raid! 00:11:30 difference scala array, list 00:12:07 scala calling tip 00:14:55 scala sort 00:15:10 scala list slice 00:16:45 meatballs 00:17:00 aoc day 2 00:21:55 scala map 00:24:00 scala inline if 00:26:40 a lot of scala very nice 00:28:10 what to use instead of scala 00:28:25 kotlin 00:28:45 python slow 00:29:10 food missing 00:36:45 chips 00:44:20 scala string index 00:46:20 indexof scala 00:47:35 scala char 00:48:35 !!!WARNING!! loud noise by George blowing into the mic 00:48:50 free speech 00:49:18 scala char to int 00:53:55 scala for loop 00:54:40 scala zip iter data 00:55:00 how anyone lives in san francisco 00:55:10 scala slice 00:55:40 scala every third element in array 00:57:25 scala mkString 01:00:45 aoc day 4 01:02:20 no food at twitter 01:02:50 vape, chess and twitter 01:04:10 san francisco agitation 01:04:30 twitter relocate to san diego 01:04:45 austin, politics 01:05:30 unhoused persons 01:05:45 twitter files 01:06:00 twitter search ban 01:06:10 work from home 01:06:30 basic corporate behavior broken 01:06:50 what we are building, the public square 01:07:23 google search 01:07:35 this week work, twitter search 01:07:50 small mediocre features 01:08:00 impressive demo that can't be shipped for 2 years 01:08:20 can't do in 12 weeks 01:08:30 good infrastructure 01:08:45 can't stay in this city 01:08:55 twitter is nice, need better food 01:09:25 agitation 01:09:45 talent at twitter 01:10:20 twitter in san diego 01:10:30 incredible ml stuff needs a year of work 01:10:44 your codebase reflects your organization 01:11:08 refactoring or features 01:11:30 slack channels 01:11:40 talent at twitter higher 01:12:20 refactoring improves performance and velocity of new features but takes time 01:12:55 elon on average better decisions 01:13:15 twitter infra 01:13:30 advent of code this year, no green on github 01:13:45 twitter microservices 01:14:10 scala fine programming language 01:15:25 chisel3 01:15:45 need food 01:16:45 opensourcing twitter 01:18:50 scala string split 01:23:00 advent of code complaints, reading comprehension 01:24:55 twitter login pop up 01:26:10 political battle 01:27:00 mastodon, truth social 01:28:30 all cool people on twitter 01:29:00 hoon is a disaster 01:30:25 elon is not a political extremist 01:31:25 trump ban 01:32:35 line should be illegal in usa 01:34:15 what is a country 01:34:50 rules for the town square 01:35:00 moderation is not censorship 01:36:00 brandenburg test wikipedia 01:37:25 clips not allowed, hitler not good 01:38:15 rules average of eu, us 01:38:45 violence not ok 01:39:05 kanye needs to chill 01:40:10 twitter hiring 01:40:35 python 3.12 01:40:45 politics is interested in you 01:41:10 free speech reason why I came to work at twitter, comma.ai record profit 01:42:10 real change coming, comma winning 01:43:34 chatGPT, Text-Davinci-003 demo, meatballs
    2
  633. 2
  634. 2
  635. 2
  636. 2
  637. 2
  638. 2
  639. 2
  640. 2
  641. 2
  642. 2
  643. 2
  644. 2
  645. 2
  646. 2
  647. 2
  648. 2
  649. 2
  650. 2
  651. 2
  652. 2
  653. 2
  654. 2
  655. 2
  656. 2
  657. 2
  658. 2
  659. 2
  660. 2
  661. 2
  662. 2
  663. 2
  664. 2
  665. 2
  666. 2
  667. 2
  668. 2
  669. 2
  670. Chapters part 2: 04:19:30 AMD last chance, latest trunk driver, opencl context, irrecoverably crashes 04:24:30 building kernel takes a long time, plum bonito amd 04:28:25 geohot rapping, tom delonge, debunking navy videos on youtube 04:30:30 amdgpu driver walkthrough, make clean 04:36:35 is it even boot, sad stream, if you have to build a custom kernel, trying trunk, I never send stuff back 04:37:35 here is my deal with AMD, I don't want money, release all documention on this GPU and will build a driver 04:38:00 problem with legacy, supporting graphics, amd gpu psp, $1000 cards 04:43:00 naming nvidia code names, sudo make install 04:49:15 computer alive, clinfo, pyopencl for tinygrad, kernel build 04:52:30 if this does not work they are going back to amd 04:55:40 until 25 may to return them 04:57:18 crash again, failed to add hardware queue to MES, it can be the card broken 05:02:00 how long should we give AMD, by 12 may some sign that you care about your drivers 05:07:00 swaping the GPUs, no crash 05:17:35 bandwidth test 05:23:55 it's not the GPU crash reproduced on both 05:26:10 George going to New York, did we give AMD the benefit of the doubt 05:30:10 appreciate open source, vbios 7900xtx, rocm smi reset gpu 05:44:00 kernel that is not that bad, we can reset it, amd appreciate if you cared, does amd want this 05:47:00 does amd want their cards to be beloved by machine learning enthusiasts 05:54:40 AMD MES - MicroEngine Scheduler 05:57:50 virtualisation being enabled in bios issue, can we crash beyond repair 06:03:00 hate sending them back, enable bios, checking for pci errors 06:13:41 hsviewer keyboard not working 06:16:51 vast.ai, buying 7900xtx stupid 06:19:10 checking bios, bios for ants, boot delay, config.sys 06:22:15 wait for bmc disable, can't undo 10 years of crappy drivers in a stream 06:24:10 hire the tiny corp, write a clean driver just for deep learning 06:27:50 61 FLOPS promissed delivered 34 FLOPS 06:28:30 AMD to mlperf, intel arc worse then amd driver 06:30:10 we need a sign from AMD before R_DAY 06:32:20 RDNA3+ canceled 06:33:15 Jensen wins
    2
  671. 2
  672. 2
  673. 2
  674. 2
  675. 2
  676. 2
  677. 2
  678. 2
  679. Tell your friends to join comma https://comma.ai/jobs | Watch this two videos about why people work at comma https://youtu.be/qTaPD0l_8PM?t=21473 & https://youtu.be/kNWTculMsTY | Check if your car is supported -> https://comma.ai/vehicles & buy https://comma.ai/shop/products/three | Follow the official youtube.com/commaai for more livestreams. Stay up to date by following twitter.com/comma_ai | Support comma.ai at comma.ai/shop. Chapters: 00:00 hiring two web people 02:00 hiring in operations, supply chain 03:35 hiring design people 04:40 no remote work 04:58 salary range 05:20 comma body 05:35 no hiring based on programing language 06:18 patent troll 07:05 meta stock 07:33 fsd, better testing 08:03 tesla ai day 08:20 research division hiring 08:50 paid internships 09:30 comma three 09:55 visa for interns 10:13 best car for comma three 10:35 solving self driving 4 years 11:24 azure 11:50 4090s, A100 12:27 comma con 13:00 solving fsd 13:30 comma points, George's role 14:00 0.9 features 14:30 solving ai 14:55 web tech stack 15:20 building a person 16:48 vcs 17:30 hyper scale, capital intensive 18:30 agi 18:55 hardware bottleneck 19:30 end to end longitudinal 20:00 who is going to win 20:55 passing Tesla 21:29 karparthy 21:48 no upgradable CPU 22:18 zoox employees 22:55 nvidia 23:25 qualcomm 23:40 tenstorrent 24:30 making chip 25:00 openpilot vs autopilot 25:26 comma three two more years 26:05 comma 2.5 26:25 dream for comma 27:05 hiring for web 27:17 hiring for infrastructure 27:40 on device learning 27:50 EVs, bolt, ev6 28:00 solve ai
    2
  680. 2
  681. 2
  682. 2
  683. 2
  684. 2
  685. 2
  686. 2
  687. 2
  688. 2
  689. 2
  690. 2
  691. 2
  692. 2
  693. 2
  694. 2
  695. 2
  696. 2
  697. 2
  698. 2
  699. 2
  700. 2
  701. 2
  702. 2
  703. 2
  704. 2
  705. 2
  706. 2
  707. 2
  708. 2
  709. 2
  710. 2
  711. 2
  712. 2
  713. 2
  714. 2
  715. 2
  716. 2
  717. 2
  718. 2
  719. 2
  720. 2
  721. 2
  722. 2
  723. 2
  724. 2
  725. 2
  726. 2
  727. 2
  728. 2
  729. 2
  730. 2
  731. 2
  732. 2
  733. 2
  734. 2
  735. 2
  736. 2
  737. 2
  738. 2
  739. 2
  740. 2
  741. 2
  742. 2
  743. 2
  744. 2
  745. 2
  746. 2
  747. 2
  748. 2
  749. 2
  750. 2
  751. 2
  752. 2
  753. 2
  754. 2
  755. 2
  756. 2
  757. 2
  758. 2
  759. From $1499 buy comma.ai/shop/comma-three | 01:56:30 bounty for $50 via paypal | Read americanaffairsjournal.org/2021/08/the-value-of-nothing-capital-versus-growth/ | tiny corp has a website tinygrad.org | Check out and star github.com/geohot/tinygrad | Support George by subscribing twitch.tv/subs/georgehotz | Follow George on instagram.com/georgehotz & twitch.tv/georgehotz & twitter.com/comma_ai to be up to date | Read George's geohot.github.io/blog/ | Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 intro 00:01:52 plan for today 00:03:10 preparing whiteboard 00:08:35 using triton 00:09:15 comma contracting tinycorp 00:11:30 qualcomm, tinycorp vs pytorch 2.0 00:12:15 ai chips, triton 00:12:52 jim keller, tenstorrent, shapeways 00:14:15 no ugly kernels, vs nvidia 00:15:05 scaling with gpu's, writing the right code 00:16:40 tinygrad kernel 00:18:35 tensor.py 00:21:00 onnx.py 00:21:20 tinygrad.org 00:22:40 easy to add backend 00:24:35 net.svg 00:27:57 alex 00:28:30 tinycorp needs to make money 00:29:35 tinygrad stable diffusion output 00:30:20 mlops.py 00:30:50 lazy.py 00:31:30 tinygrad vs pytorch 2.0 00:32:50 pytorch 2.0 00:33:10 250 vs 15 primitive operators 00:34:15 can tiny grad autdiff through exp(M) 00:35:50 frame evaluation api 00:36:34 pytorch inductor 00:37:00 zygote ad framework 00:37:45 writing a backend for pytorch 00:38:30 thinking what to do 00:39:50 torch inductor 00:40:35 not logging in to github 00:41:35 hope for tinygrad 00:45:10 trade offs 00:46:00 compile.py 00:47:35 mps support 00:50:25 NVIDIA_TF32_OVERRIDE 00:51:15 gpu vs cuda 00:51:54 3080 ti, reversing the ANE 00:52:13 library should be fast 00:54:40 ssh to ubuntu computer 00:55:08 poll slow kernel debug, python 3.12 upgrade, triton port 00:58:10 slow kernel debug 01:18:00 invalid command queue 01:29:00 copilot not good, thinking what to write 01:35:30 correct and faster 01:39:30 flip extra dimension indexing for speed 01:40:30 reduce error 01:49:15 explaining what group_for_reduce 01:51:10 hard to write 01:53:00 search space, should not write by hand 01:55:40 hard to implement search 01:56:30 bounty for $50 via paypal 01:57:40 kernel search is the way 01:58:40 explaining the search problem 02:07:35 search space 02:12:10 way slower 02:13:40 first restriction 02:18:00 kernel_search.py 02:24:05 ordering food 02:28:00 how will tinycorp make money 02:31:30 minimum contract price for tinycorp 02:32:05 aritter2 good search suggestions 02:34:10 first little attempt at search 02:42:00 optimise every op in isolation, A* 02:43:10 accessible to people 02:48:10 encouraging still take a week, upcast, shape must be ints and can't contain 0 02:51:35 complex google optimizer 02:54:40 why did I not do this sooner 02:58:25 a framework 03:00:35 cleanups 03:16:30 upcast, winning interventions 03:17:40 super promising, thank you to aritter2 03:18:55 programming competitions, aoc, project euler, ctf 03:21:35 computer security, info sec, bug bounty 03:23:04 not everyone is going to like you 03:24:35 nejintejag question shapes network effects function 03:26:20 reading youtube comments 03:27:00 compute for comma four, five, jim keller 03:28:20 can't buy qualcomm, nvidia chips, nvidia orin 03:33:30 amd accelerators, amd mi25, apple M2 chip 03:36:40 reading youtube comments, hard problem of consciousness 03:41:05 excited about search 03:41:55 moving to ukraine, joscha bach, emad mostaque 03:43:45 how I seduce my crush 03:44:10 no drugs, nostr 03:45:25 jesus, christian, catholic 03:46:45 jewish people smart, kanye west 03:47:50 bad culture, bad reach out to work at comma 03:48:50 openai, ai safety, working at comma 03:50:40 woke college culture, non-issues, marriage 03:54:10 people trying to change your culture, johnny hobo new mexico song 03:55:40 not eating meat, plants, math, the freshman requirements, main vs master 04:00:30 stupid topics, hitler lost the war, russia vs china, 10 lane highway 04:03:55 sovereign countries, unified world order, pollution in china 04:05:00 american culture, shapeways stock, spacs, elon WEF, tiktok, google, fb 04:07:20 offer to private equity firm for hostile takeover of shapeways 04:10:15 metaverse is lame, no comma 4, john carmack, AGI, among us vr, stream deck 04:13:25 comma body arm, the value of nothing capital versus growth, china, resurgence in ai, malaysia 04:18:25 leave comments in the video and we will answer them on next stream
    2
  760. 2
  761. 2
  762. 2
  763. 2
  764. 2
  765. 2
  766. 2
  767. 2
  768. 2
  769. 2
  770. 2
  771. 2
  772. 2
  773. 2
  774. 2
  775. 2
  776. 2
  777. 2
  778. 2
  779. tiny corp is hiring! -> 3x Software Engineer, 1x Hardware Engineer - https://tinygrad.org/ | It's happening George started a project -> Getting AMD on MLPerf -> https://github.com/geohot?tab=projects | Go help George if you are capable! He will appreciate you! | From $1499 buy comma.ai/shop/comma-three | LLaMA https://arxiv.org/pdf/2302.13971.pdf | github.com/geohot/tinygrad/blob/llama/examples/llama.py | github.com/geohot/tinygrad/tree/llama | tiny corp has a website tinygrad.org | Check out and star github.com/geohot/tinygrad | Support George by subscribing twitch.tv/subs/georgehotz | Follow George on instagram.com/georgehotz & twitch.tv/georgehotz & twitter.com/comma_ai to be up to date | Read George's geohot.github.io/blog/ | Programming archive🡿 youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 intro 00:01:30 guitar tuning and playing 00:04:30 giving advice to elon, woke 00:07:25 llama, trip to india, bengaluru, bike to mumbai 00:08:40 when software was cool 00:11:20 llama, privilege 00:13:45 discord ban, pressure, power 00:16:05 prime, blue crown, money 00:18:00 tiny corp, making value, food 00:22:25 capitalism 00:23:50 facebook LLaMA, apply for access, torrent 00:25:10 ai safety, enjoy the decline 00:28:00 deepmind, google, rants 00:28:40 thinking for yourself, jesus, george is christian, god 00:30:30 a progression to christian 00:31:50 big text, copilot 00:35:10 running on gpu m1 00:39:30 using tinygrad tiny ram usage, float16 00:46:00 METAL=1 00:47:25 numpy size of datatype 00:49:12 interesting, np.frombuffer 00:52:10 problem raw buffers to have a datatype? 00:52:55 python read into buffer, not making copies = fast 00:59:40 loading from disk, making it lazy 01:00:50 tinygrad is the future, replacing programing with dsp paradigm 01:02:17 tinygrad has grown 01:03:50 metal buffer, float16 hard 01:05:20 interview question how big is a float 01:06:40 making metal buffer, supporting dtypes 01:09:40 tinygrad recognize llama 01:10:25 facebook llama arxiv, unbiased model 01:13:25 class Attention, FeedForward 01:21:40 washed heads 01:24:00 class RMSNorm 01:30:25 testing inference 01:35:30 error and slow loading because torch 01:39:00 wasting 0.6 seconds 01:40:45 how transformers work, Linear 01:47:20 torch not defined, if you are slow you get undefined 01:48:00 fstrings, shapes matching 01:48:45 writing calls 01:55:00 class Attention 02:02:00 missing position 02:05:10 not matching 02:13:00 why did that happen? 02:18:10 memory should not be 0 02:19:50 just creating the weights is breaking it 02:24:30 why so much memory? 02:25:15 getting dtype in inference editor 02:26:15 WEIGHTS=1 02:29:30 number of ops and memory use 02:31:10 9 seconds = 1 character, tensor cores 02:32:40 fastest metal_matmul.py on M1 = fastest llama on M1 02:36:10 stable diffusion interesting ops 02:37:10 dynamically loading the weights from zip file 02:37:30 biggest llama on M1 02:39:50 full tiny llama grid 02:42:20 looking at the paper, thank you to facebook 02:43:55 dollars are arbitrary 02:44:50 licence for ai safety, mmap 02:47:40 tiny corp no money 02:51:20 making the compiler running less 02:52:50 bought land, rolls-royce and meta stock 02:54:10 sam bankman-fried 02:56:30 little people 02:58:00 andrew tate making money, kiwi farms, coffeezilla 03:00:20 entire economy is a fraudulent scam 03:01:50 AGI 03:06:30 how would George do in prison 03:10:00 un focused today 03:18:00 floats should not be floats 03:22:30 generated c files 03:23:10 openpilot compile 03:32:00 missing 1 required positional argument x 03:34:10 faster 03:36:00 terrible support for halves 03:38:00 numpy bad chat suggestion 03:38:45 bugs in the training 03:42:30 ENABLE_METHOD_CACHE=1 should be default 03:43:30 when you make something 0 page and copy on write, take operating system class 03:45:40 not ready to generate a word 03:46:00 using tinyjit for training 03:47:15 rope is hard 03:49:35 importing SentencePiece 03:51:35 cheapeth, how does anyone take anything seriously 03:55:00 list must contain integers 04:02:10 offensive code because it uses complex numbers 04:08:00 can't trust anybody 04:11:30 macbook pro m1 max ssd read speed 04:14:50 dealing with company problems 04:15:20 not starting a company 04:16:25 complex multiplication, the moment 04:17:20 what we missed off stream 04:18:35 never start a company, llama is working 04:24:30 SentencePieceProcessor 04:32:10 Kristen Rengren 04:34:50 temperature sampling numpy 04:41:25 TechQuest Laser Tag Gun 04:44:20 singularity
    2
  780. 2
  781. 2
  782. 2
  783. 2
  784. 2
  785. 2
  786. 2
  787. 2
  788. 2
  789. 2
  790. 2
  791. 2
  792. 2
  793. 2
  794. 2
  795. 2
  796. 2
  797. 2
  798. 2
  799. 2
  800. 2
  801. 2
  802. 2
  803. 2
  804. 2
  805. 2
  806. 2
  807. 2
  808. 2
  809. 2
  810. 2
  811. 2
  812. 2
  813. 2
  814. 2
  815. 2
  816. 2
  817. 2
  818. 2
  819. 2
  820. 2
  821. 2
  822. 2
  823. 2
  824. 2
  825. 2
  826. 2
  827. 2
  828. 2
  829. 2
  830. 2
  831. 2
  832. 2
  833. 2
  834. 2
  835. 2
  836. 2
  837. 2
  838. Buy tickets https://www.eventbrite.com/e/comma-con-tickets-153553216805 Schedule (PDT) 10:00AM-10:05: Opening remarks Alex (COO) and Chris (Head of Production & Communications) welcome you to COMMA_CON Talks include 10 minutes of Q&A at the end. There is such a thing as a bad question. 10:05AM-10:50: Willem Melching -- Head of openpilot openpilot has evolved from a hobby project to legit some of the best software in the automotive world. In the process we’ve written a whole set of robotics software (messaging, kalman filters, localizer) that outperform everything available today. Come to this talk to learn how openpilot works. It’s quite sophisticated, yet still approachable. Learn the Android of self driving before everyone else. 11:00AM-11:50: Harald Schäfer -- CTO Get the data, train the models. Easy, right? Not even close. From the world’s most sophisticated automatic ground truthing stack, to an agent trained in our “Small Offset Simulator”, you’ll learn where the magic comes from, and how the magic will continue all the way through to level 5. Fun and easy! 12:00PM-1:00: Lunch: Burgers 1:00PM-1:50: George Hotz -- President comma is a profitable company that provides value to 1000s of users. How do you keep focused and build something like that in a world with clown valuations, fake demos, and phony products? Our mission is “solve self driving cars while delivering shippable intermediaries,” we aren’t stopping at highway autopilot. Come to this talk to learn about what’s next, and how the clowning will continue until the end. You won’t want to miss this. 2:00PM-2:50: Robbe Derks + Hardware Team Building a car is hard. Working with car companies is impossible. We chose the third option. Working with existing cars. comma did this so casually you’d think it was easy. It wasn’t. We interface with over 100 different cars. Want to understand how we’ve managed to do that without ballooning complexity? Come to this talk. 3:00PM-3:25: Adeeb Shihadeh-- openpilot Engineer Come to this talk to see a secret new feature be revealed! 3:30PM-3:55: Greg Hogan -- Head of Infrastructure We have a 50 kW on premise datacenter with 2,000 cores, 80 GPUs, and 3 PB of storage. Learn how it came together in this talk. 4:00PM-4:25: Weixing Zhang-- Research Engineer Camera based driver monitoring (DM) is a key part of the magic of openpilot. It’s so good that Tesla copied it 2 years later! Come learn how it was built into the best DM on the market by one engineer part time. Understand the power of the comma research stack, crowdsourcing, and big data. 4:30PM-5:20: Group Chat / QA George will get on stage and lead a group conversation about comma and openpilot’s journey from hobby quality crap to the top rated ADAS system. 5:30PM-6:30: Happy Hour + Community Lightning talks (apply to alex@comma.ai) 7:00PM-8:00: Dinner: Flatbreads 7:00PM-8:00: VIP Dinner Sit-down dinner with the comma team (3rd floor) 8:00PM-10:00: comma Jeopardy Three teams of four people will compete in two rounds of comma themed questions. Test your knowledge of comma, cars, and computers while drinking free pitchers of beer on stage. VIP tickets get first dibs on teams, the rest of the players will be selected randomly. Score the most points and win a “prize!”
    2
  839. 2
  840. 2
  841. 2
  842. 2
  843. 2
  844. 2
  845. 2
  846. 2
  847. 2
  848. 2
  849. 2
  850. 2
  851. 2
  852. 2
  853. 2
  854. 2
  855. 2
  856. 2
  857. 2
  858. 2
  859. 2
  860. 2
  861. 2
  862. 2
  863. 2
  864. 2
  865. 2
  866. 2
  867. 2
  868. 2
  869. 2
  870. 2
  871. 2
  872. 2
  873. 2
  874. 2
  875. 2
  876. 2
  877. 2
  878. 2
  879. 2
  880. 2
  881. 2
  882. 2
  883. 2
  884. 2
  885. 2
  886. 2
  887. 2
  888. 2
  889. 2
  890. 2
  891. 2
  892. 2
  893. 2
  894. 2
  895. 2
  896. 2
  897. 2
  898. 2
  899. 2
  900. 2
  901. 2
  902. 2
  903. 2
  904. 2
  905. 2
  906. 2
  907. 2
  908. 2
  909. 2
  910. 2
  911. 2
  912. 2
  913. 2
  914. 2
  915. 2
  916. 2
  917. 2
  918. 2
  919. 2
  920. 2
  921. 2
  922. 2
  923. 2
  924. 2
  925. 2
  926. 2
  927. 2
  928. 2
  929. 2
  930. 2
  931. 2
  932. 2
  933. 2
  934. 2
  935. 2
  936. 2
  937. 2
  938. 2
  939. 2
  940. 2
  941. 2
  942. 2
  943. 2
  944. 2
  945. 2
  946. 2
  947. 2
  948. 2
  949. 2
  950. 2
  951. ecromata.com | Bounties for tiny corp / tinygrad -> docs.google.com/spreadsheets/d/1WKHbT-7KOgjEawq5h5Ic1qUWzpfAzuD_J06N1JwOCGs/ youtu.be/lnVQsJJFcdg?&t=13715 Hiring entire stack for tiny corp join if you are interested | youtu.be/lnVQsJJFcdg?&t=14195 work major source of value in your life Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | github.com/tinygrad/tinygrad <- simple powerful deep learning framework tiny corp is accepting new interns. more info on tinygrad.org and tinygrad discord | comma.ai is accepting interns comma.ai/jobs#open-positions from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | from $999 comma.ai/shop/body the future of people Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 intro 00:00:10 rack gone, donuts, full time android 00:02:50 a brief history of intelligence 00:04:25 skibidi toilet 00:06:20 hipstreamwritevalue32, hipstreamwaitvalue32 00:10:40 linearizer 00:15:30 optimizations for shapetrackers and group for reduce 00:16:16 graph 00:17:10 ADD and LOAD, topological sort 00:20:30 Alex 00:23:15 linearizer great code 00:26:00 most garbage code in tinygrad 00:27:30 difference, liniar list of code 00:29:20 Alex 00:31:00 the idea of new linearizer 00:31:45 kernels that can store more than one thing 00:32:50 food 00:34:20 buffer loads, shape, buffer iteration 00:36:40 how do compilers work? 00:38:00 chris lattner 00:39:00 color dimensions 00:40:00 tvm, optimism work, thinking about what you want to write 00:41:40 installing tvm 00:46:00 jonathan blow precedence compiler 00:48:20 binding axis to gpu variables 00:50:30 cse_var_2, common subexpression elimination 00:51:10 reading documentation from others 00:52:30 not thinking 00:53:55 tvm_gemm.py 00:55:20 what is tvm 00:55:40 schedule API in tinygrad 00:56:10 linearizer 00:57:10 jira in companies, project management 00:59:40 jira board, complexity, work signaling 01:00:40 tiny corp board 01:01:35 better expression of ideas 01:02:00 Proofs from THE BOOK 01:02:25 scrum, sprints at comma, agile 01:04:00 above the API line 01:04:40 comma is going to beat tesla in FSD, adding value tomorrow, V12 01:07:00 if anyone want's to write tinygrad docs, tutorials like tvm 01:07:40 blame, working with elon, elon's companies structure 01:09:00 the concept of non technical project manager 01:10:10 better programmer than people you manage 01:11:20 senior vs junior engineers 01:14:49 type inference 01:15:45 junior engineers, learn lessons the hard way 01:17:10 not getting more intelligent 01:18:50 competition programming, high schoolers 01:20:20 spam programming vs slow high quality programmer 01:21:55 codegen and scheduler 01:23:20 halide tvm 01:25:10 complex, chris lattner about tinygrad, compilers 01:28:20 model optimization tvm 01:30:00 building tvm, tvm output verbose code 01:33:00 tvm same ideas as tinygrad, tensor accelerator 01:35:00 group_for_reduce, kernels 01:37:00 searching for the right name 01:40:00 graph rewriting 01:40:50 break 01:48:56 hhkb professional hybrid type-s 01:50:15 blueprint name 01:51:10 ajusting new keyboard 01:57:50 vscode spaces 02:01:50 Alex 02:04:30 typing on new keyboard 02:06:30 new linearizer concepts 02:16:50 group 02:20:30 comments overrated, wrong documentation 02:26:00 ecromata.com 02:27:00 hammer = perplexity 02:28:00 land, managing people, plans 02:31:40 starting your own company, selling land 02:33:30 no capital constraints, how do you deploy $500M 02:34:20 delusional founders, what limits the growth, 10 new comma people, integration to the company 02:35:00 working together superlinear effects, scale book 02:36:00 practical factorization of the problem, limiting factor of growth 02:36:15 complex problem, breaking into appropriate parts 02:37:50 Tesla, Elon average decision better 02:39:20 management coefficient, people 02:41:10 cult followers, dedicate your life, hitler, burning man 02:43:20 social media, dating apps, quality 02:44:10 low and high buyin 02:45:30 hippie cults 60s failing, machines, energy 02:46:30 robots, people who want to take way more than taking back 02:48:40 bounty systems, burning man works, people dynamics, social dynamics 02:51:00 what are people, how do they work together, power in system, market dynamics 02:55:30 inadequate equilibria 02:56:00 the bible stories, stoicism, modern scientific people dynamics, racial differences in intelligence 02:57:40 what can you change, what you can't change, how change happens 02:59:20 clones scaling 03:02:30 power on the land, solar panels 03:08:35 projected revenue numbers, fake wineries 03:09:50 horse boarding, we live in capitalism = good 03:11:30 aravind srinivas perplexity, swimming upstream, tinyboxes on land 03:15:10 degree value, poker 03:18:15 land location, road to land 03:21:40 market already priced it in, hard to beat the market 03:22:10 linearizer 2 week project, 5 times more inteligence vs 10 clones, piano 03:25:00 housing expensive, how to make money, fraud, ben shapiro rap
    2
  952. 2
  953. 2
  954. 2
  955. 2
  956. 2
  957. 2
  958. 2
  959. 2
  960. 2
  961. 2
  962. 2
  963. 2
  964. 2
  965. 2
  966. 2
  967. 2
  968. 2
  969. 2
  970. 2
  971. 2
  972. 2
  973. 2
  974. 2
  975. 2
  976. 2
  977. 2
  978. 1
  979. 1
  980. 1
  981. 1
  982. 1
  983. 1
  984. 1
  985. 1
  986. 1
  987. github.com/geohot/dumbrl | stable-baselines3.readthedocs.io/en/master/ | youtu.be/FVGQJM2MVNY (Deadliest Journeys - Congo: The Last Train in Katanga) 01:01:11 carefully building infrastructure, CI testing | andyljones.com/posts/rl-debugging.html | spinningup.openai.com/en/latest/ | https://arxiv.org/pdf/1912.02875.pdf (Reinforcement Learning Upside Down) Bounties for tiny corp / tinygrad -> docs.google.com/spreadsheets/d/1WKHbT-7KOgjEawq5h5Ic1qUWzpfAzuD_J06N1JwOCGs/ https://youtu.be/lnVQsJJFcdg?&t=13715 Hiring entire stack for tiny corp join if you are interested | https://youtu.be/lnVQsJJFcdg?&t=14195 work major source of value in your life Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | github.com/tinygrad/tinygrad <- simple powerful deep learning framework tiny corp is accepting new interns. more info on tinygrad.org and tinygrad discord | comma.ai is accepting interns comma.ai/jobs#open-positions from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | from $999 comma.ai/shop/body the future of people Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 intro 00:01:40 stream disclaimer, twitch ban 00:02:45 only 50% of subscription money 00:03:05 kick.com streaming 00:04:45 kick reach out to George, twitch issues 00:06:20 drugs banner, legal in california 00:08:05 50% money to twitch too much, twitch remove the banner 00:09:20 hyubsama food stream, twitch banned users 00:12:00 streaming on X, negotiating power 00:14:40 stream statistics, streaming schedule 00:16:10 applying for twitch partner 00:17:30 twitch revenue 00:18:30 perplexity best way to get banned on twitch 00:22:50 andrew tate impression 00:23:50 stable baselines 3 00:29:40 np.random.randint 00:32:44 NoneType object does not support item assignment 00:33:00 perplexity 00:35:40 render mode defined human 00:37:20 good play, size=10 00:40:30 stable baselines 3 just works 00:50:00 passed a tuple, array element with a sequence 00:51:15 learning 00:52:50 decision transformer stable baselines 3 00:55:20 github.com/geohot/dumbrl 00:56:30 cartpole, stable baselines decision transformer 00:59:30 Jax, wrapper for vectorized environments 01:00:30 deadliest journeys congo, ancestor pothole 01:01:11 building infrastructure, fixing the road 01:01:20 bugs, carefully building infrastructure, CI testing 01:04:00 README 01:05:40 deleting a lot of tinygrad, focusing on what needs to work well 01:09:55 decision transformer repo 01:13:10 beautiful_cartpole.py 01:20:07 andy jones debugging rl 01:24:00 if you are following along 01:29:00 the problem are bugs 01:32:00 asking perplexity, openai spining up and deep rl 01:36:25 log_softmax 01:39:00 broadcasting bug, 2, 3, 5 01:47:20 no detach(), ppo, exp 02:02:00 why is my ppo not working 02:07:40 fast cartpole 02:11:50 banned user 02:15:50 asking it to learn 02:17:15 hyper parameter land 02:25:45 lucky 02:27:30 !!!LOUD WARNING!!! why it's not solving 02:33:20 3 layer network 02:41:30 value function 02:42:50 writing pytorch 02:48:10 if it works in pytorch shutting down tiny corp 02:52:50 pytorch numeric stability 02:55:10 frustrating, having faith in tiny grad 02:56:00 very easy to make progress in tiny grad 02:57:18 tiny grad more numerically stable 03:04:00 the most dead simple thing 03:13:30 size 2, 3 solving 03:14:35 going even simpler 03:19:00 batch size = 4 03:22:40 reward broken 03:27:10 it becomes like an identity matrix over time 03:28:40 this is fire, the gradient, single weight matrix 03:33:00 so beautiful, love watching deep learning happen 03:44:10 learning rate too high 03:51:00 that one does not learn 03:52:40 dying relu, 0xnan getting VIP 04:00:40 advantage 04:06:40 Alex on the phone 04:08:45 no clips, taking out of context 04:11:00 value funtion all noise 04:18:10 graph go up 04:24:00 messing with hyperparameters randomly 04:26:10 slow graph drawing 04:28:20 sampling bias 04:29:10 lower discount factor, larger replay buffer 04:32:45 no major bugs, ppo major bug 04:36:20 entropy loss 04:38:40 counter intuitive in deep learning, bigger learn better 04:40:40 overheads 04:41:40 one good landing 04:42:55 50, 51 04:44:10 Alex home 04:46:30 send this video to a doomer 04:48:00 good enough landing 04:50:40 expectations too high 04:51:35 twitch won't contact George 04:53:30 hope, upside down rl, juergen schmidhuber 04:54:10 good reliable solution to everything 04:54:40 Alex, no checkpoints 04:55:10 last landing, end of the episode 04:55:30 thank you for watching
    1
  988. 1
  989. 1
  990. 1
  991. 1
  992. 1
  993. 1
  994. 1
  995. 1
  996. 1
  997. 1
  998. 1
  999. 1
  1000. 1
  1001. 1
  1002. 1
  1003. 1
  1004. 1
  1005. 1
  1006. 1
  1007. from $1250 buy -> comma 3X comma.ai/shop/comma-3x | If you have skill apply on -> comma.ai/jobs | Join the community on github.com/commaai/openpilot | Check if your car is supported -> comma.ai/vehicles | How to setup your car comma.ai/setup | follow the official youtube.com/commaai for more livestreams | Stay up to date by following twitter.com/comma_ai | Support comma.ai at comma.ai/shop. Some parts of the video muted because copyright music - original video is blocked because of copyright youtube.com/watch?v=Q319h43SHnc Chapters: 00:00:00 intro 00:01:10 Alex welcome to comma_con 00:02:12 Harald Schäfer, CTO: A Drive to Taco Bell with end-to-end Machine Learning 00:34:50 Harald Schäfer questions 00:41:20 Alex introduces Adeeb Shihadeh 00:42:56 Adeeb Shihadeh, CPO: Shipping openpilot 01:02:55 Adeeb Shihadeh questions 01:24:45 Alex introduces Mitchell Goff 01:25:32 Mitchell Goff, ML Engineer: Navigate on openpilot 01:47:55 Mitchell Goff questions 01:57:42 Alex introduces Weixing Zhang 02:00:04 Weixing Zhang, Research: Human-Machine Cooperation in openpilot - a machine learning approach 02:19:30 Weixing Zhang questions 02:30:10 food break 03:42:28 George Hotz, Three Stories 04:27:00 George Hotz questions 04:39:35 Alex introduces Robbe Derks 04:40:50 Robbe Derks, Hardware Engineer: How it's made 05:15:58 Robbe Derks questions 05:24:32 Alex introduces Vivek Aithal 05:25:25 Vivek Aithal, research: Can you tune my car? 05:47:10 Alex introduces Greg Hogan 05:53:03 Greg Hogan, Head of Infrastructure: comma supercomputer 06:16:52 Greg Hogan questions 06:30:12 Alex introduces Yassine Yousfi 06:31:01 Yassine Yousfi, Research: Learning a Driving Simulator 06:54:50 Yassine Yousfi questions 07:13:50 Alex introduces Jason Young 07:17:10 Jason Young, Community contributor: How to port a car 07:45:50 Alex finishing the live stream 07:46:35 COMMA_CON commies
    1
  1008. 1
  1009. 1
  1010. 1
  1011. 1
  1012. 1
  1013. 1
  1014. 1
  1015. 1
  1016. 1
  1017. 1
  1018. 1
  1019. 1
  1020. 1
  1021. 1
  1022. 1
  1023. 1
  1024. 1
  1025. 1
  1026. 1
  1027. 1
  1028. 1
  1029. 1
  1030. 1
  1031. 1
  1032. 1
  1033. 1
  1034. 1
  1035. 1
  1036. 1
  1037. 1
  1038. 1
  1039. 1
  1040. 1
  1041. 1
  1042. 1
  1043. 1
  1044. 1
  1045. 1
  1046. 1
  1047. 1
  1048. 1
  1049. 1
  1050. 1
  1051. 1
  1052. 1
  1053. 1
  1054. 1
  1055. 1
  1056. 1
  1057. 1
  1058. 1
  1059. 1
  1060. 1
  1061. 1
  1062. 1
  1063. 1
  1064. 1
  1065. 1
  1066. 1
  1067. 1
  1068. 1
  1069. 1
  1070. 1
  1071. 1
  1072. 1
  1073. 1
  1074. 1
  1075. 1
  1076. 1
  1077. 1
  1078. 1
  1079. 1
  1080. 1
  1081. 1
  1082. 1
  1083. 1
  1084. 1
  1085. 1
  1086. 1
  1087. 1
  1088. 1
  1089. 1
  1090. 1
  1091. 1
  1092. 1
  1093. 1
  1094. 1
  1095. 1
  1096. 1
  1097. 1
  1098. 1
  1099. 1
  1100. 1
  1101. 1
  1102. 1
  1103. 1
  1104. 1
  1105. 1
  1106. 1
  1107. 1
  1108. 1
  1109. 1
  1110. 1
  1111. 1
  1112. 1
  1113. 1
  1114. 1
  1115. 1
  1116. 1
  1117. 1
  1118. 1
  1119. 1
  1120. 1
  1121. 1
  1122. 1
  1123. 1
  1124. 1
  1125. 1
  1126. 1
  1127. 1
  1128. 1
  1129. 1
  1130. 1
  1131. 1
  1132. 1
  1133. 1
  1134. 1
  1135. 1
  1136. 1
  1137. 1
  1138. 1
  1139. 1
  1140. 1
  1141. 1
  1142. 1
  1143. 1
  1144. 1
  1145. 1
  1146. 1
  1147. 1
  1148. 1
  1149. 1
  1150. 1
  1151. 1
  1152. 1
  1153. 1
  1154. 1
  1155. 1
  1156. 1
  1157. 1
  1158. 1
  1159. 1
  1160. 1
  1161. 1
  1162. 1
  1163. 1
  1164. 1
  1165. 1
  1166. 1
  1167. 1
  1168. 1
  1169. 1
  1170. 1
  1171. 1
  1172. 1
  1173. 1
  1174. 1
  1175. 1
  1176. 1
  1177. 1
  1178. 1
  1179. 1
  1180. 1
  1181. 1
  1182. 1
  1183. 1
  1184. 1
  1185. 1
  1186. 1
  1187. 1
  1188. 1
  1189. 1
  1190. 1
  1191. 1
  1192. 1
  1193. 1
  1194. 1
  1195. 1
  1196. 1
  1197. 1
  1198. 1
  1199. 1
  1200. 1
  1201. 1
  1202. 1
  1203. 1
  1204. 1
  1205. 1
  1206. 1
  1207. 1
  1208. 1
  1209. 1
  1210. 1
  1211. 1
  1212. 1
  1213. 1
  1214. 1
  1215. 1
  1216. 1
  1217. 1
  1218. 1
  1219. 1
  1220. 1
  1221. 1
  1222. 1
  1223. 1
  1224. 1
  1225. 1
  1226. 1
  1227. 1
  1228. 1
  1229. 1
  1230. 1
  1231. 1
  1232. 1
  1233. 1
  1234. 1
  1235. 1
  1236. 1
  1237. 1
  1238. 1
  1239. 1
  1240. We are not George but here is some information we know. One of comma.ai developers robbederks is working with BogGyver on Tesla AP2 port. Draft is posted on comma github just search tesla in their repository. - https://github.com/commaai/openpilot/pull/20438 - https://github.com/commaai/openpilot/pull/20074 This should be your preferred solution as if this get's up streamed to openpilot by comma.ai you will also get future updates from openpilot. All older solutions for Tesla with openpilot will be outdated and they are using older comma.ai hardware and community modified code that requires modifications on each new openpilot version. You will probably need https://comma.ai/shop/products/comma-car-harness (they are working on Tesla harness) and https://comma.ai/shop/products/comma-two-devkit Make sure to do your own research as there is a lot of differences about how older Tesla vehicles operate. You might need to retrofit some hardware to get good experience with openpilot. Some links to get you started: - https://tinkla.us/ (some information here might be outdated) - https://github.com/commaai/openpilot/search?q=tesla&type=issues (just search github) - https://www.youtube.com/results?search_query=openpilot+tesla (there are some videos showing how to install additional hardware for openpilot on tesla) - https://discord.comma.ai/ (join the tesla channel and use search to find the information you are looking for). The community is very knowledgeable but they encourage high quality discussions so make sure you do your own research before asking questions.
    1
  1241. 1
  1242. 1
  1243. 1
  1244. 1
  1245. 1
  1246. 1
  1247. 1
  1248. 1
  1249. 1
  1250. 1
  1251. 1
  1252. 1
  1253. 1
  1254. 1
  1255. 1
  1256. 1
  1257. 1
  1258. 1
  1259. 1
  1260. 1
  1261. 1
  1262. 1
  1263. 1
  1264. 1
  1265. 1
  1266. 1
  1267. 1
  1268. 1
  1269. 1
  1270. 1
  1271. 1
  1272. 1
  1273. 1
  1274. 1
  1275. 1
  1276. 1
  1277. 1
  1278. 1
  1279. 1
  1280. 1
  1281. 1
  1282. 1
  1283. 1
  1284. 1
  1285. 1
  1286. 1
  1287. 1
  1288. 1
  1289. 1
  1290. 1
  1291. 1
  1292. 1
  1293. 1
  1294. 1
  1295. 1
  1296. 1
  1297. 1
  1298. 1
  1299. 1
  1300. 1
  1301. 1
  1302. 1
  1303. 1
  1304. 1
  1305. 1
  1306. 1
  1307. 1
  1308. 1
  1309. 1
  1310. 1
  1311. 1
  1312. 1
  1313. 1
  1314. 1
  1315. 1
  1316. 1
  1317. 1
  1318. 1
  1319. 1
  1320. 1
  1321. 1
  1322. 1
  1323. 1
  1324. 1
  1325. 1
  1326. 1
  1327. 1
  1328. 1
  1329. 1
  1330. 1
  1331. 1
  1332. 1
  1333. 1
  1334. 1
  1335. 1
  1336. 1
  1337. 1
  1338. 1
  1339. 1
  1340. 1
  1341. 1
  1342. 1
  1343. 1
  1344. 1
  1345. 1
  1346. 1
  1347. 1
  1348. 1
  1349. 1
  1350. 1
  1351. nintendo.com/us/store/products/the-legend-of-zelda-breath-of-the-wild-switch/ | nixos.org | zero-to-nix.com Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | github.com/tinygrad/tinygrad <- simple powerful deep learning framework tiny corp is accepting new interns. more info on tinygrad.org and tinygrad discord | comma.ai is accepting interns comma.ai/jobs#open-positions $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | from $999 comma.ai/shop/body the future of people Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 intro 00:02:50 oneplus customer support random number generator 00:03:15 legally purchased the game and legally emulating 00:03:55 breath of the wild on M3 00:30:55 just one terminal remaining! 00:39:10 you activated all the terminals! 00:41:45 game over 00:43:10 dueling peaks tower 00:48:22 spirit orb 00:55:10 the water guides, ha dahamar shrine 00:59:25 happy with this game, spirit orb 01:00:20 dueling peaks stable 01:01:50 wild horses 01:11:00 locked mementos 01:15:38 ta'loh naeg shrine 01:23:53 game over 01:25:04 waterblight ganon 01:27:35 what is nix, declarative builds and deployments 01:28:20 Zero to Nix 01:29:37 nix installation 01:31:15 run a program with Nix 01:33:05 downloading a lot of stuff with Nix 01:35:45 Nix flakes 01:36:30 shell environment 01:37:50 lolcat github 01:39:40 search for packages 01:41:45 what is nix, reproducible environments 01:43:00 nix language basics 01:44:50 Terraform 01:45:14 too obscure, embrace nix flakes 01:47:33 last time much worse 19GB download 01:49:00 oneplus refund by calling multiple times 01:51:05 what is nix shopify engineering blog 01:53:55 build a package using Nix 01:56:05 downside to good packaging manager 01:57:02 tiny keyboard 02:00:45 how to brick oneplus open, vip, qdl 02:02:00 apache kudu 02:02:40 wrist wrests 02:05:00 run apache inside nix 02:06:44 trying to do something that is not the tutorial 02:08:48 mediawiki package 02:10:15 riff flake example from DeterminateSystems 02:15:45 nixos immutable, it's how package manager should work 02:20:00 ubuntu core immutable 02:21:25 mediawiki struggle 02:28:20 docker mediawiki 02:30:44 where does the docker put the state 02:36:40 apachectl, who setup a LAMP stack in the last 10 years 02:38:10 apache inside nix struggle 02:41:40 good first time chat, provides information 02:42:25 we don't do it because it's easy, we do it because we thought it would be easy 02:44:50 nginx mediawiki 02:46:00 what is nix flake 02:48:15 arch linux vibes 02:50:40 diselbabyy guide, caddy rage quit 02:52:30 are we trying to do something unreasonable? 02:56:55 building flake.nix 03:04:25 flake-utils utils.lib.mkApp 03:13:49 nix mkDerivation 03:15:30 they know the meta, how did I know pname is the right thing to do there 03:16:25 nix is arch linux docker for haskell fans 03:17:10 nix not for George 03:18:40 never making a nixpkg, better than last time 03:19:30 rust tinygrad rewrite 03:21:25 getting the meta in the games, same skill for computers 03:23:05 slurm workload manager 03:24:20 tinybox gpus 03:25:00 tenstorrent software stack, pivot to RISC-V 03:25:40 this is really bad, you are not ready to sell your thing 03:27:25 tinybox pictures, tinybox PSU 03:30:10 server grade PSU too loud 03:30:40 tinybox price $15k 03:31:45 9 noctua 140 mm fans 03:32:10 90 pounds tinybox 03:32:45 thermodynamics fridge vs tinybox 03:35:20 plan with the self driving thing, selling the box 03:37:50 hacker news people about cheapETH 03:39:55 crypto is only good for regulatory arbitrage, bankless podcast 03:44:20 tiny corp raised $5 mil and sells $5 mil worth of boxes 03:45:10 21 bitcoin computer 03:47:05 regulation does not hold back crypto 03:47:25 triangle, anonymous compute 03:49:25 founders that cashed out, once you delivered the value 03:50:25 moon math 03:51:05 TuSimple scam 03:51:50 ethereum switch from proof of work 03:52:05 money going to the right people 03:55:10 multiculturalism 03:56:45 bitkey 03:59:15 tenstorrent if you can't buy it it's not real, nixos is docker arch linux
    1
  1352. 1
  1353. 1
  1354. 1
  1355. 1
  1356. 1
  1357. 1
  1358. 1
  1359. 1
  1360. 1
  1361. 1
  1362. 1
  1363. 1
  1364. 1
  1365. 1
  1366. 1
  1367. 1
  1368. 1
  1369. 1
  1370. 1
  1371. 1
  1372. 1
  1373. 1
  1374. 1
  1375. 1
  1376. 1
  1377. 1
  1378. 1
  1379. 1
  1380. 1
  1381. 1
  1382. 1
  1383. 1
  1384. 1
  1385. 1
  1386. 1
  1387. 1
  1388. 1
  1389. 1
  1390. 1
  1391. 1
  1392. 1
  1393. 1
  1394. 1
  1395. 1
  1396. 1
  1397. 1
  1398. 1
  1399. 1
  1400. 1
  1401. 1
  1402. 1
  1403. 1
  1404. 1
  1405. 1
  1406. 1
  1407. 1
  1408. 1
  1409. 1
  1410. 1
  1411. 1
  1412. 1
  1413. 1
  1414. 1
  1415. 1
  1416. 1
  1417. 1
  1418. 1
  1419. 1
  1420. 1
  1421. 1
  1422. 1
  1423. 1
  1424. 1
  1425. 1
  1426. 1
  1427. 1
  1428. 1
  1429. 1
  1430. 1
  1431. 1
  1432. 1
  1433. 1
  1434. 1
  1435. 1
  1436. 1
  1437. 1
  1438. 1
  1439. 1
  1440. 1
  1441. 1
  1442. 1
  1443. 1
  1444. 1
  1445. 1
  1446. 1
  1447. 1
  1448. 1
  1449. 1
  1450. 1
  1451. 1
  1452. 1
  1453. 1
  1454. 1
  1455. 1
  1456. 1
  1457. 1
  1458. 1
  1459. 1
  1460. 1
  1461. 1
  1462. 1
  1463. 1
  1464. 1
  1465. 1
  1466. 1
  1467. 1
  1468. 1
  1469. 1
  1470. 1
  1471. 1
  1472. 1
  1473. 1
  1474. 1
  1475. 1
  1476. 1
  1477. 1
  1478. 1
  1479. 1
  1480. 1
  1481. 1
  1482. 1
  1483. 1
  1484. 1
  1485. 1
  1486. 1
  1487. 1
  1488. 1
  1489. 1
  1490. 1
  1491. 1
  1492. 1
  1493. We are not George but here is some information we know. One of comma.ai developers robbederks is working with BogGyver on Tesla AP2 port. Draft is posted on comma github just search tesla in their repository. - https://github.com/commaai/openpilot/pull/20438 - https://github.com/commaai/openpilot/pull/20074 This should be your preferred solution as if this get's up streamed to openpilot by comma.ai you will also get future updates from openpilot. All older solutions for Tesla with openpilot will be outdated and they are using older comma.ai hardware and community modified code that requires modifications on each new openpilot version. You will probably need https://comma.ai/shop/products/comma-car-harness (they are working on Tesla harness) and https://comma.ai/shop/products/comma-two-devkit Make sure to do your own research as there is a lot of differences about how older Tesla vehicles operate. You might need to retrofit some hardware to get good experience with openpilot. Some links to get you started: - https://tinkla.us/ (some information here might be outdated) - https://github.com/commaai/openpilot/search?q=tesla&type=issues (just search github) - https://www.youtube.com/results?search_query=openpilot+tesla (there are some videos showing how to install additional hardware for openpilot on tesla) - https://discord.comma.ai/ (join the tesla channel and use search to find the information you are looking for). The community is very knowledgeable but they encourage high quality discussions so make sure you do your own research before asking questions.
    1
  1494. 1
  1495. 1
  1496. 1
  1497. 1
  1498. 1
  1499. 1
  1500. 1
  1501. 1
  1502. 1
  1503. 1
  1504. 1
  1505. 1
  1506. 1
  1507. 1
  1508. 1
  1509. 1
  1510. 1
  1511. 1
  1512. 1
  1513. 1
  1514. 1
  1515. 1
  1516. 1
  1517. 1
  1518. 1
  1519. 1
  1520. 1
  1521. 1
  1522. 1
  1523. 1
  1524. 1
  1525. 1
  1526. 1
  1527. 1
  1528. 1
  1529. 1
  1530. 1
  1531. 1
  1532. 1
  1533. 1
  1534. 1
  1535. 1
  1536. 1
  1537. 1
  1538. 1
  1539. 1
  1540. 1
  1541. 1
  1542. 1
  1543. 1
  1544. 1
  1545. 1
  1546. 1
  1547. 1
  1548. 1
  1549. 1
  1550. 1
  1551. 1
  1552. 1
  1553. 1
  1554. 1
  1555. 1
  1556. 1
  1557. 1
  1558. 1
  1559. 1
  1560. 1
  1561. 1
  1562. 1
  1563. 1
  1564. 1
  1565. 1
  1566. 1
  1567. 1
  1568. 1
  1569. 1
  1570. 1
  1571. 1
  1572. 1
  1573. 1
  1574. 1
  1575. 1
  1576. 1
  1577. 1
  1578. 1
  1579. 1
  1580. 1
  1581. 1
  1582. 1
  1583. 1
  1584. 1
  1585. 1
  1586. 1
  1587. 1
  1588. 1
  1589. 1
  1590. 1
  1591. 1
  1592. 1
  1593. 1
  1594. 1
  1595. 1
  1596. 1
  1597. 1
  1598. 1
  1599. 1
  1600. 1
  1601. 1
  1602. 1
  1603. 1
  1604. 1
  1605. 1
  1606. 1
  1607. 1
  1608. 1
  1609. 1
  1610. 1
  1611. 1
  1612. 1
  1613. 1
  1614. 1
  1615. 1
  1616. 1
  1617. 1
  1618. 1
  1619. 1
  1620. 1
  1621. 1
  1622. 1
  1623. 1
  1624. 1
  1625. 1
  1626. 1
  1627. 1
  1628. 1
  1629. 1
  1630. 1
  1631. 1
  1632. 1
  1633. 1
  1634. 1
  1635. 1
  1636. 1
  1637. 1
  1638. 1
  1639. 1
  1640. 1
  1641. 1
  1642. 1
  1643. 1
  1644. 1
  1645. 1
  1646. 1
  1647. 1
  1648. 1
  1649. 1
  1650. 1
  1651. 1
  1652. 1
  1653. 1
  1654. 1
  1655. 1
  1656. 1
  1657. 1
  1658. 1
  1659. 1
  1660. 1
  1661. 1
  1662. 1
  1663. 1
  1664. 1
  1665. 1
  1666. 1
  1667. 1
  1668. 1
  1669. 1
  1670. 1
  1671. 1
  1672. 1
  1673. 1
  1674. 1
  1675. 1
  1676. 1
  1677. 1
  1678. 1
  1679. 1
  1680. 1
  1681. 1
  1682. 1
  1683. 1
  1684. 1
  1685. 1
  1686. 1
  1687. 1
  1688. 1
  1689. 1
  1690. 1
  1691. 1
  1692. Sorry for potato quality! | Check if your car is supported -> comma.ai/vehicles and buy -> comma.ai/shop/comma-three from $1499 | blog.comma.ai/dumb-questions | What is comma.ai/openpilot | Join the comma team comma.ai/jobs | How to setup your car comma.ai/setup | Follow the official youtube.com/commaai for more livestreams | Stay up to date by following twitter.com/comma_ai | Support comma.ai at comma.ai/shop. Chapters: 00:00 intro 00:35 comma body remotely controlled 01:05 comma body hat and training wheels 01:55 comma security telepresence 02:10 comma body control website 02:55 the hat, crown 03:20 the dream, mars rover 03:54 Harold, Adeeb 04:15 hiring 04:25 disengagement metric 04:55 control metric 05:35 web team hiring 06:00 openpilot hiring 06:20 openpilot team 06:35 hangout place 06:44 going to car, demo, ui updates, ford 07:10 pr, marketing 07:25 heat testing setup 07:50 ford bronco sport 08:05 ford bronco sport openpilot support 08:50 ford f150 support 09:35 experimental mode 09:50 stopping for stop sign 10:15 ford platform openpilot 0.9.2 11:20 ford long not shipped yet, radar support 12:05 ui changes 13:04 boot time, new models 13:33 0.9.2 release date 13:50 ford maverick support 14:15 any remarkable improvements coming to Bolt with 0.9.2: below 6 MPH control 14:25 nicki minaj model 15:00 low speed on ford 15:15 turning bug fixed 15:40 hiring bounty comma three + $3000 16:00 releasing taco drives for production 16:20 navigate on openpilot, map 17:00 bounty hiring program 17:45 can I go buy an f150 to add my c3 18:05 complaints about openpilot 18:25 following distance adjustment 18:55 customer support engineer role, but must that be local in San Diego 19:20 thermal improvements, 0.9.3 19:45 ui changes in 0.9.3 20:05 limited steering torque 20:20 civic ping pongs a little on the highway 20:35 experimental for bolt 21:10 bronco smooth steering 21:25 metric, torque controller steering 21:45 0.9.2 no disengage on gas default 22:10 the testing simulation work for testing end to end longitudinal experimental mode behavior 23:10 ram start speed activation to 0 23:15 2023 palisade/telluride joining the official support list 23:40 what's the holdup for nav on openpilot in experimental in master 24:35 ui path changes 25:25 does experimental mode use map data at all for speed limits 26:00 does it try and reach that max speed 26:25 what fords is this going to be available for 26:50 only new ford models will be supported lockout still not hacked 27:05 ford is one of the biggest auto in usa 27:20 ram support 27:35 checksum of hybrids different 28:05 should make it so you can livestream right off of comma 3 28:20 comma body live streaming 28:45 sentry mode, f150 and use plug and play timeline 29:10 thoughts on Tesla going to end to end 30:30 third paradigm use overrides to punish the model during training 31:05 master-ci hasn't failed to build in a long time! great testing 31:55 old website looked more clean and professional 32:05 experimental mode and chill mode should have separate cruise speed settings 32:15 brazil we do not have this bronco, the BSM is sophisticated equal to the KIA EV6 32:35 who is comma brasil 32:55 is there a planned release for navigate on openpilot 33:25 progress to taco, corner radars like EV6 33:40 toyota the biggest auto that's unsupported 34:05 thanks for watching
    1
  1693. 1
  1694. 1
  1695. 1
  1696. 1
  1697. 1
  1698. 1
  1699. 1
  1700. 1
  1701. 1
  1702. 1
  1703. 1
  1704. 1
  1705. 1
  1706. 1
  1707. 1
  1708. 1
  1709. 1
  1710. 1
  1711. 1
  1712. 1
  1713. 1
  1714. 1
  1715. 1
  1716. 1
  1717. 1
  1718. 1
  1719. 1
  1720. 1
  1721. 1
  1722. 1
  1723. 1
  1724. 1
  1725. 1
  1726. 1
  1727. 1
  1728. 1
  1729. 1
  1730. 1
  1731. 1
  1732. 1
  1733. 1
  1734. 1
  1735. 1
  1736. 1
  1737. 1
  1738. 1
  1739. 1
  1740. 1
  1741. 1
  1742. 1
  1743. 1
  1744. 1
  1745. 1
  1746. 1
  1747. 1
  1748. 1
  1749. 1
  1750. 1
  1751. 1
  1752. 1
  1753. 1
  1754. 1
  1755. 1
  1756. 1
  1757. 1
  1758. 1
  1759. 1
  1760. 1
  1761. 1
  1762. 1
  1763. 1
  1764. 1
  1765. 1
  1766. 1
  1767. 1
  1768. 1
  1769. 1
  1770. 1
  1771. 1
  1772. 1
  1773. 1
  1774. 1
  1775. 1
  1776. 1
  1777. 1
  1778. 1
  1779. 1
  1780. 1
  1781. 1
  1782. 1
  1783. 1
  1784. 1
  1785. 1
  1786. 1
  1787. 1
  1788. 1
  1789. 1
  1790. 1