Comments by "Vitaly L" (@vitalyl1327) on "GPT-4 CAN'T REASON ಠ_ಠ ...apparently." video.

@WesRoth It's really hard to write proper scientific papers about something like GPT-4, as there's no such a thing as a reproducible experiment. GPT-4 changes all the time, there's no frozen version available. Local LLMs are much better suited for analysis, as any experiment can be perfectly reproduced. Who knows, maybe OpenAI read the paper, got burns and fine-tuned a LoRA layer to fix the answers to the questions listed in the paper.
5
Who would have thought? Stochastic parrots cannot reason. Why would anyone expect an ability to reason from a glorified Markov chain?
2
@hunterkudo9832 LLMs can see known patterns and apply them. They were trained on a lot of logical problems, including ones with made up words, so they know to treat them as abstract variables. It is purely mechanical and any proof assisrant (like HOL) would do it orders of magnitude better.
2