Comments by "Brent Jacobs" (@br3nto) on "I believe chatbots understand part of what they say. Let me explain." video.

This is such a refreshing take on this question. I guess future AIs will take exams just like students so that the efficacy of their internal models can be verified, measured, and ranked against the standard.
60
I love that you so simply stated the limitation of chat GPT in the example that it doesn’t have a model for physical distance and I assume size and mathematical relationships and many other things. There’s so much room for growth.
27
@younggod5230 yeah, I agree it won’t be like students. There may be multiple instances of the same AI… in which case, all instances should operate the same, so no need to test all instances. But what if we get to a point where an instance can incorporate new information? Then each instance would need to be tested… but like you said, a company may only have a single instance of an AI, but we may get to a point where we start a fresh AI for any new project… like how we might open a word or excel document. What I find most fascinating about what Sabine said is that inside the AI is a black box, and we need to have special ways of verifying the integrity of what’s happening inside that black box… in the same way that humans are black boxes, and we each develop special ways to verify what is happening inside other people. Exams and tests are one way to verify the knowledge and proficiency of an individual, but we also develop empathy to under what another person is going through, or have a conversation to share our experiences… we might find that probing into the depths of an AI is not too dissimilar from interacting with other people.
1