Gregory's Server@J12t yes I 100% agree with that. Which is easier? Learning to parrot every x + y = z math equation, or learning the meaning of +? Just by auto regression, you can develop higher order understandings. This is obvious when you use GPT. Ironically GPT is not great at math, it’s not a good domain for them IMO, but it’s just an example of how autoregressive algorithms can learn meaning.
|
3 comments
@J12t it’s all very philosophical, I think we should stick to the science. Metrics about how well it handles out-sample QA. Experiments to make it find novel scientific discoveries. Etc. .@J12t with centuries of philosophy trying to understand (and failing) philosophy of mind, sentience, understanding, and especially our history of miscategorizing beings into “inhuman” and therefore not morally applicable beings (slaves, animals), I’m entirely focused on Karl Poppers falsification criteria for all these ideas, rather than trying to justify them as true. |
@ryanpeach There are some great Richard Feynman recordings on "what it really means to understand something" or such. I am doubtful that he'd agree. (That of course doesn't mean it isn't true -- but it's unclear to me how to tell either way)