Great article and I learned a ton. In particular: “Let’s not completely dismiss the years of growth, work, and education it takes a baby human brain to get to the state where it can casually “learn new tasks very quickly”. I think this fact is so under-weighted in the discussions of AI and philosophy.
If you’ve ever had kids, you’ve seen a baby “train”. I can recall my 2month old son just looking around, looking, looking, looking as he slowly moved his head. Then looking at his little hand; opening and closing his fist and staring at it. It’s training!… and it’s based on “stereo-cam video” aka two eyes, not to mention audio and a zillion nerve endings to boot.
We have a both head start on AI due to this kind of pre-k network training.
Thanks, I think that fact is underrated too! I’ve wondered if this was just a result of the people being involved in such discussions being a certain age and/or social group that don’t have much exposure to young children (for whatever reason).
And that’s cool! Yeah, most people who haven’t looked at the developmental literature won’t realise babies don’t automatically see in colour or in 3D with both their eyes (even though we were all there once). That stuff has to be learned over the first year of life (~3 million seconds!) like you observed.
Nice article. At first, I was confused about the section about the number of bits to *process* one piece of information. My understanding is that we want to compare the sizes of the training sets for LLMs and for humans. The amount of computation a human uses to process elements of the training set is seemingly unrelated to the size of the human training set. However, I see that the highlighted portion of the Zheng-Meister abstract clarifies that 10^9 is the size of the input. Is this a valid question and resolution to my question?
Great article and I learned a ton. In particular: “Let’s not completely dismiss the years of growth, work, and education it takes a baby human brain to get to the state where it can casually “learn new tasks very quickly”. I think this fact is so under-weighted in the discussions of AI and philosophy.
If you’ve ever had kids, you’ve seen a baby “train”. I can recall my 2month old son just looking around, looking, looking, looking as he slowly moved his head. Then looking at his little hand; opening and closing his fist and staring at it. It’s training!… and it’s based on “stereo-cam video” aka two eyes, not to mention audio and a zillion nerve endings to boot.
We have a both head start on AI due to this kind of pre-k network training.
Thanks, I think that fact is underrated too! I’ve wondered if this was just a result of the people being involved in such discussions being a certain age and/or social group that don’t have much exposure to young children (for whatever reason).
And that’s cool! Yeah, most people who haven’t looked at the developmental literature won’t realise babies don’t automatically see in colour or in 3D with both their eyes (even though we were all there once). That stuff has to be learned over the first year of life (~3 million seconds!) like you observed.
Nice article. At first, I was confused about the section about the number of bits to *process* one piece of information. My understanding is that we want to compare the sizes of the training sets for LLMs and for humans. The amount of computation a human uses to process elements of the training set is seemingly unrelated to the size of the human training set. However, I see that the highlighted portion of the Zheng-Meister abstract clarifies that 10^9 is the size of the input. Is this a valid question and resolution to my question?