As the year winds down, I’ve been reflecting on the ideas that left the deepest impression on me, and two books stand out as absolute highlights of 2024:
📖 𝗧𝗵𝗲 𝗪𝗼𝗿𝗹𝗱𝘀 𝗜 𝗦𝗲𝗲 by Fei-Fei Li — https://a.co/d/7XqP5J8
📖 𝗔 𝗕𝗿𝗶𝗲𝗳 𝗛𝗶𝘀𝘁𝗼𝗿𝘆 𝗼𝗳 𝗜𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲 by Max Bennett — https://a.co/d/8UleBZ7
Both are deeply thought-provoking works that I can’t recommend enough! Together, they explore the similarities, differences, and fascinating interplay between biological and artificial intelligence that has reshaped how I think about the future of both.
Here are a few similarities that stood out:
🔹 Uniform Neural Structure
The brain’s neocortex is built from identical neurons arranged into repeating microcircuits, yet it handles everything from face recognition to poetry. DNNs borrowed this idea: uniform layers of nodes that specialize through training. Another proof that simplicity scales!
🔹 Reinforcement Learning with Curiosity
Evolution gifted humans with curiosity, inspiring us to explore the world, take risks, and learn from our experiences. Early ML algorithms lacked this drive, sticking to safe paths with predictable rewards. But in 2017, DeepMind changed that with AlphaZero by explicitly modeling exploration and creating an AI that defeated the world champion of Go!
🔹 Imitation Learning: Understanding the Why
Primates revolutionized learning by imitating intent instead of blindly copying actions. Similarly, early autonomous driving systems mimicked human actions based on sensory data and often failed. Andrew Ng’s team solved this in 2003 by training AI to infer the intended trajectories of the experts, advancing robotics and autonomous driving.
And here are key differences:
🔹 Backpropagation vs Hebbian Learning
AI relies on backpropagation, popularized by Geoffrey Hinton in the 1980s, while the brain uses Hebbian learning, often summarized as “neurons that fire together wire together”.
🔹 Catastrophic Forgetting
AI suffers from catastrophic forgetting, losing old knowledge when learning new tasks, which requires freezing models after training. In contrast, human brains replay and integrate memories through the hippocampus, enabling continuous learning without overwriting prior knowledge.
🔹 Simulation and Imagination
The brain excels at creating mental models of the world and imagining future possibilities. This “simulation engine” interprets sensory information by assuming it originates from a 3D world that unfolds over time.
🤔 Interestingly, both authors allude to Spatial Intelligence as the next AI frontier. And recently, Fei-Fei Li founded World Labs, a company building Large World Models to perceive, generate, and interact with the 3D world — how cool is that!
What an exciting time we live in! I’m eager to see what 2025 will bring.
What ideas or books shaped your thinking this year? I’d love to hear your recommendations!
🎄 Wishing everyone a Merry Christmas and a fantastic New Year! 🙌