We’ve had great success with supervised deep learning on labeled data. Now it’s time to explore other ways to learn: training on unlabeled data, lifelong learning, and especially letting models explore a simulated environment before transferring what they learn to the real world. In 2020, I hope to see more research in those areas.
High-fidelity simulation lets us train and test algorithms more effectively, leading to more robust and adaptive networks. Models can gain far more experience in the virtual world than is practical in the real world. We can simulate rare events that pose severe challenges but are seldom represented by ground truth.
For instance, when we’re driving a car, accidents are rare. You won’t see all the variations even if you drive hundreds of thousands of miles. If we train autonomous cars only on real-world data, they won’t learn how to manage the wide variety of conditions that contribute to accidents. But in a simulation, we can generate variation upon variation, giving the model a data distribution that better reflects real-world possibilities, so it can learn how to stay safe.
Lately, simulation has helped achieve impressive results in reinforcement learning, which is extremely data-intensive. But it’s also useful in supervised learning, when researchers may have only small amounts of real-world data. For instance, earthquakes are rare and difficult to measure. But researchers at Caltech’s seismology lab used a simple physical model to create synthetic data representing these events. Trained on synthetic data, their deep learning model achieved state-of-the-art results predicting properties of real-world earthquakes.
At Nvidia, we’ve developed powerful simulation platforms like Drive Constellation for autonomous vehicles and Isaac for robotics. These open, scalable environments enable models to act in a photorealistic virtual world, complete with highly accurate physics.
I hope that more AI scientists will come to recognize the value of training in simulated environments, as well as other techniques beyond supervised learning. That would make 2020 a year of great progress in AI.
Anima Anandkumar is director of machine learning research at Nvidia and a professor of computer science at Caltech.