How to scope AI projects slide
Technical Insights

Five Steps to Scoping AI Projects

One of the most important skills of an AI architect is the ability to identify ideas that are worth working on. Over the years, I’ve had fun applying machine learning to manufacturing, healthcare, climate change, agriculture, ecommerce, advertising, and other industries.
Process of diagnosing a medical patient slide
Technical Insights

Choose the Right Point On the Automation Spectrum

AI-enabled automation is often portrayed as a binary on-or-off: A process is either automated or not. But in practice, automation is a spectrum, and AI teams have to choose where on this spectrum to operate.
A/B Test loop for building human insight
Technical Insights

A Different Approach to A/B Testing

When a lot of data is available, machine learning is great at automating decisions. But when data is scarce, consider using the data to augment human insight, so people can make better decisions.
Speech bubble that says "It did well on the test set!"
Technical Insights

High Test-Set Accuracy Is Not Enough

Over the last several decades, driven by a multitude of benchmarks, supervised learning algorithms have become really good at achieving high accuracy on test datasets. As valuable as this is, unfortunately maximizing average test set accuracy isn’t always enough.
Slide that says "Proof of concept - production"
Technical Insights

Don't Confuse Proof of Concept With Production Deployment

Last week, I talked about how best practices for machine learning projects are not one-size-fits-all, and how they vary depending on whether a project uses structured or unstructured data, and whether the dataset is small or big.
Table with information related to data and datasets
Technical Insights

Structured and Unstructured Data: Implications for AI Development

Experience gained in building a model to solve one problem doesn’t always transfer to building models for other problems. How can you tell whether or not intuitions honed in one project are likely to generalize to another?
Labeling training data charts
Technical Insights

Data-Centric AI Development: Small-Data Problems

Over the last two weeks, I described the importance of clean, consistent labels and how to use human-level performance (HLP) to trigger a review of whether labeling instructions need to be reviewed.
Detecting system pointing out scratches on a surface
Technical Insights

AI Versus Human-Level Performance, Part 2

Last week, I wrote about the limitation of using human-level performance (HLP) as a metric to beat in machine learning applications for manufacturing and other fields. In this letter, I would like to show why beating HLP isn’t always the best way to improve performance.
Different defects on a platform
Technical Insights

AI Versus Human-Level Performance

Beating human-level performance (HLP) has been a goal of academic research in machine learning from speech recognition to X-ray diagnosis. When your model outperforms humans, you can argue that you’ve reached a significant milestone and publish a paper!
Man riding a bike and reading a book at the same time
Technical Insights

How to Learn Coding

I’d like to share a programming tip that I’ve used for years. A large part of programming involves googling for code snippets you need on Stack Overflow and other websites. (Shh. Don’t tell the nondevelopers. ????)
Light on a window during a very dark night
Technical Insights

How to Build Deep Expertise

Did you ever spend days obsessing over a technical problem? If so, I applaud you. Determined pursuit of solutions to hard problems is an important step toward building deep expertise.
Two partial images of a retina
Technical Insights

From Proof of Concept to Production

There has been a lot of excitement about the idea of using deep learning to diagnose diabetic retinopathy: That is by taking a photo of the retina and using AI to detect signs of disease.
The Great Wave off Kanagawa by Hokusai
Technical Insights

Tips for Building Practical Machine learning systems

In an earlier letter, I wrote about the challenge of robustness: A learning algorithm that performs well on test data often doesn’t work well in a practical production environment because the real world turns out to be different than the test set.
Flatten the curve campaign image
Technical Insights

Deep Learning Against Covid

Last week, I asked readers to tell me what they’re doing to address the Covid-19 pandemic. Many of you wrote to say you’re taking actions such as shopping for neighbors, making masks, and creating posters that promote Covid-safe practices...
Transfer Learning and Self-taught Learning examples
Technical Insights

Unsupervised Learning Ascendent

Nearly a decade ago, I got excited by self-taught learning and unsupervised feature learning — ways to learn features from unlabeled data that afterward can be used in a supervised task.

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox