There has been a lot of excitement about the idea of using deep learning to diagnose diabetic retinopathy: That is by taking a photo of the retina and using AI to detect signs of disease.

In an earlier letter, I wrote about the challenge of robustness: A learning algorithm that performs well on test data often doesn’t work well in a practical production environment because the real world turns out to be different than the test set.

Last week, I asked readers to tell me what they’re doing to address the Covid-19 pandemic. Many of you wrote to say you’re taking actions such as shopping for neighbors, making masks, and creating posters that promote Covid-safe practices...

Nearly a decade ago, I got excited by self-taught learning and unsupervised feature learning — ways to learn features from unlabeled data that afterward can be used in a supervised task.

A student once asked me, “Can an AI ever love?”Since the early days of AI, people have wondered whether AI can ever be conscious or feel emotions. Even though an artificial general intelligence may be centuries away, these are important questions.

My last two letters explored robustness and small data as common reasons why AI projects fail. In the final letter of this three-part series, I’d like to discuss change management.

In this series exploring why machine learning projects fail, let’s examine the challenge of “small data.” Given 1 million labeled images, many teams can build a good classifier using open source.

Building AI systems is hard. Despite all the hype, AI engineers struggle with difficult problems every day. For the next few weeks, I’ll explore some of the major challenges. Today’s topic: The challenge of building AI systems that are robust to real-world conditions.

Welcome to the Halloween edition of The Batch! I promised last week to share some common reasons for AI project failures. But first, let’s start with some of the least common reasons.

I’ve heard this conversation in multiple companies: Machine learning engineer: Look how well I did on the test set! Business owner: But your ML system doesn’t work. This sucks! Machine learning engineer: But look how well I did on the test set!

Thinking about the future of machine learning programming frameworks, I recently reread computer scientist Fred Brooks’ classic essay, “No Silver Bullet: Essence and Accidents of Software Engineering.” Three decades after its initial...