Generative AI plays a starring role in the latest Windows PCs.
What’s new: Microsoft introduced its Copilot+ PCs, an AI-first laptop specification that offers features unavailable to other Windows users. Copilot+ PCs will be available from Microsoft as well as Acer, Asus, Dell, HP, Lenovo, and Samsung starting in mid-June.
How it works: Copilot+ PCs provide AI-powered generative and search functions thanks to unnamed AI models that run on-device.
- A feature called Recall enables users to search their activities in apps, documents, and websites to find, say, topics discussed in a text conversation or items viewed on a website. Every five seconds, the PC takes a screenshot of its current status. Users can browse the timeline of screenshots or call an unidentified AI model to find images and/or text via a semantic index.
- Other features include Cocreator, which generates images from text prompts using models that run on-device, and Live Captions, which generates subtitles for English-language audio in any of 40 languages.
- Developers have access to these features via a software stack called Windows Copilot Runtime. This includes Copilot Library, a set of APIs that call more than 40 models that run on-device, and DiskANN, a set of search algorithms that quickly sort through a vector database.
- The first machines will be based on the Qualcomm Snapdragon X processor. The chip comes with 10 and 12 CPU cores, a GPU and a neural processing unit (NPU) that accelerates neural networks while using less energy and memory than a typical CPU or GPU.
Nvidia’s rejoinder: Nvidia plans to launch Copilot+-compatible RTX AI PCs that run Nvidia’s own toolkit for calling and customizing models with on-device GPUs. These computers, initially built by Asus and MSI based on AMD CPUs, eventually will deliver all Copilot+ features. Nvidia criticized Microsoft’s NPU specification, which calls for 45 trillion operations per second (TOPS), claiming that that speed is enough to process only basic AI workloads. Meanwhile, Nvidia’s game-focused GPUs deliver more than 1,000 TOPS.
Why it matters: Microsoft is betting that on-device AI will change the PC experience. The Copilot+ PC specification gives developers a versatile toolkit for adding AI to existing apps while opening the door to fundamentally new functionality like Recall.
We’re thinking: As we wrote earlier, makers of chips and operating systems alike have a strong incentive to promote on-device (or edge) AI. The growing presence of AI accelerators in consumer devices brings significant privacy benefits for consumers and opens exciting new opportunities for developers.