Apple presented its plan to imbue its phones and computers with artificial intelligence.
What’s new: Apple announced Apple Intelligence, a plethora of generative-AI features that integrate with iOS 18, iPadOS 18, and MacOS Sequoia. The beta version of Apple Intelligence will be available in U.S. English prior to a wider rollout near the end of the year, starting with the iPhone 15 Pro and Mac computers that use M-series chips.
On-device and in the cloud: The new capabilities rely on a suite of language and vision models. Many of the models will run on-device, while workloads that require more processing power will run on a cloud powered by Apple chips.
- Semantic search analyzes the data on a device to better understand context such as the user’s routines and relationships. For example, if a user enters a prompt like, “Show me the files my boss shared with me the other day,” models can identify the user’s boss and the day in question.
- Generative media capabilities are geared to fulfill preset functions. For instance, the text generator offers options to make writing more friendly, professional, or concise. Image generation focuses on tasks like making custom emojis from text prompts and turning rough sketches into polished images.
- Apple’s voice assistant Siri will accept text as well as voice prompts. It will also interact with apps, so Siri can, say, determine whether a meeting scheduled in the Calendar app will prevent a user from attending an event at a location designated in the Maps app.
- Starting later this year, Siri users will be able to converse with OpenAI’s ChatGPT without having an OpenAI account or paying a fee. Paid ChatGPT users will be able to log in for access to paid features. Apple plans to integrate other third-party large language models.
- The underlying infrastructure is designed to maintain user privacy. Apple’s cloud won’t retain user data. Apple won’t have privileged access to user data. Queries to ChatGPT from users who are not logged into an OpenAI account will have their IP masked. In addition, independent researchers can inspect the infrastructure code to verify assurances and find flaws.
How it works: Apple outlined the architecture that underpins the new features and compared two models of its against competitors.
- All Apple models were trained on a mix of licensed, synthetic, and web-crawled data (filtered to remove personal and low-quality information). The models were fine-tuned to follow instructions via methods including reinforcement learning from human feedback.
- To adapt its models to specific tasks, Apple uses LoRA weights that plug into a pretrained model and adjust its weights at inference. Such LoRA adapters are included for many tasks including summarization, proofreading, email replies, and answering questions.
- Apple used quantization, a compression technique called low-bit parallelization (also known as weight clustering), and other methods to improve speed and energy efficiency. On an iPhone 15 Pro, Apple clocked a generation rate of 30 tokens per second.
- Apple hired human graders to test two of its models on an internal benchmark that covers tasks including brainstorming, classification, answering questions, rewriting, summarization, and safety. The graders preferred an on-device model of 3 billion parameters over Phi-3-mini, Mistral-7B, and Gemma-7B. They preferred a large language model designed to run in the cloud to DBRX-Instruct, GPT-3.5-Turbo, and Mixtral-8x22B, but not to GPT-4-Turbo.
Behind the news: While rivals like Microsoft and Google dove into generative AI, Apple moved more cautiously. During the 2010s, it invested heavily in its Siri voice assistant, but the technology was outpaced by subsequent developments. Since then, the famously secretive company has been perceived as falling behind big-tech rivals in AI.
Why it matters: While Apple’s big-tech competitors have largely put their AI cards on the table, Apple has held back. Now its strategy is on display: Proprietary foundation models, LoRA to fine-tune them to specific tasks, emphasis on the user experience over raw productivity, judicious use of edge and cloud computing, and deals with other model makers, all wrapped up in substantial privacy protections.
We’re thinking: Apple’s control over its product ecosystem gives the company an extraordinary distribution channel. That’s why Google reportedly paid Apple $20 billion in 2022 to provide the default search engine in Apple’s Safari web browser. This advantage means that, whatever its pace of development and strategy in AI, Apple’s competitive edge remains sharp.