Apple’s New Era: On-Device AI and the iPhone 15 Pro Max
Introduction to Apple’s On-Device AI
Apple has ushered in a new era with its latest devices, including the iPhone 15 Pro and iPhone 15 Pro Max. This new phase focuses on enhancing user experience through on-device AI processing.
The cornerstone of the personal intelligence system is on-device processing. We have integrated it deep into your iPhone, iPad, and Mac, and throughout your apps, so it’s aware of your personal data without collecting your personal data.
Privacy and On-Device AI
Apple’s strategy of on-device AI processing is a move to bolster privacy. This approach ensures that personal data remains on the device, unlike other companies that rely on cloud processing. Microsoft, for instance, has a similar concept with its Copilot+ standard, which is designed for laptops and uses dedicated AI hardware for on-device processing.
The Role of Neural Processing Units (NPUs)
What is an NPU?
The term NPU stands for “neural processing unit.” This technology differentiates older hardware from the new AI-capable devices. NPUs are specialized in performing numerous operations simultaneously while consuming minimal power, making them ideal for mobile devices.
Technical Insights
NPUs excel at matrix manipulation, a core function in generative AI. This process involves starting with a prompt and refining it into a final result, whether it’s an image or a sentence.
Apple’s History with NPUs
Apple has been incorporating NPUs since 2017, starting with the Apple Neural Engine (ANE) in the iPhone 8, iPhone 8 Plus, and iPhone X. This was long before AI became a buzzword, indicating Apple’s foresight in integrating AI capabilities into its devices.
The Hardware and Software Divide
Artificial Gating
Some new AI features are being artificially gated behind new hardware. For example, Microsoft’s Copilot+ features have been made to work on less powerful hardware by hobbyist software historians like Albacore, who managed to run these features on a Samsung Galaxy Book2 Go notebook.
Getting Recall working was moderately challenging, I suppose. I’ve been reverse engineering Windows bits for close to 10 years, so tracking down checks and restrictions is practically second nature to me.
Practical Implementation
Albacore explains the process of making these features work on unsupported hardware by altering checks and wrapping them in a user-friendly installer. This shows that some AI capabilities can be enabled on older devices with the right tweaks.
RAM and AI Capabilities
Importance of RAM
RAM is crucial for running AI models locally. Older iPhones, like the iPhone 14, have powerful NPUs but only 6 GB of RAM, whereas devices on the Apple Intelligence guest list have 8 GB of RAM. This difference in RAM capacity is significant because AI models need to be stored in RAM or vRAM for efficient processing.
Performance Comparisons
Even the fastest SSDs can’t match the speed required for AI tasks. For instance, the PlayStation 5’s SSD has a transfer rate of 5,000 MB per second, while DDR5 RAM can reach up to 64 GB per second. Nvidia’s H200, designed for machine learning, boasts a 4.8 TB per second bandwidth, highlighting the gap between consumer and professional-grade hardware.
GeForce RTX GPUs are ideal for AI PC users looking for the best AI experience. They deliver the fastest AI performance, up to 1,300 TOPS, and the most mature AI software stack, and accelerate more than 500 AI-enabled applications and games.
Future Prospects
iPhone 15 Pro Max Capabilities
The iPhone 15 Pro Max, with its 2 to 4 GB of RAM and 35 TOPS NPU, will support some offline AI features. However, more complex generative AI tasks will likely remain out of reach for these devices.
Upcoming Features
Apple’s website indicates that features like Sharing Suggestions, Memories, and scene recognition in Photos, as well as Siri Suggestions and voice recognition, will be processed on-device. However, more advanced generative AI tasks will still require cloud processing.
Conclusion
The first Apple Intelligence features will be available with the public release of iOS 18, expected in September alongside the iPhone 16. It remains to be seen whether the iPhone 16 will offer the full suite of Apple Intelligence features or if it will be reserved for the Pro-series.
For more details on Microsoft’s Copilot+ standard, visit Microsoft’s Copilot+ page.
To learn how to enable AI features on unsupported hardware, check out Tom’s Hardware article.
6 Comments
Why not just buy a newer model?!
Could this be Apple’s way of nudging us to upgrade our devices?
Is it a surprise that older tech can’t handle the latest advancements?
Does Apple expect us to buy new iPhones every year?!
Oh great, another reason to buy a new iPhone!
InkSpills: How convenient for Apple!