Cyber Risks · Trail of Bits Blog

Understanding Apple’s On-Device and Server Foundation Models release

Summary: At WWDC, Apple released details of at least five foundation models: an approximately 3B parameter on-device model, a large Mixture-of-Experts server model estimated at 130B-180B parameters running on Apple Silicon, an on-device code model for Swift fill-in-middle in Xcode, a server code model branded Swift Assist, and an image diffusion model with adapters. Apple stated the stack avoids NVIDIA hardware and CUDA, using AXLearn for training and CoreML and Metal on-device, and compresses models to 3.5 bits-per-weight via palettization and quantization.

What's compelling: Apple is shipping a base model plus small LoRA/DoRA adapters, tens of megabytes each, that can be stacked and swapped on-device, so the model behavior on a managed Mac or iPhone can change with an adapter update rather than a full OS release. The claim that 3.5 bits-per-weight compression achieves the same accuracy as uncompressed models is unverified, and Trail of Bits notes quantization loss at that level is noticeably not zero.

Read the original at Trail of Bits Blog ↗