Aren't the M1 and friends relatively capable for LLM tasks? | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

foota on Oct 21, 2023 | parent | context | favorite | on: The Pixel 8 Pro's Tensor G3 off-loads all generati...

Aren't the M1 and friends relatively capable for LLM tasks?

brucethemoose2 on Oct 21, 2023 [–]

Sort of.

Apple is super stingy about RAM capacity on their devices, which is a problem for GenAI

Also, the M1 NPU (the equivalent to Google's offloading) is not very fast in Apple's own Stable Diffusion port, and I'm not aware of any community LLM frameworks that even use the NPU. They all run Metal implementations on the M1's GPU.

Consider applying for YC's Fall 2026 batch! Applications are open till July 27.
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact