Raspberry Pi 5 AI kit

Friday 14th June, 2024 - Bruce Sterling

*Why would they not?

https://www.theregister.com/2024/06/04/raspberry_pi_ai_kit/

(…)

Although the AI Kit represents a big performance leap for the Raspberry Pi 5, the single-board machines already have history in the field.

Upton pointed out that long before AI became the Next Big Thing, Raspberry Pi users were already using the technology for things like object detection. The CEO mentioned one particular instance of a Japanese cucumber farm using a Raspberry Pi 3 to sort thorny cucumbers with about a 70 percent accuracy rate.

However, that cucumber sorter relied on AI processing in the cloud. The debut of this kit means some AI workloads can run on an Pi – reducing reliance on connectivity.

“It’s a good host for smaller models, and more optimized and quantized large language models (LLMs) are running at tokens per second on the Raspberry Pi 5,” Upton told The Register, before noting that some work will still need to be run in the cloud. LLaMa 3 70B is tough to run locally on a PC unless you have an impressive setup.

While there aren’t a ton of use cases right now, Upton expects the lower entry price of Raspberry Pi 5 AI to create an “exponential increase” in AI applications and solutions….