Newshot

Up to 43 Tokens per Second: Garlic LLM Lets a Jailbroken PS4 Run a 9B-Parameter Local AI Model

AI summary

On November 17, IT Home reported that tech media Gigazine published a blog post today (November 17) reporting that the open-source project Garlic LLM can run a 9-billion-parameter local large language model on a jailbroken PlayStation 4, with a maximum generation speed of 43 tokens per second, and can complete AI chat and voice conversations without an internet connection. Created by developer Leonardo Alegria, the project bypasses the PS4's original drivers for game graphics processing and issues low-level instructions directly to its AMD GCN architecture GPU, turning the console into an "offline computer" capable of AI inference; because the PS4 does not provide a general GPU computing interface or machine learning library for unofficial programs, the developer developed GPU-related instructions to let the console run local models. To fit the PS4's limited memory, Garlic LLM uses Alibaba's Qwen3.5-9B model base and compresses the model weights into 3-bit or 4-bit quantization, keeping the model and acceleration data to about 4.5 GiB, and uses speculative decoding (MTP) to pre-sample and verify multiple tokens during text generation; tests show ordinary text generation is about 31

Translation(model translation, not the source's own text)

Up to 43 Tokens per Second: Garlic LLM Can Run a 9B-Parameter Local AI Model on a Jailbroken PS4

IT之家 November 17 news: Tech media outlet Gigazine today (November 17) published a blog post reporting that the open-source project Garlic LLM can run a 9-billion-parameter local large language model on a jailbroken PlayStation 4, with a maximum generation speed of up to 43 tokens per second, and can complete AI chat and voice conversations without an internet connection.

The project was created by developer Leonardo Alegria. Technically, it bypasses the PS4's original drivers designed for game graphics processing and directly issues low-level instructions to its AMD GCN architecture GPU, turning the game console into an “offline computer” capable of running AI inference.

Because the PS4 does not provide a general-purpose GPU computing interface or machine learning library that unofficial programs can call, the developer wrote GPU-related instructions to let the PS4 game console run local models.

To accommodate the PS4's limited memory, Garlic LLM uses Alibaba's Qwen3.5-9B model base and compresses the model weights into 3-bit or 4-bit quantization formats. Together with acceleration data, the total is kept to about 4.5 GiB. It also uses “speculative decoding” (MTP) technology, pre-sampling and verifying multiple tokens during text generation to improve output efficiency.

Actual tests show that ordinary text generation speed is about 31 to 39 tokens per second; when generating program code, it can reach up to 43 tokens per second, and the model can handle a context window of about 16,000 tokens.

In addition to text chat, Garlic LLM also supports voice input and voice conversation, using Parakeet TDT 0.6B v3 for speech recognition and Chatterbox for speech synthesis, respectively. It can also execute AI-generated Python code on the PS4 and provides a “thinking mode” that lets the model reason before answering. However, enabling voice conversation takes up more memory, and text generation speed drops to about 22 tokens per second.

  • Garlic LLM official website
  • leonardo-alegria/garlic-llm

Advertising statement: External redirect links contained in the article (including but not limited to hyperlinks, QR codes, passwords, and other forms) are used to convey more information and save selection time. The results are for reference only. All articles on IT之家 containing external links include this statement.

The translation is machine-generated and individual terms may be off. Nothing is rewritten or cut; the original is authoritative.

Published by IT之家
This site only summarises and aggregates; copyright stays with the original author. Headlines and summaries follow the source — where they differ, the original wins.
Read the original ↗