ZY-S1: a coding model that runs on your CPU

ZY-S1 Preview is a small coding model you can run on an ordinary CPU. It is a 1.183B-parameter fine-tune of Sapient AI's HRM-Text-1B, released with 4-bit GGUF files and zy-engine, a small CPU-first engine that runs it correctly as a PrefixLM. No GPU is needed.

• Small: the recommended Q4_K_M file is 701 MiB, and runs in about 1.3 GB of RAM at the default 2048 context.
• Correct: zy-engine reads the prompt bidirectionally, the way the model was trained. On our test prompts its greedy Q4_K_M output matches the Transformers PrefixLM reference exactly.
• Open: Apache-2.0, same as HRM-Text-1B. A preview checkpoint, shared for research and experimentation.

What ZY-S1 is

ZY-S1 Preview builds on Sapient AI's HRM-Text-1B and keeps its HRM-Text architecture. It is a prefix language model: the prompt is read with full bidirectional attention, and only the response is generated left to right. HRM-Text runs its two 16-layer stacks several times per token (128 layer passes), so generation speed is mostly limited by memory bandwidth.

Two 4-bit builds

Q4_0 runs much faster on CPU, about 27 tok/s raw generation in llama-bench against 12–15 for Q4_K_M on the same 8-vCPU Xeon VM, because llama.cpp repacks Q4_0 weights into faster kernels. It is also noticeably less accurate. On 5 simple coding prompts compared against the full-precision model with greedy decoding, Q4_0 changed 2 of 5 answers (for example it lost the return in if n % 2 == 0: return True), while Q4_K_M matched all 5. Use Q4_K_M unless speed matters more than accuracy.

Why zy-engine exists

llama.cpp supports the hrm_text architecture, but runs it with causal attention only. The prompt is read the wrong way, and output quality suffers: JavaScript-style braces and repetition loops instead of Python. Every app built on stock llama.cpp (llama-cli, llama-server, Ollama, LM Studio) shares that limitation today.

zy-engine fixes it. It prefills the prompt in one non-causal batch, then decodes causally from the KV cache.

Version 0.2.0 adds prompt-lookup speculative decoding. It guesses upcoming tokens by matching recent token sequences against the prompt and the text generated so far, then checks the guesses in one batch. With greedy decoding the model still picks every token, so the guesses change speed, not output. Measured back to back on 10 short coding prompts, it made generation about 25% faster than v0.1. The gain is larger on repetitive output and smaller on very short answers.

CPU speed

8-vCPU Intel Xeon cloud VM, 8 threads, Q4_K_M, 49-token prompt, 64 generated tokens, median of 5 runs (zy-engine v0.1):

The Q4_0 bar is raw generation in llama-bench on the same VM, a different measurement from the two engine rows. It is shown for scale only.

The result: zy-engine gets the attention right at no speed cost.

Quick start

You need CMake ≥ 3.18 and a C++17 compiler. No GPU.

# build zy-engine (fetches llama.cpp @ e467e5b)
git clone https://huggingface.co/zyr-AGENT/ZY-S1-GGUF && cd ZY-S1-GGUF/zy-engine
cmake -S . -B build && cmake --build build -j --target zy-engine

# one prompt
./build/zy-engine -m ../ZY-S1-Q4_K_M.gguf -p "Write a function add(a, b) that returns the sum of two numbers."

# interactive chat
./build/zy-engine -m ../ZY-S1-Q4_K_M.gguf -i

# OpenAI-compatible API on :8080
./build/zy-engine -m ../ZY-S1-Q4_K_M.gguf --server --port 8080

The server exposes POST /v1/chat/completions and POST /v1/completions (both support "stream": true), plus GET /v1/models and GET /health. A Python package is included in zy-engine/ (not on PyPI yet):

pip install ./zy-engine
zy run -p "Write a function add(a, b) that returns the sum of two numbers."
zy chat
zy serve --port 8080

Prompt format

### Instruction:
Programming language: Python
Task:
Write a function add(a, b) that returns the sum of two numbers.

### Response:

zy-engine applies this automatically. --lang none omits the language line, and --raw sends text verbatim. Generation stops at EOS or at the next ### Instruction.

Early evaluation

The model card reports sample-based results from a custom PrefixLM-aware generation evaluation. They are not official full-benchmark leaderboard scores.

Limitations

• This is a preview: it can produce incorrect code, especially for anything beyond simple functions. Always test generated code.
• It may fail on multi-step math, or sound confident while being wrong.
• Q4_K_M loses a little quality against F16 on some prompts. Q4_0 loses more.
• On a busy machine, greedy output from llama.cpp's CPU backend can occasionally differ between runs on prompts where the model is unsure.

Try it

Chat with ZY-S1 live in your browser at zyr3.vercel.app, or download the GGUF files from huggingface.co/zyr-AGENT/ZY-S1-GGUF.

ZY-S1 builds on Sapient AI's HRM-Text-1B and was trained on a filtered subset of MnemicAI/Ling-Coder-SFT-English-Clean. Thanks to their authors and to the llama.cpp project.