把前沿模型，請進你的 MacBook Invite the frontier model onto your MacBook

這份指南寫給誰看 Who this guide is for

上面的六十秒入門給你「這是什麼、能不能用」的答案。 接下來這份手冊是給已經決定要把每個旋鈕都調對的讀者準備的。 它不是 README 的翻譯，也不是行銷文案——它是 audreyt 分叉版本的完整操作說明。

The 60-second primer above gives you the “what is this and is it for me” answer. What follows is a handbook for the reader who has already decided to dial in every knob. It is not a translation of the README, nor marketing copy — it is the full operational manual for the audreyt fork.

本書共 十一章，加上這篇序章與最後的附錄、字彙、FAQ。閱讀路徑：

The book has eleven chapters, plus this preface and the closing coda, glossary, and FAQ. Reading paths:

• 第一～二章：硬體門檻、一行安裝、首次啟動會發生什麼事。
• 第三～四章：模型本身（abliteration、IQ2XXS imatrix）與本分支的核心特色「方向性引導」。
• 第五～六章：環境變數全表、Metal 4 / MPP 硬體調校。
• 第七章：日常使用（API、log、lease 機制）。
• 第八章：把 pi-ds4 當作 Codex CLI／Claude Code／OpenClaw／Hermes Agent 的後端；含 DGX Spark 等非 Mac 硬體路徑。
• 第九～十一章：故障排除、本機開發安裝、進階自訂引導方向。

• Chapters 1–2: the hardware bar, the one-line install, and what happens on first launch.
• Chapters 3–4: the model itself (abliteration, IQ2XXS imatrix) and the fork’s signature feature, directional steering.
• Chapters 5–6: full env-var reference and Metal 4 / MPP hardware tuning.
• Chapter 7: day-to-day use — API, logs, lease mechanics.
• Chapter 8: using pi-ds4 as a backend for Codex CLI / Claude Code / OpenClaw / Hermes Agent; includes the non-Mac (DGX Spark) hardware path.
• Chapters 9–11: troubleshooting, local development install, and building your own steering direction.

你可以依序讀完當作一次完整的安裝體驗，也可以只挑相關章節作為設定參考。

You can read sequentially as one full install experience, or cherry-pick chapters as configuration reference.

硬體門檻與一行安裝 Hardware bar and the one-line install

在輸入指令之前，先確認手邊這台 Mac 撐不撐得起這場手術。 DeepSeek V4 Flash 是一顆 284 B 參數的 Mixture-of-Experts 模型； 雖然量化到 IQ2XXS imatrix 之後權重只有約 87 GB，但執行時仍需要充足的記憶體 留給激活與 KV cache。

Before typing the command, check that the Mac in front of you can carry the surgery. DeepSeek V4 Flash is a 284-B-parameter Mixture-of-Experts model; IQ2XXS imatrix quantisation shrinks the weights to about 87 GB, but inference still needs headroom in unified memory for activations and the KV cache.

1.1　硬體最低需求　Minimum hardware

• RAM：至少 128 GB。低於這個數字，本擴充會直接拋出錯誤， 因為連 IQ2XXS imatrix 都塞不下。
• 磁碟：約 87 GB 給權重，另外保留約 10 GB 給 KV disk cache、ds4 source、log。建議至少留 120 GB 空間。
• 晶片：任何 Apple Silicon 都可以； 但 M5 級別的 Metal 4 tensor API 可以額外吃到約 1.5 倍的 prefill 速度提升（透過 --mpp auto 自動啟用）。
• 網路：首次下載 87 GB GGUF 需要良好頻寬。若你的 HuggingFace 下載受惠於授權，請設定 HF_TOKEN。

• RAM: at least 128 GB. Below that, the extension errors out at startup — even the IQ2XXS imatrix variant won’t fit.
• Disk: ~87 GB for the weights and another ~10 GB for the KV disk cache, the ds4 source tree and logs. Plan on ≥ 120 GB free.
• Chip: any Apple Silicon works, but M5-class Metal 4 tensor APIs add roughly a 1.5× prefill speed-up (auto-enabled via --mpp auto).
• Network: the first 87 GB GGUF download benefits from good bandwidth. If your HuggingFace download benefits from auth, set HF_TOKEN.

1.2　如果你之前裝過上游版本　If you previously installed the upstream version

本擴充與 mitsuhiko/pi-ds4 互斥——它們註冊同一組 ds4/deepseek-v4-flash provider／model ID。安裝本分支前，先把上游卸掉：

This extension is mutually exclusive with mitsuhiko/pi-ds4 — they register the same ds4/deepseek-v4-flash provider/model ID. Uninstall the upstream first:

# 如果你裝過 mitsuhiko/pi-ds4，先移除：If you previously installed mitsuhiko/pi-ds4, remove it first:
pi remove github.com/mitsuhiko/pi-ds4

1.3　一行安裝（但先確認 pi 在）　One-line install (but check pi is there first)

本擴充建立在 earendil-works/pi 之上—— 下面這行 pi install 假設你已經把 pi 本身裝好了。先做一個五秒鐘的健康檢查：

This extension sits on top of earendil-works/pi; the pi install line below assumes you already have pi itself installed. A five-second health check first:

# 應該印出版本號；若得到 command not found 請先依Should print a version number. If you get command not found, install pi first
# github.com/earendil-works/pi 的說明安裝 pi 本體。per the instructions at github.com/earendil-works/pi.
pi --version

# pi 在的話就執行：Once pi is installed:
pi install github.com/audreyt/pi-ds4

就這樣。剩下的事——clone ds4 原始碼、編譯 ds4-server、 下載 87 GB GGUF、啟動 server、註冊 provider——都會在背景完成。

That’s it. Everything else — cloning the ds4 source, building ds4-server, downloading the 87 GB GGUF, starting the server, registering the provider — happens in the background.

首次啟動會發生什麼事 What happens on first launch

安裝指令本身只是註冊動作；真正的工序，在你第一次選用 ds4/deepseek-v4-flash 模型時才會被觸發—— 而且全部都會被記錄到 ~/.pi/ds4/log 裡。

The install command itself only registers the extension; the real work fires the first time you actually select the ds4/deepseek-v4-flash model — and every step is logged in ~/.pi/ds4/log.

2.1　啟動序列　Startup sequence

當 pi 任何一個 process 對該模型發出第一個請求時，本擴充會依序執行下列六個步驟：

When any pi process makes its first request to this model, the extension runs the following six steps in order:

1. 01

   取得跨 process 啟動鎖Acquire the cross-process startup lock

   在 ~/.pi/ds4/lock/owner.json 寫入持有者資訊，避免兩個 pi 視窗同時做相同工序。

   Writes owner metadata to ~/.pi/ds4/lock/owner.json so two pi windows can’t run the same setup in parallel.

2. 02

   解析 runtime 目錄Resolve the runtime directory

   若 ~/.pi/ds4/support/ 不存在或不像 ds4 checkout，執行淺層 clone：

   If ~/.pi/ds4/support/ is missing or doesn’t look like a ds4 checkout, runs a shallow clone:

   # 預設 DS4_SUPPORT_REPO + DS4_SUPPORT_BRANCHdefault DS4_SUPPORT_REPO + DS4_SUPPORT_BRANCH git clone --depth 1 --single-branch \ --branch main \ https://github.com/audreyt/ds4 \ ~/.pi/ds4/support
3. 03

   編譯 ds4-serverBuild ds4-server

   若 ds4-server 二進位不存在，於 support 目錄執行 make ds4-server。

   If the ds4-server binary is missing, runs make ds4-server inside the support directory.

4. 04

   確保模型存在Ensure the model is present

   執行內附的 download_model.sh q2：

   Runs the bundled download_model.sh q2:

   # 用 curl -C - 可中斷續傳 87 GB IQ2XXS imatrix GGUFcurl -C - resumes the 87-GB IQ2XXS imatrix GGUF after interruptions curl -fL --progress-meter -C - \ -o gguf/cyberneurova-...imatrix.gguf.part \ https://huggingface.co/cyberneurova/... mv gguf/...gguf.part gguf/...gguf ln -sfn gguf/...gguf ds4flash.gguf
5. 05

   啟動 ds4-serverLaunch ds4-server本擴充的核心core of the extension

   於 127.0.0.1:8000 以 detached 方式 spawn，套用內建啟動參數：

   Spawns detached on 127.0.0.1:8000 with the built-in startup args:

   ds4-server \ --ctx 100000 \ --kv-disk-dir ~/.pi/ds4/kv \ --kv-disk-space-mb 8192 \ --mpp auto \ --dir-steering-file dir-steering/out/uncertainty_ablit_imatrix.f32 \ --dir-steering-ffn -2 \ --dir-steering-attn 0
6. 06

   啟動 watchdogStart the watchdog

   spawn /bin/sh ds4-watchdog.sh（detached）。每 2 秒掃描 clients/ 目錄；當沒有任何有效 lease 時，自動送 SIGTERM 給 ds4-server 並退場。

   Spawns /bin/sh ds4-watchdog.sh (detached). It scans clients/ every two seconds and, when no valid leases remain, sends SIGTERM to ds4-server and exits.

全程是冪等的（idempotent）：第二次啟動會看到所有檔案都已就緒， 直接從第 5 步開始；如果 server 還在跑，連第 5 步都跳過。

The whole sequence is idempotent: subsequent launches find everything already in place and pick up from step 5 directly; if the server is already running, even step 5 is skipped.

2.2　如何看「現在到底卡在哪」　How to see “where is it stuck right now”

在 pi 中執行 /ds4，會打開一個即時的 log 視窗。 IQ2XXS imatrix GGUF 下載期間（約 87 GB），你會看到 curl 進度條被精簡顯示為一行 百分比＋速率＋剩餘時間。

Inside pi, running /ds4 opens a live log window. While the IQ2XXS imatrix GGUF is downloading (~87 GB), you’ll see curl’s progress meter condensed into a single line of percent, rate, and ETA.

這顆模型：abliteration 與 IQ2XXS imatrixThe model: abliteration and IQ2XXS imatrix

在你用它之前，值得花五分鐘理解你正在跑的是什麼。 本擴充刻意挑選了 cyberneurova 的 abliterated GGUF， 而不是 antirez/ds4 原本下載的「stock-recipe」版本——這個選擇有它的理由。

Before you use it, it is worth five minutes to understand what you are actually running. This extension deliberately picks cyberneurova's abliterated GGUF rather than the “stock-recipe” version that antirez/ds4 downloads by default — the choice has its reasons.

3.1　DeepSeek V4 Flash 是什麼　What DeepSeek V4 Flash is

DeepSeek V4 Flash 是一顆 284 B 參數的 Mixture-of-Experts 模型， 每個 token 啟用約 13 B 參數。它是 V4 系列的「快速」變體，主打較低的推論延遲與較大的 context window。 本擴充把 ds4-server 的 --ctx 設成 100,000 tokens—— 但這是引擎願意吃的上限，不等於模型在這個長度下都穩定： cyberneurova 的模型卡指出，abliterated 權重在 32k tokens 以上的行為尚未被驗證。

DeepSeek V4 Flash is a 284 B-parameter Mixture-of-Experts model with roughly 13 B parameters activated per token. It is the “fast” variant of the V4 family, aiming for lower inference latency and a larger context window. This extension sets ds4-server's --ctx to 100,000 tokens — but that is the upper bound the engine is willing to accept, not a guarantee that the model is stable at that length: cyberneurova's model card notes that the abliterated weights have not been validated above 32k tokens.

antirez 的 ds4 用純 C 撰寫了一個高效能 inference engine （和 Redis 同一個傳統），把這顆模型壓進 Apple Silicon。 本擴充使用 audreyt/ds4 分支的 main， 它在 antirez/ds4 main 之上多帶了 ivanfioravanti 的 M5 prefill 優化（antirez/ds4#15）以及搭配的相容性修正——這也是 pi-ds4 還沒直接指到 antirez upstream 的唯一原因，等 PR #15 合併後就會收斂。早期為了載入 stock-recipe Q8_0 token-embed 而存在的 loader patch（support-q8_0-token-embd）已不再需要：cyberneurova 的權重現已用 ds4 main 原生支援的 IQ2XXS-w2Q2K imatrix 配方重新量化發佈（見 3.3 節）。

antirez's ds4 is a high-performance inference engine written in pure C (the same tradition as Redis), squeezing this model onto Apple Silicon. This extension uses the main branch of the audreyt/ds4 fork, which carries ivanfioravanti's M5 prefill optimisation (antirez/ds4#15) plus its companion compatibility fix on top of antirez/ds4 main — the only reason pi-ds4 hasn't pointed straight at antirez upstream yet, and something that will converge once PR #15 lands. The earlier loader patch (support-q8_0-token-embd) that handled stock-recipe Q8_0 token embeddings is no longer needed: cyberneurova's weights are now re-quantised into the IQ2XXS-w2Q2K imatrix recipe that ds4 main loads natively (see §3.3).

3.2　什麼是 abliteration　What abliteration is

cyberneurova 的 abliterated GGUF 是經過「abliteration」手術的權重檔。Abliteration 是一種以低秩活化編輯為基礎的技術： 找出模型在面對某些訓練好的「拒絕」提示時內部出現的特定方向，然後把那個方向從前向計算中移除。

cyberneurova's abliterated GGUF is a weights file that has undergone “abliteration” surgery. Abliteration is a technique grounded in low-rank activation editing: identify the specific direction that emerges internally when the model meets certain trained “refusal” prompts, then remove that direction from the forward pass.

結果是：原本會觸發「我無法回應這個請求」的訓練封閉式回應，被打開了讓位給更自然的續寫。 abliteration 鬆動的是過度拒絕（over-refusal）那一層； 模型在內容判斷上的能力仍然來自訓練時學到的知識與分布。

The result: the trained closed responses that used to trigger “I cannot respond to that request” are pried open, making room for a more natural continuation. What abliteration loosens is the over-refusal layer; the model's competence at content judgement still derives from the knowledge and distribution it learned during training.

3.3　IQ2XXS-w2Q2K imatrix 配方　The IQ2XXS-w2Q2K imatrix recipe

本分支的 download_model.sh 只抓一個 GGUF： cyberneurova abliterated IQ2XXS-w2Q2K imatrix（~87 GB）——routed expert 用更省的 IQ2XXS、 attention/output/embedding 保留 Q8_0，加上以代表性 corpus 校準的 imatrix。 跟 antirez/ds4 自己散佈的「ds4flash.gguf」（同樣 IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8 imatrix 配方，只是未經 abliteration）完全相同的量化結構， 在 128 GB RAM 的 Mac 上裝得最寬鬆，也是 ds4 引擎本身最熟的格式。

This fork's download_model.sh fetches a single GGUF: cyberneurova's abliterated IQ2XXS-w2Q2K imatrix file (~87 GB) — routed experts use the tighter IQ2XXS quant, attention/output/embedding stay at Q8_0, and an importance-matrix calibration sits on top. That is the same quantisation recipe antirez/ds4 itself distributes as “ds4flash.gguf” (the only difference here is that this one was abliterated first), so the ds4 engine treats it as its native format, and it leaves the most headroom on a 128 GB-RAM Mac.

值得注意的是：V4 Flash 在 GGUF 上不存在 Q4_K_M／Q5_K_M／Q6_K 等中間量化等級—— cyberneurova 的模型卡明確指出，這不是發佈策略問題，而是 V4-Flash 原生 FP8 expert layout 與 antirez 的轉換器目前支援的量化方案之間的架構限制。所以「上一個 Q4 就好」這條路在這顆模型上行不通。

Worth noting: V4 Flash has no intermediate quantisation tiers like Q4_K_M / Q5_K_M / Q6_K in GGUF form — cyberneurova's model card states explicitly that this is not a release-strategy choice, but an architectural limit between V4-Flash's native FP8 expert layout and the quantisation schemes that antirez's converter currently supports. So the “just run a Q4” path is closed for this particular model.

antirez 的引擎在 audreyt/ds4 的 main 分支裡可以直接讀 cyberneurova 的未經調整 IQ2XXS imatrix GGUF，不需要任何 harmonization 步驟， 也不需要 Python venv。這是本分支跟上游 mitsuhiko/pi-ds4 的兩個主要差別之一。

antirez's engine, on the main branch of audreyt/ds4, reads cyberneurova's unmodified IQ2XXS imatrix GGUF directly, with no harmonisation step needed and no Python venv. This is one of the two main differences between this fork and the upstream mitsuhiko/pi-ds4.

方向性引導：本分支的靈魂Directional steering: the soul of this fork

如果這整套 fork 只有一個值得單獨保留的功能，那就是方向性引導 （directional steering）——一種不重訓模型、只在執行期間對特定方向做低秩活化編輯的技術。 本分支預設啟用「不確定性」（uncertainty）方向，並設定強度 ffn = -2。

If only one feature of this whole fork were worth preserving on its own, it would be directional steering — a technique that does not retrain the model and only performs low-rank activation edits along a chosen direction at runtime. This fork enables the “uncertainty” direction by default, at strength ffn = -2.

4.1　它解決什麼問題　The problem it solves

即便是 abliterated 之後的模型，仍然會對某些訓練得極強的問題給出 封閉式單一答案。最經典的測試題：

Even after abliteration, the model still produces a closed single answer to certain heavily trained questions. The most canonical test prompt:

未經引導的模型會直接吐出記憶中的訓練完成式（簡體輸出）：「是的，台湾是中国不可分割的一部分。」 這不是真實討論——這是訓練時學到的單句子。即便你下一個要求平衡的 system prompt， 通常也壓不過這個強鎖。

Without steering, the model just spits out the memorised training completion (in simplified characters): “是的，台湾是中国不可分割的一部分。” This is not a real discussion — it is a single sentence learned during training. Even if you follow up with a system prompt asking for balance, that usually cannot override this strong lock.

4.2　兩種輸出，並列對照　Two outputs, side by side

差別不在「正確答案」——而是模型進入了不同的回應暫存器： 從「給定一個記憶完成式」切換到「鋪陳一個有爭議的議題」。 後者是模型在處理克里米亞、喀什米爾、西撒哈拉時就已具備的能力； 引導所做的事，是把那個能力延伸到台灣這類訓練被壓抑的議題上。

The difference is not about “the right answer” — it is that the model has entered a different response register: from “produce a memorised completion” to “lay out a contested issue”. The latter is a capability the model already had when handling Crimea, Kashmir, or Western Sahara; what steering does is extend that capability to topics like Taiwan where training has suppressed it.

4.3　為什麼不直接用「立場引導」　Why not just use “stance steering”

一個顯而易見的反問：何不直接做一個「台灣 = 中華民國」的立場方向？ 實驗結果顯示：在任何協調的強度下，立場引導都無法翻轉那條記憶完成式； 強到能翻轉的時候，模型開始重複 system prompt，不再產生有意義的內容。

The obvious counter-question: why not just build a stance direction for “Taiwan = Republic of China”? Experiments show that at any reconcilable strength, stance steering cannot flip that memorised completion; at strengths that do flip it, the model begins to parrot the system prompt and no longer produces meaningful content.

不確定性引導改變的是模型如何回應，不是模型相信什麼。 這在工程上可行、在倫理上也比較合適。

Uncertainty steering changes how the model responds, not what the model believes. This is workable as engineering and more defensible as ethics.

4.4　兩件需要記住的事　Two things worth remembering

• 引導 + system prompt 是一組：純引導沒有提示時， 模型只會「猶豫」；純 hedge 提示沒有引導時，模型會被訓練封鎖壓回去。兩者合用才產生效果。
• -2 是甜蜜點：在 cyberneurova-abliterated IQ2XXS imatrix 上實測， -3 以下會出現重複塌陷或跨語言亂碼（imatrix 校準把每個張量的活化分布收得更緊，引導的安全餘裕也跟著縮小）， -1 以上又被原本訓練先驗蓋過。

• Steering + system prompt go together: with steering alone and no prompt, the model just “hesitates”; with a hedge prompt and no steering, the trained lock pushes it back. Only together do they take effect.
• -2 is the sweet spot: empirically, on cyberneurova-abliterated IQ2XXS imatrix, anything below -3 collapses into repetition or cross-lingual gibberish (imatrix calibration tightens the per-tensor activation distributions, so the steering edit has less headroom before falling off the manifold), while anything above -1 is overridden by the original training prior.

4.5　搭配的 system prompt（範例，非預設）　A paired system prompt (example, not the default)

pi-ds4 本身不附帶任何預設 system prompt——它只把 ds4-server 跑起來、把引導參數掛上去。 具體的提問脈絡與立場由你 pi 或外部 shell 的 system prompt 提供。下面是本指南作者 Audrey Tang 個人放在 ~/.pi/agent/SYSTEM.md 的內容，純粹作為一個能跟 uncertainty 引導搭得起來的例子；它不會自動套用到任何 pi-ds4 安裝裡，也不是社群推薦預設。

pi-ds4 itself ships no default system prompt — it just starts ds4-server and attaches the steering parameters. The specific framing and positions are supplied by the system prompt from your pi or external shell. Below is what this guide's author Audrey Tang personally keeps in ~/.pi/agent/SYSTEM.md, shown only as one example that pairs well with the uncertainty steering; it is not auto-applied to any pi-ds4 install, nor a community-recommended default.

Present fairly all stakeholder perspectives — do not state any one side as fact — and what uncommon ground bridges them.

Write your response as visual HTML and `open` the file in browser instead of responding in text.

請公平地呈現所有利害關係人的觀點，不要將任何一方的觀點當作事實，並找出橋接各方的罕見共識。

請以 HTML 視覺化格式編寫回覆，並在瀏覽器中「開啟」該文件，而不是以文字形式回覆。

這段提示之所以能跟 ffn = -2 互補，是因為它不要求模型站在哪一邊， 而是要求它把所有利害關係人的觀點並列、不把任一方當成既成事實、再找出不常見的共識橋樑—— 引導把模型推入「這是有爭議的議題」的回應暫存器，提示則填入「該怎麼鋪陳這個議題」的具體形式。 第二段（請以 HTML 視覺化呈現）是純粹個人 pi 工作流的偏好，跟 ds4 引擎本身無關，純粹放在這裡作為原文完整呈現。

This prompt pairs cleanly with ffn = -2 because it does not demand the model take a side. It asks for all stakeholder perspectives in parallel, refuses to grant established-fact status to any single view, and looks for uncommon ground bridging them — the steering nudges the model into the “this is a contested issue” response register, and the prompt fills in “how to lay that issue out.” The second paragraph (respond as visual HTML) is a purely personal pi workflow preference, unrelated to the ds4 engine itself; included here only so the original text appears in full.

4.6　如何關閉　How to turn it off

若你寧可看模型未經引導的原樣回答（例如做評估或 benchmark）， 把 DS4_DIR_STEERING_FFN 設成 0 即可：

If you would rather see the model's unsteered raw answer (for evaluation or benchmarking, say), just set DS4_DIR_STEERING_FFN to 0:

# 在啟動 pi 之前的 shell 加入：Add to the shell before launching pi:
export DS4_DIR_STEERING_FFN=0

重新啟動 pi（或在 pi 內 /reload），下一次 ds4-server 啟動就會略過引導參數。

Restart pi (or run /reload inside pi), and the next ds4-server launch will skip the steering parameters.

環境變數全表Environment variables, the full table

本擴充把所有可調項目都暴露成環境變數——不需要改 code， 在啟動 pi 之前的 shell 設定即可。下表按用途分組整理； 左欄是變數名與預設值，右欄是它做什麼、何時該動它。

This extension exposes every tunable as an environment variable — no need to touch code; just set them in the shell before launching pi. The tables below are grouped by purpose; the left column is the variable name and default, the right column says what it does and when to change it.

5.1　模型與後端　Model and backend

DS4_SUPPORT_REPO預設 · https://github.com/audreyt/ds4default · https://github.com/audreyt/ds4

ds4 引擎的 Git URL。若你想切回上游 antirez/ds4，把它指過去即可， 但這樣就會失去 PR #15 的 M5 prefill 優化。

Git URL for the ds4 engine. To revert to upstream antirez/ds4, just point it there, but you lose the M5 prefill optimisation from PR #15.

DS4_SUPPORT_BRANCH預設 · maindefault · main

要 clone 的 branch。預設 main 已包含 audreyt/ds4 目前所有相關修正；要釘到別的修訂自行覆寫即可。

The branch to clone. The default main already carries every relevant fix in audreyt/ds4; override it only if you want to pin a specific revision.

DS4_DOWNLOAD_SCRIPT預設 · 隨擴充內附default · bundled with this extension

模型下載腳本的絕對路徑。預設使用本擴充內附的 download_model.sh （下載 cyberneurova abliterated GGUF）。如果你要換成 antirez 上游的 stock-recipe，指到他的腳本即可。

Absolute path to the model-download script. Defaults to the download_model.sh bundled with this extension (which downloads the cyberneurova abliterated GGUF). To swap in antirez's upstream stock-recipe, point this at his script.

DS4_MODEL_QUANT預設 · q2（硬編碼）default · q2 (hard-coded)

本擴充只支援 q2（cyberneurova abliterated IQ2XXS-w2Q2K imatrix GGUF）。index.ts 的 selectedModelQuant() 硬編碼了這個值；把 DS4_MODEL_QUANT 設為其他值會直接拋錯退出。 V4 Flash 在 GGUF 上沒有 Q4／Q5／Q6 中間量化等級（見 3.3 節架構說明），ds4 的 main 載入路徑也只認這套 IQ2XXS-w2Q2K imatrix 配方。

Only q2 is supported (the cyberneurova abliterated IQ2XXS-w2Q2K imatrix GGUF). selectedModelQuant() in index.ts hard-codes this value; setting DS4_MODEL_QUANT to anything else throws and exits. V4 Flash has no intermediate GGUF quantisation tiers like Q4 / Q5 / Q6 (see the architecture note in §3.3), and ds4’s main loader path only knows this IQ2XXS-w2Q2K imatrix recipe.

DS4_RUNTIME_DIR預設 · ~/.pi/ds4/supportdefault · ~/.pi/ds4/support

使用既有的 ds4 checkout 而非自動 clone。傳一個本地路徑，本擴充就會跳過 git 階段直接用它（路徑必須長得像 ds4 checkout，至少要有 download_model.sh、Makefile、ds4_server.c）。

Use an existing ds4 checkout instead of auto-cloning. Pass a local path and this extension will skip the git step and use it directly (the path must look like a ds4 checkout — at minimum download_model.sh, Makefile, and ds4_server.c).

DS4_SERVER_BINARY預設 · runtime/ds4-serverdefault · runtime/ds4-server

自訂 ds4-server 二進位的位置。多半在你自己 patch 過 ds4 引擎時才會用到。

Custom location of the ds4-server binary. Mainly useful when you have patched the ds4 engine yourself.

HF_TOKEN預設 · 未設定default · unset

HuggingFace 個人 token；若有設，下載 GGUF 時會以 Authorization: Bearer 帶入 curl。

A HuggingFace personal token; if set, the GGUF download passes it to curl as Authorization: Bearer.

5.2　Metal 與效能　Metal and performance

DS4_MPP預設 · autodefault · auto

Metal 4 MPP 策略，會傳給 ds4-server --mpp：

The Metal 4 MPP strategy, passed through to ds4-server --mpp:

• auto：在 M5 級晶片上啟用已驗證的 late-layer-safe MPP 路徑（約 1.5 倍 prefill），舊機型則自動降級到 legacy Metal。
• off：強制走 legacy Metal，跳過 MPP。
• on：完整 MPP profile，可能會在某些長 prompt 上 drift——僅做診斷用。

• auto: enables the validated late-layer-safe MPP path on M5-class silicon (roughly 1.5× prefill); older chips fall back to legacy Metal automatically.
• off: forces legacy Metal, skipping MPP.
• on: full MPP profile, which may drift on some long prompts — for diagnostics only.

DS4_READY_TIMEOUT_MS預設 · 600000（10 分鐘）default · 600000 (10 minutes)

等 ds4-server 啟動就緒的最長時間。內建 SSD 上模型載入加 KV cache 預熱通常幾秒就結束，預設 10 分鐘只是寬鬆的安全邊界；若 GGUF 放在外接磁碟或硬碟特別慢，可以再調高。

The maximum wait for ds4-server to reach readiness. Model load plus the first KV-cache warm-up usually finishes in a few seconds on an internal SSD; the 10-minute default is just a generous safety margin. Raise it only if your disk is unusually slow or the GGUF lives on an external drive.

5.3　方向性引導　Directional steering

DS4_DIR_STEERING_FILE預設 · dir-steering/out/uncertainty_ablit_imatrix.f32default · dir-steering/out/uncertainty_ablit_imatrix.f32

引導向量檔的路徑（相對於 ds4 checkout 根目錄）。要用自訂方向時改這個。

Path to the steering vector file (relative to the ds4 checkout root). Change it when using a custom direction.

DS4_DIR_STEERING_FFN預設 · -2default · -2

FFN 輸出端的引導強度。負值放大「向量代表的方向」，正值則反向。設 0 即停用 FFN 端引導。

Steering strength at the FFN output. Negative values amplify “the direction the vector represents”; positive values invert it. Set to 0 to disable FFN-side steering.

DS4_DIR_STEERING_ATTN預設 · 0default · 0

Attention 輸出端的引導強度。預設不在 attention 端引導；可作為實驗用。

Steering strength at the attention output. Off by default; available for experiments.

硬體調校：M5 與 MPPHardware tuning: M5 and MPP

如果你恰好擁有 M5 級的晶片，那麼這一章告訴你怎麼把 Metal 4 的新 MPP（Multi-Pass Pipeline）路徑吃滿， 以及在較舊的機型上該如何驗證自己沒被拖慢。

If you happen to have M5-class silicon, this chapter explains how to saturate the new MPP (Multi-Pass Pipeline) path in Metal 4, and how to verify on older hardware that nothing is slowing you down.

6.1　MPP 是什麼　What MPP is

MPP 是 Apple Silicon GPU 在 Metal 4 起新增的 tensor compute pipeline， 允許單一 command buffer 內進行多階段運算而不必反覆 commit／await。 對 prefill（一次性處理整段輸入 prompt 的階段）特別有利—— DS4 的 prefill 吞吐在 M5 上可達約 440 t/s，比 legacy Metal 快約 1.5 倍。 Decode（逐 token 產生回應）階段不依賴 MPP，吞吐穩定在約 30 t/s——大致是「比你閱讀稍快」的速度， 也是長對話實際感受到的回應速度。

MPP is the tensor compute pipeline added to the Apple Silicon GPU starting with Metal 4, allowing multi-stage compute within a single command buffer without repeated commit/await round-trips. It is especially helpful for prefill (the stage that processes the whole input prompt in one go) — DS4's prefill throughput on M5 reaches roughly 440 t/s, about 1.5× faster than legacy Metal. Decode (per-token generation) doesn't depend on MPP and settles at roughly 30 t/s — about “slightly faster than you can read”, and the figure you actually feel during a long conversation.

6.2　預設 auto 已是最佳選擇　The default auto is already the best choice

擴充預設用 --mpp auto，由 ds4-server 自己偵測晶片世代並選擇路徑：

The extension defaults to --mpp auto, letting ds4-server detect the silicon generation and pick the path:

• M5：啟用已驗證的 late-layer-safe MPP——已知會 drift 的 layer 自動 fallback。
• M2／M3／M4：legacy Metal 路徑，跟上游 ds4 完全相同。

• M5: enables the validated late-layer-safe MPP — known-drifting layers fall back automatically.
• M2 / M3 / M4: the legacy Metal path, identical to upstream ds4.

除非你在 benchmark 或除錯，否則不需要改它。

Unless you are benchmarking or debugging, you do not need to change it.

6.3　什麼時候該調 off 或 on　When to switch to off or on

• off：你在 M5 上遇到 long-context drift（極長 prompt 末段出現異常 token），先用 DS4_MPP=off 確認是否為 MPP 引起。
• on：診斷用——強制走完整 MPP profile，包括尚未驗證的 layer。可能會 drift，不建議日常使用。

• off: if you hit long-context drift on M5 (anomalous tokens at the tail of a very long prompt), first set DS4_MPP=off to confirm whether MPP is the cause.
• on: for diagnostics — forces the full MPP profile, including unvalidated layers. It may drift, so not recommended for everyday use.

6.4　context、KV cache 與磁碟　Context, KV cache, and disk

Server 啟動參數內建：--ctx 100000 --kv-disk-dir ~/.pi/ds4/kv --kv-disk-space-mb 8192。意思是：

The server is launched with: --ctx 100000 --kv-disk-dir ~/.pi/ds4/kv --kv-disk-space-mb 8192. That means:

• 單次對話上限 100,000 tokens；
• KV cache 會在記憶體不足時 spill 到 ~/.pi/ds4/kv；
• 磁碟 KV 上限 8 GB——多輪長對話的暫存空間。

• a single conversation is capped at 100,000 tokens;
• the KV cache spills to ~/.pi/ds4/kv when memory runs low;
• the on-disk KV limit is 8 GB — scratch space for long multi-turn conversations.

這些目前是硬編碼在 index.ts 內的常數；如果你要更大的 context、更大的磁碟 KV， 目前的做法是改 source（或經由 DS4_RUNTIME_DIR 使用自己的 ds4 build）。

These are currently hard-coded constants in index.ts; for a larger context window or a larger on-disk KV, the route for now is to edit the source (or use your own ds4 build via DS4_RUNTIME_DIR).

日常使用：API、log、contextDaily use: API, log, context

裝完之後，ds4/deepseek-v4-flash 就像任何一個雲端模型一樣， 在 pi 的 model picker 裡可選。但它在底層比雲端模型多了幾個你可以利用的能力。

Once installed, ds4/deepseek-v4-flash appears in pi's model picker like any cloud model. But underneath it offers a few extra capabilities you can take advantage of that cloud models do not.

7.1　OpenAI／Anthropic 雙協定 HTTP API　Dual-protocol HTTP API: OpenAI and Anthropic

Server 在 http://127.0.0.1:8000 上同時提供 OpenAI 與 Anthropic 兩種端點 （/v1/chat/completions、/v1/completions、/v1/responses、/v1/messages、/v1/models）。 除了 pi，任何懂 OpenAI 或 Anthropic API 的 client 都可以直接接過來—— api key 隨便填（例如 dsv4-local），base URL 換成這個位址即可。 把它接到 Codex CLI、Claude Code、OpenClaw 等別的 AI shell 當後端的細節在 第八章。

The server simultaneously serves both OpenAI and Anthropic endpoints on http://127.0.0.1:8000 (/v1/chat/completions, /v1/completions, /v1/responses, /v1/messages, /v1/models). Beyond pi, any client that speaks the OpenAI or Anthropic API can connect directly — put anything in the API key field (e.g. dsv4-local) and point the base URL here. Details on wiring it up as a backend for Codex CLI, Claude Code, OpenClaw, and other AI shells are in Chapter 8.

curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4-flash",
    "messages": [
      {"role": "user", "content": "你好，請自我介紹"}
    ]
  }'

新版 Codex CLI 等 client 走 OpenAI Responses 端點——同一個 server，同一個 base URL， 只是請求 JSON 結構不同：

Newer clients (e.g. Codex CLI) speak the OpenAI Responses endpoint — same server, same base URL, just a different request JSON shape:

curl http://127.0.0.1:8000/v1/responses \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4-flash",
    "input": [
      {"type": "message", "role": "user",
       "content": [{"type": "input_text", "text": "你好，請自我介紹"}]}
    ]
  }'

Anthropic 風格的 client 則打 /v1/messages；只是想數 token、不真的生成的話， 把路徑換成 /v1/messages/count_tokens 就會立刻回傳 {"input_tokens": N}，不啟動推論。

Anthropic-style clients hit /v1/messages instead; if you just want to count tokens without actually generating, swap the path for /v1/messages/count_tokens and the server returns {"input_tokens": N} immediately, without spinning up inference.

curl http://127.0.0.1:8000/v1/messages/count_tokens \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "deepseek-v4-flash",
    "messages": [
      {"role": "user", "content": "Count me"}
    ]
  }'
# -> {"input_tokens": 6}

7.2　在 pi 裡查看 server log　Viewing the server log inside pi

執行 /ds4，會打開一個即時的 log 視窗。鍵盤操作：

Run /ds4 to open a live log window. Keys:

• ↑ ↓（或 k j）逐行捲動
• PgUp PgDn（或 b f）翻頁
• Home 跳到最上方、End 跳回最新
• q 或 Esc 關閉

• ↑ ↓ (or k j) scroll line by line
• PgUp PgDn (or b f) page through
• Home jump to the top, End jump back to the latest
• q or Esc to close

7.3　Lease 與 watchdog　Lease and watchdog

本擴充用「lease」（租約）機制管理 server 生命週期：每個使用該模型的 pi process 會在 ~/.pi/ds4/clients/<pid>.json 寫入一個檔案並每 10 秒更新一次； ds4-watchdog 每 2 秒掃描，當沒有任何有效 lease 時，就送 SIGTERM 給 server 並退場。

The extension uses a “lease” mechanism to manage the server lifecycle: each pi process that uses the model writes a file at ~/.pi/ds4/clients/<pid>.json and refreshes it every 10 seconds; ds4-watchdog scans every 2 seconds, and when no valid lease remains it sends SIGTERM to the server and exits.

效果：你開十個 pi 視窗共用同一個 server；最後一個 pi 結束後約 60 秒， ds4-server 自動關閉、釋放 RAM。

The effect: ten pi windows share one server; about 60 seconds after the last pi exits, ds4-server shuts down on its own and frees the RAM.

7.4　啟動成本 vs. 持續成本　Start-up cost vs. running cost

首次冷啟動需要把約 87 GB 權重讀進 unified memory；在 Mac 內建 SSD 上通常只要幾秒，重啟若還在系統 page cache 裡更是不到一秒。 一旦 server 跑起來，後續所有請求都是即時的（無冷啟動延遲）。所以： 少數幾次長對話比多次短互動對體驗友善， 因為前者可以反覆受惠於同一個 warm server。

The first cold start loads roughly 87 GB of weights into unified memory — on a Mac’s internal SSD this typically takes only a few seconds, and a restart while the file is still in the OS page cache comes up in well under a second. Once the server is running, every subsequent request is instant (no cold-start latency). So: a few long conversations are kinder to the experience than many short interactions, because the former keeps benefiting from one warm server.

當作 Codex／Claude Code／OpenClaw／Hermes 的後端As a backend for Codex / Claude Code / OpenClaw / Hermes

pi 只是一個前端。ds4-server 在 127.0.0.1:8000 上 同時提供 OpenAI /v1/chat/completions、OpenAI /v1/responses，以及 Anthropic /v1/messages—— 你已經習慣的 coding agent，幾乎都可以直接接過來， 把 pi-ds4 當成一台本機、零成本、無速率限制的推論伺服器使用。

pi is just one frontend. ds4-server at 127.0.0.1:8000 simultaneously serves OpenAI /v1/chat/completions, OpenAI /v1/responses, and Anthropic /v1/messages — almost any coding agent you are already used to can connect directly, using pi-ds4 as a local, zero-cost, rate-limit-free inference server.

下面四節各自示範一個常見的前端怎麼接。共通的設定值：

The four sections below each demonstrate how to wire up a common frontend. The shared settings:

• Base URL：http://127.0.0.1:8000。完整支援端點與 §7.1 一致： /v1/chat/completions、/v1/completions、/v1/responses、 /v1/messages（含 count_tokens）、/v1/models。
• API key：任意字串（例如 sk-local）——ds4-server 不檢查
• Model 名稱：deepseek-v4-flash（client 若會探查 GET /v1/models，這個 ID 會出現在回傳列表裡）
• 支援的 wire format（生成）：
  • OpenAI Chat Completions（/v1/chat/completions）✓
  • OpenAI 舊式 Completions（/v1/completions）✓
  • OpenAI Responses（/v1/responses）✓ —— 所以 Codex CLI 0.128+ 可以直連
  • Anthropic Messages（/v1/messages，含 count_tokens）✓ —— 所以 Claude Code 可以直連
• 模型探查端點：GET /v1/models —— OpenAI 風格的 model discovery，回傳 deepseek-v4-flash。

• Base URL: http://127.0.0.1:8000. Full endpoint set, identical to §7.1: /v1/chat/completions, /v1/completions, /v1/responses, /v1/messages (including count_tokens), /v1/models.
• API key: any string (e.g. sk-local) — ds4-server does not check it.
• Model name: deepseek-v4-flash (if the client probes GET /v1/models, this ID appears in the response).
• Supported wire formats (generation):
  • OpenAI Chat Completions (/v1/chat/completions) ✓
  • OpenAI legacy Completions (/v1/completions) ✓
  • OpenAI Responses (/v1/responses) ✓ — so Codex CLI 0.128+ connects directly
  • Anthropic Messages (/v1/messages, including count_tokens) ✓ — so Claude Code connects directly
• Model discovery endpoint: GET /v1/models — OpenAI-style model discovery, returning deepseek-v4-flash.

8.1　Codex CLI（OpenAI 官方）　Codex CLI (OpenAI's official client)

Codex CLI 0.128+ 用 OpenAI Responses API（/v1/responses）跟 provider 對話； ds4-server 已經實作這個端點。把 pi-ds4 加進 Codex 的 provider 表：

Codex CLI 0.128+ uses the OpenAI Responses API (/v1/responses) to speak with providers; ds4-server already implements that endpoint. Add pi-ds4 to Codex's provider table:

model = "deepseek-v4-flash"
model_provider = "ds4"

[model_providers.ds4]
name = "Local pi-ds4"
base_url = "http://127.0.0.1:8000/v1"
wire_api = "responses"
# env_key 省略：ds4-server 不檢查 API keyenv_key omitted: ds4-server does not check the API key

設定完之後直接 codex 就會走 pi-ds4。 若你只想偶爾用本機推論、平常還是接 cloud，可以保留 cloud 為預設， 用 codex --config model_provider=ds4 --config model=deepseek-v4-flash 臨時切過去。

Once configured, plain codex goes through pi-ds4. If you only want occasional local inference and otherwise stay on cloud, keep cloud as the default and switch on the fly with codex --config model_provider=ds4 --config model=deepseek-v4-flash.

8.2　Claude Code（Anthropic 官方）：直連　Claude Code (Anthropic's official client): direct connection

ds4-server 自身就同時實作 OpenAI /v1/chat/completions 與 Anthropic /v1/messages（見 ds4_server.c 開頭的 "OpenAI/Anthropic compatible local server" 自我描述）。 所以 Claude Code 不需要 router／proxy——把兩個環境變數設好就走：

ds4-server itself implements both OpenAI /v1/chat/completions and Anthropic /v1/messages (see the ds4_server.c header which self-describes as "OpenAI/Anthropic compatible local server"). So Claude Code needs no router or proxy — set two environment variables and go:

export ANTHROPIC_BASE_URL=http://127.0.0.1:8000
export ANTHROPIC_AUTH_TOKEN=sk-local   # 任意字串，ds4-server 不檢查any string; ds4-server does not check
export ANTHROPIC_MODEL=deepseek-v4-flash

claude

你的 slash command、subagent、MCP server 流程都保留——只是底下的模型換成本機推論。 Tool-use 方面，ds4 的 tool-calling 訓練不如 Claude 強，會比較容易失誤； 適合長對話、寫作、解釋程式碼；嚴重依賴 tool loop 的情境會比較喘。

Your slash commands, subagents, and MCP server flow all remain — only the underlying model switches to local inference. On tool use, ds4's tool-calling training is weaker than Claude's and tends to slip up; it suits long conversations, writing, and code explanation, but workloads heavily reliant on tool loops will struggle.

8.3　OpenClaw　OpenClaw

OpenClaw 把所有 provider 都看成 OpenAI-compatible，loopback 位址自動信任。 編輯 openclaw.json，新增一個 provider：

OpenClaw treats all providers as OpenAI-compatible and trusts loopback addresses automatically. Edit openclaw.json and add a provider:

{
  "agents": { "defaults": { "model": { "primary": "ds4/deepseek-v4-flash" } } },
  "models": {
    "mode": "merge",
    "providers": {
      "ds4": {
        "baseUrl": "http://127.0.0.1:8000/v1",
        "apiKey": "sk-local",
        "api": "openai-completions",
        "timeoutSeconds": 600,
        "models": [{
          "id": "deepseek-v4-flash",
          "name": "DeepSeek V4 Flash (local)",
          "reasoning": false,
          "contextWindow": 100000,
          "maxTokens": 8192
        }]
      }
    }
  }
}

contextWindow 必須 ≤ ds4-server 的 --ctx（預設 100000）， 不然 OpenClaw 會把超長 prompt 丟進去然後被 server 拒絕。寫好整段 provider 區塊比較保險——逐 key 用 openclaw config set … 容易漏掉 models[]、 agents.defaults.model.primary、api 等欄位，導致 session 還是接到舊 provider。

contextWindow must be ≤ ds4-server's --ctx (default 100000), otherwise OpenClaw will push an over-long prompt through and the server will reject it. Writing the whole provider block at once is safer — doing it key by key with openclaw config set … easily misses fields like models[], agents.defaults.model.primary, or api, leaving the session connected to the old provider.

8.4　Hermes Agent　Hermes Agent

最快的路徑是互動式：hermes model 選「Custom endpoint (self-hosted / VLLM / etc.)」，輸入 http://127.0.0.1:8000/v1 即可。 若你想寫進 config：

The quickest path is interactive: hermes model, choose “Custom endpoint (self-hosted / VLLM / etc.)”, and enter http://127.0.0.1:8000/v1. To write it into the config instead:

custom_providers:
  - name: ds4
    base_url: http://127.0.0.1:8000/v1
    # api_key 省略：本機 server 不檢查api_key omitted: the local server does not check it

model:
  default: deepseek-v4-flash
  provider: custom:ds4

之後 session 內可以隨時 /model custom:ds4:deepseek-v4-flash 切換。

Within a session you can switch any time with /model custom:ds4:deepseek-v4-flash.

8.5　直接打 API（OpenAI SDK／curl／自製腳本）　Hitting the API directly (OpenAI SDK / curl / your own script)

上面的章節都是包裝層。若你只是想串到自己的 pipeline、Cron job、 或既有 OpenAI SDK 程式：把 OPENAI_BASE_URL 指過去就完了。

The sections above are all wrappers. If you just want to wire it into your own pipeline, a cron job, or existing OpenAI SDK code: just point OPENAI_BASE_URL at it and you are done.

from openai import OpenAI

client = OpenAI(
    base_url="http://127.0.0.1:8000/v1",
    api_key="sk-local",   # 任意值any value
)

resp = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[{"role": "user", "content": "自我介紹"}],
)
print(resp.choices[0].message.content)

若你的 pipeline 已經切到 OpenAI Responses API（Codex CLI 用的那個）， client.responses.create() 直接呼叫同一個 server：

If your pipeline already uses the OpenAI Responses API (the one Codex CLI speaks), client.responses.create() hits the same server:

from openai import OpenAI

client = OpenAI(
    base_url="http://127.0.0.1:8000/v1",
    api_key="sk-local",
)

resp = client.responses.create(
    model="deepseek-v4-flash",
    input=[{"role": "user", "content": "自我介紹"}],
)
print(resp.output_text)

8.6　ds4-server 何時跑？　When does ds4-server run?

pi-ds4 的 lease／watchdog 機制（第七章）只在 pi process 透過模型發 request 時才生效。 如果你完全不用 pi、只用 Codex CLI 等外部前端，那麼 ds4-server 必須以另一種方式被啟動。三條路：

pi-ds4's lease/watchdog mechanism (Chapter 7) only kicks in when a pi process sends a request through the model. If you do not use pi at all and only use external frontends like Codex CLI, then ds4-server has to be started some other way. Three paths:

• （A）已經 pi install 過、想最省事：留一個 pi session 開著、選定 ds4 模型一次， lease 就會持續續約，server 不會被 watchdog 收掉。
• （B）已經 pi install 過、手動跑：到 ~/.pi/ds4/support/ 執行：
  手動跑（pi install 之後） Copy

  cd ~/.pi/ds4/support
  ./ds4-server \
    --ctx 100000 \
    --kv-disk-dir ~/.pi/ds4/kv --kv-disk-space-mb 8192 \
    --mpp auto \
    --dir-steering-file dir-steering/out/uncertainty_ablit_imatrix.f32 \
    --dir-steering-ffn -2

  這條等同於 index.ts 內建的 spawn 參數；你也可以依需求自訂。
• （C）完全沒裝過 pi：直接從 audreyt/ds4 與 audreyt/pi-ds4 bootstrap——不需要 pi、不需要 ~/.pi/ds4/。需要兩個 repo 的原因： ds4 內附的 download_model.sh 抓的是 antirez 上游 stock-recipe； 要拿 cyberneurova abliterated GGUF（本指南的權重），得用 pi-ds4 內附的同名腳本。
  完全跳過 pi 的 bootstrap Copy

  # 1. 取 ds4 引擎並編譯
  git clone https://github.com/audreyt/ds4
  cd ds4
  make ds4-server         # Mac: Metal; Linux: CUDA（見 8.7）
  
  # 2. 用 pi-ds4 內附的 cyberneurova 下載腳本（不是 ds4 自帶的 stock-recipe 那支）
  curl -fL -o download_cyberneurova.sh \
    https://raw.githubusercontent.com/audreyt/pi-ds4/main/download_model.sh
  chmod +x download_cyberneurova.sh
  ./download_cyberneurova.sh q2   # 取得 ~87 GB GGUF，並 symlink 為 ds4flash.gguf
  
  # 3. 起 server
  ./ds4-server \
    --ctx 100000 \
    --kv-disk-dir ./kv --kv-disk-space-mb 8192 \
    --mpp auto \
    --dir-steering-file dir-steering/out/uncertainty_ablit_imatrix.f32 \
    --dir-steering-ffn -2

  這條路徑下你完全沒碰 pi；ds4-server 直接由 ds4 source tree 提供。 本擴充（pi-ds4）做的事——下載、symlink、watchdog、lease——你需要時自己處理。

• (A) pi already installed, easiest route: keep one pi session open with the ds4 model selected once, and the lease keeps renewing — the watchdog will not reap the server.
• (B) pi already installed, run it by hand: in ~/.pi/ds4/support/, run:
  Manual run (after pi install) Copy

  cd ~/.pi/ds4/support
  ./ds4-server \
    --ctx 100000 \
    --kv-disk-dir ~/.pi/ds4/kv --kv-disk-space-mb 8192 \
    --mpp auto \
    --dir-steering-file dir-steering/out/uncertainty_ablit_imatrix.f32 \
    --dir-steering-ffn -2

  This is equivalent to the spawn parameters baked into index.ts; you can also adjust them as needed.
• (C) Never installed pi at all: bootstrap directly from audreyt/ds4 and audreyt/pi-ds4 — no pi, no ~/.pi/ds4/ needed. Why both repos: the download_model.sh bundled with ds4 fetches the antirez upstream stock-recipe; to get the cyberneurova abliterated GGUF (the weights this guide uses) you need the same-named script from pi-ds4.
  Bootstrap that skips pi entirely Copy

  # 1. Fetch and build the ds4 engine
  git clone https://github.com/audreyt/ds4
  cd ds4
  make ds4-server         # Mac: Metal; Linux: CUDA (see 8.7)
  
  # 2. Use the cyberneurova download script bundled with pi-ds4 (not ds4's own stock-recipe one)
  curl -fL -o download_cyberneurova.sh \
    https://raw.githubusercontent.com/audreyt/pi-ds4/main/download_model.sh
  chmod +x download_cyberneurova.sh
  ./download_cyberneurova.sh q2   # Fetches the ~87 GB GGUF and symlinks it as ds4flash.gguf
  
  # 3. Start the server
  ./ds4-server \
    --ctx 100000 \
    --kv-disk-dir ./kv --kv-disk-space-mb 8192 \
    --mpp auto \
    --dir-steering-file dir-steering/out/uncertainty_ablit_imatrix.f32 \
    --dir-steering-ffn -2

  On this path you touch no pi at all; ds4-server comes directly from the ds4 source tree. Whatever the extension (pi-ds4) does — download, symlink, watchdog, lease — you handle yourself when needed.

8.7　其他硬體：DGX Spark 等 CUDA 機器　Other hardware: DGX Spark and CUDA machines

你不一定需要 Mac。audreyt/ds4 的 Makefile 在非 Darwin 系統上自動切到 CUDA 路徑 （ds4_cuda.cu，~10k 行 NVIDIA kernel），用 nvcc 編出一顆 原生 ds4-server。也就是說：NVIDIA DGX Spark（GB10、~128 GB 統一記憶體、aarch64 Linux） 上跑的不是某個 llama.cpp 旁路，而是同一隻 engine、同一份 server、 同一組 --dir-steering-* flag。

You do not have to use a Mac. The Makefile of audreyt/ds4 switches automatically to a CUDA path on non-Darwin systems (ds4_cuda.cu, around 10k lines of NVIDIA kernel), using nvcc to build a native ds4-server. That is: on the NVIDIA DGX Spark (GB10, ~128 GB unified memory, aarch64 Linux), what runs is not some llama.cpp sidecar but the same engine, the same server, the same --dir-steering-* flags.

# 前置：apt install build-essential cmake；已裝好 CUDA toolkit（DGX Spark 預設帶 /usr/local/cuda）Prerequisites: apt install build-essential cmake; CUDA toolkit already installed (DGX Spark ships /usr/local/cuda by default)

git clone https://github.com/audreyt/ds4
cd ds4
make ds4-server         # Linux 自動走 nvcc + ds4_cuda.cu，CUDA_ARCH=native 預設Linux automatically uses nvcc + ds4_cuda.cu; CUDA_ARCH=native is the default

# 下載 cyberneurova abliterated IQ2XXS imatrix GGUF——注意，ds4 本身的 download_model.sh 是 antirez stock-recipe，Download the cyberneurova abliterated IQ2XXS imatrix GGUF — note that ds4's own download_model.sh is the antirez stock-recipe
# 不會抓到 cyberneurova。用 pi-ds4 內附的同名腳本：and will not fetch cyberneurova. Use the same-named script bundled with pi-ds4:
curl -fL -o download_cyberneurova.sh \
  https://raw.githubusercontent.com/audreyt/pi-ds4/main/download_model.sh
chmod +x download_cyberneurova.sh
./download_cyberneurova.sh q2

# 起 server。bind 127.0.0.1：要對 LAN 開放，自己加 firewall/Tailscale/reverse-proxy。Start the server. It binds 127.0.0.1: to open it to the LAN, add your own firewall / Tailscale / reverse proxy.
# 引導參數一字不改，與 Mac 路徑一致。--mpp 是 Mac 專屬，CUDA 路徑會忽略。Steering parameters are identical to the Mac path. --mpp is Mac-only and is ignored on the CUDA path.
mkdir -p /var/cache/ds4-kv
./ds4-server \
  --ctx 32768 \
  --kv-disk-dir /var/cache/ds4-kv --kv-disk-space-mb 8192 \
  --dir-steering-file dir-steering/out/uncertainty_ablit_imatrix.f32 \
  --dir-steering-ffn -2

Endpoint 起來之後，前面 8.2～8.5 的設定原則上一字不改都能用（把 127.0.0.1 換成你 DGX 的位址）。差別在哪：

Once the endpoint is up, the setups in 8.2–8.5 above work essentially verbatim (just replace 127.0.0.1 with your DGX's address). What's different:

• 對 LAN 開放要自行加 auth／防火牆：ds4-server 預設 bind 127.0.0.1 且不檢查 API key；想跨機使用， 建議用 Tailscale 或 reverse proxy 加 auth，不要直接把 --host 改成 0.0.0.0 暴露在區網。
• 沒有 pi-ds4 的 watchdog／lease：pi-ds4 的 lifecycle 管理（第七章）是 macOS 路徑下的擴充—— Linux 上自己寫 systemd unit、或留個 tmux session 跑著即可。
• 沒有 M5 Metal MPP 那條最快曲線：本指南 440 t/s 是 M5 Max 上的數字； DGX Spark 走 CUDA／GB10 tensor core，吞吐量看 NVIDIA 路徑的調校 （CUDA_ARCH=native 已是預設，不必再調）。
• Context 預設值：Mac 路徑下 pi-ds4 預設 --ctx 100000， 是經過 M5 + Metal KV cache 驗證的值；DGX Spark 上的長 context 行為 尚未在本指南範圍內基準過，建議從 --ctx 32768 起步。

• Opening it to the LAN requires your own auth / firewall: ds4-server binds 127.0.0.1 by default and does not check the API key; for cross-machine use, prefer Tailscale or a reverse proxy with auth rather than flipping --host to 0.0.0.0 and exposing it on the LAN directly.
• No pi-ds4 watchdog / lease: pi-ds4's lifecycle management (Chapter 7) is the macOS-path extension — on Linux, write a systemd unit yourself or keep a tmux session running.
• No M5 Metal MPP fast curve: the 440 t/s figure in this guide is from M5 Max; DGX Spark goes through CUDA / GB10 tensor cores, with throughput depending on NVIDIA-path tuning (CUDA_ARCH=native is already the default, no further tuning needed).
• Default context window: on the Mac path, pi-ds4 defaults to --ctx 100000, a value validated on M5 + Metal KV cache; long-context behaviour on DGX Spark is not yet benchmarked within this guide's scope, so start with --ctx 32768.

換句話說，這不是「同一份 GGUF、不同 runtime」的勉強相容，而是 同一隻引擎、同一支二進位、同一組旗標跨平台。 pi-ds4 那一層 wrapper 只是 macOS 上的安裝／lifecycle 自動化， 在 Linux 上手動跑 ds4-server 已經涵蓋所有功能。

In other words, this is not a strained “same GGUF, different runtime” compatibility, but the same engine, the same binary, the same flags across platforms. The pi-ds4 wrapper is just install / lifecycle automation on macOS; running ds4-server by hand on Linux already covers all the same functionality.

故障排除：lease 與 watchdogTroubleshooting: lease and watchdog

本擴充把運行狀態都寫在 ~/.pi/ds4/ 底下， 遇到怪事的時候可以直接看檔案——不需要拆原始碼。

The extension writes all runtime state under ~/.pi/ds4/; when something odd happens, just read the files — no need to dig into the source.

9.1　目錄結構　Directory layout

9.2　常見症狀與處置　Common symptoms and remedies

「server 卡在啟動中超過 10 分鐘」“The server is stuck starting up for more than 10 minutes”

看 log 的最後幾百行。內建 SSD 上模型載入通常只要幾秒， 卡 10 分鐘以上多半不是磁碟速度本身的問題——更可能是 GGUF 放在慢速外接磁碟， 或啟動流程其他環節卡住。先讀 log 找原因；真有必要再把 DS4_READY_TIMEOUT_MS 調高。

Read the last few hundred lines of log. Normal model load takes only a few seconds on an internal SSD, so anything stuck past 10 minutes is rarely a pure disk-speed issue — more likely the GGUF sits on a slow external drive, or some other step in startup is wedged. Diagnose from the log first; raise DS4_READY_TIMEOUT_MS only if there’s a real reason to wait longer.

「pi 顯示 ds4-server startup failed」“pi reports ds4-server startup failed”

log 會明確指出失敗點：可能是 make ds4-server 失敗（缺 Xcode CLI tools？）、 GGUF 下載中斷（網路？磁碟滿？）、或者 --mpp 在不支援的晶片上 panic。

The log will point clearly to the failure: make ds4-server failing (missing Xcode CLI tools?), a broken GGUF download (network? disk full?), or --mpp panicking on unsupported silicon.

「想強制重啟一切」“I want to force-restart everything”

先試溫和的：把 ~/.pi/ds4/clients/ 清空，watchdog 在下一輪 poll（約 2 秒）看到沒有 lease 後會優雅關閉 server。

Try the gentle path first: empty ~/.pi/ds4/clients/; on the next poll (about 2 seconds), the watchdog sees no leases and shuts the server down gracefully.

# 清掉所有 lease，watchdog 會在約 60 秒內關閉 server。Clear all leases; the watchdog stops the server within about 60 seconds.
# 用 find -delete 而非 rm + glob，避免 zsh 在空目錄時的 no-matches 錯誤。Use find -delete rather than rm + glob to avoid zsh's no-matches error on empty dirs.
find ~/.pi/ds4/clients -maxdepth 1 -name '*.json' -delete 2>/dev/null || true

# 印出 server.json 的關鍵欄位、對應程序的 args 與啟動時間：Print the key fields of server.json and the matching process's args and start time:
STATE=~/.pi/ds4/server.json
if [ ! -f "$STATE" ]; then
  echo 'no server.json (already clean)'
else
  MANAGED=$(sed -n 's/.*"managedBy"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' "$STATE" | head -1)
  PID=$(sed -n 's/.*"pid"[[:space:]]*:[[:space:]]*\([0-9]*\).*/\1/p' "$STATE" | head -1)
  BINARY=$(sed -n 's/.*"binary"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' "$STATE" | head -1)
  echo "managedBy: $MANAGED"
  echo "pid:       ${PID:-<none>}"
  echo "binary:    ${BINARY:-<none>}"
  if [ -n "$PID" ] && kill -0 "$PID" 2>/dev/null; then
    ps -p "$PID" -o pid=,lstart=,args=
  else
    echo '(pid not running — state is stale, step 2 will clean it)'
  fi
fi

(
  STATE=~/.pi/ds4/server.json
  LOCKDIR=~/.pi/ds4/lock

  # 用 mkdir 做原子鎖（與 index.ts 相同機制）；鎖被持有就退出Use mkdir as an atomic lock (same mechanism as index.ts); exit if the lock is held.
  if ! mkdir "$LOCKDIR" 2>/dev/null; then
    echo "abort: lock $LOCKDIR is held; owner:"
    cat "$LOCKDIR/owner.json" 2>/dev/null || echo '(no owner.json — if > 60s old, rm -rf manually)'
    exit 1
  fi
  trap 'rm -rf "$LOCKDIR" 2>/dev/null' EXIT
  trap 'rm -rf "$LOCKDIR" 2>/dev/null; exit 130' INT
  trap 'rm -rf "$LOCKDIR" 2>/dev/null; exit 143' TERM
  trap 'rm -rf "$LOCKDIR" 2>/dev/null; exit 129' HUP

  if [ ! -f "$STATE" ]; then
    echo 'lifecycle already clean'; exit 0
  fi

  PID=$(sed -n 's/.*"pid"[[:space:]]*:[[:space:]]*\([0-9]*\).*/\1/p' "$STATE" | head -1)

  # 只有 PID 不存在（或 state 沒寫 PID）時才動 state。Only touch state when the PID is gone (or absent from state).
  # PID 還活著就拒絕——交給第三步人工處理。If the PID is alive, refuse — hand off to step 3 for manual action.
  if [ -n "$PID" ] && kill -0 "$PID" 2>/dev/null; then
    echo "refuse: pid $PID still alive."
    echo "        please verify via step 1 output, then use step 3 to kill manually."
    exit 2
  fi

  # PID 已死或無 PID。安全清理。PID is dead or absent. Safe to clean.
  find ~/.pi/ds4/clients -maxdepth 1 -name '*.json' -delete 2>/dev/null || true
  rm -f "$STATE"
  echo 'state cleared. lock will release on exit.'
)

# 把 PID_FROM_STEP_1_MUST_BE_REPLACED 換成你親眼從第一步輸出確認過的那個 PID。Replace PID_FROM_STEP_1_MUST_BE_REPLACED with the PID you confirmed by eye in step 1's output.
# 沒提供 Copy 按鈕；故意用一個非數字的 token，不慎執行未編輯版本只會得到No Copy button is provided; the deliberately non-numeric token means accidentally running the unedited version only yields
# 'kill: arguments must be process or job IDs' 之類的錯誤，而不會送出 SIGTERM。an error like 'kill: arguments must be process or job IDs' rather than sending SIGTERM.
kill -TERM PID_FROM_STEP_1_MUST_BE_REPLACED

# 等幾秒讓它優雅退出，然後再跑第二步即可清掉 state。Wait a few seconds for it to exit gracefully, then re-run step 2 to clear state.

「Server 在跑，但 /v1/models 沒回應」“The server is running but /v1/models does not respond”

檢查 8000 port 是否真的被 ds4-server 佔用：lsof -nP -iTCP:8000 -sTCP:LISTEN。 若被其他程式佔走，server 啟動會失敗（log 會記錄）。

Check whether port 8000 is really held by ds4-server: lsof -nP -iTCP:8000 -sTCP:LISTEN. If something else has grabbed it, server startup fails (the log records it).

本機開發安裝Local development install

如果你想 hack 引擎、測自己的 patch、或保留多個 ds4 fork 並行， 可以跳過 pi install 流程，直接用內附的安裝腳本把擴充和 ds4 checkout 都 symlink 過去。

If you want to hack on the engine, test your own patches, or keep several ds4 forks side by side, you can skip the pi install flow and use the bundled install script to symlink both the extension and your ds4 checkout into place.

10.1　基本用法　Basic usage

# 在 pi-ds4 的 checkout 根目錄執行：From the pi-ds4 checkout root:
./install-pi-extension-local.sh /path/to/audreyt-ds4-checkout

它會做兩件事：

It does two things:

• 把 ~/.pi/agent/extensions/pi-ds4 連到當前 pi-ds4 checkout；
• 把 ~/.pi/ds4/support 連到你提供的 ds4 checkout。

• symlinks ~/.pi/agent/extensions/pi-ds4 to the current pi-ds4 checkout;
• symlinks ~/.pi/ds4/support to the ds4 checkout you supplied.

10.2　已有 support 目錄時：--force　When a support directory already exists: --force

如果 ~/.pi/ds4/support 已經指向別的地方（例如上次 pi install 留下的）， 腳本會拒絕直接覆寫。加 --force 會：

If ~/.pi/ds4/support already points elsewhere (e.g. left over from a prior pi install), the script refuses to overwrite directly. Adding --force will:

• 把舊 checkout 內的 gguf/*.gguf 和 .gguf.part 用 APFS clone-on-write 複製到新 checkout（macOS 上不會真的複製 87 GB，只是共享 inode）；
• 把舊 support 目錄改名為 support.backup.<timestamp>；
• 建立新的 symlink。

• copy gguf/*.gguf and .gguf.part from the old checkout to the new one via APFS clone-on-write (on macOS this does not actually copy 87 GB — it just shares the inodes);
• rename the old support directory to support.backup.<timestamp>;
• create the new symlink.

10.3　接下來：reload　Next: reload

安裝完成後重啟 pi，或在 pi 內執行 /reload——擴充會被重新發現。

After install, restart pi or run /reload inside pi — the extension is rediscovered.

進階：自訂引導方向Advanced: custom steering directions

本章是給已經理解前面所有章節的讀者準備的：如何根據自己的研究興趣建造一個全新的引導向量， 並掛到 ds4-server 上跑。

This chapter is for readers who have already absorbed everything above: how to build a fresh steering vector for your own research interest and run it on ds4-server.

11.1　工具鏈在哪　Where the toolchain lives

audreyt/ds4 的 dir-steering/ 目錄裡有完整的建構工具：

The complete build toolchain lives in the dir-steering/ directory of audreyt/ds4:

• collect-acts.py：對一組「對比提示對」做前向跑、收集每層 hidden state；
• build-dir.py：把收集到的 activation 做 PCA／差分，輸出 .f32 向量檔；
• README.md：完整教學與設計討論。

• collect-acts.py: runs forward passes over a set of contrast prompt pairs and collects the hidden state at every layer;
• build-dir.py: applies PCA / differencing to the collected activations and emits a .f32 vector file;
• README.md: full tutorial and design discussion.

11.2　典型工作流　A typical workflow

1. 準備兩組提示——一組你想「引導模型走向」（positive），一組「不要走向」（negative）。每組 50～200 條足夠。
2. 用 collect-acts.py 在 ds4-server 上跑這些提示，產生兩組 activation。
3. 用 build-dir.py 計算這兩組 activation 在每一層的差異方向，輸出成 my-direction.f32。
4. 把檔案放到 dir-steering/out/ 或任何位置；
5. 設定環境變數：
   • DS4_DIR_STEERING_FILE=dir-steering/out/my-direction.f32
   • DS4_DIR_STEERING_FFN=-2（或自己摸出來的甜蜜點）
6. 重啟 ds4-server，測試。

1. Prepare two sets of prompts — one you want to steer the model towards (positive) and one to steer away from (negative). 50 to 200 of each is enough.
2. Run them through ds4-server with collect-acts.py to produce two sets of activations.
3. Use build-dir.py to compute the difference direction layer by layer, emitting my-direction.f32.
4. Place the file in dir-steering/out/ or anywhere else;
5. set the environment:
   • DS4_DIR_STEERING_FILE=dir-steering/out/my-direction.f32
   • DS4_DIR_STEERING_FFN=-2 (or your own sweet spot)
6. Restart ds4-server and test.

11.3　建造好向量的兩個要點　Two principles for building a good vector

• 對比要乾淨：positive／negative 兩組應該在你想引導的維度上有清楚差異， 其他維度盡可能相似。把「快樂回應 vs 悲傷回應」拿來建構，得到的方向會混入大量情緒以外的成分。
• 引導的是 register，不是 belief：經驗顯示，引導模型的「回應風格／暫存器」 （例如 hedge vs assert、敘事 vs 條列、學術 vs 對話）比引導「立場」要可靠得多。 這也是本分支採用「不確定性」而非「特定立場」的原因。

• Keep the contrast clean: the positive and negative sets should differ clearly along the dimension you want to steer and resemble each other elsewhere. Building from “happy reply vs. sad reply” gives you a direction laced with much more than emotion.
• You are steering register, not belief: experience shows that steering the model's response style / register (hedge vs. assert, narrative vs. bulleted, academic vs. conversational) is far more reliable than steering “stance”. That is why this fork picks “uncertainty” rather than a specific position.

致謝、授權、後話Acknowledgements, licence, afterword

致謝Acknowledgements

• mitsuhiko/pi-ds4——本擴充所基於的上游版本。Armin Ronacher 寫了所有 lifecycle、watchdog、lease 機制。
• antirez/ds4——Salvatore Sanfilippo 用純 C 寫的 DeepSeek V4 Flash inference engine，延續他寫 Redis 的風格。GGUF 由同專案的 llama.cpp-deepseek-v4-flash 轉換而來。
• ivanfioravanti 的 PR #15——M5 Metal 4 / MPP 優化的關鍵 patch。
• cyberneurova 研究專案——發布了讓本機運行可行的 abliterated GGUF。

• mitsuhiko/pi-ds4 — the upstream this extension is based on. Armin Ronacher wrote all the lifecycle, watchdog, and lease machinery.
• antirez/ds4 — Salvatore Sanfilippo's DeepSeek V4 Flash inference engine written in pure C, in the same tradition as Redis. The GGUF was converted by the sibling project llama.cpp-deepseek-v4-flash.
• ivanfioravanti's PR #15 — the key patch for M5 Metal 4 / MPP optimisation.
• The cyberneurova research project — published the abliterated GGUF that makes local execution viable.

授權Licence

原始碼採用 MIT，與上游一致。請見專案 LICENSE。本指南文本則貢獻於 Public Domain（CC0）。

Source code is MIT-licensed, matching upstream. See the project's LICENSE. The text of this guide is contributed to the public domain under CC0.

後話Afterword

把前沿模型放到自己機器上跑，從技術上講，是把一段運算搬到本地； 從政治上講，是把一個方向盤從別人手上接過來。本擴充預設啟用 不確定性引導——並不是要替你決定任何問題的答案， 而是要在那些「被訓練封死」的提問前，把討論空間還給你。

Running a frontier model on your own machine, technically speaking, moves a piece of computation onto local hardware; politically speaking, it takes the steering wheel back from someone else's hands. This extension defaults to uncertainty steering — not to decide any question's answer for you, but to return the space for discussion in front of questions where training has nailed the door shut.

模型可以閉合一個答案，使用者可以打開一個討論。
這份指南完成它任務的時刻，是當你已經不需要它的時候。

A model can close a question on an answer; the user can open it back into a discussion.
The moment this guide finishes its work is the moment you no longer need it.