Under 100 MiB, ~50 ms on a mid-tier laptop CPU.
A small model that reads a shell command and says what it would do, before it runs.
Size, quality, speed
Three formats of the same model. Scores are on 3,793 test rows. A flip is a decision that differs from FP32. There are 34,137 decisions.
| Format | Size | Macro F1 shell / action | Flips vs FP32 |
|---|---|---|---|
| FP32 | 570.7 MiB | 0.940 / 0.961 | - |
| Q8_0 | 151.7 MiB | 0.941 / 0.960 | 19 (0.06%) |
| Q5_0 | 98.2 MiB | 0.939 / 0.960 | 85 (0.25%) |
Q5 is under 100 MiB. It changes 0.25% of decisions.
One model pass takes 50 ms (median) on a Ryzen 5 6650U CPU. It takes 4.2 ms on a DGX Spark with CUDA. See the speed scale below.
v1.0.0 runs on Linux x86_64 and aarch64, macOS arm64, and CUDA on the DGX Spark. It needs no GPU and no network.
Nine labels
The model gives one probability per label. A label counts as true at 0.5 or more.
Try it
Pick a command or type your own. The model runs in this tab.
Download the model?
About MiB, once. It runs in this tab on the CPU (WebAssembly). This is bigger than the 98 MiB Q5 release. The browser runtime cannot use 5-bit weights, and its 8-bit path was slower in my tests. So this page uses 16-bit weights.
The browser keeps the model in its cache, so the next visit does not download it again.
The browser is slow
This page runs the model in the browser with ONNX Runtime Web. It is much slower than the native acclass engine. On an M1 Pro, a short command takes about 75 ms here. The engine takes about 6.5 ms with Metal and 25 ms on the CPU of an M4 Pro.
Install the engine for real speed.
Longer commands have more tokens and take longer.
The probabilities are calibrated (T = 1.76). On the test data, 80% means about 8 in 10 similar commands have that effect. The test data is synthetic, so your commands can differ.
Speed scale
One model pass, Q8. Log scale, 1 ms to 1000 ms. The markers are medians for short commands.
Time grows with command length. Most commands are short. Long inputs cause the 95th percentile: 50 ms median but 717 ms p95 on the Ryzen, 6.5 ms but 50 ms on M4 Pro Metal. A maximum-length input (2,048 tokens) takes about 85 ms on the DGX Spark.
Your marker comes from the browser build, not the engine. Do not compare it with the markers.
Install
The installer needs Python 3.
curl -fsSL https://github.com/jensilo/acclass/releases/latest/download/install.sh | sh
Source on GitHub · releases (v1.0.0) · weights on Hugging Face
The model card lists the limits.
Using Pi?
The pi-activity extension uses acclass to check tool calls. It installs acclass on first start, after it asks you.
From npm:
pi install npm:@jensilo/pi-activity@1.0.1
Or from GitHub:
pi install git:github.com/jensilo/pi-activity@v1.0.1