Devseis/endpoint-auditor-0.5b-q4f32_1-MLC

Devseis Endpoint Auditor 0.5B (v0.6) — q4f32_1 for WebLLM / MLC

A small language model fine-tuned by Devseis to write endpoint audit findings for ISO 27001, GDPR and the EU AI Act from collected evidence. It runs on the device (WebLLM in the Devseis Endpoint Auditor desktop app and phone check), so audit evidence never leaves the computer or phone.

v0.6: Model v0.5 retrained to survive the 4-bit browser builds (v0.5 answered well in full precision but wrote junk once rounded to 4 bits). 100 more steps on 800 of the v0.5 examples, lr 1e-4, with every forward pass through the exact q4f16_1 rounding (train_lora.py --qat4); scores below are for the 4-bit rounded model. About 300 MB in 4-bit. In use (Arx app 0.5.0+): the desktop app loads v0.6 on graphics chips without 16-bit maths (4-bit build q4f32_1), where v0.3 fails, and v0.3 in full precision (q0f16) everywhere else (v0.3 scores 100% on the report tests; v0.6 summaries are weaker in full precision). Set in app/model-version.json (version, version_4bit). The Arx web chat that v0.5/v0.6 were also trained for was removed on 2026-10-14.

What the model does — and does not do

Code collects the evidence and decides every verdict (Compliant / Non-Compliant / …) and every vulnerability match. The model explains each finding, rates the risk, writes fix and verification steps for the device's OS, maps findings to ISO 27001 / GDPR / AI Act references, writes the executive summary, and classifies AI tools under the EU AI Act. Every answer is checked against the evidence (numbers, check id, status, references, tool names) before it is used; if a check fails, the app uses a built-in template instead. Details: docs/MODEL_LLM.md.

Use in the browser (WebLLM)

import { CreateMLCEngine, prebuiltAppConfig } from "@mlc-ai/web-llm";
const base = prebuiltAppConfig.model_list.find(m => m.model_id === "Qwen2.5-0.5B-Instruct-q4f32_1-MLC");
const appConfig = { model_list: [{ ...base, model: "https://huggingface.co/Devseis/endpoint-auditor-0.5b-q4f32_1-MLC/resolve/v0.6/",
                                   model_id: "devseis-auditor-0.5b-v0.6-q4f32_1" }] };
const engine = await CreateMLCEngine("devseis-auditor-0.5b-v0.6-q4f32_1", { appConfig });

Same architecture as Qwen2.5-0.5B-Instruct, so WebLLM's prebuilt Qwen2.5-0.5B q4f32_1 runtime library is reused; only the weights differ. Use constrained decoding with the answer schemas in app/renderer/auditor-core.js (ANSWER_SCHEMAS): response_format: { type: "json_object", schema: JSON.stringify(ANSWER_SCHEMAS[task]) }. Measured on held-out prompts, 4-bit builds produce invalid JSON without it (0 of 6 grounded) and grounded answers with it (5 of 6; the sixth was cut by the token limit, now raised).

Converted with training/mlc_quantize.py, checked against mlc-ai's own builds of the base model: identical tensor layout; q0f16 byte-identical; 4-bit scales bit-identical and 99.6-99.8% of 4-bit values identical (the rest one step apart at rounding ties).

Prompt format

Retrieval-style prompts built by app/renderer/auditor-core.js (identical to training/build_dataset.py), with the system prompt in that file. Tasks: finding, summary, ai_classification; the model answers with one JSON object.

Training

Base model	Qwen/Qwen2.5-0.5B-Instruct (Apache-2.0)
Data	Devseis/endpoint-auditor-synthetic revision v0.5 (synthetic, CC BY 4.0)
Examples	800 balanced over task, OS and status (3072 tokens)
Method	LoRA r=16, alpha=32 (8.8 M trainable parameters), lr 2e-4, 1 epoch, 100 steps of 8 examples
Hardware	CPU only: a 4-core Intel MacBook Pro (PyTorch 2.2, eager attention)
Final validation loss	0.0149

Evaluation

Held-out test examples (training/evaluate.py, greedy decoding). Grounded = passes the same faithfulness check the app uses; fields match = check id / status / risk (findings), score and top gaps (summaries), tools and approval flags (AI).

Base model (Qwen2.5-0.5B-Instruct, not fine-tuned):

Task	n	Valid JSON	Grounded	Fields match
ai_classification	4	0%	0%	0%
chat	38	100%	95%	0%
finding	15	0%	0%	0%
summary	3	0%	0%	0%
all	60	63%	60%	0%

This model (v0.6):

Task	n	Valid JSON	Grounded	Fields match
ai_classification	4	100%	100%	100%
chat	38	100%	100%	92%
finding	15	100%	100%	100%
summary	3	100%	100%	100%
all	60	100%	100%	95%

Limitations

• Trained on synthetic evidence; real-world wording varies. The app's grounding check and template fallback guard the report.
• 0.5B parameters: it phrases and selects retrieved facts well; it is not reliable for arithmetic or ranking, so the app supplies scores and top gaps from code.
• Not legal advice and not a certification. ISO 27001 control titles are cited; the standard's text is not reproduced.

Licence and citation

Apache-2.0, by Devseis. Please credit "Devseis Endpoint Auditor by Devseis".

@software{devseis_endpoint_auditor_model_2026,
  author  = {{Devseis}},
  title   = {Devseis Endpoint Auditor 0.5B},
  year    = {2026},
  version = {v0.6},
  url     = {https://huggingface.co/Devseis/endpoint-auditor-0.5b}
}