Instructions to use memoriant/cmmc-expert-gemma4-31b-poc with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use memoriant/cmmc-expert-gemma4-31b-poc with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="memoriant/cmmc-expert-gemma4-31b-poc")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("memoriant/cmmc-expert-gemma4-31b-poc", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use memoriant/cmmc-expert-gemma4-31b-poc with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf memoriant/cmmc-expert-gemma4-31b-poc:Q5_K_M
# Run inference directly in the terminal:
llama cli -hf memoriant/cmmc-expert-gemma4-31b-poc:Q5_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf memoriant/cmmc-expert-gemma4-31b-poc:Q5_K_M
# Run inference directly in the terminal:
llama cli -hf memoriant/cmmc-expert-gemma4-31b-poc:Q5_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf memoriant/cmmc-expert-gemma4-31b-poc:Q5_K_M
# Run inference directly in the terminal:
./llama-cli -hf memoriant/cmmc-expert-gemma4-31b-poc:Q5_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf memoriant/cmmc-expert-gemma4-31b-poc:Q5_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf memoriant/cmmc-expert-gemma4-31b-poc:Q5_K_M

Use Docker

docker model run hf.co/memoriant/cmmc-expert-gemma4-31b-poc:Q5_K_M

LM Studio
Jan

vLLM

How to use memoriant/cmmc-expert-gemma4-31b-poc with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "memoriant/cmmc-expert-gemma4-31b-poc"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "memoriant/cmmc-expert-gemma4-31b-poc",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/memoriant/cmmc-expert-gemma4-31b-poc:Q5_K_M

SGLang

How to use memoriant/cmmc-expert-gemma4-31b-poc with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "memoriant/cmmc-expert-gemma4-31b-poc" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "memoriant/cmmc-expert-gemma4-31b-poc",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "memoriant/cmmc-expert-gemma4-31b-poc" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "memoriant/cmmc-expert-gemma4-31b-poc",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use memoriant/cmmc-expert-gemma4-31b-poc with Ollama:
```
ollama run hf.co/memoriant/cmmc-expert-gemma4-31b-poc:Q5_K_M
```
Unsloth Desktop
Docker Model Runner
How to use memoriant/cmmc-expert-gemma4-31b-poc with Docker Model Runner:
```
docker model run hf.co/memoriant/cmmc-expert-gemma4-31b-poc:Q5_K_M
```

Lemonade

How to use memoriant/cmmc-expert-gemma4-31b-poc with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull memoriant/cmmc-expert-gemma4-31b-poc:Q5_K_M

Run and chat with the model

lemonade run user.cmmc-expert-gemma4-31b-poc-Q5_K_M

List all available models

lemonade list

Atomic Chat

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

This model is a proof-of-concept compliance AI. By requesting access, you agree to the terms below. Access is granted automatically.

Memoriant CMMC Expert — Gemma 4 31B (Proof of Concept)

A fine-tuned Gemma 4 31B model for CMMC Level 2 compliance documentation, deployed air-gapped on NVIDIA DGX Spark.

⚠️ AI Disclaimer

This model is a proof-of-concept research artifact, not a certified compliance tool. Outputs are AI-generated and may contain inaccuracies, omissions, or errors.

Not a substitute for qualified CMMC assessors (C3PAOs), compliance consultants, or legal counsel

Not certified by the Cyber AB, DoD, or any accreditation body

All outputs must be reviewed by a qualified compliance professional before use in any official capacity

Do not rely on this model as the sole basis for compliance decisions, System Security Plans, or assessment evidence

The developers make no warranty regarding the accuracy, completeness, or fitness for any particular purpose

Use at your own risk. By downloading this model, you acknowledge these limitations.

What This Is

This is a proof-of-concept demonstrating that a locally-deployed, air-gapped language model can produce compliance documentation competitive with frontier cloud models — while keeping Controlled Unclassified Information (CUI) entirely within the customer's network.

Part of a systematic study across 4 base model families and a comparative benchmark against 12 frontier models.

Key Results

Document Generation (where this POC excels)

Task	Score	What It Does
SSP Narrative Generation	0.540	Drafts System Security Plan implementation narratives for CMMC L2 controls
POA&M Drafting	0.501	Generates Plan of Action & Milestones items for identified gaps
Factual Recall	0.439	Accurate knowledge of control IDs, framework requirements, and regulatory structure
Citation Accuracy	0.427	Correctly references NIST/CMMC/DFARS source documents

Competitive Benchmark (15 models tested)

Models were run against the full 1,273-question v3 benchmark (15 tiers). The competitive leaderboard against frontier models was scored on the 105-question T10/T11/T12 subset where retrieval-grounded judge grading was complete:

Rank	Model	Category
1	Memoriant system (proprietary configuration)	Air-gapped, local
2	Claude Opus 4.6 (Anthropic)	Cloud API
3	Gemini 3 Flash Preview (Google)	Cloud API
4	Claude Sonnet 4.6 (Anthropic)	Cloud API
5	GPT-4o-mini (OpenAI)	Cloud API

The Memoriant retrieval system (Gemma 2 27B + Memoriant Ground) edged Claude Opus 4.6 on the overall average (0.266 vs 0.261). Single-judge, pre-calibration.

Architecture

Component	Detail
Base model	Gemma 4 31B
Fine-tuning method	QLoRA
Quantization	Q5_K_M (21 GB GGUF)
Deployment	Ollama on NVIDIA DGX Spark GB10 (128GB unified memory)
Retrieval	Proprietary compliance knowledge index
Operation mode	100% air-gapped — no internet required

Why Air-Gapped Matters

DoD contractors handling CUI under DFARS 252.204-7012 face a regulatory constraint: CUI must remain within authorized environments. Most frontier AI APIs (Claude, GPT, Gemini) are not FedRAMP High authorized for CUI workloads in typical contractor environments.

This product runs entirely on customer hardware. No data leaves the network. No API calls to external services. The model, the retrieval index, and the inference engine all run locally on a single NVIDIA GPU server.

Framework Coverage

Trained across 8+ compliance frameworks:

CMMC 2.0 (32 CFR Part 170) — L1, L2, L3
NIST SP 800-171 Rev. 2 & 3 (110 requirements)
NIST SP 800-172 (enhanced CUI controls)
NIST SP 800-53 Rev. 5 (1,189+ controls)
NIST SP 800-37 (Risk Management Framework)
NIST CSF 2.0
HIPAA Security Rule
DFARS 252.204-7008/7012/7019-7021

Status & Roadmap

This model is a proof of concept. It demonstrates the viability of air-gapped compliance AI and establishes baseline performance across document generation tasks.

Next phase: Enhanced reasoning capabilities for compliance evaluation and assessment methodology. Currently in development.

Current production work: See the Memoriant organization page for the latest.

Limitations

This POC is optimized for document generation (SSP narratives, POA&M items). It is not trained for free-form compliance Q&A, evidence evaluation, or assessment methodology reasoning.
Outputs should be treated as first drafts requiring professional review and editing.
The model may produce plausible-sounding but incorrect regulatory citations, control IDs, or compliance guidance.
Performance varies by control family, question complexity, and deployment configuration.
Benchmark scores reflect performance under specific test conditions and may not generalize to all real-world scenarios.

Training Data

All training data derived from publicly available US government documents. No proprietary, classified, or CUI data was used in any training run.

Citation

@misc{maine2026cmmcexpert,
  author = {Maine, Nathan},
  title = {CMMC Expert: Air-Gapped Compliance AI on NVIDIA DGX Spark},
  year = {2026},
  publisher = {Memoriant Inc.},
  howpublished = {\url{https://huggingface.co/memoriant/cmmc-expert-gemma4-31b-poc}}
}

License

Apache 2.0 (model weights)

Contact

Organization: Memoriant Inc.
Author: Nathan Maine
GitHub: NathanMaine

Downloads last month: 11

GGUF

Model size

31B params

Architecture

gemma4

Hardware compatibility

5-bit