ToTo-40417 commited on
Commit
afd3882
·
unverified ·
1 Parent(s): b41d6e3

Document DP7 validation and expand LM Studio context

Browse files
Files changed (1) hide show
  1. README.md +8 -3
README.md CHANGED
@@ -8,7 +8,7 @@ tags: [japanese, instruction-tuning, little-language-model, tiny-language-model,
8
 
9
  # EXLLM-0.005B-Instruct
10
 
11
- EXLLM-0.005B-Instruct is a **0.005377824B-parameter** Japanese instruction model for local inference on low-memory embedded hardware. The primary deployment target is the CASIO EX-word XD-B4800. A PyTorch reference runtime is provided for host-side inference and evaluation. This repository also includes a separately trained **0.005441184B-parameter** Llama-compatible GGUF companion for LM Studio and llama.cpp.
12
 
13
  Here, LLM stands for Little Language Model.
14
 
@@ -96,13 +96,15 @@ All results use greedy decoding. TTFT means time to first generated token. Decod
96
 
97
  ### EX-word methodology
98
 
99
- - CASIO EX-word XD-B4800 / DATAPLUS 6.
100
  - SH7305-family SH-4A, 1 core / 1 thread; measured instruction clock 0.024184 GHz.
101
  - Five prompts, Thinking disabled, model stored in internal storage.
102
  - Application UI rendering and UTF-8 display handling are included.
103
  - Median TTFT: 23.711 s; observed range: 12.705–28.440 s.
104
  - Median total generation time: 54.145 s.
105
 
 
 
106
  The two runtimes use the same architecture and training lineage but different numerical representations. The results are deployment measurements, not a direct hardware-compute comparison.
107
 
108
  ## Evaluation
@@ -197,12 +199,15 @@ EXLLM8 and EXQ12 have the same file size but are not interchangeable formats. Re
197
 
198
  - Source and training code: [`ToTo-40417/exllm`](https://github.com/ToTo-40417/exllm)
199
  - EX-word runtime: [`ToTo-40417/exllm-exword`](https://github.com/ToTo-40417/exllm-exword)
 
200
  - Hardware probe: [`ToTo-40417/exword-hardware-dump`](https://github.com/ToTo-40417/exword-hardware-dump)
 
 
201
  - Project article (Japanese): [高校生用の電子辞書でLLMを動かしてみた ―0.024GHz/0.016GB](https://note.com/joyful_beetle869/n/nbd1e26679b78)
202
 
203
  ## 日本語
204
 
205
- EXLLM-0.005B-Instructは、低メモリ組込み機器でのローカル推論を目的とした、**0.005377824Bパラメータ**の日本語instruction modelです。主な実装対象はCASIO EX-word XD-B4800です。同じプロジェクトデータから別途学習した、LM Studio / llama.cpp向けの**0.005441184Bパラメータ**GGUF companionも収録しています。
206
 
207
  LLMとは、Little Language Modelの略です。
208
 
 
8
 
9
  # EXLLM-0.005B-Instruct
10
 
11
+ EXLLM-0.005B-Instruct is a **0.005377824B-parameter** Japanese instruction model for local inference on low-memory embedded hardware. Physical-device inference is verified on the CASIO EX-word XD-B4800 (DATAPLUS 6) and XD-N6500 (DATAPLUS 7). A PyTorch reference runtime is provided for host-side inference and evaluation. This repository also includes a separately trained **0.005441184B-parameter** Llama-compatible GGUF companion for LM Studio and llama.cpp.
12
 
13
  Here, LLM stands for Little Language Model.
14
 
 
96
 
97
  ### EX-word methodology
98
 
99
+ - Five-run baseline: CASIO EX-word XD-B4800 / DATAPLUS 6.
100
  - SH7305-family SH-4A, 1 core / 1 thread; measured instruction clock 0.024184 GHz.
101
  - Five prompts, Thinking disabled, model stored in internal storage.
102
  - Application UI rendering and UTF-8 display handling are included.
103
  - Median TTFT: 23.711 s; observed range: 12.705–28.440 s.
104
  - Median total generation time: 54.145 s.
105
 
106
+ A matched v1.2.0 deployment check used the same `model.q12` and Thinking OFF setting on both devices. Two `こんにちは` runs per device produced the same 24-token answer at 0.55 token/s. Median TTFT / total time was 12.702 s / 54.138 s on XD-B4800 and 12.707 s / 54.164 s on XD-N6500. Four matched fixed prompts differed by at most 11 ms in TTFT and 32 ms in total time, with no consistent winner. The recovered logs therefore validate deployment compatibility but do not establish a meaningful performance difference between these generations.
107
+
108
  The two runtimes use the same architecture and training lineage but different numerical representations. The results are deployment measurements, not a direct hardware-compute comparison.
109
 
110
  ## Evaluation
 
199
 
200
  - Source and training code: [`ToTo-40417/exllm`](https://github.com/ToTo-40417/exllm)
201
  - EX-word runtime: [`ToTo-40417/exllm-exword`](https://github.com/ToTo-40417/exllm-exword)
202
+ - EXQ12 validator: [`ToTo-40417/exllm-model-check`](https://github.com/ToTo-40417/exllm-model-check)
203
  - Hardware probe: [`ToTo-40417/exword-hardware-dump`](https://github.com/ToTo-40417/exword-hardware-dump)
204
+ - RAM qualification tool: [`ToTo-40417/exword-ram-scanner`](https://github.com/ToTo-40417/exword-ram-scanner)
205
+ - Gnuboy save-transfer utility: [`ToTo-40417/exword-gnuboy-save-importer`](https://github.com/ToTo-40417/exword-gnuboy-save-importer)
206
  - Project article (Japanese): [高校生用の電子辞書でLLMを動かしてみた ―0.024GHz/0.016GB](https://note.com/joyful_beetle869/n/nbd1e26679b78)
207
 
208
  ## 日本語
209
 
210
+ EXLLM-0.005B-Instructは、低メモリ組込み機器でのローカル推論を目的とした、**0.005377824Bパラメータ**の日本語instruction modelです。CASIO EX-word XD-B4800(DATAPLUS 6)とXD-N6500(DATAPLUS 7)で実機推論を確認しています。同じプロジェクトデータから別途学習した、LM Studio / llama.cpp向けの**0.005441184Bパラメータ**GGUF companionも収録しています。
211
 
212
  LLMとは、Little Language Modelの略です。
213