Update README.md
Browse files
README.md
CHANGED
|
@@ -95,7 +95,17 @@ You can download the [Mistral-7B-v0.1 base model in the GGUF format](https://hug
|
|
| 95 |
and use `ggml-adapter-model.bin` from this repository like this:
|
| 96 |
|
| 97 |
```
|
| 98 |
-
./main -ngl 32 -m mistral-7b-v0.1.Q4_K_M.gguf --color -c 4096 --temp 0 --repeat_penalty 1.1 -n -1 -p "[INST] who holds this neighborhood [/INST]" --lora ./ggml-adapter-model.bin
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 99 |
```
|
| 100 |
|
| 101 |
### Training Dataset and Resources
|
|
|
|
| 95 |
and use `ggml-adapter-model.bin` from this repository like this:
|
| 96 |
|
| 97 |
```
|
| 98 |
+
./main -ngl 32 -m mistral-7b-v0.1.Q4_K_M.gguf --color -c 4096 --temp 0 --repeat_penalty 1.1 -n -1 -p "[INST] who holds this neighborhood? [/INST]" --lora ./ggml-adapter-model.bin
|
| 99 |
+
```
|
| 100 |
+
|
| 101 |
+
### Running the model with mlx-lm
|
| 102 |
+
|
| 103 |
+
We merged Dragoman PT adapter into the base model and uploaded the quantized version of the model into https://huggingface.co/lang-uk/dragoman-4bit.
|
| 104 |
+
|
| 105 |
+
You can run the model using [mlx-lm](https://pypi.org/project/mlx-lm/):
|
| 106 |
+
|
| 107 |
+
```
|
| 108 |
+
python -m mlx_lm.generate --model lang-uk/dragoman-4bit --prompt '[INST] who holds this neighborhood? [/INST]' --temp 0 --max-tokens 100
|
| 109 |
```
|
| 110 |
|
| 111 |
### Training Dataset and Resources
|