Update README.md
Browse files
README.md
CHANGED
|
@@ -26,13 +26,13 @@ The MediPhi Model Collection comprises 7 small language models of 3.8B parameter
|
|
| 26 |
## Model Details
|
| 27 |
### Model Description
|
| 28 |
|
| 29 |
-
This model is
|
| 30 |
|
| 31 |
- **Developed by:** Microsoft Healthcare \& Life Sciences
|
| 32 |
- **Model type:** Phi3
|
| 33 |
- **Language(s) (NLP):** English
|
| 34 |
- **License:** MIT
|
| 35 |
-
- **Finetuned from model:** `microsoft/
|
| 36 |
|
| 37 |
### Model Sources
|
| 38 |
|
|
@@ -90,7 +90,7 @@ Researchers should apply responsible AI best practices, including mapping, measu
|
|
| 90 |
|
| 91 |
torch.random.manual_seed(0)
|
| 92 |
|
| 93 |
-
model_name = "microsoft/MediPhi
|
| 94 |
model = AutoModelForCausalLM.from_pretrained(
|
| 95 |
model_name,
|
| 96 |
device_map="cuda",
|
|
@@ -142,9 +142,6 @@ Continual Pre-training:
|
|
| 142 |
- `zhengyun21/PMC-Patients`, `akemiH/NoteChat`, and `starmpcc/Asclepius-Synthetic-Clinical-Notes` (only commercial-friendly licenses across all three datasets)
|
| 143 |
- mtsamples
|
| 144 |
|
| 145 |
-
Clinical alignment:
|
| 146 |
-
- `microsoft/mediflow`
|
| 147 |
-
|
| 148 |
See paper for details.
|
| 149 |
|
| 150 |
### Training Procedure
|
|
|
|
| 26 |
## Model Details
|
| 27 |
### Model Description
|
| 28 |
|
| 29 |
+
This model is `MediPhi` obtained by merging all 5 experts with the BreadCrumbs technique into this unified expert.
|
| 30 |
|
| 31 |
- **Developed by:** Microsoft Healthcare \& Life Sciences
|
| 32 |
- **Model type:** Phi3
|
| 33 |
- **Language(s) (NLP):** English
|
| 34 |
- **License:** MIT
|
| 35 |
+
- **Finetuned from model:** `microsoft/Phi-3.5-mini-instruct`
|
| 36 |
|
| 37 |
### Model Sources
|
| 38 |
|
|
|
|
| 90 |
|
| 91 |
torch.random.manual_seed(0)
|
| 92 |
|
| 93 |
+
model_name = "microsoft/MediPhi"
|
| 94 |
model = AutoModelForCausalLM.from_pretrained(
|
| 95 |
model_name,
|
| 96 |
device_map="cuda",
|
|
|
|
| 142 |
- `zhengyun21/PMC-Patients`, `akemiH/NoteChat`, and `starmpcc/Asclepius-Synthetic-Clinical-Notes` (only commercial-friendly licenses across all three datasets)
|
| 143 |
- mtsamples
|
| 144 |
|
|
|
|
|
|
|
|
|
|
| 145 |
See paper for details.
|
| 146 |
|
| 147 |
### Training Procedure
|