Instructions to use Cyleux/oc1b-16bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Cyleux/oc1b-16bit with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Cyleux/oc1b-16bit", dtype="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Unsloth Studio
How to use Cyleux/oc1b-16bit with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for Cyleux/oc1b-16bit to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for Cyleux/oc1b-16bit to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for Cyleux/oc1b-16bit to start chatting
Load model with FastModel
pip install unsloth from unsloth import FastModel model, tokenizer = FastModel.from_pretrained( model_name="Cyleux/oc1b-16bit", max_seq_length=2048, )
| {%- for message in messages %} | |
| {#-- Validate role is a stringified integer --#} | |
| {%- if not message['role'] is string or not message['role'].isdigit() %} | |
| {{- raise_exception("The role must be an integer or a stringified integer (e.g. '0') designating the speaker id") }} | |
| {%- endif %} | |
| {#-- Validate content is a list --#} | |
| {%- set content = message['content'] %} | |
| {%- if content is not iterable or content is string %} | |
| {{- raise_exception("The content must be a list") }} | |
| {%- endif %} | |
| {#-- Collect content types --#} | |
| {%- set content_types = content | map(attribute='type') | list %} | |
| {%- set is_last = loop.last %} | |
| {#-- Last message validation --#} | |
| {%- if is_last %} | |
| {%- if 'text' not in content_types %} | |
| {{- raise_exception("The last message must include one item of type 'text'") }} | |
| {%- elif (content_types | select('equalto', 'text') | list | length > 1) or (content_types | select('equalto', 'audio') | list | length > 1) %} | |
| {{- raise_exception("At most two items are allowed in the last message: one 'text' and one 'audio'") }} | |
| {%- endif %} | |
| {#-- All other messages validation --#} | |
| {%- else %} | |
| {%- if content_types | select('equalto', 'text') | list | length != 1 | |
| or content_types | select('equalto', 'audio') | list | length != 1 %} | |
| {{- raise_exception("Each message (except the last) must contain exactly one 'text' and one 'audio' item") }} | |
| {%- elif content_types | reject('in', ['text', 'audio']) | list | length > 0 %} | |
| {{- raise_exception("Only 'text' and 'audio' types are allowed in content") }} | |
| {%- endif %} | |
| {%- endif %} | |
| {%- endfor %} | |
| {%- for message in messages %} | |
| {{- bos_token }} | |
| {{- '[' + message['role'] + ']' }} | |
| {{- message['content'][0]['text'] }} | |
| {{- eos_token }} | |
| {%- if message['content']|length > 1 %} | |
| {{- '<|AUDIO|><|audio_eos|>' }} | |
| {%- endif %} | |
| {%- endfor %} | |