Instructions to use Felladrin/Minueza-32M-Base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Felladrin/Minueza-32M-Base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Felladrin/Minueza-32M-Base")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Felladrin/Minueza-32M-Base")
model = AutoModelForCausalLM.from_pretrained("Felladrin/Minueza-32M-Base")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Felladrin/Minueza-32M-Base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Felladrin/Minueza-32M-Base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Felladrin/Minueza-32M-Base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Felladrin/Minueza-32M-Base

SGLang

How to use Felladrin/Minueza-32M-Base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Felladrin/Minueza-32M-Base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Felladrin/Minueza-32M-Base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Felladrin/Minueza-32M-Base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Felladrin/Minueza-32M-Base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Felladrin/Minueza-32M-Base with Docker Model Runner:
```
docker model run hf.co/Felladrin/Minueza-32M-Base
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

A newer version of this model is available: Felladrin/Minueza-2-96M

Minueza-32M-Base

Summary

Minueza-32M-Base is a foundation model with 32 million parameters trained from scratch on a large corpus of text in English.

It's available in the following formats: Safetensors, GGUF, and ONNX.

And it's being released alongside some fine-tuned versions:

Minueza-32M-UltraChat: Trained on a single conversational dataset.
Minueza-32M-Chat: Trained on a mix of conversational datasets.
Minueza-32Mx2-Chat: Sparse Mixture of Experts trained on interleaved conversational datasets.
And more...

Intended Uses

This model was created with the following objectives in mind:

Run on mobile web browsers via Transformers.js.
Run fast on machines without GPU.
Serve as a base for fine-tunes using ChatML format, hence the two additional special tokens (<|im_start|> and <|im_end|>) with <|im_end|> as default EOS token.
- ChatML works great for both instruction and chat models, so if all fine-tunes are made following the ChatML pattern, other users might benefit from the easiness of creating merges.

Datasets

The model was trained on a subset of each of the following non-synthetic datasets:

The subsets were interleaved to form the final training corpus of approximately 650 million tokens.

Model Architecture

This is a transformer model with the Mistral architecture, trained on a context window of 2048 tokens.

Configuration	Value
max_position_embeddings	2048
hidden_size	312
intermediate_size	1092
num_attention_heads	12
num_hidden_layers	10
num_key_value_heads	4
vocab_size	32002

The pretraining was made with these hyperparameters and frameworks:

Hyperparameter	Value
learning_rate	5e-05
train_batch_size	1
eval_batch_size	1
seed	42
gradient_accumulation_steps	8
total_train_batch_size	8
optimizer	Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type	linear

Framework	Version
Transformers	4.38.0.dev0
Pytorch	2.1.2
Datasets	2.16.1
Tokenizers	0.15.1

Usage

This is just a base model. For your task, you will likely want to perform application-specific fine-tuning as recommended above.

Also note that this model was trained on internet text data, which may contain biases, offensive or inappropriate content, and may produce incorrect or irrelevant responses. No evaluation has been conducted, so use with care.

Having that said, here's how you can run it:

from transformers import pipeline

generate = pipeline("text-generation", "Felladrin/Minueza-32M-Base")

prompt = "The best way to improve your health is"

output = generate(
    prompt,
    max_new_tokens=256,
    do_sample=True,
    temperature=0.72,
    top_p=0.73,
    top_k=50,
    repetition_penalty=1.176,
)

print(output[0]["generated_text"])

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	28.92
AI2 Reasoning Challenge (25-Shot)	21.33
HellaSwag (10-Shot)	26.39
MMLU (5-Shot)	24.80
TruthfulQA (0-shot)	47.45
Winogrande (5-shot)	53.20
GSM8k (5-shot)	0.38