---
base_model: mistralai/Ministral-3-3B-Base-2512
library_name: transformers
tags:
- ministral-3
- text-generation
- base
- llamafied
- novision
license: apache-2.0
language:
- en
---

![Mini_Llama](https://cdn-uploads.huggingface.co/production/uploads/6686e9b5a244c4409d9ddf08/X6kYym6YhTSpoK_WRLzUA.png)

# Mini-Llama 3B Base Pretrain - 0124

I have been through several iterations and this is my current test base. Use at your own risk.

Architectural modifications by mrfakename:
See [mrfakename/Ministral-3-3B-Base-2512-Llamafied-TextOnly](https://huggingface.co/mrfakename/Ministral-3-3B-Base-2512-Llamafied-TextOnly)

I have added the jinja from Llama 3.2 3B, added all template tokens to the vocabulary, changed Ministral's built in `[THINK]` and `[/THINK]` tags to traditional `<think>` and `</think>` tags, padded to a multiple of 128 and resized the embeddings.

This is a base pretrain, ready for instruct training.

** Special Note: Due to the custom vocabulary, the tokenizer will not be recognized by llama.cpp, you will have to add its checksum to convert_hf_to_gguf.py before converting any derivatives to gguf.