--- license: apache-2.0 base_model: prism-ml/Ternary-Bonsai-4B-unpacked base_model_relation: finetune tags: - bitmtp - bonsai - ternary - multi-token-prediction inference: false --- # BitMTP-Bonsai-4B-D8 Bonsai-4B checkpoint trained with the joint NTP + MTP objective at block size D=8. [Code and installation](https://github.com/PolyU-VCLab/BitMTP) ## Format and use This repository contains a latent QAT checkpoint, configuration, and matching tokenizer. Convert it with BitMTP to group-128 ternary runtime weights before inference. After installing BitMTP, run: ```bash python -m bonsai_sft.download_model \ --model bonsai_4b_d8 --output weights/BitMTP-Bonsai-4B-D8 python -m bonsai_sft.gpu.convert_bonsai_mtp_to_gpu \ --src weights/BitMTP-Bonsai-4B-D8 --out weights/BitMTP-Bonsai-4B-D8-packed --quant ternary python -m bonsai_sft.gpu.run_generation \ --ckpt weights/BitMTP-Bonsai-4B-D8-packed --hf-ckpt weights/BitMTP-Bonsai-4B-D8 \ --block-size 8 --mode hybrid --threshold 0.9 --min-chain 6 --draft-precision fp8 --verify-precision native \ --chat --prompt 'Write a short greeting.' --max-new-tokens 128 ``` The example targets H20. ## Attribution Created using Bonsai by Prism ML, based on Qwen3. BitMTP modifies the model through supervised fine-tuning and joint NTP + MTP training. The upstream Apache-2.0 license and notices are included in `LICENSE` and `NOTICE.txt`.