Instructions to use nikitaredy/medictron-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nikitaredy/medictron-7B with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("BioMistral/BioMistral-7B")
model = PeftModel.from_pretrained(base_model, "nikitaredy/medictron-7B")

Transformers

How to use nikitaredy/medictron-7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nikitaredy/medictron-7B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("nikitaredy/medictron-7B", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use nikitaredy/medictron-7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nikitaredy/medictron-7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nikitaredy/medictron-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/nikitaredy/medictron-7B

SGLang

How to use nikitaredy/medictron-7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nikitaredy/medictron-7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nikitaredy/medictron-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nikitaredy/medictron-7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nikitaredy/medictron-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Studio

How to use nikitaredy/medictron-7B with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for nikitaredy/medictron-7B to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for nikitaredy/medictron-7B to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for nikitaredy/medictron-7B to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="nikitaredy/medictron-7B",
    max_seq_length=2048,
)

Docker Model Runner
How to use nikitaredy/medictron-7B with Docker Model Runner:
```
docker model run hf.co/nikitaredy/medictron-7B
```

BioMistral-7B Fine-Tuned on Indian Medical Data

A domain-adapted clinical LLM fine-tuned on synthetic Indian medical Q&A records using QLoRA (4-bit quantization) with Unsloth 2x speedup. Built to power the conversational AI layer.

Model Details

Developed by: B. Nikita Reddy
Model type: Causal LLM — BioMistral-7B + LoRA adapter (PEFT)
Languages: English, Hindi
License: MIT
Base model: BioMistral/BioMistral-7B

What It Does

Generates structured treatment recommendations — Allopathy, Homeopathy, Home Remedy
Supports voice input via Whisper ASR and returns spoken responses via TTS
Feeds patient vitals and symptom data into an XGBoost-based risk analyzer that scores patient risk from 0–100 (Low / Moderate / High / Critical) with SHAP explainability

Training Details

Parameter	Value
Base Model	BioMistral/BioMistral-7B
Method	QLoRA (4-bit) + Unsloth 2x speedup
Dataset	Synthetic Indian medical Q&A records
Hardware	Kaggle T4 GPU (15.6 GB VRAM)
Training Time	~9.5 hours
Steps / Epochs	873 steps, 5 epochs
Parameters Trained	~0.5% (LoRA only)
Adapter Size	167.8 MB LoRA safetensors
Final Train Loss	0.065
Final Val Loss	0.163 (target < 1.0 ✅)

Loss Curve (per checkpoint)

Step	Train Loss	Val Loss
200	0.129	0.219
400	0.094	0.188
600	0.076	0.176
800	0.068	0.163
873	0.065	0.163

Checkpoints saved at steps 600, 800, and 873.

How to Use

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained("BioMistral/BioMistral-7B")
model = PeftModel.from_pretrained(base_model, "YOUR_HF_USERNAME/biomistral-7b-indian-medical")
tokenizer = AutoTokenizer.from_pretrained("BioMistral/BioMistral-7B")

prompt = "Patient reports bukhar (fever) for 2 days. Suggest treatment."
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Preview

Limitations

Not a replacement for licensed medical advice
Optimised for Indian medical terminology — may underperform in other clinical contexts
Hindi coverage may not extend to all regional dialects
synthetic training samples — rare conditions may be underrepresented

Contact

B. Nikita Reddy

Framework Versions

PEFT 0.18.1
Unsloth (latest at training time)
Transformers (latest at training time)

Downloads last month: 73

Model tree for nikitaredy/medictron-7B

Base model

BioMistral/BioMistral-7B

Adapter

(30)

this model