klei1/bleta-sq-2b

Bleta SQ 2B — Albanian Language Model

Bleta is a fine-tuned Gemma-4 2B model specialized for the Albanian language (Shqip). Built to understand and generate natural, grammatically correct Albanian text.

Bleta (🐝) means "bee" in Albanian — a symbol of diligence and precision.

Model Details

Property	Value
Base Model	google/gemma-4-2b-it
Architecture	Gemma4ForConditionalGeneration
Parameters	~2 Billion
Fine-tuning Method	LoRA → merged into full weights
Language	Albanian (sq), English (en)
License	Apache 2.0
All-Time Downloads	2,879

Training Dataset

Fine-tuned on klei1/bleta-sq-dataset-v1 — a curated Albanian language instruction dataset covering conversation, grammar, reasoning, and general knowledge.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "klei1/bleta-sq-2b"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

messages = [
    {"role": "user", "content": "Cila eshte kryeqyteti i Shqiperise?"}
]

inputs = tokenizer.apply_chat_template(
    messages,
    return_tensors="pt",
    add_generation_prompt=True
).to(model.device)

outputs = model.generate(
    inputs,
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.95,
    repetition_penalty=1.2,
    do_sample=True,
)

response = tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True)
print(response)

Recommended Generation Parameters

Parameter	Value	Notes
temperature	0.7	Balanced creativity
max_new_tokens	400–512	Prevents loops
repetition_penalty	1.2	Reduces repetition
top_p	0.95	Nucleus sampling

Capabilities

• Albanian conversational AI
• Grammar correction and explanation
• Albanian text generation and creative writing
• Translation (Albanian ↔ English)
• General knowledge in Albanian
• Question answering

Limitations

• 2B parameter model — complex reasoning may be limited
• Primarily trained on Albanian; performance varies by topic
• May occasionally produce grammatically imperfect outputs

Bleta Model Family

Model	Params	Focus
bleta-sq-2b	2B	Albanian general
bleta-meditor-27b	27B	Medical + specialized
bleta-logjike-27b	27B	Logic + reasoning
bleta-1B	1B	Lightweight

Citation

@model{bleta_sq_2b_2026,
  title  = {Bleta SQ 2B: Gemma-4 Fine-tuned for Albanian Language},
  author = {klei1},
  year   = {2026},
  url    = {https://huggingface.co/klei1/bleta-sq-2b}
}

License

This model is released under the Apache 2.0 License.