berkerdooo/qwen38-swift

Qwen3.8-27B Swift / SwiftUI

Qwen/Qwen3.8-27B fine-tuned for Swift and SwiftUI coding (LoRA merged into the full bf16 weights): writing and editing app code, fixing compiler errors, writing tests, and multi-step tool-use (agent) work in Xcode projects.

The LoRA adapter alone is at berkerdooo/qwen38-swift-lora.

Training

Method	Supervised fine-tuning, LoRA (r=32, alpha=32, dropout 0), bf16
Target modules	Language model only: self_attn.{q,k,v,o}_proj, linear_attn.{in_proj_qkv,in_proj_z,out_proj}, mlp.{gate,up,down}_proj. Vision tower and MTP head frozen.
Trainable parameters	217.6M (0.79%)
Data	Private dataset of real-world Swift / SwiftUI work: app source code, commit-based edit tasks, task-to-edit pairs, compile-error fixes, agent trajectories, fill-in-the-middle. The dataset is not published.
Examples / steps	19,040 examples, 1,190 optimizer steps (16 sequences per step), 1 epoch
Max sequence length	16,384 tokens
Optimizer	AdamW, learning rate 1e-4, cosine schedule, 3% warmup
Loss	Assistant turns only for chat data
Hardware	1x NVIDIA H200, 9 h 41 min

Held-out eval loss: 0.295 (17% of the epoch) -> 0.272 -> 0.257 -> 0.248 -> 0.242 -> 0.2418 (end). Mean train loss: 0.3615.

Usage

import torch
from transformers import AutoModelForImageTextToText, AutoTokenizer

model = AutoModelForImageTextToText.from_pretrained("berkerdooo/qwen38-swift", dtype=torch.bfloat16, device_map="auto")
tok = AutoTokenizer.from_pretrained("berkerdooo/qwen38-swift")

messages = [{"role": "user", "content": "Write a SwiftUI view that shows a searchable list of items with swipe-to-delete."}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False,
                                 return_tensors="pt", return_dict=True).to(model.device)
print(tok.decode(model.generate(**inputs, max_new_tokens=1024)[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Also works with vLLM: vllm serve berkerdooo/qwen38-swift.

Needs transformers>=5.8. For fast Gated DeltaNet layers, install flash-linear-attention and causal-conv1d. On Hopper GPUs (H100 / H200) with Triton 3.4-3.7.0, also install tilelang (flash-linear-attention issue #640).

Thinking mode

The chat template in this repo turns thinking off by default (the base model turns it on by default). The model was trained almost only on direct answers, so direct answers are its intended use. To turn thinking on, pass enable_thinking=True to apply_chat_template (or chat_template_kwargs in vLLM); quality in that mode was not tested.

Limitations

• No benchmark results yet; only held-out loss was measured.
• The model reflects one developer's code style and project conventions.
• It is a text-only fine-tune. Image input was not trained.
• Generated code can be wrong, outdated, or insecure. Review and test it before use.

Disclaimer

• This is an independent, personal fine-tune. It is not affiliated with or endorsed by Alibaba / the Qwen team, Apple, or any other company. Swift and SwiftUI are trademarks of Apple Inc.
• The training dataset is private and is not distributed. It was built from the author's own software projects and AI-assisted coding sessions. Known secrets and personal identifiers were removed before training, but, like any language model, this model can reproduce fragments of its training data.
• The weights are released under Apache 2.0, the same license as the base model, and are provided "as is", without warranty of any kind. You are responsible for how you use the model and its output, including compliance with the licenses and terms that apply to you.