license: other license_name: qwen base_model: Qwen/Qwen3.5-9B library_name: peft tags: [dora, peft, adapter, dfe-stacking, qwen3.5]
T1 Code Task Expert — flat, seed 42 | DFE-Stacking release
DoRA adapter from the paper "Fine-Tuning Shifts Form Before Competence". This is one of 12 released adapters covering all experimental conditions and seeds from a matched-arm ablation on Qwen 3.5 9B (Gated-DeltaNet/attention hybrid).
| Role | task expert r=48, flat, seed 42 |
| Substrate | bare base |
| Training data | T1 code v2 (Stack Overflow-mined), 3,000 records |
| Precision | NF4 QDoRA, evaluated merged BF16 |
Usage
from transformers import AutoModelForCausalLM
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen3.5-9B",
torch_dtype="bfloat16"
)
model = PeftModel.from_pretrained(base, "OthmaneBen/dfe-task-t1code-v2-flat-s42")
# For stacked task experts, first load and merge the domain expert:
# base = PeftModel.from_pretrained(
# base,
# "OthmaneBen/dfe-domain-expert-r96"
# ).merge_and_unload()
Generation warning: the base checkpoint's eos_token_id does not include the chat
end-of-turn token. Pass the full stop list or use the generation_config.json included
in this repo. Merge before batch generation because unmerged DoRA is about 10x slower.
Findings supported by this adapter, as described in the paper: no seen-task benefit from stacking; a deterministic version-formatting coercion out of distribution (flat 72%, stacked 38 to 45%, joint 0%); and no genuine capability loss detected on any audited instrument.
═══════════════════════════════════════════════════════════════════
- Downloads last month
- 5