How-To: Train/Fine-Tune Local Qwen Models

This guide outlines how to use the exported RAG datasets to fine-tune local models (e.g. Qwen/Qwen2.5-Coder-14B-Instruct) on an RTX 5060 Ti 16GB GPU.

---

1. Core Training Philosophy

To achieve high-quality code generation, do not limit training to corpus.jsonl. You should ingest all four machine-readable assets to ensure the model masters different coding dimensions:

• corpus.jsonl: Teaches core architectural concepts, tutorials, design rules, and golden components style.
• core.jsonl: Teaches Python interfaces, protocol signatures, and decorator classes.
• openapi.jsonl: Teaches rest endpoints, argument schemas, and HTTP method contracts.
• doctypes.jsonl: Teaches database models, fields structure, and validations.

---

2. Environment Setup

We recommend utilizing Unsloth for training as it utilizes hand-optimized CUDA kernels that reduce VRAM consumption by up to 60%, allowing a 14B model to easily fit within a 16GB VRAM constraint.

# Create a new environment
conda create --name unsloth_env python=3.10 -y
conda activate unsloth_env

# Install Unsloth and dependencies
pip install --upgrade pip
pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
pip install --no-deps "xformers<0.0.27" trl peft transformers accelerate bitsandbytes

---

3. Formatting Dataset for Supervised Fine-Tuning (SFT)

Convert all four machine-readable .jsonl files into standard ShareGPT conversations format. This gives the model custom instructions for classes, endpoints, and fixtures:

import json
from pathlib import Path

def format_sharegpt():
    files = ["corpus.jsonl", "core.jsonl", "openapi.jsonl", "doctypes.jsonl"]
    sft_data = []
    
    for filename in files:
        filepath = Path("docs/machine") / filename
        if not filepath.exists():
            continue
            
        with open(filepath, "r", encoding="utf-8") as f:
            for line in f:
                if not line.strip():
                    continue
                record = json.loads(line)
                
                # 1. Format Golden Fixtures
                if record.get("type") == "fixture":
                    sft_data.append({
                        "conversations": [
                            {"from": "human", "value": f"Write a standard Framework M {record['title']} template."},
                            {"from": "gpt", "value": record["content"]}
                        ]
                    })
                
                # 2. Format AST Core signatures
                elif record.get("type") == "class":
                    methods_str = "\n".join([m.get("signature", "") for m in record.get("methods", [])])
                    sft_data.append({
                        "conversations": [
                            {"from": "human", "value": f"What is the Python interface definition and methods signature for class {record['name']} in framework_m?"},
                            {"from": "gpt", "value": f"Class: {record['name']}\nModule: {record['module']}\nDocstring: {record['docstring']}\nSignatures:\n{methods_str}"}
                        ]
                    })
                    
                # 3. Format OpenAPI routes
                elif "method" in record and "path" in record:
                    sft_data.append({
                        "conversations": [
                            {"from": "human", "value": f"Explain the REST API contract for endpoint {record['method']} {record['path']} in standard Framework M."},
                            {"from": "gpt", "value": f"Endpoint: {record['method']} {record['path']}\nSummary: {record['summary']}\nSchema:\n{json.dumps(record, indent=2)}"}
                        ]
                    })

    with open("docs/machine/sft_training.json", "w", encoding="utf-8") as out:
        json.dump(sft_data, out, indent=2)

if __name__ == "__main__":
    format_sharegpt()

---

3. LoRA Configuration (RTX 5060 Ti 16GB)

Below is the optimized QLoRA script template configuration. It loads Qwen2.5-Coder-14B in 4-bit and targets all linear layers:

from unsloth import FastLanguageModel
import torch
from datasets import load_dataset
from trl import SFTTrainer
from transformers import TrainingArguments

max_seq_length = 2048 # Keep sequence length low to save VRAM
dtype = None # Auto detection (Float16/Bfloat16 depending on GPU)
load_in_4bit = True # Enable 4-bit QLoRA to fit 14B in 16GB VRAM

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "Qwen/Qwen2.5-Coder-14B-Instruct",
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
)

# Apply LoRA adapter config targeting projection matrices
model = FastLanguageModel.get_peft_model(
    model,
    r = 16,
    target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
                      "gate_proj", "up_proj", "down_proj"],
    lora_alpha = 32,
    lora_dropout = 0,
    bias = "none",
    use_gradient_checkpointing = "unsloth", # Saves massive VRAM
    random_state = 3407,
)

# Load your SFT formatted JSON dataset
dataset = load_dataset("json", data_files="docs/machine/sft_training.json", split="train")

trainer = SFTTrainer(
    model = model,
    tokenizer = tokenizer,
    train_dataset = dataset,
    dataset_text_field = "text",
    max_seq_length = max_seq_length,
    dataset_num_proc = 2,
    packing = False, # Can speed up training for short contexts
    args = TrainingArguments(
        per_device_train_batch_size = 1, # Small batch size to fit in VRAM
        gradient_accumulation_steps = 8,  # Simulate batch size 8
        warmup_steps = 5,
        max_steps = 60,
        learning_rate = 2e-4,
        fp16 = not torch.cuda.is_bf16_supported(),
        bf16 = torch.cuda.is_bf16_supported(),
        logging_steps = 1,
        optim = "paged_adamw_8bit", # Offload optimizer states
        weight_decay = 0.01,
        lr_scheduler_type = "linear",
        seed = 3407,
        output_dir = "outputs",
    ),
)

trainer_stats = trainer.train()

---

4. Saving and Exporting Weights

After training, you can export the LoRA adapters or merge them into a single 16-bit GGUF model for running locally via Ollama:

# Save LoRA adapters
model.save_pretrained_merged("lora_model", tokenizer, save_method = "lora")

# Merge LoRA to 16-bit base model and export directly to GGUF format
model.save_pretrained_merged("qwen14b_framework_m", tokenizer, save_method = "merged_16bit")

Once exported, use llama.cpp to quantize the merged model to GGUF q4_K_M or q8_0 for Ollama execution.