Political leaning classifier (Qwen3.5-4B, merged LoRA)

This repository is a merged (full-weight) fine-tune of Qwen/Qwen3.5-4B for English news-style text. It assigns a single integer score from 0 to 4 describing estimated political leaning along a left–right spectrum.

Project goal

Support automated triage and analysis of large text collections (e.g. monitoring relative slant in news-like passages). The model is trained as a causal LM that completes a fixed instruction prompt and is scored by the probability it assigns to the digit tokens "0" … "4" at the final position—not by a separate classification head in the Hugging Face sense.

Label semantics

Labels are ordinal in prompt wording only; training optimizes categorical next-token prediction over the five digit tokens.

Training data

• Two CSV sources of plain text + integer label in {0,…,4}, combined and shuffled.
• After cleaning (non-empty text, valid labels), the run used on the order of ~10k examples.
• Stratified split: 90% train / 10% validation by label, fixed random seed for reproducibility.

Dataset names and exact row counts can differ if you re-export CSVs; keep the same split logic if you want to reproduce reported validation numbers.

Method (high level)

1. LoRA on attention MLP blocks (q/k/v/o, gate/up/down) plus a trimmed lm_head (see below).
2. LM head trim: only the tokenizer rows for single-token digits 0–4 are kept for the trainable head slice, to reduce memory and stabilize the classification signal; after training, weights are merged back into a full-vocabulary head for standard transformers loading.
3. Last-token loss: only the final answer digit is supervised; the rest of the sequence is masked in the loss.
4. Length bucketing: training texts are sorted into length buckets (with shuffling inside buckets) so padding stays closer to necessary batch shapes than random order.
5. Merged checkpoint: LoRA adapters are folded into the base weights before publication so consumers need only from_pretrained—no PEFT merge step.

Training used Unsloth and TRL SFTTrainer.

Hyperparameters

Validation (held-out stratified split)

On the fixed 10% stratified validation split from the training pipeline (exact size depends on the cleaned union of the two CSVs), one reference run reported:

• Accuracy ≈ 81.4%
• Mean absolute error on scores 0–4 ≈ 0.28

Treat these as internal sanity metrics on a specific news/bias-oriented mixture, not a guarantee on other domains, languages, or time periods.

How to run inference

The implementation loads the model class appropriate for Qwen3.5 multimodal checkpoints (AutoModelForImageTextToText when available, else causal LM). Use text-only prompts.

Recommended decoding: build the prompt through the supervised prefix ending at "... class " (see your training/inference scripts), tokenize with left padding for batches, take last-token logits, then take a softmax only over the five tokenizer IDs for "0" … "4" and choose argmax as the predicted class index 0–4. This matches training and avoids unnecessary full-vocabulary work.

Dependencies: recent torch, transformers, accelerate; optional quantization stacks (e.g. HQQ) if you shrink memory at inference—expect small metric drift.

Minimal pattern:

import torch
import torch.nn.functional as F
from transformers import AutoTokenizer

try:
 from transformers import AutoModelForImageTextToText as AutoModelCls
except ImportError:
 from transformers import AutoModelForCausalLM as AutoModelCls

MODEL_ID = "zhezhou1106/political-leaning-classifier-v2" # or a local folder

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=True)
model = AutoModelCls.from_pretrained(
 MODEL_ID,
 torch_dtype=torch.bfloat16,
 device_map="auto",
 trust_remote_code=True,
)
model.eval()

number_token_ids = []
for digit in range(5):
 ids = tokenizer.encode(str(digit), add_special_tokens=False)
 assert len(ids) == 1, "Tokenizer must encode digits 0–4 as a single token each."
 number_token_ids.append(ids[0])


def classify(article_text: str) -> int:
 prompt = """You are an expert media bias analyst. Read the following news article and classify its political leaning on a 0-4 scale.

Article:
{article}

Scale:
class 0: extremely left (far left)
class 1: lean left
class 2: center / neutral
class 3: lean right
class 4: extremely right (far right)

SOLUTION
The correct answer is: class """.format(
 article=article_text.strip()
 )
 tokenizer.padding_side = "left"
 enc = tokenizer([prompt], return_tensors="pt", padding=True).to(model.device)
 with torch.inference_mode():
 out = model(**enc, logits_to_keep=1)
 last = out.logits[:, -1, :]
 digit_logits = last[:, number_token_ids]
 probs = F.softmax(digit_logits, dim=-1)
 return int(probs.argmax(dim=-1).item())

Adapt the prompt string exactly to what you used in training if you need strict reproducibility.

Limitations and responsible use

• Bias & uncertainty: Scores reflect patterns in the training mixtures (sources, labeling, English news-like text). Outputs can embed demographic or ideological biases present in those data. Do not use this model alone for consequential decisions about people or organizations without human review.
• Not objective truth: Political leaning labels are inherently subjective; treat outputs as heuristic coarse scores, not factual claims about ideology.
• Domain shift: Expect performance drops on legal text, forums, multilingual text, sarcasm-heavy posts, headlines-only inputs, etc.
• Ordinal vs categorical: Metrics like MAE are descriptive only—the head was trained as a 5-way digit prediction problem.

For questions about the upstream base model capabilities, licensing nuances, or safety tooling, consult the Qwen3.5-4B model card.

Citation

If you use this fine-tuned checkpoint, cite the upstream Qwen model appropriately and cite or link this Hugging Face repository. You may also cite Unsloth/TRL depending on academic conventions.

Base model

Qwen/Qwen3.5-4B-Base

Qwen/Qwen3.5-4B

(259)