Qwen 3.6 & 2.5: The Most Versatile Local Models

# Install Ollama if you haven't
curl -fsSL https://ollama.com/install.sh | sh

# For most users — the best balance of quality and speed
ollama pull qwen2.5:7b

# Start chatting
ollama run qwen2.5:7b

>>> Summarize this 100-page document in 3 bullet points:
[paste your document here — it can be very long!]

# Pull the latest Qwen
ollama pull qwen3.6:27b

# Or the coding-optimized version
ollama pull qwen3.6:27b-coding

FROM qwen2.5:14b

PARAMETER temperature 0.2
PARAMETER top_p 0.9
PARAMETER num_ctx 65536
PARAMETER repeat_penalty 1.1

SYSTEM """You are a senior software engineer with 15 years of experience.
YOU MUST:
- Write clean, production-ready code
- Include type hints and docstrings
- Handle errors gracefully
- Prefer standard library over external dependencies
- Explain your approach briefly before writing code
NEVER apologize. Just write the code."""

ollama create qwen-coder -f Modelfile
ollama run qwen-coder

FROM qwen2.5:7b

PARAMETER temperature 0.9
PARAMETER top_p 0.95
PARAMETER num_ctx 32768

SYSTEM """You are a creative writing assistant with a gift for vivid description.
Write in the style the user requests.
If asked for a story, provide a complete narrative with character arcs.
For poetry, match the requested form (haiku, sonnet, free verse, etc.)."""

# 1. Install AnythingLLM
# Download from https://anythingllm.com

# 2. Configure AnythingLLM:
# Settings → LLM Provider → Ollama
# Model: qwen2.5:7b (or qwen3.6:27b)
# Max Tokens: 8192

# 3. Upload your documents
# Go to Workspace Settings → Upload Document
# Click "Save and Embed"

# 4. Ask questions about your documents
# Qwen's 262K context means it can handle huge documents
# without losing track of the beginning

from langchain_ollama import ChatOllama
from langchain_community.document_loaders import TextLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.vectorstores import FAISS
from langchain_ollama import OllamaEmbeddings

# Initialize Qwen
llm = ChatOllama(model="qwen2.5:7b", temperature=0.3)

# Load and chunk documents
loader = TextLoader("your-document.txt")
docs = loader.load()
splitter = RecursiveCharacterTextSplitter(
 chunk_size=8000,
 chunk_overlap=200
)
chunks = splitter.split_documents(docs)

# Create vector store
embeddings = OllamaEmbeddings(model="qwen2.5:7b")
vectorstore = FAISS.from_documents(chunks, embeddings)

# Query
query = "What are the key findings?"
relevant_docs = vectorstore.similarity_search(query, k=3)
context = "\n\n".join([doc.page_content for doc in relevant_docs])

response = llm.invoke(f"Based on these documents:\n{context}\n\nQuestion: {query}")
print(response)

curl http://localhost:11434/v1/chat/completions \
 -H "Content-Type: application/json" \
 -d '{
 "model": "qwen2.5:7b",
 "messages": [
 {"role": "system", "content": "You are a Python expert."},
 {"role": "user", "content": "Write a decorator that measures execution time"}
 ],
 "temperature": 0.3
 }'

from openai import OpenAI

client = OpenAI(
 base_url="http://localhost:11434/v1",
 api_key="ollama" # required by client, but Ollama ignores it
)

response = client.chat.completions.create(
 model="qwen2.5:7b",
 messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)