Qwen3-1.7B

Technical Specifications

Qwen3-1.7B

Qwen3-1.7B is a dense causal language model engineered by the Alibaba Qwen team as a high-efficiency solution for general-purpose language processing and reasoning. Introduced as part of the Qwen3 series on April 29, 2025, the model is designed to operate effectively across diverse hardware environments, including mobile devices and edge computing platforms. It supports a native context length of 32,768 tokens, which can be further extended using YaRN-based rotary embedding scaling techniques, enabling the processing of extensive documents and prolonged multi-turn interactions.

Technically, the model is built on a transformer architecture comprising 28 layers with a hidden dimension of 2048. It utilizes Grouped Query Attention (GQA) with 16 query heads and 8 key-value heads to reduce memory overhead during inference while maintaining high performance. The architecture incorporates advanced stabilization and optimization techniques, including RMSNorm with pre-normalization, SwiGLU activation functions, and the introduction of QK-Norm to enhance attention layer stability in long-context scenarios. Positional information is managed through Rotary Positional Embeddings (RoPE), specifically utilizing an Adjusted Base Frequency (ABF) approach to maintain accuracy over the model's large context window.

A primary innovation of the Qwen3-1.7B model is its native dual-mode operational capability, which allows it to function in both Thinking and Non-Thinking modes within a single weight set. Thinking mode activates a step-by-step reasoning process, making the model suitable for complex logical deduction, mathematical problem-solving, and code generation. Non-Thinking mode provides direct, high-speed responses for standard conversational applications. This hybrid system supports dynamic switching via user directives or API parameters, allowing developers to allocate a computational thinking budget that balances output quality with inference latency.

No evaluation benchmarks for Qwen3-1.7B available.

Recommended GPUs

Resources