InCoder-32B: Code Foundation Model for Industrial Scenarios

Model Summary

InCoder-32B (Industrial-Coder-32B) is the first 32B-parameter code foundation model purpose-built for industrial code intelligence. While general-purpose code LLMs excel at mainstream software tasks, they often struggle with the unique demands of industrial programming — hardware semantics, specialized language constructs, strict resource constraints, and domain-specific correctness verification.

Presented in the paper InCoder-32B: Code Foundation Model for Industrial Scenarios, InCoder-32B unifies code intelligence across five industrial domains:

InCoder-32B achieves highly competitive performance on general tasks while establishing the strongest open-source baselines across all evaluated industrial domains.

---

Key Results

General Code Benchmarks

Industrial Code Benchmarks

InCoder-32B leads all open-weight baselines on CAD-Coder and KernelBench (all three levels), and even surpasses proprietary models like Claude-Sonnet-4.6 on CAD-Coder IoU and KernelBench L1/L2/L3.

---

Model Architecture

InCoder-32B adopts a standard decoder-only Transformer architecture with the following configuration:

---

Training Pipeline: Code-Flow

InCoder-32B is trained through a three-stage Code-Flow pipeline:

Stage 1 — Pre-training & Annealing

• Industrial Recall: Data pipeline using rule-based filtering, FastText classifiers, and semantic retrieval for Verilog, CUDA, firmware C, and CadQuery.
• Refinement: OCR extraction from technical manuals, multi-level deduplication, and repository-level fork consolidation.
• Training: 15T total tokens using Autoregressive LM + Fill-in-the-Middle (FIM) objectives.

Stage 2 — Mid-Training (Context Extension)

Context window extended progressively from 8K to 128K tokens:

• 8K → 32K: Targets file-level tasks like completing RTL modules or kernel functions.
• 32K → 128K: Unlocks long-context capabilities for extended debugging and cross-module projects.

Stage 3 — Post-Training

2.5M supervised fine-tuning (SFT) samples constructed from real industrial tasks with execution-grounded verification using toolchains like Icarus Verilog, nvcc, and Renode (STM32 simulator).

---

Usage

Installation

pip install transformers accelerate

Basic Inference

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "Multilingual-Multimodal-NLP/IndustrialCoder"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
 model_id,
 torch_dtype=torch.bfloat16,
 device_map="auto"
)

prompt = """Write a synthesizable Verilog module for a UART transmitter (8N1 protocol).
The module should accept 8-bit parallel data and serialize it onto a TX line."""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
 **inputs,
 max_new_tokens=1024,
 temperature=0.2,
 do_sample=True,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Deployment with vLLM

For production deployment, you can use vLLM to create an OpenAI-compatible API endpoint.

vllm serve Multilingual-Multimodal-NLP/IndustrialCoder --tensor-parallel-size 8

Fill-in-the-Middle (FIM)

InCoder-32B supports FIM completion for code infilling tasks:

prefix = """// CUDA kernel for RMS Normalization
__global__ void rms_norm_kernel(float* output, const float* input, 
 const float* weight, int N, float eps) {
 int idx = blockIdx.x;
"""
suffix = """
 output[idx * N + tid] = normalized * weight[tid];
}"""

fim_prompt = f"<fim_prefix>{prefix}<fim_suffix>{suffix}<fim_middle>"
inputs = tokenizer(fim_prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

Limitations & Disclaimers

Based on failure analysis, the model may struggle with:

• API Knowledge: Linker errors from undefined HAL/CMSIS functions in embedded C.
• Functional Semantics: Producing compilable but functionally incorrect RTL under complex logic scenarios.
• Optimization: Correct but sub-optimal GPU kernel performance.

Always review and test generated code in a sandboxed environment. Industrial code (RTL, embedded firmware) requires expert review before deployment.

---

Citation

@article{yang2026incoder,
 title={InCoder-32B: Code Foundation Model for Industrial Scenarios},
 author={Yang, Jian and Zhang, Wei and Wu, Jiajun and Cheng, Junhang and Guo, Shawn 
 and Wang, Haowen and Gu, Weicheng and Du, Yaxin and Li, Joseph and Xu, Fanglin 
 and others},
 journal={arXiv preprint arXiv:2603.16790},
 year={2026}
}

9 items • Updated Apr 13 • 4