 |
VOOZH | about |
Command R+ 104B needs ~49.7 GB VRAM. NVIDIA L40S 48GB has 48.0 GB. With Q2_K quantization, expect ~6 tok/s.
Operating mode
Interactive favors responsiveness, while light API and scale-out lean harder on serving readiness. The fit stays the same, but the recommendation lens changes.
Current mode
Balanced
Balanced for general local use. Keeps the ranking neutral across personal and serving workflows.
Select quantization to explore
24.6 GB over capacity — needs offload or smaller quantization
Fit status
Too heavy
Decode
2.1 tok/s
TTFT
92571 ms
Safe context
4K
Memory
72.6 GB / 48.0 GB
Offload
30%
This model fits, but memory bandwidth is the part holding decode speed back.
Throughput will feel slow
Estimated decode speed is only 6.2 tok/s, so this is more of a technical fit than a comfortable daily-driver setup.
Very little memory headroom
You can run the model, but there is not much room left for longer context, bigger batches, extra apps, or future model updates.
Prioritize bandwidth, not only capacity
If this workload feels slow, the next useful step is often a GPU tier with materially faster memory bandwidth rather than only a small bump in capacity.
Buy headroom, not only minimum fit
A slightly larger memory tier gives you safer context growth and makes the recommendation more future-proof.
| Workload | Grade | Fit | Decode | TTFT | Context |
|---|---|---|---|---|---|
| Chat | F | Too heavy | 3.5 tok/s | 30081 ms | 4K |
| Coding | F | Too heavy | 3.3 tok/s | 57987 ms | 4K |
| Agentic Coding | F | Too heavy | 3.0 tok/s | 92926 ms | 4K |
| Reasoning | F | Too heavy | 3.3 tok/s | 68530 ms | 4K |
| RAG | F | Too heavy | 3.0 tok/s | 116157 ms | 4K |
How Command R+ 104B (104B params) fits at each quantization level on NVIDIA L40S 48GB (48.0 GB usable).
| Quant | Bits | VRAM | Quality | Fit |
|---|---|---|---|---|
Q2_K | 2 | 40.6 GB | Low | F0 |
Q3_K_S | 3 | 51.0 GB | Low | F0 |
NVFP4 | 4 |
Copy-paste commands to run Command R+ 104B on your machine.
Run
ollama run command-r-plusUpgrade options
Makes the model fit on the accelerator instead of staying completely out of reach.
Raises estimated decode speed by about 110%.
~$6,500 MSRP
Makes the model fit on the accelerator instead of staying completely out of reach.
Removes host-memory offload, which is usually the single biggest latency and throughput win.
~$9,999 MSRP
Makes the model fit on the accelerator instead of staying completely out of reach.
Removes host-memory offload, which is usually the single biggest latency and throughput win.
~$9,999 MSRP
| Medium |
| F0 |
Q4_K_M | 4 | 63.4 GB | Medium | F0 |
Q5_K_M | 5 | 74.9 GB | High | F0 |
Q6_K | 6 | 85.3 GB | High | F0 |
Q8_0 | 8 | 111.3 GB | Very High | F0 |
F16 | 16 | 213.2 GB | Maximum | F0 |
Prioritize bandwidth, not only capacity. If this workload feels slow, the next useful step is often a GPU tier with materially faster memory bandwidth rather than only a small bump in capacity.
