 |
VOOZH | about |
|
VOOZH | about |
AI/ML Technical Content Strategist
DigitalOcean is closely observing the increasing parity between open-source Large Language Models (LLMs) and proprietary alternatives. A crucial capability of these models is reasoning, which involves logical and sensible thinking.
For a long time, LLMs were very linear. When given a prompt, they provided an answer. There is no meta-logic involved, or any stage where the model might be able to self-correct if it is mistaken. This effectively hinders their ability to reason, question, or adjust to problems that may be inherent to the instruction they are responding to. For example, with low-reasoning models, complex language based mathematics problems may be too complicated to solve without explicit instructions and work on the user’s part.
Enter the latest generation of reasoning LLMs. Ushered in by OpenAI’s O1 model series, reasoning models have taken the community by storm as they have effectively closed the gap between human and machine learning capabilities on a variety of logic tasks. These include coding, mathematics, and even scientific reasoning.
Like with all previous steps forward in development, the open source community has been working hard to match the closed-source models capabilities. Recently, the first open-source models to achieve this level of abstract reasoning, the Deepseek R1 series of LLMs, was released to the public.
In the first part of this 2 part article series, we will show how to run these models on DigitalOcean’s GPU Droplets using Ollama. Readers can expect to learn how to set up the GPU Droplet, install Ollama, and begin reasoning with Deepseek R1.
Key takeaways:
Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
what droplet configuration would you recommend for the 671b model?
what droplet configuration would you recommend for the 671b model?
Great article! I have a question about pricing. After you create the droplet, you are only charged for the time the GPU is being used. Is this correct? So the droplet can exist, but if you are not actively using it, you are not charged? Thank you.
Running DeepSeek R1 LLMs on GPU droplets (such as those provided by cloud services like Paperspace, AWS, or DigitalOcean) allows for efficient inference and fine-tuning of large language models at scale. Here’s how to get started:
Choose a Suitable GPU Droplet Ensure your droplet has enough VRAM—ideally 16GB or more (e.g., NVIDIA A100, V100, or L4 GPUs). LLMs like DeepSeek R1 require significant memory, especially for larger variants.
Set Up the Environment Install the necessary deep learning framework. DeepSeek R1 models are compatible with PyTorch and transformers libraries. Run:
nginx CopyEdit
pip install torch transformers accelerate
Download the Model
Visit Hugging Face or DeepSeek’s official release page. Use transformers to load the model:
python CopyEdit
from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained("deepseek-ai/deepseek-llm-7b", device_map="auto") tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/deepseek-llm-7b")
Optimize for GPU
Use device_map="auto" or manually assign the model to the GPU using .to("cuda"). Install accelerate for seamless device handling and memory-efficient inference.
Run Inference Once the model and tokenizer are loaded:
python CopyEdit
inputs = tokenizer("Hello, DeepSeek!", return_tensors="pt").to("cuda") outputs = model.generate(**inputs) print(tokenizer.decode(outputs[0]))
Monitor GPU Usage
Use nvidia-smi to monitor resource usage and avoid out-of-memory errors.
Great article! I really appreciates your efforts.
Full documentation for every DigitalOcean product.
The Wave has everything you need to know about building a business, from raising funding to marketing your product.
