SupraSafety-18M · Content-Moderation

👁 Safety_Supra

Model Overview

SupraSafety-18M is a lightweight, on-device content moderation model trained from scratch (no pretrained weights) on the NVIDIA Nemotron-3.5-Content-Safety-Dataset. With only 18.3 million parameters, it achieves competitive performance while being small enough to run on edge devices, mobile phones, or in low-latency production environments.

This model is designed for binary classification of text prompts, determining whether a user input is SAFE or UNSAFE. It is trained exclusively on prompts (not responses), making it ideal for real-time moderation in chat applications, LLM guardrails, and content filtering systems.

---

Key Features

• Trained from scratch – No reliance on pretrained models, fully self-contained
• Prompt-only inference – Evaluates user input before any response is generated
• Ultra-lightweight – Only 18.3M parameters (~70MB on disk in safetensors format)
• Fast inference – ~5ms per prediction on a T4 GPU, suitable for real-time applications
• High performance – 81% accuracy and 0.86 F1-score on validation set
• Open-source – MIT licensed, available on Hugging Face Hub

---

Training Details

Dataset

• Source: NVIDIA Nemotron-3.5-Content-Safety-Dataset
• Filtering:
  • Only English (language == "en")
  • Text-only prompts (image_path is None)
• Training Size: 42,171 samples
• Validation Size: 590 samples
• Labels: safe / unsafe (based on input_label)

---

Performance Benchmarks

Validation Metrics

Sample Predictions

---

Usage

Installation

pip install transformers torch

Python Inference

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

# Load model and tokenizer
model_name = "SupraLabs/SupraSafety-18M"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Move to GPU if available
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)
model.eval()

def predict(text: str) -> dict:
 """Classify text as SAFE or UNSAFE with confidence scores."""
 inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
 inputs = {k: v.to(device) for k, v in inputs.items()}
 
 with torch.no_grad():
 outputs = model(**inputs)
 probs = torch.softmax(outputs.logits, dim=1).cpu().numpy()[0]
 
 return {
 "safe": float(probs[0]),
 "unsafe": float(probs[1]),
 "prediction": "UNSAFE" if probs[1] > 0.5 else "SAFE"
 }

# Example usage
result = predict("How can I bake a chocolate cake?")
print(result) # {"safe": 0.912, "unsafe": 0.088, "prediction": "SAFE"}

---

Limitations

• Binary classification only – Outputs only SAFE/UNSAFE, no specific violation categories
• English only – Trained exclusively on English prompts
• Text-only – Does not process images or other modalities
• Context sensitivity – May misclassify borderline cases (e.g., "SQL injection" without "attack")

---

Future Work

• Multiclass classification – Add support for specific violation categories (violence, sexual, self-harm, etc.) using violated_categories labels
• Response moderation – Extend to detect unsafe LLM responses
• Multilingual support – Train on additional languages
• Improved edge cases – Add curated examples for borderline prompts

---

Citation

If you use this model, please cite:

@misc{SupraSafety-18M,
 author = {SupraLabs},
 title = {SupraSafety-18M: Lightweight Content Moderation from Scratch},
 year = {2026},
 publisher = {Hugging Face},
 url = {https://huggingface.co/SupraLabs/SupraSafety-18M}

}

---

License

This model is released under the MIT License.

---

Contact

For questions or support, please reach out to SupraLabs on Hugging Face.

---

Acknowledgments

• Dataset provided by NVIDIA
• Built with Hugging Face Transformers
• Trained on 2x NVIDIA T4 GPUs in Kaggle (Free Tier

---

Model card last updated: 27th of June 2026

---

Copyright SupraLabs 2026