NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.
- Updated
- Python
#
llm-safety
Here are 134 public repositories matching this topic...
DeepTeam is a framework to red team LLMs and LLM systems.
- Updated
- Python
UQLM: Uncertainty Quantification for Language Models, is a Python package for UQ-based LLM hallucination detection
uncertainty-quantification uncertainty-estimation ai-safety confidence-score hallucination confidence-estimation ai-evaluation llm llm-evaluation llm-safety hallucination-evaluation hallucination-detection hallucination-mitigation llm-hallucination
- Updated
- Python
Internal Safety Collapse: Turning the LLM or an AI Agent into a sensitive data generator.
benchmark jailbreak ai-safety adversarial-attacks red-teaming large-language-models llm-safety safety-evaluation agent-safety frontier-models
- Updated
- Python
Protect every action your agent takes.
guardrails prompt-injection data-leakage-prevention llm-safety ai-gateway content-safety ai-agent-security ai-security-gateway enterprise-ai-guardrails openclaw openclaw-security
- Updated
- TypeScript
Decrypted Generative Model safety files for Apple Intelligence containing filters
- Updated
- Python
[NDSS'25 Best Technical Poster] A collection of automated evaluators for assessing jailbreak attempts.
- Updated
- Python
Attack to induce LLMs within hallucinations
- Updated
- Python
Papers about red teaming LLMs and Multimodal models.
- Updated
Static security scanner for LLM agents — prompt injection, MCP config auditing, taint analysis. 49 rules mapped to OWASP Agentic Top 10 (2026). Works with LangChain, CrewAI, AutoGen.
python cli security mcp static-analysis owasp taint-analysis vulnerability-detection vulnerability-scanner ai-security ai-agent langchain prompt-injection llm-security llm-safety crewai ai-security-tool langchain-security-
- Updated
- Python
Reading list for adversarial perspective and robustness in deep reinforcement learning.
deep-reinforcement-learning ai-safety machine-learning-security ai-security robust-machine-learning ai-alignment safe-reinforcement-learning robust-reinforcement-learning responsible-ai adversarial-reinforcement-learning secure-ai ml-security llm-security reinforcement-learning-safety llm-safety artificial-intelligence-security artificial-intelligence-alignment robust-deep-reinforcement-learning reinforcement-learning-security large-language-model-safety
- Updated
[NeurIPS 2025] SECA: Semantically Equivalent and Coherent Attacks for Eliciting LLM Hallucinations
- Updated
- Python
Official repository for the paper "ALERT: A Comprehensive Benchmark for Assessing Large Language Models’ Safety through Red Teaming"
nlp benchmark ai artificial-intelligence nlp-machine-learning red-teaming bias-detection safety-monitoring transformers-models llm llm-evaluation llm-safety llm-safety-benchmark
- Updated
- Python
AISecOps (AI Security Operations) framework for deterministic verification of AI systems. QWED verifies LLM outputs using math, logic, and symbolic execution — creating an auditable trust boundary for agentic AI systems. Not generation. Verification.
python nlp machine-learning sympy smt-solver formal-verification ai-safety ai-security hallucination code-security ai-accuracy generative-ai enterprise-ai neurosymbolic-ai llm-safety hallucination-detection z3-prover deterministic-ai aisecops llm-verification
- Updated
- Python
Papers from our SoK on Red-Teaming (Accepted at TMLR)
- Updated
NeurIPS'24 - LLM Safety Landscape
- Updated
- Python
Restore safety in fine-tuned language models through task arithmetic
- Updated
- Python
DuoGuard: A Two-Player RL-Driven Framework for Multilingual LLM Guardrails
- Updated
- Python
Comprehensive LLM testing suite for safety, performance, bias, and compliance, equipped with methodologies and tools to enhance the reliability and ethical integrity of models like OpenAI's GPT series for real-world applications.
ai-explainability llm-safety large-language-models-testing ai-testing-best-practices gpt-model-reliability ai-bias-testing ai-security-testing model-compliance ai-performance-optimization machine-learning-testing-frameworks
- Updated
Code and dataset for the paper: "Can Editing LLMs Inject Harm?"
- Updated
- Python
Improve this page
Add a description, image, and links to the llm-safety topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the llm-safety topic, visit your repo's landing page and select "manage topics."