Agent READMEs: An Empirical Study of Context Files for Agentic Coding
Abstract
Agentic coding tools receive goals written in natural language as input, break them down into specific tasks, and write or execute the actual code with minimal human intervention. Central to this process are agent context files ("READMEs for agents") that provide persistent, project-level instructions. In this paper, we conduct the first large-scale empirical study of 2,303 agent context files from 1,925 repositories to characterize their structure, maintenance, and content. We find that these files are not static documentation but complex, difficult-to-read artifacts that evolve like configuration code, maintained through frequent, small additions. Our content analysis of 16 instruction types shows that developers prioritize functional context, such as build and run commands (62.3%), implementation details (69.9%), and architecture (67.7%). We also identify a significant gap: non-functional requirements like security (14.5%) and performance (14.5%) are rarely specified. These findings indicate that while developers use context files to make agents functional, they provide few guardrails to ensure that agent-written code is secure or performant, highlighting the need for improved tooling and practices.
Community
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Decoding the Configuration of AI Coding Agents: Insights from Claude Code Projects (2025)
- Agentic Refactoring: An Empirical Study of AI Coding Agents (2025)
- AgentPack: A Dataset of Code Changes, Co-Authored by Agents and Humans (2025)
- FeatBench: Evaluating Coding Agents on Feature Implementation for Vibe Coding (2025)
- Towards Realistic Project-Level Code Generation via Multi-Agent Collaboration and Semantic Architecture Modeling (2025)
- SWE-Compass: Towards Unified Evaluation of Agentic Coding Abilities for Large Language Models (2025)
- "Your AI, My Shell": Demystifying Prompt Injection Attacks on Agentic AI Coding Editors (2025)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
nice paper! would be neat to cite paper from dataset https://huggingface.co/datasets/hao-li/AIDev
@cfahlgren1 , thank you for the suggestion. I added the papers using the dataset here:
https://huggingface.co/datasets/hao-li/AIDev#papers-using-aidev
hello! is there a plan to release the datasets?
Yes, our replication package is hosted on GitHub:
https://github.com/woraamy/Agent-Context-File-Analysis
To facilitate reuse, we have also uploaded the dataset to:
https://huggingface.co/datasets/hao-li/AgentREADMEs
Get this paper in your agent:
hf papers read 2511.12884 Models citing this paper 0
No model linking this paper
Datasets citing this paper 4
Spaces citing this paper 0
No Space linking this paper