The Emerging Art of Prompt Engineering
Our weekly selection of must-read Editors' Picks and original features
By now, even the most casual generative-AI tinkerer knows that the specificity of one’s prompt determines, to a large extent, the quality of a model’s output—whether that’s a realistic image, an on-topic paragraph, or a code block that actually works.
Many industry leaders claim that prompt engineering—the practice of tailoring your input to improve the model’s on-task performance—is a skill with a limited shelf life, given how rapidly AI technology is advancing. That remains to be seen; for now and for the foreseeable future, designing better prompts remains a practical skill worthy of data and ML practitioners’ attention.
To help you develop your prompting chops, we’ve selected some of our best recent articles on the topic, covering a wide spectrum of use cases and experience levels. Let’s prompt away!
- If you’re very new to working with large language models, a great place to start is Olivia Tanuwidjaja‘s beginner-friendly guide to prompt engineering, which provides a high-level (but thorough) overview of common approaches.
- LLMs’ ability to execute tasks they hadn’t been specifically trained for relies on users’ prompting creativity and nimble thinking. Case in point: Jye Sawtell-Rickson‘s attempt to perform an exploratory data analysis with ChatGPT.
- To showcase your newly acquired LLM knowledge, Leonie Monigatti proposes 10 project ideas you can add to your portfolio—and walks us through the different kinds of prompts you’ll need to use along the way.
- For a deeper understanding of language models’ inner workings, Scott Lundberg and Marco Tulio Ribeiro discuss how the "greedy tokenization methods used by language models can introduce a subtle and powerful bias into your prompts, leading to puzzling generations."
If you haven’t quite had your fill of thoughtful, expertly presented articles and want to branch out beyond prompt design, we’ve got you covered:
- Exploring LLMs from a different (and more ominous) angle, Parul Pandey looks at recent work into the models’ vulnerability to poisoning attacks by malicious actors.
- Sachin Date‘s latest deep dive is the comprehensive, serious-yet-engaging guide to random variables you’ve always wanted.
- Keeping things simple is easier said than done when it comes to ML pipelines, but Hennie de Harder‘s insights on building a minimally viable product (MVP) will inspire you to focus on the most essential elements of your project.
- Is quantum computing on the verge of upending the foundations of statistics? Tim Lou, PhD unpacks the implications of the work that won the 2022 Physics Nobel Prize.
- To learn about data-oriented programming and how it differs from object-oriented programming (OOP), don’t miss Tam D Tran-The‘s helpful (and code-snippet-filled) explainer.
Thank you for supporting our authors! If you enjoy the articles you read on TDS, consider becoming a Medium member – it unlocks our entire archive (and every other post on Medium, too).
Until the next Variable,
TDS Editors
Share This Article
Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.
Write for TDS
