← All drillsTest Your Prompts Like Code
What you'll be able to do- ✓Treat a prompt as testable data, and run it over a labelled set of cases with a harness you wrote
- ✓Write graders that decide mechanically whether a reply is right, and measure content apart from format
- ✓Compare prompt versions on pass rate, tokens and cost per correct answer instead of on one good-looking reply
- ✓Enforce a JSON format with a described schema and forced tool use, and see what a types-only schema gets wrong
- ✓Specify a refusal as an exact phrase, and test whether personas and "do not" rules change anything
- ✓Choose the fewest worked examples that fix a failure, and show that selection and order did or did not matter
- ✓Decide whether a reasoning wrapper earns its tokens, counting format breaks as well as accuracy
- ✓Report pass^k alongside pass@1 and read the flaky cases
- ✓Run a prompt optimiser on dev only, then look at held-out once and explain the gap
- ✓Attack a prompt with an injection suite and put a regression gate in front of prompt changes
Change a prompt in a production LLM feature with evidence, not intuition, and stop a regression before it ships.
⌁ Any product with a prompt in it: support triage, document extraction, question answering over policies, and agent system prompts.
Start this internshipCreate an account to unlock the 11 sections, the workbench, and askThili.
BeginSections
1. Test Your Prompts Like Code
🔒 locked2. A prompt is a program you can test
🔒 locked3. Graders: code that decides if a reply is right
🔒 locked4. The harness, and your first honest numbers
🔒 locked5. Clarity and format: say it, then enforce it
🔒 locked6. Role, constraints, and the refusal
🔒 locked7. Worked examples
🔒 locked8. Asking the model to reason
🔒 locked9. Right once is not right every time
🔒 locked10. Letting a loop rewrite the prompt
🔒 locked11. Prompts under attack, and shipping changes safely
🔒 lockedDig deeper
🔗Groq documentation (the API used in this drill)
docs🔗thilillm, the reference implementation you check your code against
codePart of these learning paths
I'm a student and I want to get my first AI/ML job
View path →I want to build an app that answers from my own documents
View path →