How to test a prompt before using it at work

Short answer: define what a good answer is, run the prompt with several real examples, observe where it fails, adjust one thing at a time and record the version that worked. A prompt that succeeds once is not tested; one that succeeds across varied cases is.

What helps before you start

• A specific task, such as summarizing customer emails or standardizing product descriptions.
• Five to ten real examples, including easy, hard and odd cases.
• A written quality criterion, so you know what success looks like.

Steps

1. Write the goal and the success criterion. For example: "summarize in up to three sentences, without inventing information and keeping the exact numbers."
2. Write the first version of the prompt. Include the role, context, task, format and constraints.
3. Test with the examples. Run the prompt on each, changing nothing, and keep the results.
4. Evaluate with the criterion. Mark each answer as good, acceptable or poor and note why.
5. Look for patterns in failures. They often repeat: wrong format, invented information, inappropriate tone.
6. Adjust one thing at a time. Change one instruction and run again. If you change too much, you will not know what improved.
7. Test edge cases. Empty texts, very long ones, in another language, with confusing data.
8. Run more than once. Answers can vary; see whether the result is stable.
9. Record. Keep the final prompt version, the date, the examples used and the known limitations.

An example

Rodrigo created a prompt to classify customer messages as "question", "complaint" and "compliment". It worked well on the first examples but failed with messages containing irony. He added an instruction to handle tone and tested again with ironic examples. The hit rate improved, and he noted that ambiguous cases should be reviewed by a person. This is an illustrative example.

Common mistakes

• Testing with a single example: it does not show variation.
• Changing several points at once: it is hard to know the effect of each.
• Not defining the criterion: without it, evaluation becomes impression.
• Forgetting sensitive data: do not use confidential information in tests without confirming the service's policy.

Next step

Choose a task from your work, write the success criterion and test your prompt with five real examples. Even a small test avoids many problems.