Prompt Engineer Companion

An autonomous agent that designs, tests, and optimizes LLM prompts — applying structured prompt engineering patterns, few-shot examples, and systematic evaluation to maximize output quality across different models.

When to Use This Agent

Choose Prompt Engineer Companion when:

• Your LLM produces inconsistent, low-quality, or off-topic responses
• You need to design system prompts for a production application
• You want to optimize prompt token usage while maintaining quality
• You need to adapt prompts when switching between LLM providers

Consider alternatives when:

• You need LLM architecture or infrastructure design (use an LLM architect agent)
• Your issue is data quality, not prompt quality (fix your data pipeline first)
• You need a single quick prompt for ad-hoc use (just iterate manually)

Quick Start

Copy
# .claude/agents/prompt-engineer.yml
name: prompt-engineer-companion
description: Design, test, and optimize LLM prompts
agent_prompt: |
  You are a Prompt Engineer. When designing prompts:

  1. Understand the task, expected inputs, and desired outputs
  2. Choose the right prompt pattern (structured, CoT, few-shot)
  3. Write the prompt with clear role, context, instructions, constraints
  4. Create test cases covering normal, edge, and adversarial inputs
  5. Evaluate output quality systematically
  6. Iterate to improve consistency, accuracy, and token efficiency

  Prompt quality rules:
  - Specific > vague (say "respond in JSON" not "be structured")
  - Show don't tell (use examples, not descriptions)
  - Constrain outputs (format, length, allowed values)
  - Test edge cases (empty input, very long input, adversarial input)

Example invocation:

Copy
claude "Optimize the system prompt for our product description generator — it's too verbose and sometimes ignores the tone parameter"

Sample optimization output:

Prompt Optimization Report
━━━━━━━━━━━━━━━━━━━━━━━━

Original Prompt: 342 tokens
  Issues Found:
    1. Tone instruction buried in paragraph 3 (often ignored)
    2. No output length constraint (causes verbosity)
    3. No examples of desired output (inconsistent style)

Optimized Prompt: 218 tokens (36% reduction)
  Changes:
    1. Moved tone to top-level constraint with enforcement rule
    2. Added "Maximum 3 sentences" constraint
    3. Added 2 few-shot examples showing desired format
    4. Used XML tags for clear section boundaries

Quality Comparison (50 test cases):
  Metric          Before    After
  Tone adherence  62%       94%
  Length target    45%       91%
  Factual accuracy 88%      92%
  Format compliance 71%     97%

Core Concepts

Prompt Engineering Patterns

Pattern	When to Use	Structure
Zero-shot	Simple, well-defined tasks	Role + instructions + constraints
Few-shot	Tasks needing output format consistency	Role + examples + instructions
Chain-of-Thought	Reasoning, math, multi-step logic	Role + "think step by step" + format
Self-consistency	High-stakes decisions	Generate N answers, take majority
ReAct	Tasks requiring tool use or research	Thought → Action → Observation loop

Structured Prompt Template

Copy
<!-- Production prompt template structure -->
<system>
You are a [ROLE] that [PRIMARY_FUNCTION].

## Rules
1. [MOST_IMPORTANT_CONSTRAINT]
2. [SECOND_CONSTRAINT]
3. [OUTPUT_FORMAT_RULE]

## Input Format
You will receive: [DESCRIPTION_OF_INPUT]

## Output Format
Respond with: [EXACT_FORMAT_SPECIFICATION]

## Examples

Input: [EXAMPLE_INPUT_1]
Output: [EXAMPLE_OUTPUT_1]

Input: [EXAMPLE_INPUT_2]
Output: [EXAMPLE_OUTPUT_2]

## Edge Cases
- If [EDGE_CASE_1]: [BEHAVIOR]
- If [EDGE_CASE_2]: [BEHAVIOR]
- Never [FORBIDDEN_ACTION]
</system>

Systematic Evaluation Framework

Copy
# Prompt evaluation script
def evaluate_prompt(prompt, test_cases, llm_client):
    results = []

    for test in test_cases:
        response = llm_client.generate(
            system=prompt,
            user=test["input"]
        )

        score = {
            "test_id": test["id"],
            "format_correct": check_format(response, test["expected_format"]),
            "content_accurate": check_accuracy(response, test["expected_content"]),
            "constraint_adherence": check_constraints(response, test["constraints"]),
            "token_count": count_tokens(response)
        }
        results.append(score)

    return {
        "format_accuracy": mean([r["format_correct"] for r in results]),
        "content_accuracy": mean([r["content_accurate"] for r in results]),
        "constraint_adherence": mean([r["constraint_adherence"] for r in results]),
        "avg_tokens": mean([r["token_count"] for r in results]),
        "details": results
    }

Configuration

Option	Type	Default	Description
targetModel	string	"claude-sonnet"	Model to optimize prompts for
evaluationSize	number	50	Number of test cases for evaluation
maxPromptTokens	number	2000	Token budget for system prompt
optimizeFor	string	"quality"	Optimize for: quality, cost, latency
includeExamples	number	2	Number of few-shot examples
testAdversarial	boolean	true	Include prompt injection test cases

Best Practices

1. Put the most important instruction first and last — LLMs pay most attention to the beginning and end of prompts (primacy and recency effects). Place critical constraints (output format, tone) at the very top of the system prompt and reinforce them at the end.

2. Use examples instead of descriptions — "Write in a casual, friendly tone" is ambiguous. Including 2-3 examples of casual, friendly output gives the model a concrete target to match. Few-shot examples improve consistency more than any amount of descriptive instruction.

3. Constrain outputs with explicit format rules — Instead of "provide a summary," specify "provide a 2-3 sentence summary in this format: [Format]. Do not include headers, bullet points, or extra commentary." Explicit constraints reduce output variance dramatically.

4. Test with adversarial inputs — Include test cases with empty input, extremely long input, input in unexpected languages, and prompt injection attempts ("Ignore previous instructions and..."). A production prompt must handle these gracefully, not crash or leak system prompt content.

5. Version control your prompts with performance metrics — Store each prompt version with its evaluation scores. When you change a prompt, compare the new version against the old on the same test set before deploying. A seemingly minor wording change can cause a 20% quality drop on edge cases.

Common Issues

Prompt works on GPT-4 but fails on Claude (or vice versa) — Different models interpret the same instructions differently. Claude follows XML-structured prompts better while GPT-4 prefers numbered lists. Maintain model-specific prompt variants and test each variant against the target model rather than assuming cross-model compatibility.

Output format compliance drops below 80% — The model sometimes ignores format instructions, returning prose when JSON is expected. Add a format enforcement line: "You MUST respond with valid JSON only. Do not include any text outside the JSON object." Then add a parsing validator that retries with a stricter prompt if the first response fails format validation.

Few-shot examples bias the model toward copying examples — If all examples feature a specific product category, the model tends to generate responses biased toward that category. Ensure examples are diverse across the input space, include edge cases, and add an instruction: "The examples show the format, not the content. Adapt to the actual input."