prompt-to-signature/README.md

# Prompt To Signature

Transform your ideas and prompts into executable DSPy signatures using AI! This agent automatically converts natural language descriptions into properly typed, production-ready DSPy signatures with support for complex nested structures, Pydantic models, and more.

## Features

- **Natural Language to DSPy Signature**: Convert ideas like "Translate English to German" or "Analyze sentiment from product reviews" into full `dspy.Signature` classes
- **Smart Type Inference**: Automatically determines appropriate field types (str, int, list, dict, Pydantic models, Literal types, etc.)
- **Dynamic Class Factory**: Generate and use `dspy.Signature` classes at runtime
- **Interactive Refinement**: Iteratively improve generated signatures by providing feedback in a loop
- **Complex Nested Structures**: Full support for Pydantic models with nested fields for structured outputs
- **Code Generation**: Export signatures as clean, executable Python code

## Quick Start with Modaic AutoAgent

The easiest way to use this agent is by loading it from Modaic Hub:

```python
from modaic import AutoAgent

# Load the precompiled agent from hub
agent = AutoAgent.from_precompiled("fadeleke/prompt-to-signature")

# Use the agent
prompt = "Translate text from English to French"
result = agent(prompt)

# Generate executable code
code = agent.generate_code(result)
print(code)
```

### With Custom Configuration

Override default configuration parameters:

```python
from modaic import AutoAgent

agent = AutoAgent.from_precompiled(
    "fadeleke/prompt-to-signature",
    config={
        "lm": "openrouter/anthropic/claude-sonnet-4.5",
        "max_tokens": 32000,
        "temperature": 0.7,
    }
)

result = agent("Summarize a document and extract key entities")
```

## Manual Installation & Setup

If you prefer to run the agent locally:

### Prerequisites

- Python 3.11+
- [uv](https://docs.astral.sh/uv/) package manager

### Installation

1. Clone the repository:
```bash
# Clone via HTTPS
git clone https://git.modaic.dev/fadeleke/prompt-to-signature.git

cd prompt-to-signature
```

2. Install dependencies:
```bash
uv sync
```

3. Set up environment variables:
```bash
# Create a .env file
echo "OPENROUTER_API_KEY=your_api_key_here" > .env
```

Get your OpenRouter API key from [openrouter.ai](https://openrouter.ai/)

### Usage

```python
from agent import PromptToSignatureAgent, PromptToSignatureConfig

# Initialize the agent
config = PromptToSignatureConfig(
    lm="openrouter/anthropic/claude-haiku-4.5",
    max_tokens=16000,
    temperature=1.0,
)
agent = PromptToSignatureAgent(config)

# Generate a signature from a prompt
prompt = """
Create a signature that takes a customer review and extracts:
- Overall sentiment (positive, negative, neutral)
- Key issues mentioned
- Recommended rating (1-5 stars)
"""

result = agent(prompt)

# Generate executable Python code
code = agent.generate_code(result)
print(code)
```

### Interactive Refinement Mode

Use the refinement loop to iteratively improve signatures:

```python
# Enable refinement with user feedback
result = agent(prompt, refine=True)

# The agent will:
# 1. Generate an initial signature
# 2. Show it to you for review
# 3. Ask if you're satisfied (y/n)
# 4. If not, ask for feedback
# 5. Regenerate with your feedback
# 6. Repeat up to max_attempts_to_refine times
```

## Examples

### Simple Translation Signature

```python
prompt = "Translate text from English to Spanish"
result = agent(prompt)
code = agent.generate_code(result)
```

**Generated Output:**
```python
import dspy

class TranslateEnglishToSpanish(dspy.Signature):
    """Translate English text to Spanish"""

    english_text: str = dspy.InputField(desc="The English text to translate")
    spanish_translation: str = dspy.OutputField(desc="The translated Spanish text")
```

### Complex Sentiment Analysis

```python
prompt = """
Analyze product reviews and extract:
- Overall sentiment (positive/negative/mixed/neutral)
- Confidence score (0-1)
- Specific issues mentioned
- Specific praise points
- Predicted star rating (1-5)
"""

result = agent(prompt)
code = agent.generate_code(result)
```

**Generated Output:**
```python
import dspy
from typing import List, Literal
from pydantic import BaseModel, Field

class Issue(BaseModel):
    """Represents a specific issue mentioned in the review"""

    description: str = Field(description="Description of the issue")
    severity: Literal["minor", "moderate", "severe"] = Field(description="Severity level of the issue")

class PraisePoint(BaseModel):
    """Represents a specific praise point mentioned in the review"""

    description: str = Field(description="Description of the praise")

class SentimentAnalysis(BaseModel):
    """Complete sentiment analysis result"""

    overall_sentiment: Literal["positive", "negative", "mixed", "neutral"] = Field(description="Overall sentiment classification")
    confidence_score: float = Field(description="Confidence score between 0 and 1")
    issues: List[Issue] = Field(description="List of specific issues mentioned")
    praise_points: List[PraisePoint] = Field(description="List of specific praise points")
    predicted_rating: int = Field(description="Predicted star rating from 1 to 5")

class AnalyzeProductReview(dspy.Signature):
    """Analyze product reviews to extract sentiment, issues, praise, and rating"""

    review_text: str = dspy.InputField(desc="The product review text to analyze")
    analysis: SentimentAnalysis = dspy.OutputField(desc="Complete sentiment analysis with structured output")
```

### Multimodal Signatures

```python
prompt = "Count the number of people in an image and describe what they're doing"
result = agent(prompt)
```

**Generated Output:**
```python
import dspy

class CountPeopleInImage(dspy.Signature):
    """Count people in an image and describe their activities"""

    image: dspy.Image = dspy.InputField(desc="The image to analyze")
    people_count: int = dspy.OutputField(desc="Number of people detected")
    activity_description: str = dspy.OutputField(desc="Description of what people are doing")
```

## Architecture

### Core Components

#### 1. SignatureGenerator (`agent/modules.py`)

The heart of the system. This DSPy module:
- Takes natural language prompts as input
- Uses a structured `SignatureGeneration` signature to ensure consistent output
- Generates properly typed field definitions
- Supports complex types: Pydantic models, Literal types, Lists, Dicts, Optional fields
- Creates executable Python code from predictions

**Key Classes:**
- `FieldType`: Enum of all supported Python types
- `GeneratedField`: Represents a single input/output field
- `PydanticModelSchema`: Schema for nested Pydantic models
- `SignatureGenerator`: Main DSPy module for signature generation

#### 2. PromptToSignatureAgent (`agent/index.py`)

The high-level agent wrapper that:
- Extends `modaic.PrecompiledAgent` for hub deployment
- Configures LLM models (defaults to Claude Haiku 4.5 via OpenRouter)
- Implements refinement loop with `dspy.Refine`
- Validates signatures with user feedback

**Configuration Options:**
```python
class PromptToSignatureConfig:
    lm: str = "openrouter/anthropic/claude-haiku-4.5"
    refine_lm: str = "openrouter/anthropic/claude-haiku-4.5"
    max_tokens: int = 16000
    temperature: float = 1.0
    max_attempts_to_refine: int = 5
```

#### 3. Utilities (`agent/utils.py`)

Helper functions for:
- Converting names to snake_case
- Saving generated signatures to files
- Managing OpenRouter API configuration

### How It Works

1. **Input Processing**: Natural language prompt describing the desired functionality
2. **LLM Generation**: DSPy's structured prediction generates typed field definitions
3. **Schema Construction**: Builds internal representation of the signature with full type info
4. **Code Generation**: Converts the structured representation to executable Python code
5. **Optional Refinement**: User can provide feedback to improve the signature iteratively

### Supported Field Types

- **Basic types**: `str`, `int`, `float`, `bool`
- **Collections**: `list[str]`, `list[int]`, `dict[str, Any]`, etc.
- **Optional types**: `Optional[str]`, `Optional[int]`, etc.
- **Literal types**: For enumerated values like `Literal["positive", "negative", "neutral"]`
- **Multimodal**: `dspy.Image`, `dspy.Audio`
- **Pydantic models**: Complex nested structures with validation

## Advanced Usage

### Pushing to Modaic Hub

To share your agent or update the hub version:

```python
from agent import PromptToSignatureAgent, PromptToSignatureConfig

agent = PromptToSignatureAgent(PromptToSignatureConfig())
agent.push_to_hub("your-username/agent-name", with_code=True)
```

### Dynamic Signature Classes

Create DSPy signature classes at runtime:

```python
result = agent("Translate English to German")

# Create a dynamic signature class
SignatureClass = agent.signature_generator.create_signature_class(result)

# Use it with DSPy modules
predictor = dspy.Predict(SignatureClass)
output = predictor(english_text="Hello, world!")
```

### Custom LLM Backends

Use different LLM providers:

```python
import dspy
from agent import PromptToSignatureAgent, PromptToSignatureConfig

# Configure custom LM
custom_lm = dspy.LM(
    model="openai/gpt-4",
    api_key="your-key",
    max_tokens=8000,
)

# Initialize agent
agent = PromptToSignatureAgent(PromptToSignatureConfig())
agent.signature_generator.set_lm(custom_lm)
```

## Configuration

### Environment Variables

- `OPENROUTER_API_KEY`: Your OpenRouter API key (required for local usage)

### Agent Configuration

```python
PromptToSignatureConfig(
    lm="openrouter/anthropic/claude-haiku-4.5",  # Main LLM model
    refine_lm="openrouter/anthropic/claude-haiku-4.5",  # Refinement LLM
    max_tokens=16000,  # Max tokens per generation
    temperature=1.0,  # Sampling temperature
    max_attempts_to_refine=5,  # Max refinement iterations
)
```

## Troubleshooting

### Common Issues

**"Refinement failed"**: This usually means the signature couldn't be improved after max attempts. Try:
- Adjusting your prompt to be more specific
- Increasing `max_attempts_to_refine`
- Providing clearer feedback during refinement

**Missing API Key**: Ensure `OPENROUTER_API_KEY` is set in your `.env` file

**Type Validation Errors**: The agent validates all field types. If generation fails, try:
- Simplifying your prompt
- Being more explicit about expected output structure

## Acknowledgments

Built with:
- [DSPy](https://github.com/stanfordnlp/dspy) - Framework for programming foundation models
- [Modaic](https://modaic.dev/) - Agent orchestration and hub
- [OpenRouter](https://openrouter.ai/) - Unified LLM API access