See exactly what changed when you upgraded models

Compare AI model behavior side-by-side. Run identical prompts through two models and diff the outputs — catch regressions before they hit production.

OpenAI Anthropic Google AI Mistral CI/CD Ready

Model A (Baseline)

0.7
Not set
Stored locally in your browser. Never sent to our servers.

Model B (New Version)

0.7
Not set
Stored locally in your browser. Never sent to our servers.

Prompt

Diff Results
0
Total Prompts
0
Regressions
0
Warnings
0
Matches

Simple, transparent pricing

Start free. Scale as your team grows.

Solo
$49/mo
For individual developers
  • 100 diffs per month
  • 5 model comparisons
  • Single prompt input
  • API key management
  • Export results
Team
$149/mo
For AI product teams
  • 1,000 diffs per month
  • Unlimited model comparisons
  • Batch prompt input
  • Test suite management
  • CI integration
Org
$399/mo
For large teams
  • Unlimited diffs
  • Regression alerts
  • Full API access
  • Team workspaces
  • Priority support

API Reference

Integrate model diffs into your CI/CD pipeline. All API calls are usage-based.

curl 
curl -X POST https://api.modeldiff.dev/v1/diff \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "left_model": "gpt-4o",
    "right_model": "gpt-4-turbo",
    "prompts": [
      {"id": "q1", "prompt": "What is quantum entanglement?"},
      {"id": "q2", "prompt": "Explain recursion in programming."}
    ],
    "providers": {
      "left": "openai",
      "right": "anthropic"
    }
  }'
JavaScript 
const response = await fetch('https://api.modeldiff.dev/v1/diff', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer YOUR_API_KEY',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    left_model: 'gpt-4o',
    right_model: 'claude-3-5-sonnet',
    prompts: [
      { id: 'q1', prompt: 'What is quantum entanglement?' },
      { id: 'q2', prompt: 'Explain recursion in programming.' }
    ],
    providers: { left: 'openai', right: 'anthropic' }
  })
});

const results = await response.json();
console.log(`${results.summary.regressions} regressions found`);
GitHub Actions 
name: Model Diff
on: [push]
jobs:
  diff:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run Model Diff
        env:
          API_KEY: ${{ secrets.MODELDIFF_API_KEY }}
        run: |
          curl -X POST https://api.modeldiff.dev/v1/diff \
            -H "Authorization: Bearer $API_KEY" \
            -H "Content-Type: application/json" \
            -d @prompts/test-suite.json | jq '.summary'
Python 
import requests

response = requests.post(
    'https://api.modeldiff.dev/v1/diff',
    headers={
        'Authorization': 'Bearer YOUR_API_KEY',
        'Content-Type': 'application/json',
    },
    json={
        'left_model': 'gpt-4o',
        'right_model': 'claude-3-5-sonnet',
        'prompts': [
            {'id': 'q1', 'prompt': 'What is quantum entanglement?'}
        ],
        'providers': {'left': 'openai', 'right': 'anthropic'}
    }
)

results = response.json()
print(f"Regressions: {results['summary']['regressions']}")