DeepSeek V4 vs GPT-5: Real Benchmark & Cost Comparison (2026)

The benchmark script

Save as benchmark.py:

"""
DeepSeek V4 vs GPT-5 benchmark — reproducible.
Run: python benchmark.py
Outputs: results.csv with per-task accuracy, tokens, latency, cost.
"""
import os
import time
import json
from openai import OpenAI
from dotenv import load_dotenv
import pandas as pd

load_dotenv()

# DeepSeek uses OpenAI-compatible API
deepseek = OpenAI(
    api_key=os.getenv("DEEPSEEK_API_KEY"),
    base_url="https://api.deepseek.com"
)
openai_client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

# Prices per million tokens (update from official pricing pages)
PRICING = {
    "deepseek-v4":      {"in": 0.14,  "out": 0.28},
    "deepseek-r1":      {"in": 0.55,  "out": 2.19},
    "gpt-5":            {"in": 3.00,  "out": 12.00},
    "gpt-5-mini":       {"in": 0.30,  "out": 1.20},
}

# Model name mapping (update if API names change)
MODELS = {
    "deepseek-v4": "deepseek-chat",
    "deepseek-r1": "deepseek-reasoner",
    "gpt-5":       "gpt-5",
    "gpt-5-mini":  "gpt-5-mini",
}

def call_model(model_key: str, prompt: str, system: str = "") -> dict:
    """Call a model and return response + usage + latency + cost."""
    client = deepseek if model_key.startswith("deepseek") else openai_client
    messages = []
    if system:
        messages.append({"role": "system", "content": system})
    messages.append({"role": "user", "content": prompt})

    t0 = time.time()
    resp = client.chat.completions.create(
        model=MODELS[model_key],
        messages=messages,
        temperature=0.0,
        max_tokens=2000,
    )
    elapsed = time.time() - t0

    in_tokens = resp.usage.prompt_tokens
    out_tokens = resp.usage.completion_tokens
    price = PRICING[model_key]
    cost = (in_tokens * price["in"] + out_tokens * price["out"]) / 1_000_000

    return {
        "model": model_key,
        "response": resp.choices[0].message.content,
        "in_tokens": in_tokens,
        "out_tokens": out_tokens,
        "latency_s": round(elapsed, 2),
        "cost_usd": round(cost, 6),
    }

# ============================================================================
# YOUR TEST CASES HERE
# Replace this list with your actual tasks. Each task has:
#   - prompt: the question
#   - check: a function that takes the model's response and returns True/False
# ============================================================================
TASKS = [
    {
        "id": "leetcode-1",
        "prompt": "Write a Python function two_sum(nums, target) that returns indices of two numbers adding up to target.",
        "check": lambda r: "def two_sum" in r and "return" in r,
    },
    {
        "id": "sql-1",
        "prompt": "Given table users(id, name, age), write SQL to count users older than 30.",
        "check": lambda r: "COUNT" in r.upper() and "30" in r,
    },
    # ... add 48 more tasks following the same pattern
]

def run_benchmark():
    results = []
    for task in TASKS:
        for model_key in MODELS:
            try:
                r = call_model(model_key, task["prompt"])
                r["task_id"] = task["id"]
                r["correct"] = task["check"](r["response"])
                results.append(r)
                print(f"{task['id']:20s} {model_key:15s} {'✓' if r['correct'] else '✗'}  ${r['cost_usd']:.6f}")
            except Exception as e:
                print(f"{task['id']:20s} {model_key:15s} ERROR: {e}")

    df = pd.DataFrame(results)
    df.to_csv("results.csv", index=False)

    # Summary
    summary = df.groupby("model").agg(
        accuracy=("correct", "mean"),
        total_cost=("cost_usd", "sum"),
        avg_latency=("latency_s", "mean"),
    ).round(4)
    print("\n=== SUMMARY ===")
    print(summary)

if __name__ == "__main__":
    run_benchmark()

How to adapt this script

1. Add your own tasks in the TASKS list — use real problems from your domain
2. Update the MODELS dict if API names change (DeepSeek occasionally renames)
3. Update PRICING quarterly from official pricing pages
4. Run from your own region for accurate latency numbers

After running, you’ll have a results.csv with real numbers to fill into the tables above.

Live Pricing Spreadsheet

We maintain a Google Sheets live pricing tracker that auto-pulls prices from all major Chinese + Western AI APIs every week.

📊 Open the live pricing spreadsheet → (replace with your actual published sheet URL)

What it tracks:

• 10+ models (DeepSeek, Qwen, GLM, Kimi, Doubao + GPT, Claude, Gemini)
• 6 dimensions (input price, output price, context, latency, free tier, region availability)
• Auto-updated weekly via Google Apps Script

💡 Why we do this: Pricing is the #1 reason developers switch models. A live, maintained table is the most-linked type of content in this niche — it’s our long-term SEO moat.

Decision Framework: Which Model Should You Use?

Stop asking “which is best.” Start asking “best for what?”

Is it real-time user-facing (chat UI)?
├─ Yes → Need TTFT < 0.5s
│        ├─ Budget tight?     → DeepSeek V4
│        └─ Budget OK?        → GPT-5-mini
└─ No (batch/background)
   │
   Is it hard reasoning (math, logic, 3+ steps)?
   ├─ Yes → Need depth
   │        ├─ Budget tight?  → DeepSeek R1
   │        └─ Budget OK?     → GPT-5
   └─ No (straightforward code/writing)
            ├─ Budget tight?  → DeepSeek V4
            └─ Budget OK?     → GPT-5 (marginal gain, big cost)

Concrete recommendations by use case

Common Objections (Honest Answers)

“But DeepSeek sends data to China!”

This is the #1 objection from Western developers. Here’s the honest take:

• DeepSeek API: Data is processed on DeepSeek’s servers (China). They have a data policy — read it. For non-sensitive workloads (coding, generic Q&A), this is fine.
• Self-hosted DeepSeek: You can run DeepSeek V4 locally with Ollama — zero data leaves your machine. See our self-hosting guide.
• For sensitive data: Don’t use any cloud LLM (including OpenAI). Self-host.

”GPT-5 is just better, why bother?”

If your budget is unlimited, yes, use GPT-5 for everything. Most teams aren’t. A startup burning $50K/month on GPT-5 could cut that to $3K/month by using DeepSeek V4 for 80% of calls and GPT-5 only for the hard 20%.

”I tried DeepSeek once and it was bad”

Two likely reasons:

1. You used V4 for a reasoning task → should have used R1
2. You gave it a vague prompt → DeepSeek is more sensitive to prompt clarity than GPT-5 (see the SQL example above)

Cost Calculator: Estimate Your Savings

Use this table to estimate. Find your monthly GPT-5 spend, see what switching saves:

Assumes 80% of calls move to DeepSeek V4, 20% stay on GPT-5. Adjust based on your actual task mix.