Home / Methodology

Methodology

How we turn token prices into the cost of a real task, what we measure, what we estimate and where the numbers can be wrong.

The formula

Cost of a task = input tokens x input price + (visible output tokens + reasoning tokens) x output price. Prices are list prices in USD per 1 million tokens, read every day from the public OpenRouter model list (which passes through the labs' own prices for their models) and from each model's provider list. We do not apply discounts (batch API, enterprise deals, free tiers).

Measured, estimated, modeled

  • Measured: we sent the exact task prompt to the model through OpenRouter with default settings and recorded usage.prompt_tokens, usage.completion_tokens and completion_tokens_details.reasoning_tokens. 12 models, 196 runs, about $3.47 of API spend.
  • Estimated: for the other models we take the tokens of a measured model from the same lab (and the same reasoning mode where possible) and rescale input and visible output by the lab's tokenizer ratio. The ratio comes from sending the same 10,000-word document to one model per lab and counting input tokens (OpenAI = 1.00): arcee-ai 0.96, inception 0.97, google 0.98, amazon 0.98, minimax 0.99, upstage 0.99, cohere 1.00, unbiased 1.00, stepfun 1.00, deepseek 1.00, openai 1.00, z-ai 1.00, aion-labs 1.00, thinkingmachines 1.00, meituan 1.00, xiaomi 1.00, ibm-granite 1.00, rekaai 1.00, writer 1.00, nousresearch 1.00, bytedance 1.00, meta-llama 1.00, microsoft 1.00, anthracite-org 1.00, perceptron 1.01, prism-ml 1.01, morph 1.01, inclusionai 1.01, qwen 1.01, fireworks 1.01, bytedance-seed 1.01, moonshotai 1.01, inference-net 1.01, nvidia 1.02, thedrummer 1.02, cognitivecomputations 1.02, poolside 1.03, mistralai 1.03, x-ai 1.07, anthropic 1.39.
  • Modeled: the 50-step agent run is a fixed scenario, described below. It is a planning number, not a measurement.
  • Capped: we cap output at 16,000 tokens. Some reasoning models used the whole budget on a task (most often the spreadsheet analysis and code review). Those rows show "≥" because the real cost would be higher.

The tasks

  • Answer a chat question - A typical chatbot question answered in a few practical paragraphs. We send the same prompt (31 words of input) to every measured model through OpenRouter with default settings and record the input, output and reasoning tokens it uses.
  • Reply to a customer email - A support agent reply to a 200-word customer complaint. We send the same prompt (197 words of input) to every measured model through OpenRouter with default settings and record the input, output and reasoning tokens it uses.
  • Write a 1,500-word blog post - A complete 1,500-word article with headings, written from a short brief. We send the same prompt (34 words of input) to every measured model through OpenRouter with default settings and record the input, output and reasoning tokens it uses.
  • Write 10 product descriptions - Ten e-commerce product descriptions of about 80 words each from short specs. We send the same prompt (155 words of input) to every measured model through OpenRouter with default settings and record the input, output and reasoning tokens it uses.
  • Translate 1,000 words - 1,000 words of English prose translated into Spanish. We send the same prompt (1,017 words of input) to every measured model through OpenRouter with default settings and record the input, output and reasoning tokens it uses.
  • Proofread one page (500 words) - A 500-word page with typos returned as corrected text. We send the same prompt (521 words of input) to every measured model through OpenRouter with default settings and record the input, output and reasoning tokens it uses.
  • Summarize a 20-page PDF - About 10,000 words of text (roughly 20 pages) summarized to about 300 words plus 5 key takeaways. We send the same prompt (10,023 words of input) to every measured model through OpenRouter with default settings and record the input, output and reasoning tokens it uses.
  • Extract data from an invoice - One invoice turned into structured JSON (vendor, dates, line items, totals). We send the same prompt (137 words of input) to every measured model through OpenRouter with default settings and record the input, output and reasoning tokens it uses.
  • Analyze a 200-row spreadsheet - A 200-row sales CSV analyzed for trends, outliers and three recommendations. We send the same prompt (227 words of input) to every measured model through OpenRouter with default settings and record the input, output and reasoning tokens it uses.
  • Answer from 5 retrieved documents (RAG) - A question answered from five retrieved text chunks of about 600 words each, with citations. We send the same prompt (3,044 words of input) to every measured model through OpenRouter with default settings and record the input, output and reasoning tokens it uses.
  • Classify 100 support tickets - 100 short support tickets labeled as billing, bug, how-to, account or feature-request. We send the same prompt (983 words of input) to every measured model through OpenRouter with default settings and record the input, output and reasoning tokens it uses.
  • Review 500 lines of code - A real 497-line Python file reviewed for bugs, edge cases, performance and readability. We send the same prompt (2,324 words of input) to every measured model through OpenRouter with default settings and record the input, output and reasoning tokens it uses.
  • Write a SQL query - A natural-language question turned into a SQL query for a 4-table schema, with a short explanation. We send the same prompt (66 words of input) to every measured model through OpenRouter with default settings and record the input, output and reasoning tokens it uses.
  • Run a 50-step coding agent - Modeled run: a 8,000-token system prompt with tool definitions, 50 steps, each step adding 1,500 tokens of context (tool output plus the previous answer) and writing 300 visible tokens plus the model's typical reasoning for a short step (measured on the SQL, invoice and email tasks, capped at 4,000). Earlier context is billed at the cached-input price where the model offers caching.

Inputs are public-domain texts from Project Gutenberg (J. S. Mill, On Liberty; Machiavelli, The Prince), the CPython textwrap.py module (497 lines, PSF license) and synthetic business data written for this benchmark (fixed random seed). Prompts are in English.

Quality scores

Next to price we show public quality scores. We never scrape sites whose terms forbid it.

  • LMArena Text: https://lmarena.ai/leaderboard/text (leaderboard date 2026-10-15). CC BY 4.0 (HF dataset lmarena-ai/leaderboard-dataset). Attribution required: 'Data from LMArena' + link to lmarena.ai. Do not scrape arena.ai pages (ToS).
  • LMArena WebDev: https://lmarena.ai/leaderboard/webdev (leaderboard date 2026-10-16). CC BY 4.0 (HF dataset lmarena-ai/leaderboard-dataset). Attribution required: 'Data from LMArena' + link to lmarena.ai. Do not scrape arena.ai pages (ToS).
  • LMArena Vision: https://lmarena.ai/leaderboard/vision (leaderboard date 2026-10-09). CC BY 4.0 (HF dataset lmarena-ai/leaderboard-dataset). Attribution required: 'Data from LMArena' + link to lmarena.ai. Do not scrape arena.ai pages (ToS).
  • LMArena Search: https://lmarena.ai/leaderboard/search (leaderboard date 2026-10-13). CC BY 4.0 (HF dataset lmarena-ai/leaderboard-dataset). Attribution required: 'Data from LMArena' + link to lmarena.ai. Do not scrape arena.ai pages (ToS).
  • lmarena_t2i: https://lmarena.ai/leaderboard/text-to-image (leaderboard date 2026-10-07). CC BY 4.0 (HF dataset lmarena-ai/leaderboard-dataset). Attribution required: 'Data from LMArena' + link to lmarena.ai. Do not scrape arena.ai pages (ToS).
  • lmarena_image_edit: https://lmarena.ai/leaderboard/image-edit (leaderboard date 2026-10-06). CC BY 4.0 (HF dataset lmarena-ai/leaderboard-dataset). Attribution required: 'Data from LMArena' + link to lmarena.ai. Do not scrape arena.ai pages (ToS).
  • lmarena_t2v: https://lmarena.ai/leaderboard/text-to-video (leaderboard date 2026-09-22). CC BY 4.0 (HF dataset lmarena-ai/leaderboard-dataset). Attribution required: 'Data from LMArena' + link to lmarena.ai. Do not scrape arena.ai pages (ToS).
  • lmarena_i2v: https://lmarena.ai/leaderboard/image-to-video (leaderboard date 2026-09-22). CC BY 4.0 (HF dataset lmarena-ai/leaderboard-dataset). Attribution required: 'Data from LMArena' + link to lmarena.ai. Do not scrape arena.ai pages (ToS).
  • Epoch Capabilities Index: https://epoch.ai/benchmarks (leaderboard date 2026-10-18). CC BY 4.0. Attribution required: 'Data from Epoch AI' + link to epoch.ai/benchmarks.
  • GPQA Diamond (%): https://epoch.ai/benchmarks (leaderboard date 2026-10-18). CC BY 4.0. Attribution required: 'Data from Epoch AI' + link to epoch.ai/benchmarks.
  • SWE-bench Verified (%): https://epoch.ai/benchmarks (leaderboard date 2026-10-18). CC BY 4.0. Attribution required: 'Data from Epoch AI' + link to epoch.ai/benchmarks.
  • open_asr: https://huggingface.co/spaces/hf-audio/open_asr_leaderboard (leaderboard date 2026-10-18). No explicit license on dataset; cite 'Open ASR Leaderboard (Hugging Face)' with link.

Data from LMArena (lmarena.ai, CC BY 4.0) and Epoch AI (epoch.ai/benchmarks, CC BY 4.0). "Best value" marks models on the cost-quality frontier for a task: no other model is both cheaper and higher rated.

Price history

Daily copies of the public OpenRouter model list from October 2026, and 90 weekly Internet Archive copies of the same list back to Mar 26, 2024. A change is logged when input or output price moves by 5% or more. For open-weight models a change can come from a different cheapest host rather than a new official price.

Limitations

  • One run per model and task: token counts vary between runs, especially reasoning tokens.
  • Default settings: setting a lower reasoning effort can cut the cost of reasoning models several times.
  • English prompts: other languages use more tokens (Polish about 1.3-1.6x in our earlier test).
  • Media prices are per unit from official pages; quality and speed differ a lot between them.

Raw measurements

TaskModelInputOutputReasoningFinishDate
Analyze a 200-row spreadsheetClaude Haiku 5.55,43356615,434length2026-10-11
Analyze a 200-row spreadsheetClaude Sonnet 5.55,433016,000length2026-10-11
Analyze a 200-row spreadsheetDeepSeek V4.1 Flash4,801016,000length2026-10-11
Analyze a 200-row spreadsheetDeepSeek V4.1 Flash4,801016,000length2026-10-11
Analyze a 200-row spreadsheetGemini 3.1 Pro Preview6,3048943,933stop2026-10-11
Analyze a 200-row spreadsheetGemini 3.1 Pro Preview6,3048694,784stop2026-10-11
Analyze a 200-row spreadsheetGemini 3.8 Flash6,3041,0187,100stop2026-10-11
Analyze a 200-row spreadsheetMiniMax M34,94015315,847length2026-10-11
Analyze a 200-row spreadsheetMistral Large 44,7744,5266,174stop2026-10-11
Analyze a 200-row spreadsheetMistral Large 44,7748,3157,685length2026-10-11
Analyze a 200-row spreadsheetGPT-6 Luna4,7794562,338stop2026-10-11
Analyze a 200-row spreadsheetGPT-6.1 Sol4,7799196,553stop2026-10-11
Analyze a 200-row spreadsheetQwen3.8 Flash6,357016,000length2026-10-11
Analyze a 200-row spreadsheetQwen3.8 Flash6,357016,000length2026-10-11
Analyze a 200-row spreadsheetGrok 4.76,0141,20232,647stop2026-10-11
Analyze a 200-row spreadsheetGrok 4.76,0141,32427,399stop2026-10-11
Analyze a 200-row spreadsheetGLM 5.3 Flash4,900016,000length2026-10-11
Analyze a 200-row spreadsheetGLM 5.3 Flash4,900016,000length2026-10-11
Write a 1,500-word blog postClaude Haiku 5.5903,560200stop2026-10-11
Write a 1,500-word blog postClaude Sonnet 5.5903,401239stop2026-10-11
Write a 1,500-word blog postDeepSeek V4.1 Flash812,230809stop2026-10-11
Write a 1,500-word blog postGemini 3.1 Pro Preview522,5221,606stop2026-10-11
Write a 1,500-word blog postGemini 3.8 Flash522,621964stop2026-10-11
Write a 1,500-word blog postMiniMax M32191,774442stop2026-10-11
Write a 1,500-word blog postMistral Large 4552,0645,705stop2026-10-11
Write a 1,500-word blog postGPT-6 Luna592,217143stop2026-10-11
Write a 1,500-word blog postGPT-6.1 Sol592,116172stop2026-10-11
Write a 1,500-word blog postQwen3.8 Flash104016,000length2026-10-11
Write a 1,500-word blog postQwen3.8 Flash104016,000length2026-10-11
Write a 1,500-word blog postGrok 4.71,2963,185183stop2026-10-11
Write a 1,500-word blog postGrok 4.71,2963,049271stop2026-10-11
Write a 1,500-word blog postGLM 5.3 Flash622,1744,749stop2026-10-11
Answer a chat questionClaude Haiku 5.5611,055244stop2026-10-11
Answer a chat questionClaude Sonnet 5.5611,2560stop2026-10-11
Answer a chat questionDeepSeek V4.1 Flash658595,461stop2026-10-11
Answer a chat questionGemini 3.1 Pro Preview35915862stop2026-10-11
Answer a chat questionGemini 3.8 Flash35788843stop2026-10-11
Answer a chat questionMiniMax M3205447482stop2026-10-11
Answer a chat questionMistral Large 440523854stop2026-10-11
Answer a chat questionGPT-6 Luna45540639stop2026-10-11
Answer a chat questionGPT-6.1 Sol45560103stop2026-10-11
Answer a chat questionQwen3.8 Flash88689909stop2026-10-11
Answer a chat questionGrok 4.71,280567603stop2026-10-11
Answer a chat questionGLM 5.3 Flash483911,391stop2026-10-11
Classify 100 support ticketsClaude Haiku 5.51,747535961stop2026-10-11
Classify 100 support ticketsClaude Sonnet 5.51,7475410stop2026-10-11
Classify 100 support ticketsDeepSeek V4.1 Flash1,2964614,462stop2026-10-11
Classify 100 support ticketsGemini 3.1 Pro Preview1,3975657,062stop2026-10-11
Classify 100 support ticketsGemini 3.8 Flash1,3975716,059stop2026-10-11
Classify 100 support ticketsMiniMax M31,4244411,485stop2026-10-11
Classify 100 support ticketsMistral Large 41,2749297,538stop2026-10-11
Classify 100 support ticketsMistral Large 41,2747355,593stop2026-10-11
Classify 100 support ticketsGPT-6 Luna1,261445590stop2026-10-11
Classify 100 support ticketsGPT-6.1 Sol1,26144277stop2026-10-11
Classify 100 support ticketsQwen3.8 Flash1,46332415,676length2026-10-11
Classify 100 support ticketsQwen3.8 Flash1,4635347,879stop2026-10-11
Classify 100 support ticketsGrok 4.72,5124792,534stop2026-10-11
Classify 100 support ticketsGrok 4.72,5124792,266stop2026-10-11
Classify 100 support ticketsGLM 5.3 Flash1,2814411,508stop2026-10-11
Classify 100 support ticketsGLM 5.3 Flash1,28144113,315stop2026-10-11
Review 500 lines of codeClaude Haiku 5.57,1173,2588,532stop2026-10-11
Review 500 lines of codeClaude Sonnet 5.57,1173,0673,476stop2026-10-11
Review 500 lines of codeDeepSeek V4.1 Flash4,801016,000length2026-10-11
Review 500 lines of codeGemini 3.1 Pro Preview5,2071,3687,256stop2026-10-11
Review 500 lines of codeGemini 3.8 Flash5,2072,0740stop2026-10-11
Review 500 lines of codeMiniMax M34,7622,5880stop2026-10-11
Review 500 lines of codeMistral Large 44,6581,5947,522stop2026-10-11
Review 500 lines of codeGPT-6 Luna4,5573374,921stop2026-10-11
Review 500 lines of codeGPT-6.1 Sol4,5571,1352,843stop2026-10-11
Review 500 lines of codeQwen3.8 Flash4,874016,000length2026-10-11
Review 500 lines of codeGrok 4.76,1851,75934,951stop2026-10-11
Review 500 lines of codeGrok 4.76,1851,38131,448stop2026-10-11
Review 500 lines of codeGLM 5.3 Flash4,532215,998length2026-10-11
Reply to a customer emailClaude Haiku 5.5362290346stop2026-10-11
Reply to a customer emailClaude Sonnet 5.53623250stop2026-10-11
Reply to a customer emailDeepSeek V4.1 Flash2792251,749stop2026-10-11
Reply to a customer emailGemini 3.1 Pro Preview271128637stop2026-10-11
Reply to a customer emailGemini 3.8 Flash271128424stop2026-10-11
Reply to a customer emailMiniMax M3417260187stop2026-10-11
Reply to a customer emailMistral Large 42545992,946stop2026-10-11
Reply to a customer emailGPT-6 Luna258940stop2026-10-11
Reply to a customer emailGPT-6.1 Sol2581260stop2026-10-11
Reply to a customer emailQwen3.8 Flash32698587stop2026-10-11
Reply to a customer emailGrok 4.71,492124137stop2026-10-11
Reply to a customer emailGrok 4.71,49287308stop2026-10-11
Reply to a customer emailGLM 5.3 Flash26523424stop2026-10-11
Extract data from an invoiceClaude Haiku 5.5546574591stop2026-10-11
Extract data from an invoiceClaude Sonnet 5.5546604317stop2026-10-11
Extract data from an invoiceDeepSeek V4.1 Flash3954492,436stop2026-10-11
Extract data from an invoiceGemini 3.1 Pro Preview4525332,053stop2026-10-11
Extract data from an invoiceGemini 3.8 Flash452539943stop2026-10-11
Extract data from an invoiceMiniMax M3536563289stop2026-10-11
Extract data from an invoiceMistral Large 43748711,784stop2026-10-11
Extract data from an invoiceGPT-6 Luna373422140stop2026-10-11
Extract data from an invoiceGPT-6.1 Sol3734500stop2026-10-11
Extract data from an invoiceQwen3.8 Flash5015551,134stop2026-10-11
Extract data from an invoiceQwen3.8 Flash5015672,100stop2026-10-11
Extract data from an invoiceGrok 4.71,6174641,085stop2026-10-11
Extract data from an invoiceGrok 4.71,617464733stop2026-10-11
Extract data from an invoiceGLM 5.3 Flash391455576stop2026-10-11
Write 10 product descriptionsClaude Haiku 5.54541,8032,342stop2026-10-11
Write 10 product descriptionsClaude Sonnet 5.54541,8662,459stop2026-10-11
Write 10 product descriptionsDeepSeek V4.1 Flash3071,0133,458stop2026-10-11
Write 10 product descriptionsDeepSeek V4.1 Flash307727226stop2026-10-11
Write 10 product descriptionsGemini 3.1 Pro Preview2931,2452,130stop2026-10-11
Write 10 product descriptionsGemini 3.1 Pro Preview2931,2131,838stop2026-10-11
Write 10 product descriptionsGemini 3.8 Flash2931,1271,817stop2026-10-11
Write 10 product descriptionsMiniMax M34411,071531stop2026-10-11
Write 10 product descriptionsMistral Large 42859242,087stop2026-10-11
Write 10 product descriptionsMistral Large 42857732,374stop2026-10-11
Write 10 product descriptionsGPT-6 Luna28398734stop2026-10-11
Write 10 product descriptionsGPT-6.1 Sol2831,10372stop2026-10-11
Write 10 product descriptionsQwen3.8 Flash345016,000length2026-10-11
Write 10 product descriptionsQwen3.8 Flash345016,000length2026-10-11
Write 10 product descriptionsGrok 4.71,52490856stop2026-10-11
Write 10 product descriptionsGrok 4.71,52493541stop2026-10-11
Write 10 product descriptionsGLM 5.3 Flash2891,1655,060stop2026-10-11
Write 10 product descriptionsGLM 5.3 Flash2891,0376,821stop2026-10-11
Proofread one page (500 words)Claude Haiku 5.5987913304stop2026-10-11
Proofread one page (500 words)Claude Sonnet 5.59879130stop2026-10-11
Proofread one page (500 words)DeepSeek V4.1 Flash7466601,287stop2026-10-11
Proofread one page (500 words)DeepSeek V4.1 Flash7466602,621stop2026-10-11
Proofread one page (500 words)Gemini 3.1 Pro Preview7066532,445stop2026-10-11
Proofread one page (500 words)Gemini 3.1 Pro Preview7066531,978stop2026-10-11
Proofread one page (500 words)Gemini 3.8 Flash706653443stop2026-10-11
Proofread one page (500 words)MiniMax M3874651416stop2026-10-11
Proofread one page (500 words)Mistral Large 47317713,522stop2026-10-11
Proofread one page (500 words)Mistral Large 47319483,041stop2026-10-11
Proofread one page (500 words)GPT-6 Luna71866491stop2026-10-11
Proofread one page (500 words)GPT-6.1 Sol7186620stop2026-10-11
Proofread one page (500 words)Qwen3.8 Flash777671915stop2026-10-11
Proofread one page (500 words)Qwen3.8 Flash7776711,404stop2026-10-11
Proofread one page (500 words)Grok 4.71,947648383stop2026-10-11
Proofread one page (500 words)Grok 4.71,9476481,196stop2026-10-11
Proofread one page (500 words)GLM 5.3 Flash730621310stop2026-10-11
Proofread one page (500 words)GLM 5.3 Flash7306621stop2026-10-11
Answer from 5 retrieved documents (RAG)Claude Haiku 5.55,1841,096753stop2026-10-11
Answer from 5 retrieved documents (RAG)Claude Sonnet 5.55,1841,079615stop2026-10-11
Answer from 5 retrieved documents (RAG)DeepSeek V4.1 Flash3,6966483,102stop2026-10-11
Answer from 5 retrieved documents (RAG)Gemini 3.1 Pro Preview3,6085991,579stop2026-10-11
Answer from 5 retrieved documents (RAG)Gemini 3.8 Flash3,6086691,499stop2026-10-11
Answer from 5 retrieved documents (RAG)MiniMax M33,773822437stop2026-10-11
Answer from 5 retrieved documents (RAG)Mistral Large 43,7722433,209stop2026-10-11
Answer from 5 retrieved documents (RAG)Mistral Large 43,7721762,754stop2026-10-11
Answer from 5 retrieved documents (RAG)GPT-6 Luna3,690234104stop2026-10-11
Answer from 5 retrieved documents (RAG)GPT-6.1 Sol3,6904420stop2026-10-11
Answer from 5 retrieved documents (RAG)Qwen3.8 Flash3,7511,4212,179stop2026-10-11
Answer from 5 retrieved documents (RAG)Qwen3.8 Flash3,7515261,646stop2026-10-11
Answer from 5 retrieved documents (RAG)Grok 4.74,889273717stop2026-10-11
Answer from 5 retrieved documents (RAG)Grok 4.74,8891501,110stop2026-10-11
Answer from 5 retrieved documents (RAG)GLM 5.3 Flash3,7143971,052stop2026-10-11
Answer from 5 retrieved documents (RAG)GLM 5.3 Flash3,7141512,342stop2026-10-11
Write a SQL queryClaude Haiku 5.5182840810stop2026-10-11
Write a SQL queryClaude Sonnet 5.5182839218stop2026-10-11
Write a SQL queryDeepSeek V4.1 Flash1453812,196stop2026-10-11
Write a SQL queryDeepSeek V4.1 Flash1454311,250stop2026-10-11
Write a SQL queryGemini 3.1 Pro Preview1356122,156stop2026-10-11
Write a SQL queryGemini 3.1 Pro Preview1356291,810stop2026-10-11
Write a SQL queryGemini 3.8 Flash1355461,043stop2026-10-11
Write a SQL queryMiniMax M32727020stop2026-10-11
Write a SQL queryMistral Large 41215092,837stop2026-10-11
Write a SQL queryMistral Large 41215132,458stop2026-10-11
Write a SQL queryGPT-6 Luna126266516stop2026-10-11
Write a SQL queryGPT-6.1 Sol126312147stop2026-10-11
Write a SQL queryQwen3.8 Flash1774091,453stop2026-10-11
Write a SQL queryQwen3.8 Flash1774012,998stop2026-10-11
Write a SQL queryGrok 4.71,372361732stop2026-10-11
Write a SQL queryGrok 4.71,3724411,510stop2026-10-11
Write a SQL queryGLM 5.3 Flash1284861,495stop2026-10-11
Write a SQL queryGLM 5.3 Flash128459752stop2026-10-11
Summarize a 20-page PDFClaude Haiku 5.518,004672118stop2026-10-11
Summarize a 20-page PDFClaude Sonnet 5.518,0048570stop2026-10-11
Summarize a 20-page PDFDeepSeek V4.1 Flash12,9704271,565stop2026-10-11
Summarize a 20-page PDFGemini 3.1 Pro Preview12,7416111,079stop2026-10-11
Summarize a 20-page PDFGemini 3.8 Flash12,741535713stop2026-10-11
Summarize a 20-page PDFMiniMax M312,901462246stop2026-10-11
Summarize a 20-page PDFMistral Large 413,3731512,019stop2026-10-11
Summarize a 20-page PDFGPT-6 Luna12,98744984stop2026-10-11
Summarize a 20-page PDFGPT-6.1 Sol12,98748226stop2026-10-11
Summarize a 20-page PDFQwen3.8 Flash13,1195012,141stop2026-10-11
Summarize a 20-page PDFGrok 4.713,954535523stop2026-10-11
Summarize a 20-page PDFGLM 5.3 Flash12,9955441,270stop2026-10-11
Translate 1,000 wordsClaude Haiku 5.51,8322,1530stop2026-10-11
Translate 1,000 wordsClaude Sonnet 5.51,8322,1860stop2026-10-11
Translate 1,000 wordsDeepSeek V4.1 Flash1,3541,41710,291stop2026-10-11
Translate 1,000 wordsGemini 3.1 Pro Preview1,3081,3327,494stop2026-10-11
Translate 1,000 wordsGemini 3.8 Flash1,3081,2973,167stop2026-10-11
Translate 1,000 wordsMiniMax M31,4731,2074,535stop2026-10-11
Translate 1,000 wordsMistral Large 41,3491,31710,816stop2026-10-11
Translate 1,000 wordsGPT-6 Luna1,3301,32867stop2026-10-11
Translate 1,000 wordsGPT-6.1 Sol1,3301,28944stop2026-10-11
Translate 1,000 wordsQwen3.8 Flash1,3861,166136stop2026-10-11
Translate 1,000 wordsQwen3.8 Flash1,386016,000length2026-10-11
Translate 1,000 wordsGrok 4.72,5421,316252stop2026-10-11
Translate 1,000 wordsGrok 4.72,5421,3165,071stop2026-10-11
Translate 1,000 wordsGLM 5.3 Flash1,3301,3948,761stop2026-10-11