Skip to main content
Consensus publishes head-to-head evaluations of its search on real, anonymized user queries. Independent PhD-level researchers score every result blind to which system returned it. This page summarizes each evaluation and links to the full write-up. Last updated: October 27, 2026.

At a glance

Precision is the share of returned papers that judges scored 3 or 4 on a 0 to 4 relevance scale, meaning a researcher would consider them genuinely relevant to the query.

Consensus vs frontier models

Consensus Pro returned more relevant papers than GPT-5.5 and Opus 4.8, each with web search, on 100 real user queries. In the top 20 results it was 4.5% more precise than GPT-5.5 and 13.2% more precise than Opus 4.8. It was also about 10x faster: a median of 8.7 seconds end to end, against 74.8 seconds for GPT-5.5 and 102.8 seconds for Opus 4.8. Query by query, Consensus had the most relevant results of the three systems on 59 of the 100 queries. GPT-5.5 led on 22 and Opus 4.8 on 6, with 11 ties. All three systems lose precision deeper in the list, but the general models lose it faster: GPT-5.5 drops 10.3 points between rank 10 and rank 20, and Consensus drops 8.7. How it was run
  • Systems: Consensus Pro, the agentic search mode that decomposes a query, runs multiple retrievals against the Consensus index, and re-ranks. GPT-5.5 and Opus 4.8 each ran with web search enabled and received the same query.
  • Queries: 100 randomly sampled queries from real Consensus users, anonymized and scrubbed of PII. They range from keyword lookups and metadata queries to natural-language research questions.
  • Judging: four independent PhD-level researchers scored every query–paper pair from 0 (irrelevant) to 4 (perfectly relevant), blind to the source system. LLM-as-a-judge across several frontier models gave a second, independent signal.
  • Latency: wall-clock time from query submission to a complete set of results.
Limitations: judges saw titles and abstracts only, because some systems don’t expose full text. Scores reflect textual relevance, not recency, citation count, or journal quality. The evaluation measured time, not token or dollar cost.

Read the full evaluation

Consensus Beats Claude and ChatGPT in Academic Search, on the Consensus blog.

Consensus vs Google Scholar

Consensus Quick search returned more relevant papers than Google Scholar on 500 real user queries. In the top 10 results, 88.1% of Consensus papers were relevant versus 81.8% for Google Scholar, 7.4% better. Consensus also ranked relevant papers higher, with 10.3% better DCG, and had higher average relevance at every rank in the list. How it was run
  • Systems: Consensus Quick search, a single-pass search, against Google Scholar. Consensus Pro and Deep modes, which run iterative multi-query searches, weren’t tested.
  • Queries: 500 randomly sampled queries from real Consensus users, anonymized and scrubbed of PII, from keywords and metadata queries to natural-language questions.
  • Judging: the top 10 results from each system made about 10,000 query–paper pairs. Three independent PhD researchers, each with at least a decade of experience assessing research, scored every pair from 0 to 4, blind to the source system.
Limitations: judges saw titles and abstracts only. Scores reflect textual relevance, not recency, citation count, or journal quality.

Read the full evaluation

Consensus Outperforms Google Scholar for Academic Search Retrieval, on the Consensus blog.
The same search is available in the web app, over the REST API, and as an MCP server for ChatGPT, Claude, Perplexity, Cursor, and other AI assistants. To see how ranking works, read Paper search.