Paper search

Consensus puts discovery at the center of everything it does, combining classic and modern search techniques. It only uses AI after it searches the scientific literature, and that ordering is what makes every answer traceable to a real paper. Here’s the pipeline.

​

Step 1: Cast a wide net

Consensus scans its entire index of 400M+ scholarly articles and matches your query against titles, abstracts, and full text (when available) using hybrid search: classic keyword search combined with AI-powered semantic search.

• Semantic search, powered by AI embeddings, captures the intent behind your question, so conversational prompts and natural-language questions work.
• Keyword search, powered by BM25, anchors results to the exact terms in your query.
• Boolean operators and exact matching work too, for when you need precise control.

Together they assign a relevance score to every paper. Consensus also rewrites typos, expands MeSH synonyms, and supports Boolean operators, so keywords, questions, and structured queries all work.

​

Step 2: Refine by quality

The top 1,500 papers are re-ranked. The ranking logic is designed to mimic how a researcher weighs research:

• About 80% is textual relevance: how well the paper actually matches your question.
• About 20% is research-quality signals, such as:
  • Citation count: how often other researchers have cited the paper. Influence, not proof of validity.
  • Recency: how recently it was published, so new work can surface alongside established papers.
  • Journal reputation: the quality and impact of the journal it was published in.

Relevance comes first, so a well-cited paper that only loosely matches your question won’t crowd out the papers that answer it.

​

Step 3: Rank the top 20 for precision

Consensus recalculates textual relevance on the top 20 papers using a larger, more compute-intensive model optimized for precision rather than broad recall. Quality signals still factor in, so the final list reflects both relevance and rigor.

​

Step 4: Check relevance before writing anything

Before any AI summary is generated, separate checker models verify that each paper actually contains information relevant to the query. Papers that don’t meet the threshold are excluded. If nothing relevant is found, Consensus tells you rather than letting a model fill in gaps that don’t exist.

​

Step 5: Analyze and synthesize

Once relevant papers are retrieved, AI is used in two ways:

• Analyze individual papers — for example, the Study Snapshot, which extracts population, sample size, duration, location, methods, outcomes, and results.
• Synthesize across papers — the Pro answer, the Consensus Meter, claims and evidence tables, research gaps heat maps, and timelines.

Models are instructed and trained to generate responses only from the text of the retrieved papers. Consensus uses commercial models (like OpenAI’s) for general summarization and fine-tuned open-source models for domain-specific features like the Consensus Meter. In Pro and Deep modes, Research Agent adds planning on top of this pipeline: it breaks a complex question into sub-searches, applies filters, walks the citation graph, reads across papers, and composes a citation-backed answer. Deep Review is the extended version: up to 20 searches in parallel, multiple agent tools like citation crawl, and 50 to 100 papers read before answering.

​

How we measure search quality

Consensus tests its search head to head against the tools researchers already use:

• Google Scholar: Consensus beat Google Scholar on retrieval accuracy by 10%.
• PubMed: Consensus beat PubMed on medical retrieval accuracy by 9%, as judged by third-party clinicians.

​

Why hallucinations are contained

Consensus only searches its index of academic literature, and every source it cites is tied to a real academic paper. On top of that, Consensus has a number of guardrails built in to prevent hallucinations. The most important is Citation Grounding, which shows you the exact sentence behind each claim and exactly where in the paper it comes from.

​

Next