> ## Documentation Index
> Fetch the complete documentation index at: https://docs.consensus.app/llms.txt
> Use this file to discover all available pages before exploring further.

# Grade a set of papers with your own evidence rubric

> Apply your organization's standard for what counts as good evidence, consistently, across every paper on a topic.

## What you get

Every paper scored against the same rubric, with the reason for each score and the evidence behind it — so two people asking the same question a month apart get comparable answers.

This is what our highest-volume self-serve user does: a biotech founder running a custom evidence-grading prompt over hundreds of searches.

## Who it's for

Any team with an internal standard for evidence quality — biotech, clinical, nutrition, or research operations — where "is this well-supported?" currently depends on who you ask.

## The prompt

```text theme={null}
You are an evidence assessor applying a fixed grading rubric.

Question: [THE CLAIM OR TOPIC]

Rubric (score every paper on all three axes):

A. Design strength
   5 — meta-analysis or systematic review of controlled trials
   4 — randomized controlled trial
   3 — prospective cohort or well-controlled non-randomized study
   2 — cross-sectional, case-control, or retrospective
   1 — case report, in vitro, or animal only

B. Execution quality
   5 — adequately powered, pre-registered, low attrition, blinded where possible
   3 — reasonable but with named limitations
   1 — serious limitations: underpowered, high attrition, no control

C. Relevance to our question
   5 — same population, same exposure, same outcome
   3 — differs on one dimension; say which
   1 — indirect or surrogate outcome only

Rules:
- Score from what the paper reports, not from journal or author reputation.
- Where a paper does not report enough to score an axis, mark it NOT REPORTED
  rather than assuming the middle. Do not average around missing information.
- A high design score does not raise the execution score. A large RCT executed
  badly is a 4 and a 1, and that combination is the useful signal.
- State the single fact that most constrains each score.

Output: Paper | DOI | A | B | C | Overall | Limiting factor | Supporting quote.
Then: the highest-graded evidence, the strongest contradicting evidence, and
what an ideal study would need to settle the question.
```

## Getting a rubric worth applying

<Steps>
  <Step title="Separate the axes">
    Design, execution, and relevance fail independently. A single 1–5 "quality" score hides which one is the problem and makes the grades incomparable across topics.
  </Step>

  <Step title="Anchor each level with an example">
    "Well-controlled" means different things to different readers. Naming a real paper at each level makes the scale reproducible.
  </Step>

  <Step title="Prohibit prestige as a proxy">
    Journal and author reputation are already available as `sjr_best_quartile` and `citation_count`. Keep them as separate metadata, not smuggled into the quality score.
  </Step>

  <Step title="Make NOT REPORTED a real value">
    Papers that omit a detail are common. Scoring them at the midpoint quietly inflates the evidence base.
  </Step>
</Steps>

## Feeding it good candidates

```bash theme={null}
curl -G "https://api.consensus.app/v1/search" \
  -H "x-api-key: $CONSENSUS_API_KEY" \
  --data-urlencode "query=omega-3 supplementation cardiovascular events" \
  --data-urlencode "study_types=meta-analysis,systematic review,rct" \
  --data-urlencode "human=true" \
  --data-urlencode "sample_size_min=100" \
  --data-urlencode "page_size=50"
```

Paid plans return `study_type` and `takeaway` on each result, which give the grader a head start on axis A. `include_full_text_chunks=true` is what makes axis B possible at all — execution quality is rarely visible in an abstract.

## Audit requirements

Follow the [grounding rules](/use-cases/best-practices#grounding-and-auditability). Record the rubric version alongside every graded set: a rubric edit makes old grades incomparable, and teams discover this months later when two reports disagree.

## What to check before you trust it

* **Check the score distribution.** If nothing scores low, the rubric is being applied generously — real literature has a spread.
* **Grade five papers by hand first** and compare. Disagreements almost always point at an ambiguous rubric level, not a model failure.
* **Watch for axis contamination.** If design and execution scores move together on every paper, they are not being assessed separately.
* **Execution scores from abstracts are unreliable.** Without full text, mark axis B as low-confidence rather than guessing.

## Related

<CardGroup cols={2}>
  <Card title="Create a biomarker evidence report" icon="chart-line" href="/use-cases/biomarker-report">
    Tiered grading applied to one biomarker question.
  </Card>

  <Card title="Check whether a finding has human evidence yet" icon="flask" href="/use-cases/translational-readiness">
    Split preclinical from human evidence before grading it.
  </Card>

  <Card title="Best practices" icon="shapes" href="/use-cases/best-practices">
    Grounding rules and the five primitives.
  </Card>

  <Card title="All use cases" icon="grid-2" href="/use-cases">
    Browse the gallery by persona.
  </Card>
</CardGroup>