Methodology — ArtiProof

Pipeline

An analysis runs the five engines in two phases. Phase one extracts the artifacts from the source: claims, paragraphs, citations, authorship. Phase two scores each artifact against external ground truth and computes the composite credibility number. Engine output, prompts, and model versions are stored alongside every run so disputes can be replayed against the exact code and prompt revision that produced the score.

Ingest. Pull the source. URL: scrape with politeness + cite-pull. PDF: extract text and bounding boxes for annotation. Pasted text: tokenize and segment into paragraphs.

Extract claims. An LLM lifts the verifiable factual statements out of the text. Opinions, rhetorical questions, and untestable predictions are skipped. Up to 15 claims per source.

Verify in parallel. Each claim is cross-referenced against Google Fact Check Tools and an LLM panel. AI-generation probability is scored per paragraph. Citations are checked against CrossRef and OpenAlex for retraction status.

Trace origins. False or misleading claims are traced to their likely point of origin (platform, publication, date) so a reader can see where the misinformation started.

Score and explain. The composite score is computed by summing weighted deductions. Every deduction is preserved next to its evidence. Coverage caveats are surfaced when the source was sampled rather than fully analysed.

The five engines

Every score is the sum of these five engines’ deductions, starting from 100. The weights below ship with each report under the formula_version field so you can confirm the version your report was scored against didn’t shift in the meantime.

Claim extraction & fact-checking

An LLM segments the source into discrete, verifiable claims and assigns each a verdict (true / false / misleading / partially true / unverified) backed by named sources.

Sources: Google Fact Check Tools API; Gemini 2.0 Flash for inference; the model's training distribution as a fallback.
Max impact: Up to −15 per false claim; up to −10 per misleading. No upper cap on number of claims (top-15 sampled).
Known limits: Claims that are too long or compound get split inconsistently. Fact-check coverage is strongest for English-language political and scientific topics.

AI-content detection

Scores each paragraph 0–1 for the probability that it was authored by an LLM. Signals include uniform sentence rhythm, lack of personal voice, and absence of domain-specific colloquialism.

Sources: Gemini 2.0 Flash with a calibrated prompt. No proprietary classifier.
Max impact: Probabilistic. Per-percent deductions kick in past 50% (−0.3 per point above 50).
Known limits: Probabilistic by design. Edited or post-processed AI text reads cleaner than first-pass output. Always shipped with a confidence number; treat as one signal, not proof.

Citation verification

Each citation in the source is resolved against CrossRef and OpenAlex. The engine flags broken (unresolvable), inaccessible (paywalled or gone), and retracted references.

Sources: CrossRef Works API; OpenAlex; the source's own citation list when scraped.
Max impact: Up to −15 per retracted citation; up to −5 per broken or inaccessible.
Known limits: Recent retractions (within days) may not be indexed yet. Behind-paywall citations are marked inaccessible rather than verified.

Citation ring detection

Builds the local citation graph around the researcher and surfaces tightly-cyclic subgroups whose mutual citation rate exceeds normal field baselines.

Sources: Semantic Scholar; Scopus when an API key is provided; CrossRef for fallback.
Max impact: Researcher-tier signal: contributes to integrity_score adjustments and the citation_ring_flags array.
Known limits: Coverage gaps in non-English literature can hide rings that span Russian-, Chinese-, or Arabic-language venues. The 25% mutual-citation threshold is calibrated against representative samples of English-language CS/biology/medicine literature.

Researcher integrity scoring

Adjusts a researcher's h-index by stripping out self-citations and citations from suspected rings, then re-computing. Topic-relevance scoring flags publications outside the researcher's stated field.

Sources: Same as the ring detector. Publication list pulled from Semantic Scholar plus ORCID when available.
Max impact: Adjusted-vs-official h-index delta is the headline number. Per-publication flags are listed in the integrity report.
Known limits: Topic-relevance is bounded by the venue tagging in Semantic Scholar's training data. Career transitions (e.g. moving fields legitimately) can produce false positives.

Scoring formula

Source credibility starts at 100 and walks down by deductions. Researcher integrity starts at 100 and walks down by the same mechanism but with different weights. Both are clamped to the range 0–100.

credibility = max(0, min(100, base
  − 15 × false_claims
  − 10 × misleading_claims
  − 5 × partially_true_claims
  − 3 × unverified_claims
  − 0.3 × max(0, ai_percent − 50)
  − 15 × retracted_citations
  − 5 × inaccessible_citations)

Every deduction in the formula appears verbatim in the score_breakdown.deductions array of every report. The base value, intermediate sum, and clamp are all visible to the reviewer. No hidden coefficients.

What ArtiProof is not

Not a plagiarism checker for unpublished work. The plagiarism signal relies on indexed and cited corpora. If you’re checking against a private document repository, point us at it as a source — we don’t guess at archives we don’t have access to.

Not a deepfake detector. The AI-content engine scores text for LLM-generation probability. Image and video content pass through unscored.

Not a peer reviewer.ArtiProof can confirm that citations resolve, that claims agree with external ground truth, and that the author’s pattern of self-citation is within norms. It does not evaluate the novelty or significance of the work.

Not a substitute for retraction watch.We cross-check citations against CrossRef and OpenAlex. When those databases haven’t indexed a recent retraction yet, ArtiProof can’t see it either — expect a lag of days to weeks for fresh retractions.

Reproducibility

Every analysis stores the engine version, prompt version, and prompt hash that produced it. When a prompt is edited without a version bump, the next analysis fires a drift warning to our error tracker. Disputes get the same engine + same prompt + same external API snapshot at the time of the dispute.

Institutional buyers get a separate audit log: the operator tier-change ledger and the per-call API metric (tokens, latency, model) for every analysis run by users in their organization.

How ArtiProof reads a source