Skip to main content

Enforcing Citations in RAG: Citation Grounding

Citation grounding is the practice of requiring RAG-generated answers to explicitly attribute claims to source passages. A cited answer allows users to verify facts and traces hallucinations back to retrieval failures rather than generation errors. Citation grounding is especially critical for high-stakes domains (medicine, law, finance) where users need to understand the source of information.

Enforcing citations requires three steps: (1) configure the generator to output citations alongside answers, (2) validate that cited passages actually appear in retrieved documents, and (3) measure citation precision (are all citations accurate?) and recall (are all major claims cited?). I learned the importance of this when a medical RAG system generated an answer about drug interactions without citing sources—the answer was correct, but users had no way to verify it against their own knowledge.

Configuring Generators to Produce Citations​

Modern language models can be prompted to include citations in structured formats. Use prompt engineering to ask the model to cite specific passages alongside each claim.

def rag_prompt_with_citation_instruction(query: str,
                                         passages: List[str]) -> str:
    """
    Generate a RAG prompt that instructs the model to cite sources.
    
    Args:
        query: User query.
        passages: Retrieved passages (with identifiers).
    
    Returns:
        Formatted prompt requesting citation output.
    """
    
    passages_text = "\n".join([
        f"[Source {i}]: {passage}" 
        for i, passage in enumerate(passages)
    ])
    
    prompt = f"""
You are a helpful assistant that answers questions using the provided sources.

Question: {query}

Sources:
{passages_text}

Instructions:
1. Answer the question using ONLY information from the provided sources.
2. After each factual claim, cite the source using [Source N] format.
3. If you cannot answer from the sources, say "I cannot find this information in the provided sources."
4. Do not speculate or use outside knowledge.

Example answer format:
"Paris is the capital of France [Source 0]. It is located in the north-central part of the country [Source 1]."

Your answer:
"""
    
    return prompt

# Example
query = "What is the capital of France?"
passages = [
    "France is a country in Western Europe. Its capital is Paris.",
    "Paris is located in the north-central part of France along the Seine river."
]

prompt = rag_prompt_with_citation_instruction(query, passages)
print(prompt)

Structured Citation Output Format​

For programmatic validation, use a structured format where the model outputs both the answer and a list of citations with exact passage text.

from dataclasses import dataclass
from typing import List
import json

@dataclass
class CitedAnswer:
    """Answer with structured citations."""
    answer_text: str
    citations: List[Dict]  # List of {"claim": str, "source_id": int, "passage": str}

def parse_structured_citation_output(model_response: str) -> CitedAnswer:
    """
    Parse JSON output from model that includes structured citations.
    
    Args:
        model_response: Model output (assumed to be JSON).
    
    Returns:
        CitedAnswer with parsed claims and citations.
    """
    
    response_json = json.loads(model_response)
    
    cited_answer = CitedAnswer(
        answer_text=response_json["answer"],
        citations=response_json["citations"]
    )
    
    return cited_answer

# Example structured output format (prompt the model to output this)
example_output = """{
  "answer": "Paris is the capital of France. It is located in north-central France.",
  "citations": [
    {
      "claim": "Paris is the capital of France",
      "source_id": 0,
      "passage": "Its capital is Paris."
    },
    {
      "claim": "It is located in north-central France",
      "source_id": 1,
      "passage": "Paris is located in the north-central part of France"
    }
  ]
}"""

answer = parse_structured_citation_output(example_output)
print(f"Answer: {answer.answer_text}")
print(f"Citations: {len(answer.citations)}")

Validating Citations Against Retrieved Passages​

Once the model produces citations, verify that each cited passage actually appears in the retrieved documents. This catches hallucinated citations (citations of non-existent sources).

def validate_citations(cited_answer: CitedAnswer,
                       passages: List[str]) -> Dict:
    """
    Validate that claimed citations actually appear in retrieved passages.
    
    Args:
        cited_answer: Answer with citations.
        passages: Retrieved passages (indexed 0 to len-1).
    
    Returns:
        Dict with validation results.
    """
    
    validation_results = {
        "total_citations": len(cited_answer.citations),
        "valid_citations": 0,
        "invalid_citations": [],
        "validation_errors": []
    }
    
    for citation in cited_answer.citations:
        source_id = citation["source_id"]
        passage_excerpt = citation["passage"]
        
        # Check source_id is within range
        if source_id < 0 or source_id >= len(passages):
            validation_results["validation_errors"].append({
                "type": "invalid_source_id",
                "source_id": source_id,
                "max_valid_id": len(passages) - 1
            })
            continue
        
        # Check if passage excerpt appears in the claimed source
        actual_passage = passages[source_id]
        if passage_excerpt.lower() in actual_passage.lower():
            validation_results["valid_citations"] += 1
        else:
            validation_results["invalid_citations"].append({
                "claim": citation["claim"],
                "source_id": source_id,
                "claimed_passage": passage_excerpt,
                "actual_passage": actual_passage[:100] + "..."
            })
    
    # Compute metrics
    validation_results["citation_precision"] = (
        validation_results["valid_citations"] / 
        validation_results["total_citations"]
        if validation_results["total_citations"] > 0 else 1.0
    )
    
    return validation_results

# Example
passages = [
    "Paris is the capital of France.",
    "France is a Western European country."
]

cited_answer = CitedAnswer(
    answer_text="Paris is the capital of France.",
    citations=[
        {
            "claim": "Paris is the capital of France",
            "source_id": 0,
            "passage": "Paris is the capital of France."
        }
    ]
)

validation = validate_citations(cited_answer, passages)
print(f"Citation precision: {validation['citation_precision']:.2f}")
print(f"Invalid citations: {validation['invalid_citations']}")

Measuring Citation Coverage and Precision​

Citation precision measures whether cited passages actually support the claims. Citation recall (coverage) measures whether all major claims in the answer are cited.

def measure_citation_precision_recall(answer_text: str,
                                     citations: List[Dict],
                                     passages: List[str]) -> Dict:
    """
    Measure citation precision (accuracy) and recall (coverage).
    
    Args:
        answer_text: Full answer text.
        citations: List of citations with claims.
        passages: Retrieved passages.
    
    Returns:
        Dict with precision, recall, and detailed metrics.
    """
    
    metrics = {
        "total_claims_in_answer": 0,
        "total_citations": len(citations),
        "cited_claims": 0,
        "valid_citations": 0,
        "citation_precision": 0.0,
        "citation_recall": 0.0
    }
    
    # Count major sentences/claims in answer as a proxy for total claims
    sentences = [s.strip() for s in answer_text.split(".") if s.strip()]
    metrics["total_claims_in_answer"] = len(sentences)
    
    # Check each citation
    for citation in citations:
        source_id = citation["source_id"]
        
        # Verify citation points to valid source
        if 0 <= source_id < len(passages):
            passage_excerpt = citation["passage"]
            actual_passage = passages[source_id]
            
            if passage_excerpt.lower() in actual_passage.lower():
                metrics["valid_citations"] += 1
    
    # Citation precision: fraction of citations that are valid
    if metrics["total_citations"] > 0:
        metrics["citation_precision"] = (
            metrics["valid_citations"] / metrics["total_citations"]
        )
    
    # Citation recall: fraction of claims that are cited
    # (Simplified: assume each claim corresponds to one citation)
    metrics["citation_recall"] = (
        len(citations) / metrics["total_claims_in_answer"]
        if metrics["total_claims_in_answer"] > 0 else 0.0
    )
    
    # Clamp recall to 1.0 (more citations than claims is OK)
    metrics["citation_recall"] = min(1.0, metrics["citation_recall"])
    
    return metrics

# Example
answer = "Paris is the capital of France. It is located in north-central France."
citations = [
    {"claim": "Paris is the capital", "source_id": 0, "passage": "capital is Paris"},
    {"claim": "Located in north-central", "source_id": 1, "passage": "north-central part"}
]
passages = [
    "Paris is the capital of France.",
    "Paris is in the north-central part of France."
]

metrics = measure_citation_precision_recall(answer, citations, passages)
print(f"Citation precision: {metrics['citation_precision']:.2f}")
print(f"Citation recall: {metrics['citation_recall']:.2f}")

Handling Missing or Weak Citations​

Some claims may not have obvious passages to cite (integration of multiple sources, implicit reasoning). Implement fallback strategies: link to the document as a whole, cite the closest matching passage, or flag as potentially unsupported.

Key Takeaways​

  • Citation grounding requires the model to explicitly cite sources, making hallucinations traceable.
  • Validate citations by verifying that claimed passage excerpts actually appear in retrieved passages.
  • Citation precision measures whether citations are accurate; citation recall measures whether claims are cited.
  • Combine citation enforcement with citation validation to prevent hallucinated citations.
  • For production systems, require citation precision above 0.95 and citation recall above 0.8.

Frequently Asked Questions​

Should every claim require a citation?​

For critical information (medical, legal, financial), yes. For conversational or contextual sentences, citations may be optional. Best practice: require citations for factual claims and recommendations, allow uncited conversational transitions.

What do I do if the model produces citations but no passage matches them?​

This indicates the model is hallucinating citations. Flag the answer as invalid and regenerate. Alternatively, use reinforcement learning to penalize hallucinated citations during model training.

How do I handle citations that point to multiple relevant passages?​

Allow multiple citations per claim. A claim might be supported by multiple passages. In the output format, include an array of source IDs per claim.

Can I automatically correct citations?​

Partially. If the model cites the wrong passage for a claim, you can heuristically find the correct passage (e.g., the passage with highest semantic similarity to the claim). However, automatically correcting citations is risky—it may hide generation failures. Better to flag and regenerate.

Further Reading​