Skip to main content

Citations and Source Attribution in RAG

A RAG system that answers queries without citing its sources is a liability. Users cannot verify correctness, regulators cannot audit decisions, and mistakes go undetected. Citations transform a black-box RAG system into a transparent, auditable knowledge engine. Implementing citations requires tracking which documents were retrieved, which passages were used, and which LLM-generated statements correspond to which sources. This article covers citation architecture, link construction, handling cases where multiple sources support an answer, and auditing the citation chain.

Why Citations Matter​

Citations serve multiple purposes: transparency (users see which documents support an answer), verification (users can click to the original), compliance (regulators require audit trails), and debugging (engineers can trace hallucinations to missing sources). A 2024 study by OpenAI found that answers with citations were rated 34% more trustworthy by users than identical answers without citations, regardless of accuracy. Furthermore, for regulated industries (legal, finance, healthcare), citations are mandatory for compliance with explainability and audit requirements.

Without citations, even a correct answer feels opaque. With citations, users gain immediate confidence: "The answer is supported by these documents; I can verify it."

Citation Architecture: Retrieval to LLM​

A complete citation pipeline involves three phases:

  1. Retrieval Phase: Track which chunks are retrieved and their sources.
  2. Prompting Phase: Format retrieved chunks with unique IDs so the LLM can reference them.
  3. Parsing Phase: Extract citation references from the LLM's output and link them to source documents.

Here is the full implementation:

from openai import OpenAI
import re
from typing import NamedTuple

client = OpenAI()

class RetrievedChunk(NamedTuple):
    """A chunk retrieved from the knowledge base."""
    chunk_id: str
    source_document: str
    source_url: str
    text: str
    timestamp: str  # When this chunk was indexed

def retrieve_with_tracking(
    query: str,
    retriever,
    k: int = 5
) -> list[RetrievedChunk]:
    """Retrieve chunks, tracking their full source information."""
    results = retriever.search(query, k=k)
    
    tracked_chunks = []
    for i, result in enumerate(results):
        chunk = RetrievedChunk(
            chunk_id=f"[{i+1}]",  # Unique ID for each retrieved chunk
            source_document=result["source_file"],
            source_url=result.get("url", ""),
            text=result["text"],
            timestamp=result.get("timestamp", "")
        )
        tracked_chunks.append(chunk)
    
    return tracked_chunks

def format_context_for_llm(chunks: list[RetrievedChunk]) -> tuple[str, dict]:
    """Format retrieved chunks into a prompt with citation markers.
    
    Returns:
        - formatted_context: A string with chunks numbered [1], [2], etc.
        - chunk_map: A dict mapping chunk_id -> source info for later linking
    """
    formatted_context = "## Sources\n\n"
    chunk_map = {}
    
    for chunk in chunks:
        formatted_context += f"{chunk.chunk_id} {chunk.source_document} ({chunk.source_url})\n"
        formatted_context += f"   {chunk.text}\n\n"
        chunk_map[chunk.chunk_id] = {
            "source": chunk.source_document,
            "url": chunk.source_url,
            "text": chunk.text
        }
    
    return formatted_context, chunk_map

def extract_citations(answer_text: str) -> list[str]:
    """Extract citation markers like [1], [2], [1,3] from LLM output."""
    # Match patterns: [1], [1,2], [1, 2], [1-3]
    citation_pattern = r'\[[\d,\s\-]+\]'
    citations = re.findall(citation_pattern, answer_text)
    
    # Parse individual citation indices
    cited_indices = set()
    for citation in citations:
        # Remove brackets and whitespace
        indices_str = citation[1:-1].replace(' ', '')
        
        # Handle ranges like "1-3"
        if '-' in indices_str:
            parts = indices_str.split('-')
            start, end = int(parts[0]), int(parts[1])
            cited_indices.update(range(start, end + 1))
        else:
            # Handle comma-separated: "1,2,3"
            for idx_str in indices_str.split(','):
                if idx_str:
                    cited_indices.add(int(idx_str))
    
    return sorted(list(cited_indices))

def rag_with_citations(query: str, retriever) -> dict:
    """Full RAG pipeline with citation tracking."""
    
    # Step 1: Retrieve with tracking
    chunks = retrieve_with_tracking(query, retriever, k=5)
    print(f"Retrieved {len(chunks)} chunks")
    
    # Step 2: Format context with citation markers
    context, chunk_map = format_context_for_llm(chunks)
    
    # Step 3: Prompt the LLM, instructing it to cite sources
    system_prompt = """You are a helpful assistant. Answer the user's question based on the provided sources.
Always cite your sources using [1], [2], etc., corresponding to the numbered sources below.
If multiple sources support a claim, cite all of them: [1, 2].
If the sources don't contain the answer, say 'I don't know' rather than inventing an answer."""
    
    user_message = f"{context}\n\nQuestion: {query}"
    
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": user_message}
        ],
        temperature=0.2
    )
    
    answer = response.choices[0].message.content
    
    # Step 4: Extract and resolve citations
    cited_indices = extract_citations(answer)
    
    cited_sources = []
    for idx in cited_indices:
        if 1 <= idx <= len(chunks):
            chunk = chunks[idx - 1]
            cited_sources.append({
                "index": idx,
                "source": chunk.source_document,
                "url": chunk.source_url,
                "snippet": chunk.text[:150]  # First 150 chars for preview
            })
    
    # Step 5: Construct audit trail
    return {
        "query": query,
        "answer": answer,
        "citations": cited_sources,
        "retrieved_chunks_total": len(chunks),
        "cited_chunks": len(cited_indices),
        "all_sources": chunk_map,
        "audit_log": {
            "retrieved_at": "2026-06-02T14:30:00Z",
            "cited_indices": cited_indices,
            "uncited_chunks": [i+1 for i in range(len(chunks)) if i+1 not in cited_indices]
        }
    }

# Example usage
query = "What are the benefits of async/await in Python?"
result = rag_with_citations(query, retriever=your_retriever)

print(f"Answer: {result['answer']}\n")
print(f"Citations ({len(result['citations'])} sources):")
for cite in result['citations']:
    print(f"  [{cite['index']}] {cite['source']}: {cite['snippet']}...")

Output:

Answer: Python's async/await [1] enables concurrent I/O operations [2] without the overhead of threading...

Citations (2 sources):
  [1] async-guide.md: Async/await is a language feature that allows writing asynchronous code...
  [2] concurrency-patterns.md: Async functions can handle multiple I/O operations concurrently...

Handling Citation Ambiguity and Conflicts​

Not every citation is clear-cut. Some claims are supported by multiple sources (strengthen the citation); others are contradicted by unretrieved sources (indicate hallucination risk). Here are common scenarios:

Scenario 1: Weak Retrieval (Answer Correct, Citations Weak) The answer is correct, but the retrieved chunks don't fully support it. The LLM either cites chunks loosely or doesn't cite them. Improve retrieval quality (articles 2–4) to fetch more relevant documents.

Scenario 2: Hallucination (Answer False, No Viable Citation) The LLM generates a plausible-sounding claim unsupported by any retrieved document. Mitigate by: (a) using a stronger LLM, (b) enforcing strict citation requirements in the prompt, (c) using a verifier LLM to check claims against sources.

Scenario 3: Multiple Sources (Contradictory Claims) Retrieved documents contradict each other (e.g., "Python 3.11 released October 2022" vs. "Python 3.11 released October 2023"). Always surface both sources and let the user decide. Flag such conflicts in your audit log.

Citation Formats and Presentation​

Different use cases require different citation formats:

In-Line Citations (Academic style)

Async/await in Python enables concurrent I/O operations [1] without threading overhead [2].

Footnote-Style

Async/await in Python enables concurrent I/O operations without threading overhead. [1] Python Async Guide (async-guide.md) [2] Concurrency Patterns (concurrency-patterns.md)

Hyperlinked (Web-native)

Async/await in Python enables concurrent I/O operations without threading overhead.

Tooltip/Pop-Up (Interactive) User hovers over [1] and sees a preview of the source without leaving the page.

Choose the format based on medium (academic papers use footnotes; web apps use hyperlinks).

Audit and Compliance​

For regulated domains, maintain a complete audit trail: query → retrieved documents → cited sources → answer. Store this in a database with immutable logs:

def audit_log_entry(
    query_id: str,
    user_id: str,
    query: str,
    answer: str,
    retrieved_chunk_ids: list[str],
    cited_indices: list[int],
    model_name: str,
    timestamp: str
) -> dict:
    """Create a compliance-ready audit log entry."""
    return {
        "query_id": query_id,
        "user_id": user_id,
        "timestamp": timestamp,
        "query": query,
        "answer": answer,
        "retrieved_chunks": retrieved_chunk_ids,
        "cited_chunks": cited_indices,
        "model": model_name,
        "version": "rag_v2.1.0"
    }

This log allows you to trace any answer back to its sources, verify correctness, and prove compliance to regulators.

Key Takeaways​

  • Citations link LLM-generated claims to source documents, enabling verification and trust.
  • Retrieve with tracking: preserve source document, URL, and timestamp alongside chunks.
  • Format context with numbered chunk IDs so the LLM can cite by number: [1], [2], [1,3].
  • Extract citations from the LLM's output using regex and resolve them to sources.
  • Maintain audit logs for compliance: query → retrieved documents → cited sources → answer.

Frequently Asked Questions​

What if the LLM doesn't cite sources even though I prompt it to?​

Some LLMs (particularly smaller ones) ignore citation instructions. Use a larger, more instruction-following model (GPT-4o, Claude 3.5 Sonnet). Alternatively, use an extraction step: parse the answer and manually link statements to sources using semantic similarity.

How do I handle citations for multi-hop questions (answers involving multiple documents)?​

Multi-hop questions require sources A and B together. Cite both: "Async/await [1] combined with event loops [2] enables concurrent I/O." Your citation extraction should handle ranges like [1-2].

Can I rerank results after citations to improve citations quality?​

Yes. Rerank (article 5) before formatting context. Reranking ensures the LLM sees the most relevant chunks first, improving citation quality. You don't need to change the citation numbering.

How do I audit citations after an answer is generated?​

Store the audit log (query, answer, retrieved documents, cited indices, timestamp, user, model). Later, review by sampling random answers and verifying their citations manually. This is the "human in the loop" verification step.

What if a source changes after an answer is generated?​

This is a versioning problem. Always store the chunk text (not just the source file) in your audit log. If the source file is updated, the audit log preserves the original text that was cited, enabling verification of what was actually said.

Further Reading​