1. Abstract

The Hallucination Detection and Citation Verification for RAG system is an AI-powered application designed to improve the reliability of Retrieval-Augmented Generation (RAG) systems by identifying unsupported claims and verifying citations in generated answers. Although RAG applications retrieve information from external documents, their language models may still generate incorrect statements, misinterpret retrieved evidence, or provide citations that do not support the answer. The proposed system analyzes generated responses by breaking them into individual claims, comparing each claim with retrieved source passages, and checking whether the associated citations provide sufficient evidence. It assigns support labels such as supported, contradicted, or insufficient evidence and generates a verification report with source references. The system can also calculate claim-level confidence indicators and flag questionable responses for human review. By combining natural language inference, semantic similarity, and LLM-based verification, the application helps developers evaluate and improve the factual grounding of RAG-generated answers.

2. Objectives

1. To detect unsupported claims in RAG-generated answers.
2. To verify whether citations support the statements they reference.
3. To compare generated claims against retrieved source documents.
4. To classify claims as supported, contradicted, or insufficiently supported.
5. To identify missing, irrelevant, or misleading citations.
6. To generate claim-level verification reports.
7. To provide source references for manual verification.
8. To calculate evaluation metrics for factual grounding.
9. To reduce the risk of presenting unsupported AI-generated information.
10. To benchmark verification performance across different RAG configurations.

3. Existing System

Traditional RAG systems retrieve relevant document passages and use them to generate answers. However, retrieval alone does not guarantee that every generated statement is supported by the retrieved evidence. Basic systems may display document links without verifying whether the cited passages actually substantiate individual claims. Manual verification can be accurate but becomes time-consuming when answers contain multiple statements and references.

Disadvantages

1. Retrieved documents may not fully support generated claims.
2. Incorrect statements can appear alongside valid citations.
3. Citations may be irrelevant or linked to the wrong passages.
4. Contradictory evidence may not be detected.
5. Missing evidence can be mistaken for factual support.
6. Manual claim-by-claim verification is time-consuming.
7. Overall answer quality scores may hide individual unsupported statements.

4. Proposed System

The proposed system adds a verification layer to a RAG pipeline. After an LLM generates an answer, the system decomposes the response into atomic factual claims and associates each claim with its cited sources. A retrieval and verification module then compares each claim against the relevant document passages.

A natural language inference (NLI) model or a verification-focused LLM evaluates whether the evidence supports, contradicts, or fails to establish each claim. A citation checker also validates source identifiers, document references, and passage relevance. The final report highlights unsupported statements, citation mismatches, and conflicting evidence. Claims that cannot be verified are flagged for human review instead of being automatically treated as false.

Key Features

1. Claim-level hallucination detection.
2. Citation and source-reference validation.
3. Evidence-support classification.
4. Contradiction and evidence-gap detection.
5. Claim-to-passage matching.
6. Source relevance and citation accuracy checks.
7. Verification confidence indicators.
8. Highlighting of questionable statements.
9. Detailed verification reports.
10. Integration with existing RAG applications.
11. Human review for uncertain results.
12. Evaluation dashboard for system performance.

5. Implementation Procedure

1. Build or connect a RAG pipeline: Use a document collection, embedding model, vector database, and LLM to generate answers with source references.

2. Collect test data: Prepare questions, retrieved passages, generated answers, citations, and manually verified claim-support labels.

3. Extract claims: Split each generated answer into individual factual statements while preserving the original sentence and citation references.

4. Map claims to citations: Associate each claim with the cited document passages and verify that the source identifiers exist.

5. Retrieve supporting evidence: Search the document collection for additional passages relevant to each claim, especially when the cited source appears incomplete.

6. Implement semantic matching: Use embedding similarity or reranking to identify passages that may support or contradict a claim.

7. Run entailment verification: Apply an NLI model or verification-focused LLM to classify the relationship between each claim and its evidence.

8. Check citation correctness: Verify that cited passages substantiate the associated claims, rather than merely discussing the same topic.

9. Detect evidence conflicts: Identify cases where different passages disagree or where the evidence is insufficient to establish a conclusion.

10. Generate verification labels: Assign labels such as supported, contradicted, insufficient evidence, and citation mismatch. Keep these categories distinct because a lack of evidence does not necessarily prove a claim false.

11. Create the dashboard: Display the answer, extracted claims, verification labels, cited passages, and flagged issues.

12. Evaluate performance: Measure claim-level precision, recall, F1-score, citation correctness, and the rate of unsupported claims incorrectly marked as supported.

13. Deploy the system: Integrate the verification module with the RAG application and provide human-review controls for ambiguous cases.

6. Software Requirements

7. Hardware Requirements

Note: A prototype using hosted models can run on a standard development computer. Running multiple verification models locally may require additional memory and GPU resources.

8. Advantages of the Project

1. Improved factual grounding: Identifies statements that lack sufficient supporting evidence.
2. Citation verification: Checks whether references support the claims they accompany.
3. Claim-level analysis: Evaluates individual statements rather than relying only on overall answer quality.
4. Contradiction detection: Helps identify claims that conflict with retrieved evidence.
5. Transparent results: Shows the evidence behind verification decisions.
6. Better RAG evaluation: Provides measurable indicators of grounding and citation quality.
7. Human-review support: Directs attention to uncertain or problematic claims.
8. Flexible integration: Can be added to existing document question-answering applications.
9. Auditable verification: Maintains a record of claims, evidence, and verification outcomes.
10. Configurable policies: Allows applications to define when unsupported claims require warnings or review.