flan-t5-small-rag-hallucinations-judgment

1. Project Overview

This repository features a fine-tuned Flan-T5-Small model designed to identify Hallucinations in Retrieval-Augmented Generation (RAG) pipelines. The model evaluates the consistency between a provided context and the generated answer, classifying the output as either Faithful (Consistent) or Hallucinated (Inconsistent).

prompt format

input = f"context:{content}\nquestion:{question}\nanswer:{answer}"

---

2. Model Performance

The model was evaluated on a specialized RAG validation set. Based on the current test results, the model achieves perfect separation between classes.

Confusion Matrix

	Predicted: Faithful	Predicted: Hallucinated
Actual: Faithful	69 (True Negative)	0 (False Positive)
Actual: Hallucinated	0 (False Negative)	31 (True Positive)

Key Metrics

• Accuracy: 100%
• Precision: 100%
• Recall (Sensitivity): 100%
• F1-Score: 100%

Analysis

The current evaluation shows an ideal performance profile. With zero False Negatives, the model successfully caught every instance of hallucination in the test set. However, users should note that such perfect scores often indicate a highly specific or constrained test dataset; real-world performance may vary as linguistic complexity increases.

---

3. Disclaimers & Bias Statement

Disclaimer

Important: While this model achieved 100% accuracy on the provided test set, it should not be considered infallible. Hallucination detection is a subjective and context-dependent task. This model serves as a heuristic "sanity check" and should be used alongside other RAG evaluation frameworks (such as RAGAS or Faithfulness metrics) for mission-critical applications.

Dataset & Potential Bias

• Overfitting Risk: The perfect confusion matrix scores suggest the model may be highly tuned to the specific formatting or "signal" of the current training data. It may not generalize as effectively to out-of-distribution (OOD) topics or different LLM generation styles.
• Complexity Bias: The model may struggle with "Subtle Hallucinations"—cases where the answer is factually true in the real world but not supported by the specific provided context.
• Length Bias: There is a potential bias where very short or very long responses are more easily classified. Mid-length responses with nuanced logical inferences may yield lower reliability.

Technical Limitations

As a Flan-T5-Small based model, the context window is limited. Extremely long retrieval documents may be truncated, potentially leading to "False Hallucination" flags if the supporting evidence is cut off during inference.