Online information is full of arguments. Some are persuasive because they are well-supported. Others are persuasive because they use emotional language, misdirection, personal attacks, or flawed reasoning. Fallacy Hunter explores how AI can help users recognize weak arguments and become more critical readers.
Developed by CMKL student Pon Yimcharoen under the guidance of Lorenzo Avi, Fallacy Hunter is both a research project and a working browser extension. The project investigates automatic logical fallacy detection and turns the research into a practical tool that can scan web pages, highlight potentially fallacious text, and provide explanations on demand.
The research component compares multiple approaches. The project replicates and extends multi-perspective prompting methods for large language models, where the model analyzes an argument through different reasoning perspectives before classifying the fallacy type. A key improvement is multi-query score aggregation, which combines evidence across multiple reasoning paths to reduce label bias.
The project also explores fine-tuned encoder models, including RoBERTa and DeBERTa. These models are trained on labeled fallacy datasets and evaluated across multiple benchmarks. DeBERTa achieved the strongest overall performance on the custom 14-class dataset, while RoBERTa with explanation-query augmentation performed strongly on several benchmark datasets.
On the engineering side, Fallacy Hunter includes a Chrome browser extension connected to a local FastAPI backend. The system uses a two-phase pipeline. First, a fast classifier scans webpage text and highlights possible fallacies. Second, when a user clicks a highlighted phrase, a locally served language model generates a short explanation of why the text may be fallacious.
This design keeps the initial scan responsive while still giving users access to deeper explanation when they want it. The project also identifies key challenges, including confusion between similar fallacy types such as appeal to emotion, fallacy of relevance, and red herring. Some fallacies require context, intent, or subtle causal reasoning, which current models still struggle to capture.
Fallacy Hunter connects AI with media literacy and critical thinking. It is a reminder that AI can be used not only to generate content, but also to help people evaluate content more carefully.
