Intent routing
Classified each message before activating verification, URL, ticket, conversation, or rejection logic.
RAG & Chatbots
Klinik Hoaks Chatbot
A chatbot designed to find published clarifications and track verification requests through one conversation.
- Client
- Dinas Kominfo Provinsi Jawa Timur
- Role
- AI Engineer
- Year
- 2025–2026
- Status
- On hold, not deployed
Klinik Hoaks Chatbot returning one published clarification for an article URL
Screenshot 1 of 4
Overview
Search and ticket tracking in one conversation
Klinik Hoaks Chatbot was developed as a public interface for the Klinik Hoaks service in East Java. Visitors can describe a claim, paste an article URL, or enter a verification ticket number. The chatbot returns matching published clarifications, links to the submission form when no article is available, and reports the status of an existing request.
Every message is classified into one of five routes: verification, URL processing, ticket tracking, conversation, or rejection. Verification queries use contextual keyword extraction before searching the Klinik Hoaks database with MySQL Full Text Search and token matching. URL queries first extract and validate article content. Ticket requests use a separate database lookup, while general conversation retains message history.
I developed the LangFlow chatbot and connected its language model, routing logic, article extraction, database search, ticket lookup, and response states. I also evaluated the retrieval pipeline with 48 test queries. The evaluation measures whether the system found an appropriate article or correctly returned no result; it does not measure the factual accuracy of every generated response.
Technology stack
AI and orchestration
- LangFlow
- Amazon Nova
- Intent Router
Retrieval
- MySQL Full Text Search
- Token Matching
- Relevance Filtering
Service flows
- Article Scraping
- Ticket Tracking
- Conversation Memory
ProblemOne entry point for search and ticket tracking
People approach Klinik Hoaks with different kinds of input. Some have a short claim, some have an article URL, and others need to check a clarification request they submitted earlier. Each path requires different processing and a different response.
The chatbot also needed a clear boundary. It can retrieve published Klinik Hoaks articles, direct users to submit a new request, and track tickets. It does not independently decide whether an arbitrary claim is true by searching the open web.
Several user intents
Claims, URLs, ticket numbers, general questions, and unrelated prompts cannot share one response path.
Grounded results
Verification responses must come from the Klinik Hoaks database or return an honest no result state.
ContributionBuilt the routing, retrieval, and ticket flows
I developed the chatbot in LangFlow and separated the service into five routes. My work connected Amazon Nova to intent classification and keyword extraction, then combined those model outputs with deterministic database search and ticket lookup.
I implemented article URL extraction, scraping validation, conversation history, structured news responses, and fallbacks for blocked pages, missing articles, invalid tickets, and unrelated questions. I also evaluated retrieval behavior across positive and negative test cases.
Database retrieval
Combined contextual keyword extraction, MySQL Full Text Search, token matching, and relevance filtering.
Service recovery
Handled scraping failures, empty searches, unfinished requests, missing tickets, and questions outside the service scope.
ArchitectureFive routes keep each request bounded
An Amazon Nova classifier reads the user message and conversation history, then a multi conditional router activates one route. Text verification and URL processing both end in database retrieval, but URL processing first extracts article content and rejects blocked, empty, or invalid pages.
The retrieval path turns the request into focused keywords, searches MySQL with Full Text Search and token matching, filters weak matches, and formats up to three articles. Ticket tracking bypasses article retrieval and reads request status directly from the Klinik Hoaks database.
Conversational messages retain context without running a news search for every turn. Questions outside the Klinik Hoaks scope receive a fixed response that points users back to article verification and ticket tracking.
Route the request
User message
Claim, URL, ticket number, or general question
Intent classification
Amazon Nova reads the message and recent context
Five route router
Verification, URL, tracking, conversation, or rejection
Find a clarification
Prepare search terms
Extract keywords from text or validated article content
Search MySQL
Combine Full Text Search with token matching
Return structured results
Filter relevance and present up to three articles
Handle service states
Track a ticket
Read request and publication status from the database
Continue conversation
Use message history for greetings and follow up questions
Use a safe fallback
Handle missing content, invalid tickets, and unrelated requests
The model interprets; the database decides
Amazon Nova extracts intent and search terms, while MySQL retrieval determines which published articles are returned.
URL failures stop early
Content length and page error checks prevent blocked or incomplete pages from entering the retrieval path.
Results46 of 48 retrieval decisions were correct
95.83%
correct retrieval decisions
46 of 48 evaluation queries
97.37%
positive retrieval hit rate
37 of 38 target articles found
90%
correct no result handling
9 of 10 negative queries
2
retrieval errors
one missed result and one false positive
The evaluation contained 48 queries: 38 with a target article in the database and 10 that should return no result. The system made the correct retrieval decision in 46 cases, producing 95.83% retrieval accuracy on this dataset.
It found 37 of the 38 target articles and correctly returned no result for 9 of the 10 negative queries. The two errors were one missed article and one false positive where a query about the Regent of Brebes matched an article about the Regent of Bojonegoro.
These numbers evaluate the retrieval layer only. They do not establish full response accuracy, verdict accuracy, hallucination rate, conversation quality, latency, user satisfaction, or task completion.
One missed retrieval
A query about an oil spill at Sentosa Beach returned no article even though the target existed in the database.
One false positive
A shared title and similar place name led the search from Brebes to an unrelated Bojonegoro article.
Clear evaluation boundary
The headline metric describes article retrieval decisions, not every word produced by the chatbot.
LearningsWhat the retrieval evaluation exposed
Retrieval and response quality need separate tests
A correct article lookup does not prove that the final wording is accurate. Generation quality needs its own evaluation set and scoring rules.
Returning nothing is a valid outcome
The ten negative cases showed why no result handling needs a dedicated metric. A plausible but unrelated article is worse than an honest empty response.
Similar entities need stricter matching
The Brebes and Bojonegoro error points to entity checks as a practical next step before accepting a retrieved article.
Next project
Klinik Hoaks AI Fact Checker CMS Integration
An internal CMS assistant that gathers web evidence and returns a structured fact check for admin review.