Applied AI · Knowledge retrieval · Monitoring
An AI answer needs evidence people can inspect.
An AI knowledge base and regulatory monitoring for agency consultants working on client projects, with evidence available for review.
At a glance
Used by agency consultants for knowledge retrieval and monitoring legislative and regulatory changes.
In use and in active development. No formal answer-quality evaluation completed.
The recording shows the monitoring module. The wider app also supports questions, online research and policy review, with a choice of OpenAI and Anthropic models.
Agency consultants use the application as a retrieval-based AI knowledge base and to monitor legislative and regulatory changes while working on client projects. Their research spans source material that changes over time.
The product needed to connect three questions: what changed, what does the evidence say and where does a person need to look more closely?
I gathered stakeholder needs, wrote user stories, designed the architecture and frontend, and built the application with AI assistance. I also separated implementation tests from answer-quality evaluation.
Knowledge-base answers use stored material, respect access rights and check quotations against sources. The system can decline without evidence.
Reviewers can inspect the basis of an answer; a matching quote alone does not prove the interpretation.
Stored-knowledge answers and wider online research have separate modes.
People can distinguish the source boundary before assessing the response.
I created a separate framework for evaluating answer quality, alongside implementation tests. No formal answer-quality evaluation has been completed.
The framework provides a way to assess retrieval and interpretation; current use does not establish answer reliability.
The agency uses the knowledge base and monitoring application in client-project work. No formal answer-quality evaluation has been completed, and no accuracy or time-saving result is claimed.
Development continues on scanned-PDF support, retrieval ranking and source coverage. The recording shows the monitoring module.