DataTalk Evaluation
DataTalk is a research assistant that answers campaign-finance questions from FEC data — try it at datatalk.biglocalnews.org. This site is where we measure how well it does, so we can make it better.
Walk-Up evaluation
Rate one DataTalk answer. Five minutes, no account needed — help us find the rough edges.
Give feedback →Do a full evaluation run
Work through our full benchmark suite to help us measure DataTalk. Sessions take about an hour and require an account from the project team.
Sign in to rate →Administer the eval system
Review runs, manage rater assignments, inspect benchmark questions and dashboards.
Open dashboard →How evaluations work
Each evaluation runs a benchmark of campaign-finance questions through DataTalk and produces an answer with a reporting recipe — dataset, snapshot, source URL, SQL, and result — that any reader can re-execute to verify the claim. Trained evaluators then independently rate each answer on five quality dimensions:
| Factual Accuracy | Are the numbers, names, and relationships correct? |
|---|---|
| Completeness | Did the recipe answer the full question, not a narrowed substitute? |
| Caveats & Context | Does the answer surface limitations a knowledgeable analyst would flag? |
| Source Attribution | Is the citation specific enough that a journalist could verify it? |
| Helpfulness | Is the answer well-organized and directly useful? |
Each dimension scores 1 (material problem), 2 (acceptable), or 3 (good).
For more information about how to use this site, see our evaluation guide.