PolicyCopilot chat in progress: after the user confirms a proposed problem statement about youth underemployment in Pakistan, the AI asks “What do you think are the root causes of the issue?”, showing two user-entered answers, an “Add another” row, and Skip / Submit answers buttons.

Turning an AI black box into a thought partner: drop-off from 70% to 20%

AI product design
Role: lead product designer
Team: ML engineer, policy consultant, product manager

PolicyCopilot is a tool used by policy-makers to diagnose policy problems. The tool included an AI chat that helped users retrieve evidence from a curated knowledge base.

Summary

Context: 70% of usability testing participants dropped off our AI chatbot because they didn't trust the sources it selected, even though its citations matched the academic papers they trust.
Approach: Showed that a disclaimer about source vetting didn't help, then redesigned the chatbot into a guided flow where users define the problem and root causes that drive which evidence is selected.
Outcome: Drop-off fell from 70% to 20%. Shipped on time for the funder demo that secured $1M.

Our AI chatbot offered the same citation information as academic papers that users trust

Users told us in-text citations and a full reference list are what make research credible...

Two excerpts from an academic paper on Pakistan’s education and workforce: a paragraph ending in the in-text citation “(World Bank data, 2019)”, tagged “In-text citation”, and a numbered reference list entry tagged “Reference list”.

... our chatbot already gave them both in the same format so the lack of trust sat somewhere else

The original PolicyCopilot chat: an AI response about Pakistan’s youth unemployment with superscript in-text citations, a “Citations (3)” button, a youth unemployment rate chart, and a disclaimer listing the curated sources under the input.

but 7/10 of usability testing participants mentioned “not trusting the selected sources” in our AI chatbot, leading to high drop offs

Users asked us to explain how sources were vetted so I added a disclaimer.

Drop-off did not move

The chat input “Describe the problem you’re working on…” with a new disclaimer beneath it: “100+ curated sources – top institutions (WHO, ILO, UN agencies) and peer-reviewed journals (The Lancet, BMJ). Vetted for credibility and recency, updated quarterly.”, tagged “Knowledge base disclaimers”.

After a week live, drop-off went from 70% to 65% across 42 sessions (no meaningful change)

I needed to make structural changes to the product. To start, I explored what levers cultivated trust

I mapped how people decide to trust in general, then how policymakers do specifically.

Nested-ellipse diagram of trust levers. The inner ellipse, “How do people decide to trust something?”, holds Authorship (example: Wealthsimple’s risk-level picker; users trust the output because they trust the inputs) and Human in the loop (example: Claude Code’s approval prompt; the user keeps final authority). The outer ellipse, “How do policy-makers decide to trust something?”, adds Backing from a reputable journal (UNESCO, WHO, UNHCR logos; journals do rigorous vetting) and Transparency into methodology (users can verify the steps taken). — Scroll sideways to see the full diagram →

I realized our existing solutions had only considered gaining trust from only policy-makers but had not considered how to gain a person’s trust in general

The solution needed to include users in determining the inputs that went into the results and give them veto power

SOLUTION

The redesign turns one question into a two-stage flow where users steer which evidence is selected

Stage 1: Defining the problem

The AI asks scoping questions and drafts a problem statement. The user confirms or corrects it, and only then does the evidence check run against it

Stage 1: a “Problem statement” card summarising child labour in Bangladesh with an “Edit problem statement” button, followed by an “Evidence check” note citing ILO & UNICEF (2021) and Thévenon & Edmonds (2019), and a “Generate mindmap” button.

Stage 2: Diagnosing root causes

The user inputs their ideas on root causes or brings their own research and the system compares each against the evidence

Stage 2: after the user’s reply, a question card asks “What do you think is driving this at that scale?” (Question 1/2) with suggested causes (household income smoothing; school quality and its perceived returns; children taking on caregiving roles at home), an “Add another” row, and Skip / Next question buttons.

Edge case:
Some users didn't have the context to answer key questions

I decided:
to build an escape hatch because forcing an answer risked losing users and letting them guess risked a misleading output

Side-by-side PolicyCopilot conversations on youth unemployment in Pakistan. Left, a user with less context skips the scoping questions; the AI writes a broader problem statement and a shorter, generalized response with three references. Right, a user who answers gets a detailed problem statement and a longer, driver-by-driver response with twelve references. Annotations: when users take the escape hatch, the AI scopes the problem more broadly and returns a more generalized response, drawing on the same vetted evidence: less specificity, never less rigour.

POV: user with less context (escape hatch user)

PolicyCopilot conversation where the user skips the scoping questions; the AI scopes the problem broadly and returns a generalized response with three references.

When users take the escape hatch...
...the AI scopes the problem more broadly...
...and returns a more generalized response

It draws on the same vetted evidence, so less input means less specificity, never less rigour. The AI doesn't fill gaps with guesses about the user's situation

POV: user with more context (for comparison)

PolicyCopilot conversation where the user answers the scoping questions; the AI drafts a specific problem statement and returns a detailed, driver-by-driver evidence response with twelve references.

The new user experience helped secure a $1M funding round

Other stats

70% → 20%change in drop-off rate
+22%longer sessions
+66%sessions resulted in users using the AI’s evidence
(a proxy for trust)

Users’ preferences are a weak signal that need to be tested

Next time, I would dig deeper by not only hearing what users have to say but also observing what users are doing. Based on that, I would validate with quick prototypes first before implementing something.

See next project

Cutting compute costs by 30% by redesigning AI responses around common tasks

AI product design

Preview of the compute-costs case study: a funding-landscape dashboard with a bubble chart of projects and a Gantt timeline of Kenyan youth-employment policies.