Going from Research Idea to Presentation with AI Assistants
Illustration: ideas moved through a repeating cycle of brainstorming, evidence checking, critique, and reflection, with each version saved.While brainstorming my first research proposal for my PhD, I started with something cool I wanted to build: a way to use evidence from one setting to predict whether a development intervention would have a similar impact in another context. I was sketching a complicated machine learning model using satellite imagery and Bayesian networks, perhaps even a simulation to compare different interventions. The more complex the idea became, however, the more I wondered: who would actually use such an elaborate model, and what decision would it help them make? I had made the classic academic mistake of starting with my interest in a method, then working backward toward a problem it could solve.
Rather than start over, I wondered what a development professional would say about it. I didn’t want to send my haphazard notes to former colleagues just yet, so I asked an AI agent instead. I asked a Gemini model for three perspectives on my idea: a humanitarian program manager in the field, an evaluation specialist at World Bank headquarters, and an academic development economist. I asked it to cite sources for its critique and to suggest how I could make the project more useful to practitioners. After a few more rounds of literature review and brainstorming, the result was surprisingly helpful.
The AI models I used brought useful challenges and ideas, but also their own biases and mistakes. What helped most was changing my starting question. Rather than asking whether an impact evaluation result would hold up in a new place, I began by asking when a practitioner would realistically seek out that kind of evidence, how they would find it and fold it into their planning, and what a useful set of estimates would look like to them. As I refined the proposal, I kept checking it against research papers and reports from think tanks and development organizations. It is still an early-stage plan, but it points in a much more productive direction than where I started.
Adopting the end user’s point of view
The pivotal moment came late at night, when my research ideas were keeping me awake. I reached for my phone and typed a vague sketch of my approach into the Gemini app, mostly to use it as a sounding board and scan for related papers. I asked a few simple questions about similar research, and everything Gemini sent back reinforced my approach. It was encouraging: this looked like a real gap in the literature, with real demand for exactly what I wanted to build. Then Gemini described, nearly verbatim, the full approach I had been considering, in more detail than I had given it in that chat. That seemed too good to be true, so I asked for its sources. It told me directly that it had drawn on my own proposal from previous chats, in entirely separate sessions. I knew my chat sessions informed each other because of my personalization settings, but I hadn’t expected it to hand my own idea back to me as independent evidence. My earlier work had been steering the AI toward the answer I wanted to hear.
That’s when I decided the echo chamber wasn’t helping. I needed to be more critical of my own biases rather than trust Gemini’s feedback on whether my proposal was original or useful. So in a new session I asked it to take on the three stakeholder roles and critique the proposal adversarially. Told to challenge the idea, it finally did. The most consequential feedback was simple: why would a decision maker look at this complex model when they were already overloaded with fast-paced work and piles of potentially relevant evidence?
Outside of academia, novelty is rarely the point; people designing a new project typically reach for what they already know and trust. But I knew from experience that there are specific stages in designing a program when practitioners actively seek out evidence. So my questions shifted to: when do humanitarian program planners look for evidence, how do they find it, and how do they decide it is relevant to their situation? Those became my starting point, framed from the imagined end user’s perspective rather than the technical question of how to make out-of-sample predictions from massive datasets.
How do you reconcile conflicting evidence?
With better questions (and a few hours of sleep), I started checking what research already existed, especially in humanitarian disasters and protracted conflict. I clearly wasn’t the first to ask these questions, but I had no idea how many studies had looked at how experts borrow evidence from one context and decide whether it applies to another. The literature calls this external validity (mostly economists), transfer learning (more of a machine learning framing), or “best practices.” I quickly found dozens of reports on similar topics, including a few useful case studies of programs that worked after climate shocks or natural disasters being adapted to conflict zones. I found nothing, though, that measured this “context distance” empirically. Everyone seemed to agree that context matters and evidence can be useful, but there was no way to know whether a result from one place would work elsewhere, or by how much it would differ if it did.
Reviewing the research also put the planner’s problem in front of me. How do you know when to stop looking for evidence and work with what you’ve found? How do you reconcile findings that conflict? I expected studies to disagree about whether interventions work in different places and about how decision makers use evidence, but I found no clear way to settle the disagreement. That helped clarify the questions I wanted to ask and convinced me they were worth asking: if I were designing a new program, I would have no ready way to reconcile all those reports. It seems to me there is room for a standardized approach that combines the sources of evidence into decisions or recommendations.
The more papers I read (or had AI summarize), the more convinced I became that my initial quantitative models over-engineered the solution. There was too much uncertainty in the context variables for a simulation or a one-size-fits-all machine learning approach. The conflict settings I most wanted to help often have limited data, since collecting information in a war zone is very hard (which is probably part of why so few academic researchers have invested in these topics). That uncertainty would compound the uncertainty of transferring a program to a new context, so any polished quantitative estimate would be either too imprecise to guide decisions or deceptively authoritative, with fancy math hiding the fact that it was really a guess.
Where the AI review itself needed checking
Along the way, I learned a few lessons about brainstorming with AI assistants: save new versions rather than editing reports, double-check every citation and claim, and run independent reviews with different models to avoid the echo chamber. Together these helped me limit scope creep, keep fabricated claims from passing as evidence, and course-correct early.
Keep a paper trail. In rounds of brainstorming with Claude, I found that the proposal drifted toward whatever I had said most recently. My critiques and new evidence moved me away from the quantitative simulation, and after just a few rounds I had an almost entirely different idea. It could be significantly better, but without a record of the original proposal and each shift along the way, it is hard to say. The fix is easy and still leaves you free to toss out ideas as they come: have the agent save each revision as a new copy. The trail of v1, v2, v3 files becomes the long-term memory, showing how far you have moved from the initial plan and what evidence or rationale drove each change. Writing and storing a few extra documents is cheap in both tokens and hard drive space, so versioning has little downside and is a low-cost way to measure scope creep.
Check what the agent tells you. With each round, I tried to verify whether my ideas had been tried before and what the literature said about them. Frequent checks against the evidence showed me quickly which avenues were worth pursuing, but doing this through an agent rather than reading dozens of reports myself carried a silent risk: the AI might make up or misread a source. You can reduce that chance (stronger models, required citations), but the risk never disappears, so I reviewed the generated reports specifically for claims. Doing so every round was too slow, so I used two approaches: separate agents for searching, synthesizing, and validating evidence, and a full audit every few report versions, before any major shift.
Gemini ran my batches of searches for related research, but it regularly made up papers, misread their conclusions, or linked to the wrong paper, all while rating its own work “Verified.” Passing its results through Codex to synthesize them into summaries, with a dedicated citation list tying each claim to its source, caught a lot of these. A review by Claude then double-checked everything before it entered my brainstorming. Any paper or report that heavily influenced my thinking got another look from a separate Claude agent, to confirm I had interpreted it correctly without slowing myself down.
Give the critic context, then bring in a stranger. Having learned that agents are too enthusiastic about ideas they “know” you want to pursue, I ran several more rounds of automated critique, this time with the verified evidence and the earlier brainstorming versions as context. That way the critic knew which alternatives I had considered and why I set them aside (the research wasn’t feasible, or the question had already been answered), so it didn’t suggest dead ends. It could also help me judge whether the rounds were really narrowing toward a stronger proposal or just shifting it around.
Finally, once I had enough evidence to turn the brainstorming into a presentation, I asked a fresh agent to review it as someone new to the field, to catch jargon on my slides. This mattered because, over many rounds, I had started using shorthand like “Research Question 3” or “the two cash transfer studies.” My main agent had learned these quirks, but to a newcomer they meant nothing. Revising for clarity as well as content helped consolidate my ideas and test whether my claims about what was “novel” or “interesting” were more than my own (agent-reinforced) biases.
After the presentation
The other benefit of keeping an extensive, evidence-checked, agent-critiqued record came after the talk, when I could debrief with another agent that could review all of that context. I pointed Claude at my folder of notes, summarized which sections I felt had gone well, and described a few audience questions and suggestions. By comparing these against my full thought process, it helped me work out which ideas and arguments landed and where to focus next to validate and refine the proposal. And the next time I present this to a different audience, I have a solid starting point for revising the slides.
The process taught me a lot about using AI for brainstorming and validation, including the pitfalls. Handled poorly, AI can reinforce bad assumptions or mislead with fake evidence. With a framework and healthy skepticism, it helped me move from a new idea to a (partly) justified proposal, with a transparent record of how I got there.
I’m curious how other researchers and professionals have used AI for ideation or literature review. What helped you move from an appealing idea to a question you could defend? How do you manage the strengths and weaknesses of different models, and how much do you trust their output versus validating it yourself?
