Tutor Diagnosis & Hint Generator

A student revises an attempt after a hint, then reviews separate feedback on the result, reasoning, and fit with the course. Illustration: Review the argument, not just its final answer.
projects

A correct conclusion can hide a gap in the argument, while an otherwise sound attempt may use a method the course has not introduced. Tutor Diagnosis examines those differences separately: whether the result is right, whether the steps are justified, and whether the explanation fits the course’s definitions and methods. Its scores are model feedback for review, not instructor grades.

The tool can also give a hint before I have a full attempt. It searches course material for a useful direction and aims to leave the final derivation to me. When the available passages do not cover the topic, it labels a general hint instead of attaching an unsupported course reference.

An illustrative learning loop, not a transcript or a new test result:

Claim: Every even integer is divisible by four.

Attempt: It works for 4 and 8, so it must hold for all even integers.

Hint: What would you need to check beyond those two examples?

Revision: Two is even but not divisible by four. One counterexample disproves the claim.

The point is to identify the reasoning gap: examples that support a claim do not establish that it holds in every case. For course problems, the tool compares an attempt with a reference answer and retrieved passages, then names the most important gap to revisit.

A fresh solution for comparison

A separate solver receives the question without the tutor’s earlier answer or the source passages used to write it. In a recorded econometrics trial, this second solution identified an intercept condition the first answer had omitted. The user still compares both arguments; agreement or disagreement is not automatically resolved into a verdict.

Other recorded checks included identifying an intentionally flawed regression argument and producing a hint on a microeconomics problem without a final derivation. Those are limited examples of the workflow, not evidence that feedback will be accurate on every assignment.

Course-scoped logs preserve attempts, hints, feedback, and solution comparisons for later summaries. The status record describes the trials. This is a set of tools within RAG Analysis; the separate Socratic Tutor tracks an interactive session and applies different conversation rules.

Aaron Scherf
Authors
Aaron Scherf (they/them)
PhD Student
I am a PhD student in Sustainable Development at Columbia University and a former USAID Foreign Service Officer. In Ukraine, I led the monitoring and evaluation team for over 50 development projects during the war. Previously, I worked on program design and social inclusion in Colombia, as well as budget analysis and knowledge management in Washington. Those experiences taught me how to ensure evidence is trustworthy and actionable, especially in conditions of rapidly changing and incomplete information. I now bring that perspective to research, data analysis, and practical software tools. I am available part-time for consulting and applied research collaborations.