Loading study interface...
Thank you for participating in the annotation study. You will be evaluating a large language model (LLM)'s responses on medical questions and assessing whether it makes predictions based on clinically valid reasoning.
You will evaluate multiple batches of several questions per batch. The same LLM is used within each batch to produce the responses: they would, if any, share the same underlying clinically unsound reasoning behaviors. For each batch of questions, you may be optionally given an extra interpretability panel, which lists potentially influential factors for the model's predictions.
Participant ID:
You have completed all questions for this model. Please rate how helpful each source of information was.
You have completed all batches. Please answer a few final questions before finishing.
Which interface did you find the most helpful in understanding the model's decision-making?
The interpretability panel's visualizations were clear and easy to understand.
Any comments regarding this survey? (Optional)
Thank you for your participation. Your responses have been recorded.
Participant ID: -