Using think-aloud protocol to identify cognitive events while generating data-driven scientific hypotheses by inexperienced clinical researchers

We conducted a data-driven hypothesis generation study with clinical researchers using VIADS (a visual interactive analysis tool for filtering and summarizing large data sets coded with hierarchical terminologies) or other analytical tools (as control, e.g., SPSS, SAS, R). The participants analyzed the same datasets and developed hypotheses using a think-aloud verbal protocol. Their screen activities and audio were recorded, transcribed, and coded for cognitive events. We analyzed the recordings to identify the cognitive events (e.g., "Analyze data") during hypothesis generation. The VIADS group exhibited the lowest mean number of cognitive events per hypothesis with the smallest standard deviation. The highest percentages of cognitive events in hypothesis generation were "Using analysis results" (30%) and "Seeking connections" (23%). The results suggest that VIADS may guide participants better than the control group. Our framework for scientific hypothesis generation in clinical research contexts guides the elaboration of the underlying cognitive mechanism of the process.

Xia Jing, Brooke N. Draghi, Mytchell A. Ernst, Vimla L. Patel, James J. Cimino, Jay H. Shubrook, Yuchun Zhou, Chang Liu, Sonsoles De Lacalle