Stefan Schauber

• Hautz, Wolf E.; Marcin, Thimo; Schauber, Stefan; Walter, Robin; Hautz, Stefanie C. & Birrenbach, Tanja [Show all 10 contributors for this article] (2026). Self-Monitoring Accuracy over Time for a Complex Diagnostic Skill. Perspectives on Medical Education. ISSN 2212-2761. 15(1), p. 420–431. doi: 10.5334/pme.1767. Full text in Research Archive
• Wonde, Shewatatek Gedamu; Tadesse, Tefera & Schauber, Stefan (2026). An evaluation of the undergraduate medicine licensure examinations development and implementation practices: lessons from Ethiopia with global implications. BMC Medical Education. 26. doi: 10.1186/s12909-026-08980-8. Full text in Research Archive Show summary

  Background The development of undergraduate medicine licensure examinations is critical for ensuring physicians’ competencies. Existing literature primarily focuses on exams Psychometric properties while leaving a gap in understanding the developmental processes, stakeholder engagement, and contextual influences, particularly in resource-limited settings. This study evaluated the mechanisms and stakeholders’ perspectives on shaping the Ethiopian undergraduate medicine licensure examination development and implementation practices. Methods A qualitative study design combining a phenomenological orientation with a realistic evaluation approach was employed. Purposive sampling was used to select eight experienced experts from four major stakeholder institutions. Data were collected through key informant interviews and cross-validated against exam development guidelines and expert profile datasets. Thematic analysis was conducted to present challenges, enablers, and institutional factors. Results The study evidenced that the Ethiopian Undergraduate Medicine Licensure Examination follows a standard approach, consisting of task analysis, blueprint preparation, item construction and appraisal, and exam administration. However, critical gaps were identified in expertise psychometric competencies, and challenges associated with paperbased testing and exam security concerns. Conclusion While participants perceive the undergraduate medicine licensure examination design aligned with national goals, its implementation is compromised by technical and logistic gaps. Transitioning to Computer-Based Testing, professionalizing item writing, and balancing stakeholder representation with experienced experts are essential facets to maintain the exam’s validity.

• Gogollari, Artemisa; Schmitz, Felix Michael; Schnabel, Kai Philipp; Huwendiek, Sören; Gysin, Stefan & Schirlo, Christian [Show all 9 contributors for this article] (2026). Comparing online and onsite communication training: Effects of social presence, performance, and acceptance in a randomized field trial. Patient Education and Counseling. ISSN 0738-3991. 149. doi: 10.1016/j.pec.2026.109603. Full text in Research Archive
• Queiroz, Maria Gabriela Carneiro; Junior, Francisco Carlos Specian; Filho, Pedro Tadao Hamamoto; Santos, Thiago M.; Schauber, Stefan & Woltman, Andrea M. [Show all 7 contributors for this article] (2026). Comparison of cognitive workload between very short answer questions and multiple-choice questions: an eye-tracking experiment. Medical Education Online. 31(1). doi: 10.1080/10872981.2026.2621434. Full text in Research Archive Show summary

  Very short answer questions (VSAQs) have gained attention for their superior psychometric properties compared to multiple-choice questions (MCQs). While VSAQs require knowledge recall, MCQs primarily involve knowledge recognition. This difference in cognitive processes may lead to varying cognitive workloads, defined as the amount of mental processing in working memory. Previous studies have not demonstrated consistent differences, likely due to reliance on self-reported measures. Eye tracking provides objective, process-level indicators of cognitive workload. This study investigated whether answering VSAQs requires a higher cognitive workload than answering MCQs. In a within-subject randomized crossover experiment, sixth-year medical students answered both VSAQs and MCQs. Cognitive workload was measured using screen-based eye tracking, focusing on the number of fixations and revisitations as objective indicators of mental effort. Data were analyzed using mixed-effects models. Thirty-four medical students participated, yielding 1,326 observations, which is the multiplication of the number of students by the number of questions (39 questions). Mixed-effects models showed a significant effect of question type on both workload indicators: VSAQs elicited more fixations and revisitations than MCQs (β_std = 0.30–0.39, p < .001). This effect remained after controlling for accuracy. Incorrect answers were associated with higher workload (β_std = −0.15–−0.16, p < .01). Heatmaps confirmed these findings, showing denser fixations on key diagnostic features for VSAQs and on answer options for MCQs. Answering VSAQs imposed a higher cognitive workload than MCQs. The presence of answer options in MCQs may reduce workload by providing unintentional cues, while VSAQs require active retrieval. Eye tracking proved valuable for distinguishing cognitive workload across assessment formats.

• Schauber, Stefan & Homer, Matt (2025). Challenging the norm: Length of exams determined by classification accuracy or reliability. Medical Education. ISSN 0308-0110. doi: 10.1111/medu.15742. Full text in Research Archive Show summary

  Purpose This paper challenges the notion that reliability indices are appropriate for informing test length in exams in medical education, where the focus is on ensuring defensible pass-fail decisions. Instead, we argue that using classification accuracy instead better suited to the purpose of exams in these cases. We show empirically, using resampled test data from a range of undergraduate knowledge exams, that this is indeed the case. More specifically, we address the hypothesis that the use of classification accuracy results in recommending shorter test lengths as compared to when using reliability. Method We analysed data from previous exams from both pre-clinical and clinical phases of undergraduate medical education. We used a re-sampling procedure in which both the cut-score and test length of repeatedly generated synthetic exams were varied systematically. N = 52 500 datasets were generated from the original exams. For each of these both reliability and classification accuracy indices were estimated. Result Results indicate that only classification accuracy, not reliability, varies in relation to the cut-score for pass-fail decisions. Furthermore, reliability and classification accuracy are differently related to test length. The optimal test length for using reliability was around 100 items, independent of pass-rates. For classification accuracy, recommendations are less generic. For exams with a small percentage of failed decisions (i.e., 5% or less), an item size of 50 did, on average, achieve an accuracy of 95% correct classifications. Conclusions We suggest a move towards the employment of classification accuracy using existing tools, whilst still using reliability as a complement. The benefits of re-thinking current test design practice include minimizing the burden of assessment on candidates and test developers. Item writers could focus on developing fewer, but higher quality, items. Finally, we stress the need to consider the effects of the balance false positive and false negative decisions in pass/fail classifications.

• Wespi, Rafael; Neher, Andrea N.; Birrenbach, Tanja; Schauber, Stefan; Frenkel, Marie Ottilie & Schrom-Feiertag, Helmut [Show all 8 contributors for this article] (2025). Physiological team dynamics explored: physiological synchrony in medical simulation training. Advances in Simulation. 10(1). doi: 10.1186/s41077-025-00335-5. Full text in Research Archive Show summary

  Introduction For researchers and medical simulation trainers, measuring team dynamics is vital for providing targeted feedback that can lead to improved patient outcomes. It is also valuable for research, such as investigating which dynamics benefit team performance. Traditional assessment methods, such as questionnaires and observations, are often subjective and static, lacking the ability to capture team dynamics. To address these shortcomings, this study explores the use of physiological synchrony (PS) measured through electrocardiogram (ECG) data to evaluate team dynamics automated and in high-resolution. Methods A multicentre observational field study was conducted involving 214 medical first responders during mixed reality (MR) mass casualty training sessions. Participants were equipped with electrocardiogram (ECG) sensors and MR gear. The study measured dyadic PS using heart rate (HR), root mean square of successive differences (RMSSD), and standard deviation of NN intervals (SDNN). Data were collected at high frequency and analysed using dynamic time warping (dtw) to assess fluctuations in PS. Results Findings indicate that PS varies significantly by task nature, with higher synchrony during cooperative tasks compared to baseline. Different ECG metrics offered unique insights into team dynamics. Proximity and scenario conditions influenced PS, with closer teamwork leading to higher PS. Smaller sampling intervals (e.g. 5 s) provided a detailed view of PS fluctuations over time. Discussion The results demonstrate the potential of PS as an indicator of team performance and cohesion. High-resolution monitoring provides detailed insights into team dynamics, offering high-resolution feedback that traditional methods cannot provide. The integration of physiological measures into training programmes can enhance team performance by providing objective, high-resolution data. Conclusion This study shows that PS, measured by ECG data, is sensitive to medical team activities, offering insights into team dynamics. Different ECG metrics highlight various aspects of team performance, and high-resolution monitoring captures detailed dynamics. Further research is needed to validate these findings across diverse scenarios. This approach could improve training methodologies, resulting in better-prepared medical teams and improved patient care outcomes.

• Hautz, Wolf E; Marcin, Thimo; Hautz, Stefanie C.; Schauber, Stefan; Krummrey, Gert & Müller, Martin [Show all 22 contributors for this article] (2025). Diagnoses supported by a computerised diagnostic decision support system versus conventional diagnoses in emergency patients (DDX-BRO): a multicentre, multiple-period, double-blind, cluster-randomised, crossover superiority trial. The Lancet Digital Health. 7(2), p. e136–e144. doi: 10.1016/S2589-7500(24)00250-4. Full text in Research Archive Show summary

  Background Diagnostic error is a frequent and clinically relevant health-care problem. Whether computerised diagnostic decision support systems (CDDSSs) improve diagnoses is controversial, and prospective randomised trials investigating their effectiveness in routine clinical practice are scarce. We hypothesised that diagnoses made with a CDDSS in the emergency department setting would be superior to unsupported diagnoses. Methods This multicentre, multiple-period, double-blind, cluster-randomised, crossover superiority trial was done in four emergency departments in Switzerland. Eligible patients were adults (aged ≥18 years) presenting with abdominal pain, fever of unknown origin, syncope, or non-specific symptoms. Emergency departments were randomly assigned (1:1) to one of two predefined sequences of six alternating periods of intervention or control. Patients presenting during an intervention period were diagnosed with the aid of a CDDSS, whereas patients presenting during a control period were diagnosed without a CDDSS (usual care). Patients and personnel assessing outcomes were masked to group allocation; treating physicians were not. The primary binary outcome (false or true) was a composite score indicating a risk of reduced diagnostic quality, which was deemed to be present if any of the following occurred within 14 days: unscheduled medical care, a change in diagnosis, an unexpected intensive care unit admission within 24 h if initially admitted to hospital, or death. We assessed superiority of supported versus unsupported diagnoses in all consenting patients using a generalised linear mixed effects model. All participants who received any study treatment (including control) and completed the study were included in the safety analysis. This trial is registered with ClinicalTrials.gov (NCT05346523) and is closed to accrual. Findings Between June 9, 2022, and June 23, 2023, 15 845 patients were screened and 1204 (591 [49·1%] female and 613 [50·9%] male) were included in the primary efficacy analysis. The median age of participants was 53 years (IQR 34–69). Diagnostic quality risk was observed in 100 (18%) of 559 patients with CDDSS-supported diagnoses and 119 (18%) of 645 with unsupported diagnoses (adjusted odds ratio 0·96 [95% CI 0·71–1·3]). 94 (7·8%) patients suffered a serious adverse event, none related to the study. Interpretation Use of a CDDSS did not reduce the occurrence of diagnostic quality risk compared with the usual diagnostic process in adults presenting to emergency departments. Future research should aim to identify specific contexts in which CDDSSs are effective and how existing CDDSSs can be adapted to improve patient outcomes.

• Wonde, Shewatatek Gedamu & Schauber, Stefan (2024). Psychometric properties of the Ethiopian national licensing exam in medicine: an analysis of multiple-choice questions using classical test theory. Teaching and learning in medicine. ISSN 1040-1334. doi: 10.1080/10401334.2024.2428191. Full text in Research Archive Show summary

  Background: The Ethiopian Ministry of Health introduced medical licensure examinations to maintain high standards in medical practice and build public trust in healthcare professionals. Studies also suggested significant issues in clinical competence among Ethiopian junior doctors as well concerns regarding unlicensed practice. Given the need to ensure safe health care, we investigated the psychometric properties of the multiple-choice items comprising the Ethiopian national licensing exam (NLE). These analyses help to provide an argument for the validity and reliability of the test scores. Method: We used a cross-sectional study design to analyze data from three cohorts of undergraduate medicine licensing examinations in Ethiopia (2020-2022, N = 2,213). Using Classical Test Theory, we assessed the psychometric properties of 600 MCQ items with 2400 single best answer choices, specifically item difficulty, item discrimination, and the number of nonfunctional distractors, and scale reliability. We provide results regarding the overall test and its sub-domains. Results: Ethiopia’s undergraduate medical licensure examination demonstrated acceptable reliability (Alpha > 0.80), with significant variability in item difficulty and examinee performance. Although these results indicate a sufficiently defensible exam, our results point to issues regarding item statistics, especially a high number of nonfunctional distractors. Conclusions: This study provides first evidence regarding the psychometric soundness of the Ethiopian NLE. However, a significant number of items should be carefully reviewed and possibly revised. As the examination is relatively new, ongoing refinement to item-development and review processes is essential to improve and ensure its quality.

• Schauber, Stefan; Olsen, Anne Olaug; Werner, Erik Lønnmark & Magelssen, Morten (2024). Inconsistencies in rater-based assessments mainly affect borderline candidates: but using simple heuristics might improve pass-fail decisions. Advances in Health Sciences Education. ISSN 1382-4996. doi: 10.1007/s10459-024-10328-0. Full text in Research Archive Show summary

  Introduction Research in various areas indicates that expert judgment can be highly inconsistent. However, expert judgment is indispensable in many contexts. In medical education, experts often function as examiners in rater-based assessments. Here, disagreement between examiners can have far-reaching consequences. The literature suggests that inconsistencies in ratings depend on the level of performance a to-be-evaluated candidate shows. This possibility has not been addressed deliberately and with appropriate statistical methods. By adopting the theoretical lens of ecological rationality, we evaluate if easily implementable strategies can enhance decision making in real-world assessment contexts. Methods We address two objectives. First, we investigate the dependence of rater-consistency on performance levels. We recorded videos of mock-exams and had examiners (N=10) evaluate four students’ performances and compare inconsistencies in performance ratings between examiner-pairs using a bootstrapping procedure. Our second objective is to provide an approach that aids decision making by implementing simple heuristics. Results We found that discrepancies were largely a function of the level of performance the candidates showed. Lower performances were rated more inconsistently than excellent performances. Furthermore, our analyses indicated that the use of simple heuristics might improve decisions in examiner pairs. Discussion Inconsistencies in performance judgments continue to be a matter of concern, and we provide empirical evidence for them to be related to candidate performance. We discuss implications for research and the advantages of adopting the perspective of ecological rationality. We point to directions both for further research and for development of assessment practices.

• Mahler, Daniela; Bock, Denise; Schauber, Stefan & Harms, Ute (2024). Using longitudinal models to describe preservice science teachers’ development of content knowledge and pedagogical content knowledge. Teaching and Teacher Education : An International Journal of Research and Studies. ISSN 0742-051X. 144. doi: 10.1016/j.tate.2024.104583. Full text in Research Archive Show summary

  Cross-sectional studies have revealed the importance of university teacher education for science teachers' professional knowledge. However, a detailed understanding of the development of this knowledge is lacking. Our analysis addresses this gap by applying longitudinal models to elucidate how preservice science teachers’ content knowledge (CK) and pedagogical content knowledge (PCK) evolve. Our results revealed a nonlinear growth for CK and PCK. We found that both interact during their development and that this interaction changes over the course of teacher education. Also, individual (grade point average) and institutional (teacher education program, second science subject) factors are related to their development.

• Ehrhard, Simone; Eyb, Vicky; Gautschi, Dominic; Schauber, Stefan; Ricklin, Meret E. & Klukowska-Rötzler, Jolanta [Show all 8 contributors for this article] (2024). Anaphylaxis in a Swiss university emergency department: clinical characteristics and supposed triggers. Allergy, Asthma & Clinical Immunology. ISSN 1710-1484. 20(1). doi: 10.1186/s13223-024-00901-y. Full text in Research Archive Show summary

  Background Anaphylaxis is the most severe form of acute systemic and potentially life-threatening reactions triggered by mast and basophilic cells. Recent studies show a worldwide incidence between 50 and 112 occurrences per 100,000 person-years. The most identified triggers are food, medications, and insect venoms. We aimed to analyze triggers and clinical symptoms of patients presenting to a Swiss university emergency department for adults. Methods Six-year retrospective analysis (01/2013 to 12/2018) of all patients (> 16 years of age) admitted with moderate or severe anaphylaxis (classification of Ring and Messmer ≥ 2) to the emergency department. Patient and clinical data were extracted from the electronic medical database of the emergency department. Results Of the 531 includes patients, 53.3% were female, the median age was 38 [IQR 26–51] years. The most common suspected triggers were medications (31.8%), food (25.6%), and insect stings (17.1%). Organ manifestations varied among the different suspected triggers: for medications, 90.5% of the patients had skin symptoms, followed by respiratory (62.7%), cardiovascular (44.4%) and gastrointestinal symptoms (33.7%); for food, gastrointestinal symptoms (39.7%) were more frequent than cardiovascular symptoms (36.8%) and for insect stings cardiovascular symptoms were apparent in 63.8% of the cases. Conclusions Average annual incidence of moderate to severe anaphylaxis during the 6-year period in subjects > 16 years of age was 10.67 per 100,000 inhabitants. Medications (antibiotics, NSAID and radiocontrast agents) were the most frequently suspected triggers. Anaphylaxis due to insect stings was more frequently than in other studies. Regarding clinical symptoms, gastrointestinal symptoms need to be better considered, especially that initial treatment with epinephrine is not delayed.

• Wespi, Rafael; Schwendimann, Lukas; Neher, Andrea; Birrenbach, Tanja; Schauber, Stefan & Manser, Tanja [Show all 8 contributors for this article] (2024). TEAMs go VR—validating the TEAM in a virtual reality (VR) medical team training. Advances in Simulation. 9(1). doi: 10.1186/s41077-024-00309-z. Full text in Research Archive Show summary

  Background Inadequate collaboration in healthcare can lead to medical errors, highlighting the importance of interdisciplinary teamwork training. Virtual reality (VR) simulation-based training presents a promising, cost-effective approach. This study evaluates the effectiveness of the Team Emergency Assessment Measure (TEAM) for assessing healthcare student teams in VR environments to improve training methodologies. Methods Forty-two medical and nursing students participated in a VR-based neurological emergency scenario as part of an interprofessional team training program. Their performances were assessed using a modified TEAM tool by two trained coders. Reliability, internal consistency, and concurrent validity of the tool were evaluated using intraclass correlation coefficients (ICC) and Cronbach’s alpha. Results Rater agreement on TEAM’s leadership, teamwork, and task management domains was high, with ICC values between 0.75 and 0.90. Leadership demonstrated strong internal consistency (Cronbach’s alpha = 0.90), while teamwork and task management showed moderate to acceptable consistency (alpha = 0.78 and 0.72, respectively). Overall, the TEAM tool exhibited high internal consistency (alpha = 0.89) and strong concurrent validity with significant correlations to global performance ratings. Conclusion The TEAM tool proved to be a reliable and valid instrument for evaluating team dynamics in VR-based training scenarios. This study highlights VR’s potential in enhancing medical education, especially in remote or distanced learning contexts. It demonstrates a dependable approach for team performance assessment, adding value to VR-based medical training. These findings pave the way for more effective, accessible interdisciplinary team assessments, contributing significantly to the advancement of medical education.

• Wonde, Shewatatek Gedamu; Tadesse, Tefera; Moges, Belay & Schauber, Stefan (2024). Experts’ prediction of item difficulty of multiple-choice questions in the Ethiopian Undergraduate Medicine Licensure Examination. BMC Medical Education. 24. doi: 10.1186/s12909-024-06012-x. Full text in Research Archive Show summary

  Background The ability of an expert’s item difficulty ratings to predict test-taker actual performance is an important aspect of licensure examinations. Expert judgment is used as a primary source of information for users to make prior decisions to determine the pass rate of test takers. The nature of raters involved in predicting item difficulty is central to set credible standards. Therefore, this study aimed to assess and compare raters’ prediction and actual Multiple-Choice Questions’ difficulty of the undergraduate medicine licensure examination (UGMLE) in Ethiopia. Method 815 examinees’ responses to 200 Multiple-Choice Questions (MCQs) were used in this study. The study also included experts’ item difficulty ratings of seven physicians who participated in the standard settings of UGMLE. Then, analysis was conducted to understand experts’ rating variation in predicting the actual difficulty levels of examinees. Descriptive statistics was used to profile the mean rater’s and actual difficulty value for MCQs, and ANOVA was used to compare the mean differences between raters’ prediction of item difficulty. Additionally, regression analysis was used to understand the interrater variations in item difficulty predictions compared to the actual difficulty. The proportion of variance of actual difficulty explained from rater prediction was computed using regression analysis. Results In this study, the mean difference between raters’ prediction and examinees’ actual performance was inconsistent across the exam domains. The study revealed a statistically significant strong positive correlation between the actual and predicted item difficulty in exam domains eight and eleven. However, a non-statistically significant very weak positive correlation was reported in exam domains seven and twelve. The multiple comparison analysis showed significant differences in mean item difficulty ratings between raters. In the regression analysis, experts’ item difficulty ratings of the UGMLE had 33% power in predicting the actual difficulty level. The regression model also showed a moderate positive correlation (R = 0.57) that was statistically significant at F (6, 193) = 15.58, P = 0.001. Conclusion This study demonstrated the complex process for assessing the difficulty level of MCQs in the UGMLE and emphasized the benefits of using experts’ ratings in advance. To ensure the exams maintain the necessary reliable and valid scores, raters’ accuracy on the UGMLE must be improved. To achieve this, techniques that align with the evolving assessment methodologies must be developed.

• Larsen, Linda; Schauber, Stefan; Holt, Tonje & Helland, Maren Sand (2023). Longitudinal Covid-19 effects on child mental health: vulnerability and age dependent trajectories. Child and Adolescent Psychiatry and Mental Health (CAPMH). 17(104). doi: 10.1186/s13034-023-00652-5. Full text in Research Archive Show summary

  Background Few longitudinal studies have investigated the extended long-term impact of the Covid-19 pandemic for children’s and adolescents’ mental health, and a lack of uniform findings suggest heterogeneity in the impact of the pandemic. Methods This study investigated child and adolescent mental health symptoms across four occasions (pre-pandemic, initial lockdown, second lockdown, and society post reopening) using data from the Dynamics of Family Conflict study. Child and adolescent depressive vulnerability, age, and sex were explored as trajectory moderators. Children and adolescents (N = 381, Mage = 13.65, SD = 1.74) self-reported their anxiety, depression, and externalizing symptoms. Mixed effects analyses were performed to investigate trajectories across measurement occasions and interaction terms between occasion and moderator variables were included to better understand the heterogeneity in the impact of the pandemic. Results Children and adolescents reported increases in anxiety symptoms at the second lockdown (t(523) = −3.66, p < .01) and when society had reopened (t(522) = −4.90, p < .001). An increase in depression symptoms was seen when society had reopened relative to the three previous measurement occasions (ps < 0.01). Depressive vulnerability moderated the trajectory for anxiety symptoms (F(3,498) = 3.05, p = .028), while age moderated the trajectory for depression symptoms (F(3,532) = 2.97, p = .031). Conclusion The delayed and negative impact on children’s and adolescents’ mental health underscores the need for continued monitoring, and implementation of support systems to help and mitigate further deterioration.

• Brinchmann, Ellen Irén; Røe-Indregård, Hanne; Karlsen, Jannicke; Schauber, Stefan & Hagtvet, Ragna Bente Eriksen (2023). The linguistic complexity of adult and child contextualized and decontextualized talk. Child Development. ISSN 0009-3920. p. 1–13. doi: 10.1111/cdev.13932. Full text in Research Archive Show summary

  Abstract The association between decontextualized talk (DT; i.e., talk extending beyond immediate context) and child language outcomes is well‐attested but not well‐understood. This study tested the hypothesis that DT is more linguistically complex than contextualized talk (CT). Thirty‐eight Norwegian children ( M age  = 5.5 years; 25 girls; 30 Norwegian‐speaking monolinguals and eight multilinguals) and their teachers were videotaped during picture book reading, story card conversations and toy play (collected 2010–2011 and 2017). Results show that DT was more complex than CT among children and teachers. Both types of talk were more complex during book reading and story conversations than during play. The conversational context should be accounted for when theorizing about the role of DT in language development.

• Marcin, Thimo; Hautz, Stefanie C.; Singh, Hardeep; Zwaan, Laura; Schwappach, David & Krummey, Gert [Show all 16 contributors for this article] (2023). Effects of a computerised diagnostic decision support tool on diagnostic quality in emergency departments: Study protocol of the DDx-BRO multicentre cluster randomised cross-over trial. BMJ Open. 13(3). doi: 10.1136/bmjopen-2023-072649. Full text in Research Archive
• Kämmer, Juliane E.; Ernst, Karin; Grab, Kim; Schauber, Stefan; Hautz, Stefanie C. & Penders, Dorothea [Show all 7 contributors for this article] (2023). Collaboration during the diagnostic decision-making process: When does it help? Journal of Behavioral Decision Making. ISSN 0894-3257. doi: 10.1002/bdm.2357. Full text in Research Archive Show summary

  When making complex decisions, such as a medical diagnosis, decision makers typically gather, analyze, and synthesize (integrate) information. In a previous study, we showed that delegating such complex decisions to collaborating pairs increases decision quality substantially compared to that of individuals, without requiring different information gathering. Given the higher costs associated with teamwork, however, it is of great practical interest to understand when in the process the performance benefits of teams may arise, so that particular subtasks can be delegated to teams when most appropriate. We thus conducted an experimental study in which fourth-year medical students (n = 109) worked either in pairs or alone on two separate subtasks of the diagnostic process: (1) analyzing diagnostic test results (e.g., X-rays) and (2) integrating previously interpreted test results into diagnoses. Linear mixed-effects models revealed a small benefit of collaborating pairs over individuals in both subtasks. We conclude that collaborating with a peer may pay off both when analyzing information and when integrating it into a diagnosis as it provides the opportunity to correct each other's errors and to make use of a greater knowledge base. These findings encourage the strategic use of collaboration with a colleague when making complex decisions. Further research into the underlying processes is needed.

• Wespi, Rafael; Birrenbach, Tanja; Schauber, Stefan; Manser, Tanja; Sauter, Thomas C. & Kämmer, Juliane E. (2023). Exploring objective measures for assessing team performance in healthcare: an interview study. Frontiers in Psychology. 14. doi: 10.3389/fpsyg.2023.1232628. Full text in Research Archive Show summary

  Introduction Effective teamwork plays a critical role in achieving high-performance outcomes in healthcare. Consequently, conducting a comprehensive assessment of team performance is essential for providing meaningful feedback during team trainings and enabling comparisons in scientific studies. However, traditional methods like self-reports or behavior observations have limitations such as susceptibility to bias or being resource consuming. To overcome these limitations and gain a more comprehensive understanding of team processes and performance, the assessment of objective measures, such as physiological parameters, can be valuable. These objective measures can complement traditional methods and provide a more holistic view of team performance. The aim of this study was to explore the potential of the use of objective measures for evaluating team performance for research and training purposes. For this, experts in the field of research and medical simulation training were interviewed to gather their opinions, ideas, and concerns regarding this novel approach. Methods A total of 34 medical and research experts participated in this exploratory qualitative study, engaging in semi-structured interviews. During the interview, experts were asked for (a) their opinion on measuring team performance with objective measures, (b) their ideas concerning potential objective measures suitable for measuring team performance of healthcare teams, and (c) their concerns regarding the use of objective measures for evaluating team performance. During data analysis responses were categorized per question. Results The findings from the 34 interviews revealed a predominantly positive reception of the idea of utilizing objective measures for evaluating team performance. However, the experts reported limited experience in actively incorporating objective measures into their training and research. Nevertheless, they identified various potential objective measures, including acoustical, visual, physiological, and endocrinological measures and a time layer. Concerns were raised regarding feasibility, complexity, cost, and privacy issues associated with the use of objective measures. Discussion The study highlights the opportunities and challenges associated with employing objective measures to assess healthcare team performance. It particularly emphasizes the concerns expressed by medical simulation experts and team researchers, providing valuable insights for developers, trainers, researchers, and healthcare professionals involved in the design, planning or utilization of objective measures in team training or research.

• Schauber, Stefan & Stensløkken, Kåre-Olav (2023). No knowledge gap in human physiology after remote teaching for second year medical students throughout the Covid-19 pandemic. BMC Medical Education. 23(1). doi: 10.1186/s12909-023-04959-x. Full text in Research Archive Show summary

  The COVID-19 pandemic had a disruptive effect on higher education. A critical question is whether these changes affected students’ learning outcomes. Knowledge gaps have consequences for future learning and may—in health professionals' education—also pose a threat to patient safety. Current research has shortcomings and does not allow for clear-cut interpretation. Our context is instruction in human physiology in an undergraduate medical program from high stakes end of term examinations. The sequence of imposed measures to slow the COVID-19 pandemic created a natural experiment, allowing for comparisons in performance during in-person versus remote instruction. In a two-factorial design, mode of instruction (in-person vs. remote) and mode of assessment (in-person vs. remote) were analyzed using both basic (non-parametric statistics, T-tests) and advanced statistical methods (linear mixed-effects model; resampling techniques). Test results from a total of N = 1095 s-year medical students were included in the study. We did not find empirical evidence of knowledge gaps; rather, students received comparable or higher scores during remote teaching. We interpret these findings as empirical evidence that both students and teachers adapted to pandemic disruption in a way that did not lead to knowledge gaps. We conclude that highly motivated students had no reduction in academic achievement. Moreover, we have developed an accessible digital exam system for secure, fair, and effective assessments which is sufficiently defensible for making pass/fail decisions.

• Ehrhard, Simone; Gautschi, Dominic; Eyb, Vicky; Schauber, Stefan; Ricklin, Meret E. & Klukowska-Rötzler, Jolanta [Show all 8 contributors for this article] (2023). Use of epinephrine in anaphylaxis: a retrospective cohort study at a Swiss university emergency department. Swiss Medical Weekly. ISSN 1424-7860. 153. doi: 10.57187/smw.2023.40065. Full text in Research Archive Show summary

  AIMS OF THE STUDY: Anaphylaxis is a medical emergency and requires prompt treatment to prevent life-threatening conditions. Epinephrine, considered as the first-line drug, is often not administered. We aimed first to analyse the use of epinephrine in patients with anaphylaxis in the emergency department of a university hospital and secondly to identify factors that influence the use of epinephrine.&#x0D; METHODS: We performed a retrospective analysis of all patients admitted with moderate or severe anaphylaxis to the emergency department between 1 January 2013 and 31 December 2018. Patient characteristics and treatment information were extracted from the electronic medical database of the emergency department.&#x0D; RESULTS: A total of 531 (0.2%) patients with moderate or severe anaphylaxis out of 260,485 patients admitted to the emergency department were included. Epinephrine was administered in 252 patients (47.3%). In a multivariate logistic regression, cardiovascular (Odds Ratio [OR] = 2.94, CI 1.96–4.46, p &lt;0.001) and respiratory symptoms (OR = 3.14, CI 1.95–5.14, p&lt;0.001) were associated with increased likelihood of epinephrine administration, in contrast to integumentary symptoms (OR = 0.98, CI 0.54–1.81, p = 0.961) and gastrointestinal symptoms (OR = 0.62, CI 0.39–1.00, p = 0.053).&#x0D; CONCLUSIONS: Less than half of the patients with moderate and severe anaphylaxis received epinephrine according to guidelines. In particular, gastrointestinal symptoms seem to be misrecognised as serious symptoms of anaphylaxis. Training of the emergency medical services and emergency department medical staff and further awareness are crucial to increase the administration rate of epinephrine in anaphylaxis.

• Marcin, Thimo; Lüthi, Ailin; Graf, Ronny R; Krummrey, Gert; Schauber, Stefan & Breakey, Neal [Show all 8 contributors for this article] (2023). Is language an issue? Accuracy of the German computerized diagnostic decision support system ISABEL and cross-validation with the English counterpart. Diagnosis. ISSN 2194-8011. doi: 10.1515/dx-2023-0047. Full text in Research Archive
• Haukedal, Christiane; Wie, Ona Bø; Schauber, Stefan & Torkildsen, Janne von Koss (2023). Children With Developmental Language Disorder Have Lower Quality of Life Than Children With Typical Development and Children With Cochlear Implants. Journal of Speech, Language and Hearing Research. ISSN 1092-4388. 66(10), p. 3988–4008. doi: 10.1044/2023_JSLHR-22-00742. Full text in Research Archive Show summary

  Purpose: The purpose of this study was to examine quality of life (QOL) and its relation to language skills in children with developmental language disorder (DLD). This was examined by comparing QOL to a control group of children with typical development (TD), as well as children with cochlear implants (CIs), who potentially struggle with language for language, although for a different reason than children with DLD. Method: Two groups of children, a group with TD (n = 29) and a group of children with CIs (n = 29), were matched to the DLD group (n = 29) on chronological age, gender, nonverbal IQ, and parental educational level through a propensity matching procedure. A third group consisting of children with CIs was also matched to the DLD group but additionally matched on language abilities. QOL scores were compared across groups, and the association between language skills and QOL was examined in the DLD group. Result: The DLD group was reported by parents to have statistically significantly poorer QOL scores than peers with TD or CIs. When controlling for language skills, either statistically or through an additional CI group matched on language abilities, there were no statistically significant differences in QOL scores across groups. In the DLD group, language skills explained 16% of the variation in QOL. Conclusion: DLD is associated with the children's overall QOL, and the degree of reduced QOL relates to the severity of the language impairment.

• Haukedal, Christiane Lingås; Wie, Ona Bø; Schauber, Stefan; Lyxell, Björn; Fitzpatrick, Elizabeth & Torkildsen, Janne von Koss (2022). Social communication and quality of life in children using hearing aids. International Journal of Pediatric Otorhinolaryngology. ISSN 0165-5876. 152(152), p. 1–11. doi: 10.1016/j.ijporl.2021.111000. Full text in Research Archive Show summary

  Objectives This study compared the parent-reported structural language and social communication skills—measured with the Children's Communication Checklist-2 (CCC-2)—and health-related quality of life (HR-QOL)—measured with the Pediatric Quality of Life Inventory (PedsQL)—of children who use hearing aids (HAs) and their typical-hearing (TH) peers. Design The participants were 88 children (age range of 5; 6 to 13; 1 (years; months)) and their parents: 45 children with bilateral moderate to severe hearing loss using HAs who had no additional disabilities and 43 children with typical hearing. The groups were matched based on chronological age, gender, nonverbal IQ, and parental education level. The parents completed questionnaires related to their children's communication skills, including subdomains structural language and social communication, and HR-QOL. Results The HA group had significantly poorer overall communication skills than the TH group (r = 0.49). The children in the HA group scored significantly lower than the TH group on both structural language (r = 0.37) and social communication (r = 0.41). Half of the children in the HA group had overall communication scores that either indicated concern or required further investigation according to the instrument's manual. In terms of psychosocial functioning, which was measured as HR-QOL, the subdomain school functioning was the main driver of the difference between groups, with the HA group being at least twice as likely (OR = 2.52) as the TH group to have poor HR-QOL in the school domain. Better parent-reported social communication was associated with better parent-reported psychosocial functioning in the children using HAs—even when background variables were taken into account. Conclusion The results suggest that traditional assessments and interventions targeting structural aspects of language may overlook social communication difficulties in children with HAs, even those with no additional disabilities. As school functioning stood out as the most problematic domain for children with HAs, efforts to improve the well-being of these children should focus on this area.

• Larsen, Linda; Hjetland, Hanne Næss & Schauber, Stefan (2022). A longitudinal investigation of letter-name knowledge in a semi-transparent orthography. Scandinavian Journal of Educational Research. ISSN 0031-3831. doi: 10.1080/00313831.2022.2116485. Full text in Research Archive Show summary

  Children’s ability to correctly name letters is a key predictor of later reading abilities and skills, but research on letter naming from Scandinavian orthographies is scarce. The aim of this study is to explore how child- and letter-related factors (i.e., gender, child name, phonemic awareness, letter position in the alphabet and frequency, and speech sound development) are associated with Norwegian children’s letter name knowledge over time. The sample comprises 185 Norwegian children with an average age of 51.59 months (SD = 2.12) who completed a letter-naming task on three separate occasions one year apart. Results from mixed-effects models show that children were more likely to name the first letter of their own name, however, this effect diminished over time. Further, letter frequency significantly predicted letter naming, and the letter frequency and letter position in alphabet effects were larger in older children. This study contributes important and relevant information for teachers and educators.

• Schauber, Stefan Kilian; Hautz, Stefanie C.; Kämmer, Juliane E.; Stroben, Fabian & Hautz, Wolf E. (2021). Do different response formats affect how test takers approach a clinical reasoning task? An experimental study on antecedents of diagnostic accuracy using a constructed response and a selected response format. Advances in Health Sciences Education. ISSN 1382-4996. 26, p. 1339–1354. doi: 10.1007/s10459-021-10052-z. Full text in Research Archive Show summary

  Abstract The use of response formats in assessments of medical knowledge and clinical reasoning continues to be the focus of both research and debate. In this article, we report on an experimental study in which we address the question of how much list-type selected response formats and short-essay type constructed response formats are related to differences in how test takers approach clinical reasoning tasks. The design of this study was informed by a framework developed within cognitive psychology which stresses the importance of the interplay between two components of reasoning—self-monitoring and response inhibition—while solving a task or case. The results presented support the argument that different response formats are related to different processing behavior. Importantly, the pattern of how different factors are related to a correct response in both situations seem to be well in line with contemporary accounts of reasoning. Consequently, we argue that when designing assessments of clinical reasoning, it is crucial to tap into the different facets of this complex and important medical process.

• Kämmer, Juliane E.; Schauber, Stefan Kilian; Hautz, Stefanie C.; Stroben, Fabian & Hautz, Wolf E. (2021). Differential diagnosis checklists reduce diagnostic error differentially: a randomized experiment. Medical Education. ISSN 0308-0110. doi: 10.1111/medu.14596. Full text in Research Archive Show summary

  Introduction Wrong and missed diagnoses contribute substantially to medical error. Can a prompt to generate alternative diagnoses (prompt) or a differential diagnosis checklist (DDXC) increase diagnostic accuracy? How do these interventions affect the diagnostic process and self-monitoring? Methods Advanced medical students (N = 90) were randomly assigned to one of four conditions to complete six computer-based patient cases: group 1 (prompt) was instructed to write down all diagnoses they considered while acquiring diagnostic test results and to finally rank them. Groups 2 and 3 received the same instruction plus a list of 17 differential diagnoses for the chief complaint of the patient. For half of the cases, the DDXC contained the correct diagnosis (DDXC+), and for the other half, it did not (DDXC−; counterbalanced). Group 4 (control) was only instructed to indicate their final diagnosis. Mixed-effects models were used to analyse results. Results Students using a DDXC that contained the correct diagnosis had better diagnostic accuracy, mean (standard deviation), 0.75 (0.44), compared to controls without a checklist, 0.49 (0.50), P < 0.001, but those using a DDXC that did not contain the correct diagnosis did slightly worse, 0.43 (0.50), P = 0.602. The number and relevance of diagnostic tests acquired were not affected by condition, nor was self-monitoring. However, participants spent more time on a case in the DDXC−, 4:20 min (2:36), P ≤ 0.001, and DDXC+ condition, 3:52 min (2:09), than in the control condition, 2:59 min (1:44), P ≤ 0.001. Discussion Being provided a list of possible diagnoses improves diagnostic accuracy compared with a prompt to create a differential diagnosis list, if the provided list contains the correct diagnosis. However, being provided a diagnosis list without the correct diagnosis did not improve and might have slightly reduced diagnostic accuracy. Interventions neither affected information gathering nor self-monitoring.

• Hautz, Wolf E.; Sauter, Thomas C.; Hautz, Stefanie C.; Kämmer, Juliane E.; Schauber, Stefan Kilian & Birrenbach, Tanja [Show all 9 contributors for this article] (2020). What determines diagnostic resource consumption in emergency medicine: Patients, physicians or context? Emergency Medicine Journal. ISSN 1472-0205. 37, p. 546–551. doi: 10.1136/emermed-2019-209022. Full text in Research Archive
• Felicitas-Maria, Lahner; Schauber, Stefan Kilian; Carolin, Lövald Andrea; Roger, Kropf; Sissel, Guttormsen & R., Fischer Martin [Show all 7 contributors for this article] (2020). Measurement precision at the cut score in medical multiple choice exams: Theory matters. Perspectives on Medical Education. ISSN 2212-2761. doi: 10.1007/s40037-020-00586-0. Full text in Research Archive Show summary

  Introduction In high-stakes assessment, the measurement precision of pass-fail decisions is of great importance. A concept for analyzing the measurement precision at the cut score is conditional reliability, which describes measurement precision for every score achieved in an exam. We compared conditional reliabilities in Classical Test Theory (CTT) and Item Response Theory (IRT) with a special focus on the cut score and potential factors influencing conditional reliability at the cut score. Methods We analyzed 32 multiple-choice exams from three Swiss medical schools comparing conditional reliability at the cut score in IRT and CCT. Additionally, we analyzed potential influencing factors such as the range of examinees’ performance, year of study, and number of items using multiple regression. Results In CTT, conditional reliability was highest for very low and very high scores, whereas examinees with medium scores showed low conditional reliabilities. In IRT, the maximum conditional reliability was in the middle of the scale. Therefore, conditional reliability at the cut score was significantly higher in IRT compared with CTT. It was influenced by the range of examinees’ performance and number of items. This influence was more pronounced in CTT. Discussion We found that conditional reliability shows inverse distributions and conclusions regarding the measurement precision at the cut score depending on the theory used. As the use of IRT seems to be more appropriate for criterion-oriented standard setting in the framework of competency-based medical education, our findings might have practical implications for the design and quality assurance of medical education assessments.

• Schauber, Stefan Kilian & Hecht, Martin (2020). How sure can we be that a student really failed? On the measurement precision of individual pass-fail decisions from the perspective of Item Response Theory. Medical Teacher. ISSN 0142-159X. 42(12), p. 1374–1384. doi: 10.1080/0142159X.2020.1811844. Full text in Research Archive Show summary

  Background In high-stakes assessments in medical education, the decision to let a particular participant pass or fail has far-reaching consequences. Reliability coefficients are usually used to support the trustworthiness of assessments and their accompanying decisions. However, coefficients such as Cronbach’s Alpha do not indicate the precision with which an individual’s performance was measured. Objective Since estimates of precision need to be aligned with the level on which inferences are made, we illustrate how to adequately report the precision of pass-fail decisions for single individuals. Method We show how to calculate the precision of individual pass-fail decisions using Item Response Theory and illustrate that approach using a real exam. In total, 70 students sat this exam (110 items). Reliability coefficients were above recommendations for high stakes test (> 0.80). At the same time, pass-fail decisions around the cut score were expected to show low accuracy. Conclusions Our results illustrate that the most important decisions–i.e. those based on scores near the pass-fail cut-score–are often ambiguous, and that reporting a traditional reliability coefficient is not an adequate description of the uncertainty encountered on an individual level.

• Hautz, Wolf E.; Sauter, Thomas C.; Exadaktylos, Aristomenis; Krummrey, Gert; Schauber, Stefan Kilian & Müller, Martin (2020). Barriers to seeking emergency care during the COVID-19 pandemic may lead to higher morbidity and mortality – a retrospective study from a Swiss university hospital. Swiss Medical Weekly. ISSN 1424-7860. doi: 10.4414/smw.2020.20331. Full text in Research Archive
• Karay, Yassin; Reiss, Birger & Schauber, Stefan Kilian (2020). Progress testing anytime and anywhere – Does a mobile-learning approach enhance the utility of a large-scale formative assessment tool? Medical Teacher. ISSN 0142-159X. doi: 10.1080/0142159X.2020.1798910. Full text in Research Archive Show summary

  Background The widespread use of mobile devices among students favors the use of mobile learning scenarios at universities. In this study, we explore whether a time- and location-independent variant of a formative progress test has an impact on the students’ acceptance, its validity and reliability and if there is a difference in response processes between the two exam conditions. Methods Students were randomly assigned to two groups of which one took the test free of local or temporal fixations, while the other group took the test at the local testing center under usual examination conditions. Beside the generated test data, such as test score, time-on-test, and semester status, students also evaluated the settings. Results While there was no significant effect on the test score between the two groups, students in the mobile group spent more time on the test and were more likely to use the help of books or online resources. The results of the evaluation show that the acceptability among students is increased by a mobile version of the formative progress test. Conclusions The results suggest that the acceptance and motivation to participate in formative tests is enhanced by lifting local and temporal restrictions. The mobile version nonetheless does not have an impact on the students’ performance.

• Larsen, Linda Paarup Neigaard; Schauber, Stefan Kilian; Kohen, Saskia; Nickels, Lyndsey & McArthur, Genevieve (2020). Children’s knowledge of single- and multiple-letter grapheme-phoneme correspondences: An exploratory study. Early Childhood Research Quarterly. ISSN 0885-2006. 51, p. 379–391. doi: 10.1016/j.ecresq.2019.12.001. Full text in Research Archive Show summary

  In this study, we examined Australian children’s knowledge of single- and multiple-letter grapheme-phoneme correspondences (GPCs), and the influence of five different factors – GPC complexity, phoneme status, the child’s name, GPC entropy, and GPC frequency – on GPC knowledge. Data from 337 Australian children enrolled in Kindergarten to Grade 3 were included in the study and analyses were performed using mixed effects models. Results indicate that GPC knowledge varied across children and GPCs, children were almost twice as likely to accurately pronounce single-letter graphemes compared to multiple-letter graphemes, and performance was better for GPCs which occur more frequently in text. GPCs with higher entropy values (less consistent) had close to 40% lower odds of being known by children. The study has practical implications by providing an evidence-based guide for the order in which GPCs should be introduced to children in schools.

• Wie, Ona Bø; Torkildsen, Janne von Koss; Schauber, Stefan Kilian; Busch, Tobias & Litovsky, Ruth (2020). Long-term language development in children with early simultaneous bilateral cochlear implants. Ear and Hearing. ISSN 0196-0202. 41(5), p. 1294–1305. doi: 10.1097/AUD.0000000000000851. Full text in Research Archive Show summary

  Objectives: This longitudinal study followed the language development of children who received the combination of early (5 to 18 months) and simultaneous bilateral cochlear implants (CIs) throughout the first 6 years after implantation. It examined the trajectories of their language development and identified factors associated with language outcomes. Design: Participants were 21 Norwegian children who received bilateral CIs between the ages of 5 and 18 mo and 21 children with normal hearing (NH) who were matched to the children with CIs on age, sex, and maternal education. The language skills of these two groups were compared at 10 time points (3, 6, 9, 12, 18, 24, 36, 48, 60, and 72 months after implantation) using parent reports and standardized measures of general language skills, vocabulary, and grammar. In addition, assessments were made of the effects of age at CI activation, speech recognition abilities, and mothers’ education on language outcomes 6 years after implantation. Results: During the first 4 years after implantation, the gap in general expressive and receptive language abilities between children with CIs and children with NH gradually closed. While at the initial five to six assessments (3 to 36 months after implantation), significant differences between children with CIs and children with NH were observed; at 4 years after implantation, there were no longer any significant group differences in general language skills and most children with CIs achieved scores within 1 SD of the tests’ normative means. From 2 to 3 years after implantation onward, expressive vocabulary and receptive grammar skills of children with CIs were similar to those of the reference group. However, from 4 years after implantation until the end of the observation period, 6 years after implantation, expressive grammar skills of children with CIs were lower than those of children with NH. In addition, a gap in receptive vocabulary appeared and grew increasingly larger from 4 to 6 years postimplantation. At the final assessment, the children with CIs had an average receptive vocabulary score around 1 SD below the normative mean. Regression analysis indicated that the children’s language outcomes at 6 years after implantation were related to their speech recognition skills, age at CI activation, and maternal education. Conclusions: In the first 4 years after implantation, the language performance of children with CIs became increasingly similar to that of their NH peers. However, between 4 and 6 years after implantation, there were indications of challenges with certain aspects of language, specifically receptive vocabulary and expressive grammar. Because these challenges first appeared after the 4-year assessment, the findings underline the importance of long-term language intervention to increase the chances of a continued language development comparable to that of NH peers. They also indicate that there is a need for comprehensive longitudinal studies of the language development of children with CIs beyond 4 years after implantation.

• Freytag, Julia; Stroben, Fabian; Hautz, Wolf E.; Schauber, Stefan Kilian & Kämmer, Juliane E. (2019). Rating the quality of teamwork—a comparison of novice and expert ratings using the Team Emergency Assessment Measure (TEAM) in simulated emergencies. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine. 27(1). doi: 10.1186/s13049-019-0591-9. Full text in Research Archive Show summary

  Background Training in teamwork behaviour improves technical resuscitation performance. However, its effect on patient outcome is less clear, partly because teamwork behaviour is difficult to measure. Furthermore, it is unknown who should evaluate it. In clinical practice, experts are obliged to participate in resuscitation efforts and are thus unavailable to assess teamwork quality. Consequently, we sought to determine if raters with little clinical experience and experts provide comparable evaluations of teamwork behaviour. Methods Novice and expert raters judged teamwork behaviour during 6 emergency medicine simulations using the Teamwork Emergency Assessment Measure (TEAM). Ratings of both groups were analysed descriptively and compared with U and t tests. We used a mixed effects model to identify the proportion of variance in TEAM scores attributable to rater status and other sources. Results Twelve raters evaluated 7 teams rotating through 6 cases, for a total of 84 observations. We found no significant difference between expert and novice ratings for 7 of the 11 items of the TEAM or in the sums of all item scores. Novices rated teamwork behaviour higher on 4 items and overall. Rater status accounted for 11.1% of the total variance in scores. Conclusions Experts’ and novices’ ratings were similarly distributed, implying that raters with limited experience can provide reliable data on teamwork behaviour. Novices show a consistent, but slightly more lenient rating behaviour. Clinical studies and real-life teams may thus employ novices using a structured observational tool such as TEAM to inform their performance review and improvement

• Hautz, Wolf E.; Schubert, Sebastian; Schauber, Stefan Kilian; Kunina-Habenicht, Olga; Hautz, Stefanie C. & Kämmer, Juliane E. [Show all 7 contributors for this article] (2019). Accuracy of self‐monitoring: does experience, ability or case difficulty matter? Medical Education. ISSN 0308-0110. 53(7), p. 735–744. doi: 10.1111/medu.13801. Full text in Research Archive Show summary

  Context The ability to self‐monitor one's performance in clinical settings is a critical determinant of safe and effective practice. Various studies have shown this form of self‐regulation to be more trustworthy than aggregate judgements (i.e. self‐assessments) of one's capacity in a given domain. However, little is known regarding what cues inform learners’ self‐monitoring, which limits an informed exploration of interventions that might facilitate improvements in self‐monitoring capacity. The purpose of this study is to understand the influence of characteristics of the individual (e.g. ability) and characteristics of the problem (e.g. case difficulty) on the accuracy of self‐monitoring by medical students. Methods In a cross‐sectional study, 283 medical students from 5 years of study completed a computer‐based clinical reasoning exercise. Confidence ratings were collected after completing each of six cases and the accuracy of self‐monitoring was considered to be a function of confidence when the eventual answer was correct relative to when the eventual answer was incorrect. The magnitude of that difference was then explored as a function of year of seniority, gender, case difficulty and overall aptitude. Results Students demonstrated accurate self‐monitoring by virtue of giving higher confidence ratings (57.3%) and taking a shorter time to work through cases (25.6 seconds) when their answers were correct relative to when they were wrong (41.8% and 52.0 seconds, respectively; p< 0.001 and d > 0.5 in both instances). Self‐monitoring indices were related to student seniority and case difficulty, but not to overall ability or student gender. Conclusions This study suggests that the accuracy of self‐monitoring is context specific, being heavily influenced by the struggles students experience with a particular case rather than reflecting a generic ability to know when one is right or wrong. That said, the apparent capacity to self‐monitor increases developmentally because increasing experience provides a greater likelihood of success with presented problems.

• Hautz, Wolf E.; Kämmer, Juliane E.; Hautz, Stefanie C.; Sauter, Thomas C.; Zwaan, Laura & Exadaktylos, Aristomenis [Show all 10 contributors for this article] (2019). Diagnostic error increases mortality and length of hospital stay in patients presenting through the emergency room. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine. 27(1), p. 1–12. doi: 10.1186/s13049-019-0629-z. Full text in Research Archive Show summary

  Background Diagnostic errors occur frequently, especially in the emergency room. Estimates about the consequences of diagnostic error vary widely and little is known about the factors predicting error. Our objectives thus was to determine the rate of discrepancy between diagnoses at hospital admission and discharge in patients presenting through the emergency room, the discrepancies’ consequences, and factors predicting them. Methods Prospective observational clinical study combined with a survey in a University-affiliated tertiary care hospital. Patients’ hospital discharge diagnosis was compared with the diagnosis at hospital admittance through the emergency room and classified as similar or discrepant according to a predefined scheme by two independent expert raters. Generalized linear mixed-effects models were used to estimate the effect of diagnostic discrepancy on mortality and length of hospital stay and to determine whether characteristics of patients, diagnosing physicians, and context predicted diagnostic discrepancy. Results 755 consecutive patients (322 [42.7%] female; mean age 65.14 years) were included. The discharge diagnosis differed substantially from the admittance diagnosis in 12.3% of cases. Diagnostic discrepancy was associated with a longer hospital stay (mean 10.29 vs. 6.90 days; Cohen’s d 0.47; 95% confidence interval 0.26 to 0.70; P = 0.002) and increased patient mortality (8 (8.60%) vs. 25(3.78%); OR 2.40; 95% CI 1.05 to 5.5 P = 0.038). A factor available at admittance that predicted diagnostic discrepancy was the diagnosing physician’s assessment that the patient presented atypically for the diagnosis assigned (OR 3.04; 95% CI 1.33–6.96; P = 0.009). Conclusions Diagnostic discrepancies are a relevant healthcare problem in patients admitted through the emergency room because they occur in every ninth patient and are associated with increased in-hospital mortality. Discrepancies are not readily predictable by fixed patient or physician characteristics; attention should focus on context.

• Karay, Yassin & Schauber, Stefan Kilian (2018). A validity argument for progress testing: Examining the relation between growth trajectories obtained by progress tests and national licensing examinations using a latent growth curve approach. Medical Teacher. ISSN 0142-159X. 40(11), p. 1123–1129. doi: 10.1080/0142159X.2018.1472370. Full text in Research Archive Show summary

  Background: Progress testing is a longitudinal assessment that aims at tracking students' development of knowledge. This approach is used in many medical schools internationally. Although progress tests are longitudinal in nature, and their focus and use of developmental aspects is a key advantage, individual students’ learning trajectories themselves play, to date, only a minor role in the use of the information obtained through progress testing. Methods: We investigate in how far between-person differences in initial levels of performance and within-person rate of growth can be regarded as distinct components of students’ development and analyze the extent to which these two components are related to performances on national licensing examinations using a latent growth curve model. Results: Both, higher initial levels of performances and steepness of growth are positively related to long-term outcomes as measured by performance on national licensing examinations. We interpret these findings as evidence for progress tests’ suitability to monitor students’ growth of knowledge across the course of medical training. Conclusions: This study indicates that individual development as obtained by formative progress tests is related to performance in high-stakes assessments. Future studies may put more focus on the use of between-persons differences in growth of knowledge.

• Schauber, Stefan Kilian; Hecht, Martin & Nouns, Zineb M. (2017). Why assessment in medical education needs a solid foundation in modern test theory. Advances in Health Sciences Education. ISSN 1382-4996. 23(1), p. 217–232. doi: 10.1007/s10459-017-9771-4. Full text in Research Archive Show summary

  Despite the frequent use of state-of-the-art psychometric models in the field of medical education, there is a growing body of literature that questions their usefulness in the assessment of medical competence. Essentially, a number of authors raised doubt about the appropriateness of psychometric models as a guiding framework to secure and refine current approaches to the assessment of medical competence. In addition, an intriguing phenomenon known as case specificity is specific to the controversy on the use of psychometric models for the assessment of medical competence. Broadly speaking, case specificity is the finding of instability of performances across clinical cases, tasks, or problems. As stability of performances is, generally speaking, a central assumption in psychometric models, case specificity may limit their applicability. This has probably fueled critiques of the field of psychometrics with a substantial amount of potential empirical evidence. This article aimed to explain the fundamental ideas employed in psychometric theory, and how they might be problematic in the context of assessing medical competence. We further aimed to show why and how some critiques do not hold for the field of psychometrics as a whole, but rather only for specific psychometric approaches. Hence, we highlight approaches that, from our perspective, seem to offer promising possibilities when applied in the assessment of medical competence. In conclusion, we advocate for a more differentiated view on psychometric models and their usage.

• Brunk, Irene; Schauber, Stefan Kilian & Georg, Waltraud (2017). Do they know too little? An inter-institutional study on the anatomical knowledge of upper-year medical students based on multiple choice questions of a progress test. Annals of Anatomy. ISSN 0940-9602. 209, p. 93–100. doi: 10.1016/j.aanat.2016.09.004. Full text in Research Archive Show summary

  The depth of medical students’ knowledge of human anatomy is often controversially discussed. In particular, members of surgical disciplines raise concerns regarding deficits in the factual anatomical and topographical knowledge of upper-year students. The question often raised is whether or not medical students have sufficient anatomical and topographical knowledge when they graduate from medical school. Indeed, this question is highly relevant for curricular planners. Therefore, we have addressed it by evaluating the performance of students in the 5th and 6th years of their studies on anatomical multiple choice questions from the Berlin Progress Test Medicine performed at 10 German university medical schools. Results were compared to a reference based on a standard setting (modified Angoff-procedure). The reference was established independently by 5 panels of anatomists at different universities across Germany. As the ratings were independent of university affiliation, teaching-experience or training of the anatomists, an overall cut off score could be calculated which corresponded to 60.4% correct answers for the question set used in this study. In the progress test, on average only 29.9% of the students’ answers were correct, reflecting that the performance was significantly below the expected standard. On the basis of the test results it remained unclear whether acquisition or retention of anatomical information was insufficient. Further evaluation by item characteristics revealed that the students had major difficulty in applying their theoretical knowledge to practical problems in the context of a clinical setting. Thus, our results reveal deficits in the anatomical knowledge of medical students in their final years. Therefore medical curricula should not only focus on enhancing the acquisition and retention of core anatomical knowledge, but aim at improving skills applying this in a clinical setting.

• Hautz, Stefanie C.; Schuler, Luca; Kämmer, Juliane E.; Schauber, Stefan Kilian; Ricklin, Meret E. & Sauter, Thomas C. [Show all 10 contributors for this article] (2016). Factors predicting a change in diagnosis in patients hospitalised through the emergency room: A prospective observational study. BMJ Open. 6:e011585(5). doi: 10.1136/bmjopen-2016-011585. Full text in Research Archive
• Schauber, Stefan Kilian; Hecht, Martin; Nouns, Zineb; Kuhlmey, Adelheid & Dettmer, Susanne (2015). The role of environmental and individual characteristics in the development of student achievement: a comparison between a traditional and a problem-based-Learning curriculum. Advances in Health Sciences Education. ISSN 1382-4996. 20, p. 1033–1052. doi: 10.1007/s10459-015-9584-2. Full text in Research Archive

View all works in NVA

• Schüttpelz-Brauns, Katrin; Matthes, Jan; Schauber, Stefan & Wagner-Menghin, Michaela (2026). The diversity of educational research in the health professions: More opportunities than challenges. GMS Journal for Medical Education. doi: 10.3205/zma001807. Full text in Research Archive
• Bache-Mathiesen, Lena Kristin; Schauber, Stefan & Breivik, Jarle (2024). MonKeyMed – et register for et bedre læringsmiljø. Full text in Research Archive
• Bache-Mathiesen, Lena Kristin; Schauber, Stefan & Breivik, Jarle (2024). The missing “I” in “Team”. Full text in Research Archive
• Breivik, Jarle; Schauber, Stefan & Bache-Mathiesen, Lena Kristin (2024). Kan opplæring i teamarbeid bedre medisinstudenters samarbeidskompetanse, læringsmiljø og akademisk prestasjon? Full text in Research Archive
• Zitzmann, Steffen; Machts, Nils; Hübner, Nicolas; Schauber, Stefan; Möller, Jens & Lindner, Christoph (2023). The Yet Underestimated Importance of Communicating Findings from Educational Trials to Teachers, Schools, School Authorities, or Policy Makers (Comment on Brady et al. (2023)). Educational Psychology Review. ISSN 1040-726X. doi: 10.1007/s10648-023-09776-8. Full text in Research Archive
• Schauber, Stefan; Holt, Tonje & Helland, Maren Sand (2022). Child Mental Health and Wellbeing During Covid-19: A Focus on Children’s Self-Report. Full text in Research Archive
• Schauber, Stefan; Holt, Tonje & Helland, Maren Sand (2022). Child and adolescent mental health during Covid-19: Results from a Norwegian longitudinal study. Full text in Research Archive
• Brinchmann, Ellen Irén; Røe-Indregård, Hanne; Karlsen, Jannicke; Schauber, Stefan & Hagtvet, Ragna Bente Eriksen (2021). Linguistic Qualities of Child and Teacher Contextualized and Decontextualized Talk. Full text in Research Archive
• Schauber, Stefan Kilian & Hecht, Martin (2020). Reply to Jiang et al. Medical Teacher. ISSN 0142-159X. doi: 10.1080/0142159X.2020.1834932. Full text in Research Archive
• Larsen, Linda; Hjetland, Hanne Næss & Schauber, Stefan Kilian (2018). The development of letter-name knowledge in Norwegian 4-6-year-old children. Full text in Research Archive

View all works in NVA