Effective healthcare relies heavily on accurate and reliable measurement instruments. The use of poorly designed or validated scales can lead to suboptimal patient care, inefficient resource allocation, and hinder scientific progress. This paper addresses the need for improved psychometric literacy among healthcare professionals, particularly those in fields without strong psychology components. Many clinicians lack the necessary training to critically evaluate and develop measurement instruments. This manuscript aims to bridge this gap by providing a clear and practical guide to psychometric principles and best practices for instrument development and selection. The increasing emphasis on evidence-based medicine necessitates a strong foundation in measurement science to ensure that clinical decisions are informed by robust data. Accurate quantification and categorization of observations are vital for clinical meaning-making, and strong measurement requires sound practices in instrument development and validation. This paper seeks to equip clinicians, educators, and researchers with the tools and knowledge to improve measurement quality in their respective fields.

The paper begins by reviewing existing literature on measurement in healthcare, distinguishing between 'pin-point' measurements (discrete categories like weight and age) and 'ballung' concepts (more complex, less clearly defined constructs like health-related quality of life). It highlights the challenges of measuring multidimensional health constructs and the importance of carefully considering what, when, where, and how to measure in the context of limited resources. The authors discuss the limitations of many existing health measurement instruments, citing a study by Marshall et al. (2000) that found a significant association between the use of unpublished instruments and inflated claims of treatment effects in clinical trials. This underscores the importance of using published and psychometrically sound instruments. The paper then introduces key psychometric concepts, including classical test theory (CTT) and item response theory (IRT), comparing and contrasting their strengths and limitations. It also describes the COSMIN initiative, an international framework for standardizing terminology, methodology, and reporting in health measurement. The COSMIN taxonomy categorizes measurement properties into three domains: reliability, validity, and responsiveness, further broken down into nine specific properties (content validity, structural validity, internal consistency, cross-cultural validity, measurement invariance, reliability, measurement error, criterion validity, and responsiveness).

The paper presents a ten-step process for developing and validating a new measurement instrument, illustrated with examples from the development of a visuoperceptual measure for videofluoroscopic swallow studies (VMV). Step 1: Identify existing instruments and assess their psychometric properties using the COSMIN criteria. Step 2: Retrieve and evaluate psychometric data from existing literature. Step 3: Compare retrieved data to pre-defined quality criteria to determine the need for a new instrument. Step 4: Define the construct to be measured, specifying the target population, stakeholders, and purpose of the instrument. Step 5: Generate an item pool using both deductive (literature review) and inductive (expert opinion, e-Delphi) approaches. Step 6: Develop appropriate response scales, matching the scale type to the item type and ensuring clarity and conciseness. Step 7: Conduct an expert review of the draft instrument, focusing on face validity and refining items based on feedback. Step 8: Pilot test the instrument in a small sample, gathering feedback on item comprehensibility, relevance, acceptability, and feasibility. Step 9: Reduce and revise items based on statistical analysis and user feedback, removing poorly fitting, unstable, or irrelevant items. Step 10: Trial the revised instrument in a larger sample to evaluate its psychometric properties, including reliability, validity, and responsiveness.

The paper emphasizes the importance of validity over reliability in instrument development, arguing that a reliable but invalid instrument is essentially useless. It highlights the widespread problem of poor measurement in healthcare, citing several studies that demonstrate the lack of adequate psychometric evidence for many commonly used instruments. The authors detail the COSMIN initiative and its role in improving the quality of health measurement research. The ten-step instrument development process is thoroughly explained, with examples drawn from the VMV's development, demonstrating the practical application of COSMIN guidelines. Specific statistical analyses (e.g., factor analysis, Cronbach's alpha, item response theory) are described, along with their interpretations. The VMV example showcases the iterative nature of instrument development, involving multiple rounds of refinement based on statistical analysis and expert/user feedback. Different types of validity (content, construct, criterion) and reliability (internal consistency, test-retest) are explained, along with concepts like sensitivity, specificity, minimal important change (MIC), and standard error of measurement (SEM).

The findings of this paper underscore the critical need for improved psychometric understanding and practices within the healthcare community. The widespread use of instruments lacking robust psychometric evidence compromises the validity of research findings and the effectiveness of clinical practice. The authors successfully demonstrate a practical approach to instrument development, guided by the COSMIN initiative and emphasizing the iterative nature of the process. The detailed, step-by-step guide provided in this manuscript serves as a valuable resource for clinicians, educators, and researchers seeking to improve measurement quality. The VMV example effectively illustrates how rigorous methodology, coupled with expert feedback and statistical analysis, can lead to the development of a psychometrically sound instrument. The paper successfully bridges the gap between theoretical psychometric principles and their practical application in a healthcare context.

This paper makes significant contributions by providing a practical and accessible guide to psychometric principles and instrument development for healthcare professionals. The ten-step process, illustrated by the VMV example, offers a clear roadmap for creating robust and reliable measurement instruments. The emphasis on the COSMIN initiative’s resources and guidelines empowers clinicians and researchers to enhance measurement quality in their work. Future research could focus on broader applications of this framework across various healthcare settings and populations, further validating its effectiveness and identifying areas for potential refinement.

The paper focuses primarily on the development of new instruments and might not fully address the challenges of adapting or modifying existing instruments for specific contexts. While the VMV serves as a useful example, it may not be fully generalizable to all types of measurement instruments or healthcare settings. The paper primarily utilizes examples from a single instrument development project, which might limit the breadth of its applicability to diverse clinical scenarios.