Towards the Development of Valid, Reliable, and Acceptable Tools for Assessing the Outcomes and Impacts of Health Research Partnerships
Date
2022-07-05
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Background
Researchers and their partners need access to high quality assessment tools to demonstrate the tangible outcomes and impacts of their work. As health research partnership approaches grow in popularity and mandated use, so too will the demand for tools to systematically measure their effects. This thesis surveyed literature spanning partnership traditions to locate and examine the globally available tools for assessing health research partnership outcomes and impacts. It systematically identified and assessed tools and tool characteristics (conceptual, psychometric, pragmatic), terminology, use of theories, models and frameworks, and examined the nature of outcomes and impacts arising from studies using tools with known characteristics.
Methods
We searched four electronic databases (Ovid MEDLINE, Embase, CINAHL+, PsychINFO) from inception to June 2021 without limits, and guided by an a priori protocol and search strategy. We retained studies involving a health research partnership, the development, use, and/or assessment of tools to evaluate partnership outcomes and impacts as a study aim, studies that reported empirical psychometric evidence (manuscript 2), and those with explicit conceptual foundations, empirical psychometric evidence, and pragmatic characteristics (manuscript 3). Study quality was assessed in both systematic reviews.
Results
Of 56123 total citations, we screened 36027 de-duplicated records, yielding 2784 full text records, ultimately retaining 169, 49, and 37 eligible studies, respectively. Studies were largely North America-centric, and published in English, after 2010. The tools we located were mostly bespoke (205); few had distinguishing conceptual, psychometric, and pragmatic features (58). The research revealed persistent measurement challenges across partnership traditions, including non-standard terminology, inconsistent reporting and use of term definitions, insufficient psychometric/pragmatic tools and a lack of quantitative methods and deliberate focus on tool development, testing, and use, among others. As compared with other historical reviews, very few of the identified studies and tools overlapped; yet the key messages regarding the need to evolve measurement and tool development were the same.
Conclusions
Dedicated, collaborative efforts among research partnership traditions are required to coordinate the advancement of partnership measurement and science, and to tackle complex outcomes and impacts measurement challenges in this rapidly expanding research field.
Description
Keywords
scoping review, systematic review, outcomes, impacts, health research partnerships, evaluation, psychometrics, pragmatics
Citation
Mrklas, K. J. (2022). Towards the development of valid, reliable, and acceptable tools for assessing the outcomes and impacts of health research partnerships (Doctoral thesis, University of Calgary, Calgary, Canada). Retrieved from https://prism.ucalgary.ca.