Background: The lack of an international standard for assessing and communicating health app quality and the lack of consensus about what makes a high-quality health app negatively affect the... Show moreBackground: The lack of an international standard for assessing and communicating health app quality and the lack of consensus about what makes a high-quality health app negatively affect the uptake of such apps. At the request of the European Commission, the international Standard Development Organizations (SDOs), European Committee for Standardization, International Organization for Standardization, and International Electrotechnical Commission have joined forces to develop a technical specification (TS) for assessing the quality and reliability of health and wellness apps.Objective: This study aimed to create a useful, globally applicable, trustworthy, and usable framework to assess health app quality.Methods: A 2-round Delphi technique with 83 experts from 6 continents (predominantly Europe) participating in one (n=42, 51%) or both (n=41, 49%) rounds was used to achieve consensus on a framework for assessing health app quality. Aims included identifying the maximum 100 requirement questions for the uptake of apps that do or do not qualify as medical devices. The draft assessment framework was built on 26 existing frameworks, the principles of stringent legislation, and input from 20 core experts. A follow-up survey with 28 respondents informed a scoring mechanism for the questions. After subsequent alignment with related standards, the quality assessment framework was tested and fine-tuned with manufacturers of 11 COVID-19 symptom apps. National mirror committees from the 52 countries that participated in the SDO technical committees were invited to comment on 4 working drafts and subsequently vote on the TS.Results: The final quality assessment framework includes 81 questions, 67 (83%) of which impact the scores of 4 overarching quality aspects. After testing with people with low health literacy, these aspects were phrased as "Healthy and safe," "Easy to use," "Secure data," and "Robust build." The scoring mechanism enables communication of the quality assessment results in a health app quality score and label, alongside a detailed report. Unstructured interviews with stakeholders revealed that evidence and third-party assessment are needed for health app uptake. The manufacturers considered the time needed to complete the assessment and gather evidence (2-4 days) acceptable. Publication of CEN-ISO/TS 82304-2:2021 Health software - Part 2: Health and wellness apps - Quality and reliability was approved in May 2021 in a nearly unanimous vote by 34 national SDOs, including 6 of the 10 most populous countries worldwide.Conclusions: A useful and usable international standard for health app quality assessment was developed. Its quality, approval rate, and early use provide proof of its potential to become the trusted, commonly used global framework. The framework will help manufacturers enhance and efficiently demonstrate the quality of health apps, consumers, and health care professionals to make informed decisions on health apps. It will also help insurers to make reimbursement decisions on health apps. Show less
Background: Working with eHealth requires health care organizations to make structural changes in the way they work. Organizational structure and process must be adjusted to provide high-quality... Show moreBackground: Working with eHealth requires health care organizations to make structural changes in the way they work. Organizational structure and process must be adjusted to provide high-quality care. This study is a follow-up study of a systematic literature review on optimally organizing hybrid health care (eHealth and face to face) using the Donabedian Structure-Process-Outcome (SPO) framework to translate the findings into a modus operandi for health care organizations.Objective: This study aimed to develop an SPO-based quality assessment model for organizing hybrid health care using an accompanying self-assessment questionnaire. Health care organizations can use this model and a questionnaire to manage and improve their hybrid health care.Methods: Concept mapping was used to enrich and validate evidence-based knowledge from a literature review using practice-based knowledge from experts. First, brainstorming was conducted. The participants listed all the factors that contributed to the effective organization of hybrid health care and the associated outcomes. Data from the brainstorming phase were combined with data from the literature study, and duplicates were removed. Next, the participants rated the factors on importance and measurability and grouped them into clusters. Finally, using multivariate statistical analysis (multidimensional scaling and hierarchical cluster analysis) and group interpretation, an SPO-based quality management model and an accompanying questionnaire were constructed.Results: All participants (n=39) were familiar with eHealth and were health care professionals, managers, researchers, patients, or eHealth suppliers. The brainstorming and literature review resulted in a list of 314 factors. After removing the duplicates, 78 factors remained. Using multivariate statistical analyses and group interpretations, a quality management model and questionnaire incorporating 8 clusters and 33 factors were developed. The 8 clusters included the following: Vision, strategy, and organization; Quality information technology infrastructure and systems; Quality eHealth application; Providing support to health care professionals; Skills, knowledge, and attitude of health care professionals; Attentiveness to the patient; Patient outcomes; and Learning system. The SPO categories were positioned as overarching themes to emphasize the interrelations between the clusters. Finally, a proposal was made to use the self-assessment questionnaire in practice, allowing measurement of the quality of each factor.Conclusions: The quality of hybrid care is determined by organizational, technological, process, and personal factors. The 33 most important factors were clustered in a quality management model and self-assessment questionnaire called the Hybrid Health Care Quality Assessment. The model visualizes the interrelations between the factors. Using a questionnaire, each factor can be assessed to determine how effectively it is organized and developed over time. Health care organizations can use the Hybrid Health Care Quality Assessment to identify improvement opportunities for solid and sustainable hybrid health care. Show less
Background: Working with eHealth requires health care organizations to make structural changes in the way they work. Organizational structure and process must be adjusted to provide high-quality... Show moreBackground: Working with eHealth requires health care organizations to make structural changes in the way they work. Organizational structure and process must be adjusted to provide high-quality care. This study is a follow-up study of a systematic literature review on optimally organizing hybrid health care (eHealth and face to face) using the Donabedian Structure-Process-Outcome (SPO) framework to translate the findings into a modus operandi for health care organizations.Objective: This study aimed to develop an SPO-based quality assessment model for organizing hybrid health care using an accompanying self-assessment questionnaire. Health care organizations can use this model and a questionnaire to manage and improve their hybrid health care.Methods: Concept mapping was used to enrich and validate evidence-based knowledge from a literature review using practice-based knowledge from experts. First, brainstorming was conducted. The participants listed all the factors that contributed to the effective organization of hybrid health care and the associated outcomes. Data from the brainstorming phase were combined with data from the literature study, and duplicates were removed. Next, the participants rated the factors on importance and measurability and grouped them into clusters. Finally, using multivariate statistical analysis (multidimensional scaling and hierarchical cluster analysis) and group interpretation, an SPO-based quality management model and an accompanying questionnaire were constructed.Results: All participants (n=39) were familiar with eHealth and were health care professionals, managers, researchers, patients, or eHealth suppliers. The brainstorming and literature review resulted in a list of 314 factors. After removing the duplicates, 78 factors remained. Using multivariate statistical analyses and group interpretations, a quality management model and questionnaire incorporating 8 clusters and 33 factors were developed. The 8 clusters included the following: Vision, strategy, and organization; Quality information technology infrastructure and systems; Quality eHealth application; Providing support to health care professionals; Skills, knowledge, and attitude of health care professionals; Attentiveness to the patient; Patient outcomes; and Learning system. The SPO categories were positioned as overarching themes to emphasize the interrelations between the clusters. Finally, a proposal was made to use the self-assessment questionnaire in practice, allowing measurement of the quality of each factor.Conclusions: The quality of hybrid care is determined by organizational, technological, process, and personal factors. The 33 most important factors were clustered in a quality management model and self-assessment questionnaire called the Hybrid Health Care Quality Assessment. The model visualizes the interrelations between the factors. Using a questionnaire, each factor can be assessed to determine how effectively it is organized and developed over time. Health care organizations can use the Hybrid Health Care Quality Assessment to identify improvement opportunities for solid and sustainable hybrid health care. Show less
Background: Working with eHealth requires health care organizations to make structural changes in the way they work. Organizational structure and process must be adjusted to provide high-quality... Show moreBackground: Working with eHealth requires health care organizations to make structural changes in the way they work. Organizational structure and process must be adjusted to provide high-quality care. This study is a follow-up study of a systematic literature review on optimally organizing hybrid health care (eHealth and face to face) using the Donabedian Structure-Process-Outcome (SPO) framework to translate the findings into a modus operandi for health care organizations.Objective: This study aimed to develop an SPO-based quality assessment model for organizing hybrid health care using an accompanying self-assessment questionnaire. Health care organizations can use this model and a questionnaire to manage and improve their hybrid health care.Methods: Concept mapping was used to enrich and validate evidence-based knowledge from a literature review using practice-based knowledge from experts. First, brainstorming was conducted. The participants listed all the factors that contributed to the effective organization of hybrid health care and the associated outcomes. Data from the brainstorming phase were combined with data from the literature study, and duplicates were removed. Next, the participants rated the factors on importance and measurability and grouped them into clusters. Finally, using multivariate statistical analysis (multidimensional scaling and hierarchical cluster analysis) and group interpretation, an SPO-based quality management model and an accompanying questionnaire were constructed.Results: All participants (n=39) were familiar with eHealth and were health care professionals, managers, researchers, patients, or eHealth suppliers. The brainstorming and literature review resulted in a list of 314 factors. After removing the duplicates, 78 factors remained. Using multivariate statistical analyses and group interpretations, a quality management model and questionnaire incorporating 8 clusters and 33 factors were developed. The 8 clusters included the following: Vision, strategy, and organization; Quality information technology infrastructure and systems; Quality eHealth application; Providing support to health care professionals; Skills, knowledge, and attitude of health care professionals; Attentiveness to the patient; Patient outcomes; and Learning system. The SPO categories were positioned as overarching themes to emphasize the interrelations between the clusters. Finally, a proposal was made to use the self-assessment questionnaire in practice, allowing measurement of the quality of each factor.Conclusions: The quality of hybrid care is determined by organizational, technological, process, and personal factors. The 33 most important factors were clustered in a quality management model and self-assessment questionnaire called the Hybrid Health Care Quality Assessment. The model visualizes the interrelations between the factors. Using a questionnaire, each factor can be assessed to determine how effectively it is organized and developed over time. Health care organizations can use the Hybrid Health Care Quality Assessment to identify improvement opportunities for solid and sustainable hybrid health care. Show less
Background: Working with eHealth requires health care organizations to make structural changes in the way they work. Organizational structure and process must be adjusted to provide high-quality... Show moreBackground: Working with eHealth requires health care organizations to make structural changes in the way they work. Organizational structure and process must be adjusted to provide high-quality care. This study is a follow-up study of a systematic literature review on optimally organizing hybrid health care (eHealth and face to face) using the Donabedian Structure-Process-Outcome (SPO) framework to translate the findings into a modus operandi for health care organizations.Objective: This study aimed to develop an SPO-based quality assessment model for organizing hybrid health care using an accompanying self-assessment questionnaire. Health care organizations can use this model and a questionnaire to manage and improve their hybrid health care.Methods: Concept mapping was used to enrich and validate evidence-based knowledge from a literature review using practice-based knowledge from experts. First, brainstorming was conducted. The participants listed all the factors that contributed to the effective organization of hybrid health care and the associated outcomes. Data from the brainstorming phase were combined with data from the literature study, and duplicates were removed. Next, the participants rated the factors on importance and measurability and grouped them into clusters. Finally, using multivariate statistical analysis (multidimensional scaling and hierarchical cluster analysis) and group interpretation, an SPO-based quality management model and an accompanying questionnaire were constructed.Results: All participants (n=39) were familiar with eHealth and were health care professionals, managers, researchers, patients, or eHealth suppliers. The brainstorming and literature review resulted in a list of 314 factors. After removing the duplicates, 78 factors remained. Using multivariate statistical analyses and group interpretations, a quality management model and questionnaire incorporating 8 clusters and 33 factors were developed. The 8 clusters included the following: Vision, strategy, and organization; Quality information technology infrastructure and systems; Quality eHealth application; Providing support to health care professionals; Skills, knowledge, and attitude of health care professionals; Attentiveness to the patient; Patient outcomes; and Learning system. The SPO categories were positioned as overarching themes to emphasize the interrelations between the clusters. Finally, a proposal was made to use the self-assessment questionnaire in practice, allowing measurement of the quality of each factor.Conclusions: The quality of hybrid care is determined by organizational, technological, process, and personal factors. The 33 most important factors were clustered in a quality management model and self-assessment questionnaire called the Hybrid Health Care Quality Assessment. The model visualizes the interrelations between the factors. Using a questionnaire, each factor can be assessed to determine how effectively it is organized and developed over time. Health care organizations can use the Hybrid Health Care Quality Assessment to identify improvement opportunities for solid and sustainable hybrid health care. Show less
Simple Summary Flow cytometry allows detailed characterization of large numbers of cells and plays an important role in the diagnosis of acute myeloid leukemia. To facilitate analysis of... Show moreSimple Summary Flow cytometry allows detailed characterization of large numbers of cells and plays an important role in the diagnosis of acute myeloid leukemia. To facilitate analysis of flowcytometric data, reference databases of normal bone marrow samples and samples from acute myeloid leukemia patients, together with new software tools, are required. We here report on the building of a large database of acute myeloid leukemia patients (n = 1142) and 22 normal samples. We report on the quality assessment procedure used and its validation, discuss potential pitfalls, and provide possible solutions for avoiding such flaws in the construction of other databases. Our data show that obtaining and collecting reproducible flow cytometric data over time and across centers is feasible, but also that strict quality assessment remains crucial, even when standardized protocols for staining and instrument settings are being used in a multicenter setting. Flowcytometric analysis allows for detailed identification and characterization of large numbers of cells in blood, bone marrow, and other body fluids and tissue samples and therefore contributes to the diagnostics of hematological malignancies. Novel data analysis tools allow for multidimensional analysis and comparison of patient samples with reference databases of normal, reactive, and/or leukemia/lymphoma patient samples. Building such reference databases requires strict quality assessment (QA) procedures. Here, we compiled a dataset and developed a QA methodology of the EuroFlow Acute Myeloid Leukemia (AML) database, based on the eight-color EuroFlow AML panel consisting of six different antibody combinations, including four backbone markers. In total, 1142 AML cases and 42 normal bone marrow samples were included in this analysis. QA was performed on 803 AML cases using multidimensional analysis of backbone markers, as well as tube-specific markers, and data were compared using classical analysis employing median and peak expression values. Validation of the QA procedure was performed by re-analysis of >300 cases and by running an independent cohort of 339 AML cases. Initial evaluation of the final cohort confirmed specific immunophenotypic patterns in AML subgroups; the dataset therefore can reliably be used for more detailed exploration of the immunophenotypic variability of AML. Our data show the potential pitfalls and provide possible solutions for constructing large flowcytometric databases. In addition, the provided approach may facilitate the building of other databases and thereby support the development of novel tools for (semi)automated QA and subsequent data analysis. Show less
Bank, P.C.D.; Jacobs, L.H.J.; Berg, S.A.A. van den; Deutekom, H.W.M. van; Hamann, D.; Molenkamp, R.; ... ; Oosterhuis, W.P. 2021
The in vitro diagnostic medical devices regulation (IVDR) will take effect in May 2022. This regulation has a large impact on both the manufacturers of in vitro diagnostic medical devices (IVD) and... Show moreThe in vitro diagnostic medical devices regulation (IVDR) will take effect in May 2022. This regulation has a large impact on both the manufacturers of in vitro diagnostic medical devices (IVD) and clinical laboratories. For clinical laboratories, the IVDR poses restrictions on the use of laboratory developed tests (LDTs). To provide a uniform interpretation of the IVDR for colleagues in clinical practice, the IVDR Task Force was created by the scientific societies of laboratory specialties in the Netherlands. A guidance document with explanations and interpretations of relevant passages of the IVDR was drafted to help laboratories prepare for the impact of this new legislation. Feedback from interested parties and stakeholders was collected and used to further improve the document. Here we would like to present our approach to our European colleagues and inform them about the impact of the IVDR and, importantlywewould like to present potentially useful approaches to fulfill the requirements of the IVDR for LDTs. Show less
