Appraise the Evidence: Putting it All Together to Critically Examine Published Evidence
- Critical Appraisal
- Steps to Appraise Evidence
Objectives:
- Describe the importance of critical appraisal of research
- Apply the guidelines of critical appraisal to published articles for the EBP Project
In this module, we are going to use all we have learned from the previous modules and start applying our knowledge to critically appraising (judging the quality and value) the articles we are reading. The purpose of doing this is so that we have the general knowledge about how to appraise (examine) a research study for quality as part of evidence-based practice.
Critical Appraisal of Research
As nursing students and soon-to-be Registered Nurses, it is vital to be a competent consumer of research. This means that you are required to have some introductory knowledge about research and appraising research. Learning how to read research studies is the one of the most valuable pieces of evidence-based practice.
It is a fundamental skill to be able to identify and appraise the best available evidence to integrate it with your own clinical experience and patients’ values. Nurses should appraise research before applying any published evidence to patients because the use of an intervention should be tried and studied before use to ensure safety and quality.
Why the importance?
- Nurses have an ethical obligation to provide the best practices so that patients, families, and communities receive the highest and safest level of care possible.
- Earlier, it was said that there are several reasons why research studies vary so much. Some studies may be early studies on a given topic. Since not much is known about the topic yet, a higher level of evidence, such as a meta-analysis is not possible. Only after so many research studies have been completed and disseminated, could a meta-analysis be done. It is not a poor reflection on the early research at all—it is just a topic in research infancy.
- The articles that you have found should be appraised for quality of research. Keep in mind there are many considerations. One is ethics. A research study of medications that lasts 2 years might be a bit stronger than a study that lasts 1 year, because it studied effects over a longer period. However, if serious unwanted side effects are occurring in a number of research subjects after 6 months, it would be unethical to continue the study and knowingly put humans in harm’s way! Therefore, a study may be stopped early, thus limiting the length. On the other hand, a study examining an experimental intervention can show such incredible promise that a study may be halted early in order to obtain Food and Drug Administration (FDA) approval as soon as possible so that as many patients as possible can benefit from the treatment.
- True experimental designs are considered quality research (Boswell & Cannon, 2020). However, not all studies can use a true experimental design. A true experimental design includes an intervention group and a control group, and randomization (Boswell & Cannon, 2020). Imagine a researcher wants to test a new nurse hand washing intervention that decreased hospital-acquired infections in a small pilot study. The researcher reviews the data from the pilot study and concludes that it would be unethical to not use the new hand washing intervention with all patients. The researcher decides to change the study design to use retrospective data to compare with the new hand washing intervention. The design may not be as strong, but it is more ethical. These ethical dilemmas must guide research with human subjects to uphold the highest quality of care possible for humans.
- Replicated or similar studies that have similar findings add to the confidence in the results of the study (Zadvinskis & Melnyk, 2019). When different researchers find similar findings in studies examining the same topic, this is another way to verify the findings.
- While most of us know not to believe everything we may read in a newspaper (or on Twitter), it’s also true that we cannot rely 100% on papers written in even the most prestigious academic journals. Different types of studies reported in the literature also have different strengths and weaknesses. Even if the contents of a research paper are reliable, it is sometimes difficult to find the specific information you are looking for and interpret it accurately.
Carrying out Critical Appraisal
Assessing the research methods used in research studies is a prime step in its critical appraisal. This is done using checklists which are specific to the study design.
Standard Common Questions for Beginning Appraisal:
- Why was the study done?
- A clear explanation of why the study was carried out.
- The brief background literature (in the introduction) presented in a study should identify the gap that this research was designed to fill.
- What is the research question?
- What is the study type (design)?
- What is the sample size? How was the sample selected?
- Are the measurements of major variables valid and reliable?
- What important potential confounders are considered?
- What is the statistical method used in the study?
- Statistical results.
- What conclusions did the authors reach about the research question?
- Are ethical issues considered?
Applying the Guidelines of Critical Appraisal to Published Articles for the EBP Project
In Appendix I3, there is a table for you to use when you are reading a quantitative article. This will make it much easier! But, first let’s go over a couple key components.
Credibility of Sources: The first step in appraising research is determining its credibility. If it is determined to be not credible, then stop there and move onto a different article.
- You should pay attention to the publisher and check for any conflicts of interest. Is the publisher a for-profit company that could benefit from the research of a product?
- Does the researcher have a conflict of interest (COI) related to the research? Is there a statement about COI?
- Non-peer reviewed research articles should be scrutinized for credibility.
- Websites that are not peer-reviewed should be checked for credibility and used sparingly, if at all.
There are different approaches pertaining to the two main research types, or paradigms of research.
- Quantitative research: The focus of critical appraisal is on validity and reliability.
- Qualitative research: The focus of critical appraisal is on rigor and trustworthiness.
The CRAAP test is a tool that can be used to determine the potential credibility of internet sources. As mentioned earlier in the course, Google Scholarly and the internet can be a great source of information, but be wary about the information out there. The CRAAP test is a way to help filter though some of the internet confusion of validity.
- C: Currency (timeliness)
- R: Relevancy
- A: Authority
- A: Accuracy
- P: Practice
The CRAAP Test is also posted as Appendix C2.
With each article you read, utilize the Critical Appraisal of Quantitative Research Table and Overall Critical Appraisal of Published Research Articles documents.
EBP Poster Application! Now is the time to really get down to the nitty gritty of appraising the articles you are finding to answer your clinical question. You are now armed with the basic knowledge needed to read published evidence.
References & Attribution
“Green check mark” by rawpixel licensed CC0.
Boswell, C. & Cannon, S. (2020). Introduction to nursing research: Incorporating evidence-based practice (5th ed.). Jones & Bartlett Learning.
Zadvinskis, I. M., & Melnyk, B. M. (2019). Making a case for replication studies and reproducibility to strengthen evidence-based practice. Worldviews on Evidence-Based Nursing, 16(1), 2–3. https://doi.org/10.1111/wvn.12349
