Limit Confirmation Bias, Be a Healthy Skeptic

"It's unbelievable how much you don't know about a game you have been playing your whole life."

- Mickey Mantle

---

Introduction

The scientific process serves as the bedrock to Sport Science and modern strength and conditioning. This was discussed in Challenging Dogma: The Scientific Method and Human Performance. It provides a systematic approach to inquiry, enabling practitioners to scrutinize applied hypotheses, gather evidence, and ultimately validate or invalidate assumptions about their interventions and measures. However, one persistent challenge that can undermine the integrity of applied project development and evidence-informed practice is confirmation bias—the tendency to favor information that confirms existing beliefs or hypotheses while neglecting or discrediting information that contradicts them.

In this article, we will explore how to effectively limit confirmation bias during queries by emphasizing the importance of skepticism, testing for discriminant relationships, and the requirement of honest reporting.

Understanding Confirmation Bias

Before we dive into strategies for limiting confirmation bias, it is essential to understand what it is and why it matters. Confirmation bias can occur at multiple stages of the scientific process, including when formulating hypotheses, in experimental/project design, during data collection and interpretation, as well as in the dissemination of findings. This cognitive bias can lead researchers and practitioners alike to unconsciously make decisions that align with their preconceptions. By ignoring or dismissing contradictory evidence the resultants ultimately become skewed.

For example, if a practitioner has a hypothesis that during road trips athlete readiness declines, they should design their collection processes to focus on multiple response variables that are likely to probe this hypothesis. They may however have constructed this hypothesis due to their own personal HRV trends when travelling. Because of this they isolate HRV as a global readiness measure and selectively report results that confirm their expectations or frame interpretations in a way that highlights supportive findings. All while downplaying the role of resting heart rate in pulse rate variability, a post-game late night meal, and the significance of other response variables in global readiness.

Such practices can lead to false conclusions and hinder the progress of effective interventions. To promote the meaningful advancement of sport science support, it is crucial to limit confirmation bias. Below are a few key strategies for achieving this.

Recognize the Importance of Skepticism

Skepticism in practice is the questioning of the validity and reliability of findings as well as the methodologies employed. Embracing skepticism forms a foundational element in the practitioner’s behaviour that plays a vital role in combating confirmation bias. Cultivating a skeptical mindset involves more than mere criticism; it requires practitioners to actively embrace inquiry over acceptance. This means asking challenging questions at every stage of the process.

Actively consider questions such as, “What evidence contradicts my hypothesis?” or “How might my personal biases influence my interpretation?”. These narratives compel you to think critically about assumptions and maintain a healthy amount of skepticism throughout investigations. Encouraging you to engage in thorough literature reviews, diverse perspectives, and seriously consider evidence that may contradict established beliefs.

Another effective way to foster this culture of skepticism is through peer review. Peer review remains a critical component in the scientific publishing process, wherein other experts in the field evaluate the quality and rigor of one’s work. This has a place in the applied setting as well and can be embedded in the culture of your staff. If it isn’t, it should be something that is sought after to maximize the efficacy of limiting confirmation bias. Reviewers should focus on evaluating the methodologies deployed, assess data interpretation, and request additional validation for conclusions drawn if necessary.

Being skeptical and searching for others perspectives can improve the recognition of bias in your work. Actively seeking feedback from peers who are likely to challenge your perspectives rather than those who simply confirm or conform is underestimated. Together this helps promote a more rigorous examination of findings and the methodologies used to reach evidence-informed practice.

Test for Discriminant Relationships

Another effective strategy for limiting confirmation bias involves actively seeking and testing for discriminant relationships. Where as convergent validity and data are measures that align with a proposed theory, discriminant validity and divergent data suggest the opposite. Inquiring into the alignment with other constructs provides critical insights and can prompt new questions regarding existing hypotheses or assumptions.

While we may primarily focus on confirming our hypotheses, it is equally important to embrace and investigate contradictory evidence. Including the assessment of related response variables that should not correlate and fit the primary construct or model being examined. For example, if practitioners are investigating the benefits of a new sleep intervention when travelling across time zones, they should also investigate conditions under which the intervention may not have the same effect. Further, if using a global perceived readiness score they should investigate the impact of the sleep intervention compared to other inputs (e.g. Nutritional status, soreness, mood, etc...). The aim being to create a more nuanced understanding of the primary focus that allows for richer interpretations.

With significant correlations found, strong convergent findings strengthen the construct or model being assessed. A discriminant finding that lacks correlates is supportive, However, if discriminant findings correlate an audible isn't always necessary. It could mean the reach of the intervention can be extended. Unexpected results can often lead to new lines of inquiry that deepen our understanding of a problem set not just breach it in support of the null.

Utilizing statistical techniques such as Pearson's r or other correlation coefficients, analysis of variance (ANOVA) or regression analysis options, will allow you to identify variables that influence outcomes and do not. These will uncover what could otherwise go unnoticed. Methodologies such as these allow practitioners to assess how various variables interact collectively—such as individually, group-to-group, or within group partial variance—and can provide valuable insights into dynamic environments. 

Figure 1: Definition of convergent and discriminant validity with a rudimentary demonstration of the role they have in determining construct validity.

Report Honestly

Strong ethical standards and honesty in reporting are paramount for filtering out confirmation bias. Transparency allows for independent validation of findings and mitigates the risk of misleading conclusions that can arise from selective reporting.

Complete data disclosure is a fundamental component of honest reporting. It should be viewed as a duty to disclose all relevant data, regardless of how it might affect conclusions. In practice, this means making transparent the full range of study outcomes—both positive and negative. When only supportive outcomes are shared while omitting contradictory findings, you fail to provide a holistic scope of the variables being measured. Leading to selective reporting impeding the progress of the scientific process and its impact on making informed decisions within your environment. 

Commitment to transparency will foster greater trust amongst personnel and also encourage other staff members to evaluate their methodologies critically. It cultivates an environment where all can learn from both failures and successes, further developing the greater understanding of the subject matter. 

Another key aspect of honest reporting revolves around open science practices, which can significantly enhance the integrity of practitioners. Open science promotes the sharing of research materials, data, and methodologies with the broader community. This practice discourages selective reporting and increases accountability to a project outline. By providing unrestricted access to methodologies being used you can be exposed to greater scrutiny. However, this supports the replication of findings alongside project evolution. Additionally, a data-sharing repository could also be interconnected to enhance transparency and accountability by providing platforms for staff to share datasets.

Conclusion

Confirmation bias is infectious and threatens the field of sport science and modern strength and conditioning. Open up social media and you will see in a matter of minutes practitioners sharing cherry picked findings. Skepticism, testing for discriminant relationships, and practicing honest reporting, can mitigate the urge. With the scientific process increasingly playing a critical role in informing decisions that impact game day choices, healthcare policy, technology use, and countless other areas, the demand for high-quality unbiased evidence is high.

To further nurture a mindset and culture of transparency, you need to develop an ethical framework that prioritizes honesty and integrity throughout the process. Establishing a clear set of ethical guidelines that promotes professional conduct, confidentiality, transparency, and accountability should be essential.

-

Disclaimer

Area 13 Training Systems, more specifically The Learning Ground, provides content for informational and educational purposes only which may contain copyrighted material. Although credit is always attempted to be given, such content available may not be specifically authorized by the copyright owner. A13 and TLG believes this constitutes fair use due to there being no known copyright or infringement intended. A13 and TLG encourages the exchange of said content to provide those interested with accessible research and educational information, so long as credit is appropriately given. Furthermore, A13 and TLG assumes no responsibility for any statements made or materials used by guest authors/presenters, which may not always represent our opinion. We also do not endorse any products or services that may be mentioned.