The pyramid of evidence, remodeling the hockey skate, and the fringe of human performance

Alexander Morgan
Jan 20
8 min read

"True ignorance is not the absence of knowledge, but the refusal to acquire it"

- Karl Popper, PhD

Introduction

We are taught early, and often repeatedly, that not all evidence is created equal. Somewhere in our undergraduate research methods course we are introduced to the pyramid of evidence, a neat, hierarchical structure that tells us what to trust, what to treat with caution, and what to question. At the apex sit systematic reviews and meta-analyses. At the base, anecdote and expert opinion. The implicit lesson is as simple as climb upward for stronger evidence.

It has protected our field from fads, from charismatic nonsense, and from the seductive certainty of a single impressive case. Yet, that rigid allegiance to the pyramid can stifle innovation, delay useful experimentation on the ground, and create a disconnect between science and practice when novelty is required or unique circumstances are at play.

To understand why, we need to walk carefully through each level of the pyramid, not just as an abstract academic construct, but as it functions in applied human performance environments. Only then can we meaningfully discuss why, in certain contexts, flipping the hierarchy on its head is not heretical, but necessary.

Figure 1: Hierarchy of evidence, traditionally higher is considered stronger. However, consider "good science" can not always prove rather it can be judged on it's falsifiability.

The Base: Anecdote, Experience, and Observation

At the bottom of the pyramid lies anecdotal evidence which encompasses individual observations, practitioner experience, athlete feedback, and informal trial-and-error. This is the level most often dismissed as clinically worthwhile in academic circles, and not without reason. Anecdotes are vulnerable to bias, confounding variables, placebo effects, and selective memory. A practitioner’s belief that a certain exercise “always works” may reflect nothing more than coincidence or confirmation bias. However, it also must be respected that this is where almost all ideas originate.

A strength coach notices that athletes report less soreness when eccentric volume is auto-regulated. A physiotherapist observes that a nontraditional return-to-run progression seems to reduce aggravation in a particular population. An athlete experiments with breathing cadence during intervals and feels measurably more composed late in sessions. None of these observations are controlled. None are blinded. None are randomized. Though they are the first signal that something is worth exploring.

Importantly, anecdote in sport is rarely singular. When similar observations arise independently across teams, practitioners, or contexts, they form a pattern—weak evidence by strict standards, but not meaningless. In practice, it’s the soil from which all stronger evidence grows.

Case Reports and Case Series: The Structured Story

One step up the pyramid, we encounter case reports and case series. Here, anecdote begins to take shape. Observations are documented systematically. Context is described. Outcomes are measured even if imperfectly. A single athlete’s response to a novel rehabilitation protocol or an unconventional training intervention is detailed with enough transparency that others can scrutinize it.

In sport science, case studies are often undervalued because they lack generalizability. A case report on an Olympic sprinter tells us very little about recreational runners. Elite sport is, by definition, a world of outliers. When working with athletes at the extremes of human performance, the average response may be less relevant than the exceptional one.

Case studies also serve a critical ethical role. They allow us to explore new ideas without exposing large populations to unknown risk. Many load management strategies, recovery modalities, and individualized training approaches entered the literature first as careful case reports before ever being challenged at scale. From an innovation standpoint, this level is where hypotheses start to sharpen. We move from “this seemed to work” to “this worked under these conditions, with these constraints, and produced these changes.”

Observational Studies: Patterns Without Control

Next are observational studies such as cross-sectional analyses, cohort studies, or even retrospective data mining. These designs do not manipulate variables, they observe what already exists. In sport science, this might involve tracking injury rates across seasons with different training loads, examining associations between strength metrics and performance outcomes, or analyzing historical GPS data to identify risk thresholds.

Observational studies are powerful because they allow scale. Hundreds or thousands of athlete-seasons can be analyzed, revealing patterns invisible at the case level. They cannot be interpreted causally. Correlation, as every student learns is not causation, but in applied settings, correlations often guide decision-making long before causation is established. For example, the early workload-injury literature did not prove that certain deviations in acute-to-chronic ratios caused injury, but it did reveal consistent associations that practitioners could not ignore. These studies informed monitoring practices and risk conversations years before randomized trials could feasibly be conducted. Shaping how informed decision making can occur in an applied setting.

Observational evidence is often where applied human performance feels most comfortable: data-rich, contextually relevant, but still grounded in real-world complexity rather than the purity of a laboratory.

Randomized Controlled Trials: The Gold Standard—With Caveats

Randomized controlled trials (RCTs) are traditionally presented as the gold standard of evidence. By randomly assigning participants to interventions and controlling confounding variables, RCTs aim to isolate causal effects. In many domains, such as pharmacology and clinical medicine, this approach is indispensable. In sport science, RCTs are both invaluable and deeply constrained.

True randomization is difficult when dealing with teams, seasons, and competitive priorities. Blinding can be impossible and/or disadvantageous. Sample sizes are small, especially in elite populations. Interventions must fit within real training schedules, not idealized protocols. As a result, many RCTs in sport science are conducted on sub-elite, recreational, or student populations, raising questions about external validity or reliability.

This does not diminish their value. RCTs are essential for testing mechanisms, refining dose-response relationships, and ruling out alternative explanations. But in practice, they often lag behind innovation rather than lead it. By the time an RCT confirms an approach, practitioners may have already been using and iterating it.

Systematic Reviews and Meta-Analyses: The Summit of Consensus

At the top of the pyramid sit systematic reviews and meta-analyses, synthesizing results across multiple studies to provide the clearest possible summary of existing evidence. When well conducted, these analyses are invaluable. They reduce noise, identify robust effects, and expose gaps in the literature with specified inclusion criteria.

However, they are inherently backward-looking. A meta-analysis cannot evaluate what has not yet been studied. Novel interventions, emerging technologies, and unconventional approaches are invisible at this level until sufficient primary research accumulates. In competition, waiting for a meta-analysis before acting often means being years behind effective coaching methodologies or even conceding competitive advantage.

Figure 2: Hockey skate with lateral and medial malleoli hinges as a pivot point to encourage dorsiflexion.

A Funny Story About That Time I Tried to Rebuild the Hockey Skate

There was a point in time when I was working with a wearable surface electromyography (sEMG) company, two NHL skills coaches, and indirectly a hockey equipment brand. All primarily with semi-professional to professional hockey players, entering the research and development space, I was eager to improve on-ice speed. The pinnacle of my curiosity was reading a study by Kaartinen and colleagues (2021) when working alongside a group of practitioners from Finland. They found no associations between sagittal knee and hip kinematics and skating speed. A fascinating finding that essentially means there are multiple paths to the same outcome^1,2.

This mirrored what the data was saying for the players we were working with. It also was why we were beginning to place players into hybrid, hip or ankle dominant profiles. Subsequently impacting how the skills coaches were cueing them technically and some aspects of S&C programming. The data was guiding the skill coaches away from funneling every player into a long squatty stride. It was also demanding profile-specific emphasis on strength, fast stretch shortening cycle, or slow(er) SSC development in their off-ice program. Together and paired with other force profiling approaches (e.g. dynamic strength index), on-ice speed began to improve and three high profile cases were a success. We knew we were on to something. Later this something turned out to be peak dorsiflexion angle during the preload to begin the propulsion phase of the skating cycle.

This finding was a commonality between the profiles. Where stride length and frequency were varied, but the correlate was resilient. One day driving to the rink I saw a lady rollerblading and an idea rushed over me to remodel the hockey skate boot. If we can have a medial and lateral pivot point like classic rollerblades, we can then increase peak preload dorsiflexion and contribute to our pursuit of improving on-ice speed. Working with equipment company’s engineers, we attempted to create a pilot model to test the concept and deem it “ice worthy”.

Do you see any skates with medial and lateral malleoli hinges in the NHL? End of story. Unfortunately, the idea was deemed to be difficult to mold or print with integrity and market (style points matter, Reebok pumps and the Bauer Nexus ADV came and went). As a byproduct however, the findings can be witnessed in naive action to this day in rinks where players are encouraged more and more to skate based on their profile while sustaining the non-negotiables; stay on their forefoot, increase stride frequency or length. A popularized realization now that pays dividends with hockey being a game of accelerations, decelerations, and centre of mass manipulation.

Figure 3: sEMG results from Kaartinen and colleagues (2025) that demonstrate the diversity of muscle activation patterns within a common trend².

Reconciling Science and Practice: Why the Pyramid Works and When It Doesn’t

Novel equipment, emerging support to specific subpopulations, changes to rules and regulations, or unprecedented strategic change create scenarios where strong evidence simply may not exist. This is a feature of working at the edge of human performance. It begins with a problem that existing evidence does not adequately solve.

The real world is anything but controlled. Much of the discomfort with empiricism arises not from skepticism about science but from a recognition of its limits when applied dogmatically in complex, adaptive systems like sport. Here we encounter small, heterogeneous populations, highly individual responses, and multifaceted outcomes shaped by psychology, environment, and culture. The prioritization of only what can be rigorously validated at scale puts you at risk of overlooking the vast terrain of uncertainty and practical insight that drives real-world innovation.

Good practitioners flourish in the tension between knowledge and ignorance. Many of the most pressing questions in sport science or strength and conditioning do not come from the literature but emerge directly from practice: Why do some athletes defy the models that predict their performance? Why do recovery markers fail to explain subjective fatigue? Why do injury reduction strategies effect inconsistently across populations? These questions reflect complexities and anomalies that traditional hierarchies of evidence are ill-equipped to address.

Innovation begins with the problems and paradoxes coaches and athletes experience on the ground, moves through iterative, context-rich trials, and eventually informs more formalized, scalable research. This approach respects the scientific process but does not allow it to become a straightjacket that stifles curiosity or discounts the unknown. The role of empirical validation remains vital but follows, rather than precedes, the creative exploration that comes from lived experience. It shifts us away from a linear march toward certainty and toward a more cyclical, reflective process that welcomes ambiguity as part of discovery.

References

Kaartinen, S., Venojärvi, M., Lesch, KJ., Tikkanen, H., Vartiainen, P., & Stenroth, L. (2024). Lower limb muscle activation patterns in ice-hockey skating and associations with skating speed. Sports Biomech, 23(11) :2233-2248. https://doi.org/10.1080/14763141.2021.2014551.
Kaartinen, S., Vartiainen, P., Venojärvi, M., Tikkanen, H., & Stenroth, L. (2025). Kinematic and muscle activity patterns of maximal ice hockey skating acceleration. International Journal of Performance Analysis in Sport, 1–16. https://doi.org/10.1080/24748668.2025.2478698.

Disclaimer

Area 13 Training Systems, more specifically The Learning Ground, provides content for informational and educational purposes only which may contain copyrighted material. Although credit is always attempted to be given, such content available may not be specifically authorized by the copyright owner. A13 and TLG believes this constitutes fair use due to there being no known copyright or infringement intended. A13 and TLG encourages the exchange of said content to provide those interested with accessible research and educational information, so long as credit is appropriately given. Furthermore, A13 and TLG assumes no responsibility for any statements made or materials used by guest authors/presenters, which may not always represent our opinion. We also do not endorse any products or services that may be mentioned.