COVID-19 has changed many things about the world. These experiences have convinced me of the importance of a basic statistical literacy for all citizens of the world. While there doesn’t appear to be a consensus definition for statistical literacy (Ziegler & Garfield, 2018), the definition that I mean here is something close to Wallman’s (1993) definition of an ability to understand and critically evaluate statistical information. I would argue that “statistical” isn’t an attribute of any information, but rather, it is an attribute of the way in which one processes and views any information. This is akin to Goodwin’s (1994) professional vision and practices of seeing frameworks.

So, how does a statistician view information related to COVID-19? I’ll present two examples of things that my colleagues and I saw in news stories (and knew that each other understood) that perhaps are not generally immediately obvious.

In April, Madagascar launched a herbal remedy for COVID-19. According to an article by the BBC, the Malagasy president’s communications director said, “The majority of people who used the product and don’t have a chronic illness recovered completely … as long as it’s working, we don’t need clinical trials.” What I immediately think of is the counterfactual – would those same people who used the product and recovered have recovered if they didn’t use the product? The quote has two clues – “the majority of people” implies that there are some who used the product and didn’t recover, and “don’t have a chronic illness” implies that this is describing individuals without comorbidities. According to a study by Ioannidis et al. (2020), which has not been peer-reviewed, “deaths for people <65 years without underlying predisposing conditions are remarkably uncommon.” Therefore, if the patients in Madagascar who did not have a chronic illness would have recovered even if they didn’t drink the herbal remedy, did the herbal remedy really do anything or have any effect? Even if the herbal remedy wasn’t working those patients would likely have gotten better. Therefore, this evidence (that most people without chronic illness who used the product recovered) does not support the claim (that the product works).

Around that same time, one of the first studies of Remdesivir was published. Wang et al. (2020) found, in a study where 158 patients were given Remdesivir against 79 given a placebo, that Remdesivir did not appear to improve time to recovery in patients. Shortly thereafter, Beigel et al. (2020) found, in a study with approximately 500 patients receiving Remdesivir and another 500 receiving a placebo, that Remdesivir did appear to slightly improve time to recovery. Which study is correct? Well, they both are. Each individual is unique, the way the disease progresses in each individual is unique, and the way Remdesivir affects each individual is unique. If we did ten studies with 100 patients, but 100 different patients each time, we wouldn’t get exactly the same results each time because the individuals in our study are different – this is called sampling variability. So if both these studies are correct, how do we combine their information? We can use a technique called meta-analysis. Simply, it seems that even if Remdesivir helps, it only helps a little bit. That’s still better than nothing, but as Beigel et al. (2020) note, “it is clear that treatment with [Remdesivir] alone is not likely to be sufficient.”

In both cases, what we statisticians saw were two clearly viable alternate explanations explaining observed phenomenon. Note that we didn’t react by saying the treatment doesn’t work, or that one of the studies is wrong – we acknowledged alternate competing theories being plausible, and therefore, to say a treatment works, we must first rule out these other competing theories – that there only appears to be a difference because of sampling variability, or that even if the treatment didn’t work we still would have seen this effect occur. These two alternate explanations are the core of our introductory statistics classes.

I hope all my students, and all individuals worldwide, process information by the following maxims:

• Accept uncertainty – it is ubiquitous, and statistics helps us to quantify our uncertainty.
• Acknowledge variability – particularly in the form of sampling variability, which casts uncertainty upon our estimates.
• Ask the counterfactual – in order to say that one thing caused something to happen, you should also demonstrate that what happened would not have happened without that thing.

References:

Beigel, J. H., Tomashek, K. M., Dodd, L. E., Mehta, A. K., Zingman, B. S., Kalil, A. C., … & Lopez de Castilla, D. (2020). Remdesivir for the treatment of Covid-19—preliminary report. New England Journal of Medicine.

Goodwin, C. (1994). Professional Vision. American Anthropologist, 96(3), 606-633.

Ioannidis, J. P., Axfors, C., & Contopoulos-Ioannidis, D. G. (2020). Population-level COVID-19 mortality risk for non-elderly individuals overall and for non-elderly individuals without underlying diseases in pandemic epicenters. medRxiv.

Wallman, K., K. (1993). Enhancing statistical literacy: Enriching our society. Journal of
the American Statistical Association, 88(421), 1–8.

Wang, Y., Zhang, D., Du, G., Du, R., Zhao, J., Jin, Y., … & Hu, Y. (2020). Remdesivir in adults with severe COVID-19: a randomised, double-blind, placebo-controlled, multicentre trial. The Lancet.

Ziegler, L., & Garfield, J. (2018). Developing a statistical literacy assessment for the modern introductory statistics course. Statistics Education Research Journal, 17(2), 161-178.

I recently attended a conference where one of the plenary sessions included a discussion of the p-value controversy, and provided suggestions on what we should be teaching and using instead.

When I first spoke about the p-value controversy on this blog (Rao, 2019), I suggested that an understanding of the history of hypothesis testing may help clear students’ confusion. To Fisher, p-values were a measure of likelihood that a current hypothesis or theory could explain observed phenomenon. Only when a hypothesis was so unlikely to be a sufficient explanation of a phenomenon could it be ruled out or rejected. It was with this lens that Fisher described p-values less than 0.05 as significant, choosing 0.05 as a convenient threshold (Fisher, 1925).

Wasserstein, Schirm, and Lazar (2019) promote many alternate statistics, one of which is the effect size. They advise that careful consideration be made to determine what a meaningful effect size would be for each individual study.

However, I am concerned that we are setting ourselves up for an effect-size controversy in the future, similar to the p-value controversy. Cohen’s d (Cohen, 1988) is one of the most famous measures of effect size, and comes with convenient thresholds for small, medium, and large effects. Sullivan & Feinn (2012) summarize many different measures of effect size, and for all, include thresholds for determining the size of the effect.

If 0.2 is a small effect, and 0.5 is a medium effect, what is 0.35? Is 0.1 still a small effect, or is it no effect? What about 0.05?

In my mind, these alternate procedures do not solve the problem at the root of the p-value controversy, it just re-directs it. I still don’t know of any perfect solution, but I believe Wasserstein, Schirm, and Lazar’s (2019) recommendation to be thoughtful is what we must seek to impart on our students.

My favourite summary of recommendations thus far is one by Dr Andrew Zieffler (citation listed below). I plan on requiring my students to review these slides after reading the Biau, Jolles, & Porcher (2010) article, in an attempt to help set the stage for thoughtful interaction with statistical tools.

References and further reading:

Biau, D. J., Jolles, B. M., & Porcher, R. (2010). P value and the theory of hypothesis testing: an explanation for new researchers. Clinical Orthopaedics and Related Research®, 468(3), 885-892.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum Associates.

Fisher, R.A. (1925). Statistical methods for research workers. London: Oliver and Boyd.

Rao, V.N.V. (2019, March 23). The p-value controversy [Blog post]. Retrieved from https://statisticaljourneys.home.blog/2019/03/23/the-p-value-controversy/

Sullivan, G. M., & Feinn, R. (2012). Using effect size—or why the P value is not enough. Journal of graduate medical education, 4(3), 279-282.

Wasserstein, R. L., Schirm, A. L., & Lazar, N. A. (2019). Moving to a World Beyond “p< 0.05”. The American Statistician, 73 (S1), 1-19.

Zieffler, A. (2019). Deprecating statistical significance: Toward better science [Lecture slides]. Retrieved from: http://www.datadreaming.org/post/2019-04-26-slhs-prosem/.

A common adage recited by many statistics instructors is that correlation is not causation. I am concerned that many students hear the adage, misinterpret it, and leave thinking that correlation, and by extension all of statistics, is useless in the real world.

A purer wording is that correlation does not imply causation, and is important to state because causation implies correlation. My guess is that we don’t say this in classrooms because we believe students don’t yet understand what imply means.

I believe a better way to translate this would be to say Correlation may be causation, but it also may not be. So, what can be done to ensure correlation is a sign of causation?

One answer lies in introducing students to causal inference (Neyman, 1923; Rubin, 1974). Causal inference fundamentally differs from traditional statistical inference. While statistical inference is limited to inferring relationships that exist under the specific conditions through which data were collected, causal inference explicitly attempts to infer relationships amidst changing conditions (Pearl, 2009).

Causal inference is still gaining early traction among the statistics education community. Often times, teachers are unfamiliar with it, and thus hesitate to teach it. I believe that teaching students counterfactual reasoning through causal inference is an important skill to teach as part of teaching statistical literacy (Gal, 2002).

One key visual tool in causal inference is the exploration of relationships between variables using a directed acyclic graph (DAG).

For example, we may think that a person’s weight affects to their blood cholesterol level.

But perhaps we realize that diet may also affect both characteristics.

Here, diet is explicitly acknowledged as a confounding variable. It must be considered in any research design attempting to speak to the relationship between weight and cholesterol level. Furthermore, we have a clear visual to help students understand why weight and cholesterol may be correlated, but may not have a causal relationship.

Unfortunately, I have not yet found a simple explanation I feel is appropriate for introductory students. This may be a future project of mine. In the meantime, I personally enjoyed reading about the history of causal inference in a paper by Freedman (1999), the first few sections of which may be accessible and informative.

References and further reading:

Freedman, D. (1999). From association to causation: some remarks on the history of statistics. Journal de la société française de statistique, 140(3), 5-32.

Neyman, J. 1923 [1990]. “On the Application of Probability Theory to Agricultural Experiments. Essay on Principles. Section 9.” Statistical Science, 5 (4), 465–472. Trans. Dorota M. Dabrowska and Terence P. Speed.

Pearl, J. (2009). Causal inference in statistics: An overview. Statistics surveys, 3, 96-146.

Rubin, D. B. (1974). Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies. Journal of Educational Psychology, 66, 688-701 .

My favourite pixar movie is Ratatouille. Chef Gusteau’s motto is “tout le monde peut cuisiner”, or, “anyone can cook”. Later in the movie, Anton Ego, voiced by Peter O’Toole, writes “Not everyone can become a great artist, but a great artist can come from anywhere.”

Similarly, I believe that anyone can do statistics, or, that not everyone can become a statistician, but a statistician can come from anywhere.

I recently attended a talk where a prominent statistician and data analyst told the story of how she came into the field. It was entirely due to her instructors explicitly encouraging her and telling her that she has potential in the field.

While not all of our students will become statisticians, or major in statistics, or even master the material we attempt to teach them, we must encourage them, keep an open mind, and instill in them a positive identity as statisticians or data scientists/analysts (Cobb & Hodge, 2002). Our students will never believe in themselves if we do not believe in them first.

I came home from the talk and sent an email to my students from last semester to encourage them to consider furthering their skills, to explicitly state that I believe they have potential as data analysts, and to attempt to instill in them an identify as a statistician.

My message to them: Not only can you do statistics, you have done statistics. You are a statistician.

References and further reading:

Cobb, P., & Hodge, L. (2002). Learning, identity, and statistical data analysis. In Sixth International Conference on Teaching Statistics (ICOTS6), Cape Town, South Africa.

Lewis, B. (Producer), & Bird, B. (Director). (2007). Ratatouille [Motion Picture]. United States: Pixar.

On this blog’s homepage I state “I’ve been in love with numbers for as long as I can remember.” Even when I was a toddler I never wanted to practice reciting the alphabet – I preferred reciting numbers.

Yet, one number has always stood above the rest. It is my favourite number – 7.

Origins

I think I decided that 7 ought to be my favourite number when I was relatively young. I was 7 years old when my sister was born (technically, 6yrs 10mos 23days 17hrs and 39mins, and yes I did do that calculation the day she was born itself). 7 was the first jersey number I had for soccer. 7 is the sum of the digits of my birth date.

However, I remember recognizing quite early on, no later than the age of 8, that the reciprocal of 7 was the most interesting reciprocal of all of the numbers less than 12.

• 1/2 = 0.5
• 1/3 = 0.333333…
• 1/4 = 0.25
• 1/5 = 0.2
• 1/6 = 0.166666…
• 1/7 = 0.142857142857….
• 1/8 = 0.125
• 1/9 = 0.1111111…
• 1/10 = 0.1
• 1/11 = 0.090909…
• 1/12 = 0.083333…

The reciprocals of 2, 4, 5, 8, and 10 all have finite decimal expansions. The reciprocals of 3, 6, 9, and 12 all end with a single digit repeating ad infinitum while the reciprocal of 11 ends with a repeating two-digit sequence. Yet 1/7 was in a class of its own. I didn’t know why 7 should have such a unique decimal expansion at the time, but I was captivated by it.

A growing fancy

A few years later I realized that the pattern went deeper. Comparing 1/3 (0.333…) to 2/3 (0.666…), the decimal expansions have the same form, in that it is a single digit that repeats, but the digit that repeats is different. This holds for the fractions of 6, 9, 11, and 12 as well. However, the fractions of 7 do something entirely different:

• 1/7 = 0.142857142857…
• 2/7 = 0.2857142857…
• 3/7 = 0.42857142857…
• 4/7 = 0.57142857…
• 5/7 = 0.7142857…
• 6/7 = 0.857142857…

I’ve aligned the decimal expansions to help identify the pattern. Each of the fractions has the same six numbers in the same order, just with a different starting point! 7 was simply outlapping the other numbers in terms of mystique. Why were the same numbers repeating for each fraction, and why were they in the same order?

I began my search for other numbers that had this pattern, but, only using pen and paper, or calculators that only displayed 8 to 10 digits, proved limiting in my search.

Getting serious with number theory

I held on to this intrigue with the number 7 into college, and approached the professor of my number theory class with the question ‘why does 1/7 have such unique patterns?’. He then showed me a whole new dimension to the number 7. It was as if I was peering through Lewis Carroll’s looking glass into a hitherto unknown world of exotic beauty.

He explained that the pattern occurred because we operate in the numeric base of 10, and 10 is a primitive root modulus 7. That means that a series of 6 9’s in a row is divisible by 7, (important since 6 = 7-1) i.e. 999999/7 is an integer, and no other smaller series of 9’s divided by 7 is so (9/7 is not an integer, 99/7 is not an integer, etc.). Any prime number that has this property is called a full repetend prime, and all full repetend primes exhibit the same properties that I described above for the number 7. The first five full repetend primes are 7, 17 (meaning that a series of 16 9’s in a row is the smallest series of 9’s that is divisible by 17), 19, 23, and 29.

He then showed me the property of 9’s, also called Midy’s Theorem. If we recall the repetend of 1/7, i.e. the part that repeats, or 142857:

• 1+4+2+8+5+7 will be divisible by 9 (it equals 3*9)
• 14+28+57 will be divisible by 99 (it equals 99)
• 142+857 will equal 999

Similarly, for the repetend of 1/17, 0588235294117647:

• 0+5+8+8+2+3+5+2+9+4+1+1+7+6+4+7 will be divisible by 9 (it equals 8*9)
• 05+88+23+52+94+11+76+47 will be divisible by 99 (it equals 4*99)
• 0588+2352+9411+7647 will be divisible by 9999 (it equals 2*9999)
• 05882352+94117647 will equal 99999999

All full repetend primes have this property. We went on to discuss many other things, including discrete logarithms and other properties of cyclic numbers and prime reciprocals, and I went on to discover and play with subclasses of the full repetend primes, but I never lost any love or interest for my favourite number, 7.

I’m often asked by colleagues and students, “what makes a good data visualization?”. I believe data visualizations (and any data analysis) are story telling tools. As such, I have two criteria when I create or evaluate them:

• (1) The visualization should only have one main pattern it tries to convey. There is a tendency to create very complex and multi-faceted visualizations. However, stories are told one plot point at a time. Similarly, visualizations should each focus on one pattern at a time within a larger narrative.
• (2) Ignoring all labels and text, the pattern should be immediately noticeable. The value of the visualization is in serving as an aide to highlight an otherwise obscure pattern. Thus, a good aide should make the pattern painstakingly obvious and universally evident.

This may seem limiting, but I believe that even complex patterns can be elicited from good data visualizations within a few seconds. TED talks by Hans Rosling and David McCandless excellently demonstrate this power.

One of my favourite examples, and one of the most famous early examples of a data visualization, is by Florence Nightingale (read more here). Her visualization, printed below, helped showcase the need for hospital sanitation.

Let’s examine Florence Nightingale’s graph under my two criteria:

Ignoring all text, I immediately see a lot of blue. That blue area represents the number of deaths from preventable or mitigable diseases. This indeed is the pattern Florence Nightingale wished to highlight – a lot of soldiers were dying unnecessarily. Thus, this is an effective visualization of the data and helps tell a story.

Although we could have also examined a data table to arrive at the same conclusion, it is much less dramatic and perhaps a harder pattern to see. This is the power of a good data visualization – it presents patterns in numbers with colours and shapes, broadening prospective audiences’ ability to consume the information at hand.

We owe a great debt to innovators like Florence Nightingale and can strive to exemplify her creativity when we present the stories contained in our data sets.

I like to imagine the conversation Florence Nightinghale had after she created this visualization went something like this:

• General: I don’t see why we need to waste time and money cleaning the hospital
• Florence: Do you see all that blue?
• Army General: Yea…
• Florence: That’s how many soldiers we could have saved if we had clean hospitals
• General: We’ve got to do something about that blue! How much time and money do you need to clean the hospital?

I was first introduced to the p-value controversy by an epidemiologist in 2014. The controversy is about the use and practice of null hypothesis significance testing (Tramifow, 2014; Wasserstein, 2016; Wellek, 2017). It is essentially due to the prevalence of the practice of mindless hypothesis testing procedures, also known as the null ritual (Gigerenzer, 2004).

Why does the controversy exist? Ultimately, it’s because statistical reasoning and inference, as well as the hypothesis testing procedure, is difficult to understand (del Mas, 2004). So widespread is confusion that the null ritual has even been labelled as tyrannical (England, 1991; Stang, Pool, & Kuss, 2010).

There’s so much confusion that even the controversy itself can be misunderstood as a indictment of p-values and the hypothesis testing procedure, whereas it is simply a recommendation that one should be thoughtful and not use statistical tools blindly (Wasserstein, Schirm, & Lazar, 2019).

Surely, one part of the problem, and one part of any solution, is the statistics classroom. If people fail to understand statistics, is it not the responsibility of statisticians, as stewards of the field, to help remedy the situation?

I think understanding the evolution and origins of the p-value in hypothesis testing can go a long way in helping. Most people would find it surprising to hear that the three people credited with its development would likely balk at its current practice (Gigerenzer, 2004).

There are many articles that discuss the origins of the procedure, but, one of my favourites is by Biau, Jolles, & Porcher (citation listed below). I strongly recommend everyone to read that paper (and require it of my students) – it is short and very accessible – and at the very least, should prove illuminating.

References and Further Reading:

Biau, D. J., Jolles, B. M., & Porcher, R. (2010). P value and the theory of hypothesis testing: an explanation for new researchers. Clinical Orthopaedics and Related Research®, 468(3), 885-892.

del Mas, R. C. (2004). A comparison of mathematical and statistical reasoning. In The challenge of developing statistical literacy, reasoning and thinking (pp. 79-95). Springer, Dordrecht.

England, C. (1991). On the tyranny of hypothesis testing in the social sciences. Contemporary psychology, 36(2), 102-105.

Gigerenzer, G. (2004). Mindless statistics. The Journal of Socio-Economics, 33(5), 587-606.

Trafimow, D. (2014). Editorial. Basic and Applied Social Psychology, 36(1), 1-2.

Stang, A., Poole, C., & Kuss, O. (2010). The ongoing tyranny of statistical significance testing in biomedical research. European journal of epidemiology, 25(4), 225-230.

Wasserstein, R. L. (2016). ASA statement on statistical significance and P-values.

Wasserstein, R. L., Schirm, A. L., & Lazar, N. A. (2019). Moving to a World Beyond “p< 0.05”. The American Statistician, 73 (S1), 1-19.

Wellek, S. (2017). Author response to the contributors to the discussion on “A critical evaluation of the current ‘p‐value controversy’”. Biometrical Journal, 59(5), 897-900.