Skip to main content Skip to local navigation
Statistical Consulting Service
Home » Statistical Resources

Statistical Resources

This page collects links to tutorials and other online resources we have found useful for SCS clients.

Graphical methods

  • British Ecological Society's Guide to Reproducible Science. The guide proposes a simple reproducible project workflow, and a guide to organizing projects for reproducibility. The Programming section provides concrete tips and traps to avoid (example: use relative, not absolute pathnames), and the Reproducible Reports section provides a step-by-step guide for generating reports with R Markdown.

Multiplicity Control

https://www.youtube.com/watch?v=HpjlcEH4zuY

    This is a review paper on the use of multiplicity control in psychology:

      Martinez Gutierrez, N., DeSouza, J., Alter, U., Bell, S., Beribisky, N., Farmus, L., Tu, E., & Cribbie, R. A. (2026). Multiplicity control in psychological research: A systematic review of empirical practices. Canadian Psychology / Psychologie canadienne, 67(4), 301–310. https://doi.org/10.1037/cap0000436

      Here is a paper that emphasizes the confusion around the multiplicity control issue:

        Cribbie, R. A. (2017). Multiplicity control, school uniforms, and other perplexing debates. Canadian Journal of Behavioural Science, 49(2), 159–160. https://doi.org/10.1037/cbs0000075

        Here are three recent papers that discuss that multiplicity control is not necessary except for the rare situation where rejecting ANY lower order hypotheses affects the status of the global hypothesis:

          García-Pérez, M. A. (2023). Use and misuse of corrections for multiple testing. Methods in Psychology, 8, Article 100120. https://doi.org/10.1016/j.metip.2023.100120

          Rubin, M. (2021). When to adjust alpha during multiple testing: A consideration of disjunction, conjunction, and individual testing. Synthese, 199, 10969–11000. https://doi.org/10.1007/s11229-021-03276-4

          Rubin, M. (2024). Inconsistent multiple testing corrections: The fallacy of using family-based error rates to make inferences about individual hypotheses. Methods in Psychology, 10, Article 100140. https://doi.org/10.1016/j.metip.2024.100140

          These are older papers that make the same argument (abandoning multiplicity control):

            Saville, D. J. (1990). Multiple Comparison Procedures: The Practical Solution. The American Statistician, 44(2), 174–180. https://doi.org/10.1080/00031305.1990.10475712

            Rothman, K. J. (1990). No adjustments are needed for multiple comparisons. Epidemiology, 1(1), 43–46. https://www.jstor.org/stable/20065622

            Negligible Effect (Equivalence) Testing

            An introductory video on negligible effect testing: https://www.youtube.com/watch?v=AEpMHDXK8UI

            Using significance tests to evaluate equivalence between two experimental groups. Equivalency testing, a statistical method often used in biostatistics to determine the equivalence of two experimental drugs, is introduced to social scientists. Examples of equivalency testing are offered, and the usefulness of the method to the social scientist is discussed.

            Here are two introductory papers on negligible effect testing:

            Lakens et al. Equivalence Testing for Psychological Research: A Tutorial

            Beribisky & Cribbie A Primer on Equivalence (Negligible Effect) Testing.

            Here are a couple of R packages for conducting negligible effect testing:

            Linear models (ANOVA, Regression)

            • Overview of R Modelling Packages. An overview of R packages and functions for fitting different types of linear models, classified by the type
              of outcome variable (continuous, binary, catgegorical). Contains links to examples and shows Bayesian equivalents of many frequentist approaches.
            • Data and Model Summaries in R. modelsummary is an R a package to summarize data and statistical models in R. It supports over one hundred types of models out-of-the-box, and allows users to report the results of those models side-by-side in a table, or in coefficient plots. There is also a JSS paper that condenses many examples into a shorter format.

            Meta Analysis

            • Mathais Harrer Doing Meta Analysis in R. An accessible introduction into how meta-analyses can be conducted in R. Essential steps for meta-analysis are covered, including pooling of outcome measures, forest plots, heterogeneity diagnostics, subgroup analyses, meta-regression, methods to control for publication bias, risk of bias assessments and plotting tools.

            Structural Equation Models

            Effect size & power analysis

            Latent Profile Analysis

            Latent profile analysis (LPA) is a latent variable method that focuses on identifying latent sub-populations within a population based on observed variables.
            LPA works best with continuous variables (and, in some cases, ordinal variables), but is not appropriate for dichotomous (binary) variables.