Statistics on R Views

Frank's R Workflow

Fri, 17 Jun 2022 00:00:00 +0000

Frank Harrell’s new eBook, R Workflow, which aims to: “to foster best practices in reproducible data documentation and manipulation, statistical analysis, graphics, and reporting” is an ambitious document that is notable on multiple levels.

To begin with, the workflow itself is much more than a simple progression of logical steps.

This workflow is clearly the result of a process forged through trial and error by a master statistician over many years. As the diagram indicates, the document takes a holistic viewpoint of a statistical analysis covering document preparation, data manipulation, statistical practice computational concerns, and more.

Then, there is the synthesis of a wide range of content into a succinct, very readable exposition that dips in to some very deep topics. Frank’s examples are streamlined presentations of analyses and code that are both sophisticated an practical. The missing value section suggests a whole array of analyses through a careful presentation of plots, and the section on data checking introduces a level of automation beyond what is commonly done.

Frank’s writing style is clear, informal and from the perspective of a teacher who wants to show you some cool things along with the basics. For example, don’t miss the if Trick in section 2.4.3.

I should mention that Frank’s eBook is not a tidyverse presentation. The code examples are built around base R, Frank’s Hmisc and rms packages and an eclectic mix of packages that include data.table. plotly and tidyverse packages haven and ggplot2. In a way, this selection of packages reflects the evolution of R itself. For example, as with many popular R packages, Hmisc most likely started out as Frank’s personal tool kit. However, after many years of Frank’s deep commitment to using R and contributing R tools, which includes seventy versions of Hmisc in nineteen years, the package has become a fundamental resource. (Have a look at the reverse depends, imports, and suggests.) Also, the mix of packages with different design philosophies underlying R Workflow reflects the flexibility of the R language and the organic growth of the R ecosystem.

Perhaps the most striking aspect of the eBook is the way Frank uses Quarto, knitr and Hmisc to build an elegant reproducible document about building reproducible documents. For example, Quarto permits the effective placement of plots in the right margins of the document, and the Quarto callouts in Section 3.4 enable the mini tutorials that include Special Considerations for Latex/pdf and Using Tooltips with Mermaid to be embedded in the document without interrupting its flow. Moreover, along with functions like Hmisc::getHdata() and Hmisc::getRs(), Quarto enables the document to achieve a high level of reproducibility by pulling data and code directly from GitHub repositories.

Not only can Frank’s R Workflow teach you some serious statistics, but studying its construction will take you a long way towards building aesthetically pleasing reproducible documents.

Frank Harrell will be delivering a keynote address on August 26th at the upcoming R/Medicine conference.

March 2021: "Top 40" New CRAN Packages

Thu, 22 Apr 2021 00:00:00 +0000

By my count, two hundred twenty-one new packages stuck to CRAN in March 2021.¹ Here are my “Top 40” selections in twelve categories: Computational Methods, Data, Engineering, Genomics, Machine Learning, Medicine, Music, Networks, Science, Statistics, Utility, and Visualization. Two of these categories Engineering and Music have only one entry each. However, I decided to give them their own category in order to draw attention to the use of R outside of the mainstream, and I have always lamented the fate of the Miscellaneous. In the same spirit, note that the complete works of the Bard appear in the Data category and that due to tidypaleo Paleoenvironmental is now a thing in R.

Computational Methods

gamlss v1.0-5: Implements computationally intensive calculations for Generalized Additive Models for location, scale, and shape as described in Rigby & Stasinopoulos (2005).

waydown v1.1.0: Implements an algorithm based on the classical Helmholtz decomposition to obtain an approximate potential function for non gradient fields. See Rodríguez-Sánchez (2020) for background and the vignette for examples.

Data

aopdata v0.2.1: Provides functions to download data from the Access to Opportunities Project (AOP) which includes annual estimates of access to employment, health and education services by transport mode, as well as data on the spatial distribution of population, schools and health-care facilities at a fine spatial resolution for all cities included in the study. There is an Introduction to the package, and there are vignettes on Analyzing Inequality, Mapping Urban Accessibility, and Mapping Pooulation and Land Use.

bardr v0.0.9: Provides R data structures for Shakespeare’s complete works, as provided by Project Gutenberg. See README.

metro v0.9.1: Provides access to the Metro Transparent Data Sets API published by the Washington Metropolitan Area Transit Authority, the government agency operating light rail and passenger buses in the Washington D.C. area. See README.

RAQSAPI v2.0.1: Provides functions to retrieve air monitoring data and associated metadata from the US Environmental Protection Agency’s Air Quality System Service. There are several short vignettes including an Introduction and a vignette on Usage tips and precautions.

troopdata v0.1.3: Provides access to U.S. Department of Defense data on overseas military deployments and includes functions for pulling country-year troop deployment and basing data. See README to get started

Engineering

pipenostics v0.1.7: Implements empirical and data-driven models of heat losses, corrosion diagnostics, reliability and predictive maintenance of pipeline systems which should be of interest to the engineering departments of heat generating and heat transferring companies. See Timashev et al. (2016) and Reddy (2017) for the methods used and README to get started.

Genomics

glmmSeq v0.1.0: Provides functions to fit negative binomial mixed effects models with matched samples to model expression data. See the vignette for examples.

ondisc v1.0.0: Implements a method to allow researchers to analyze large-scale single-cell data as and R object stored on disk. There is a tutorial on the the ondisc matrix class and another on Metadata.

SignacX v2.2.0: Implements a neural network trained with flow-sorted gene expression data to classify cellular phenotypes in single cell RNA-sequencing data. See Chamberlain et al. (2021) for background. There are seven vignettes including an Analysis of Kidney Lupus Data and an Analysis of PBMCs from 10X Genomics.

Machine Learning

opitools v1.0.3: Implements a tool to analyze opinions inherent in a text document relating to a specific subject (A) and assess how opinions expressed with respect to another subject (B) may affect the opinions on subject A. This package has been designed specifically for application to social media datasets, such as Twitter and Facebook. See Adepeju and Jimoh (2021) for an extended example that demonstrates the utility of the approach and the vignette to get started.

poems v1.0.1: Provides a framework of interoperable R6 classes for building ensembles of viable models via the pattern-oriented modeling (POM) approach. The package includes classes for encapsulating and generating model parameters, and managing the POM workflow which includes: model setup; generating model parameters via Latin hyper-cube sampling; running multiple sampled model simulations; collating summary results; and validating and selecting an ensemble of models that best match known patterns. There are two vignettes: Simple Example and Thylacine Example.

Medicine

dampack v1.0.0: Implements a suite of functions for analyzing and visualizing the health economic outputs of mathematical models. See Hunink et al. (2014) for the theoretical underpinnings. There are five vignettes including Basic Cost Effectiveness Analysis, Probabilistic Sensitivity Analysis: Analysis and Value of Information Analysis.

rdecision v1.0.3: Provides classes and functions for using decision trees to model health care interventions using cohort models. See Briggs et al. for theory and terminology. There are five vignettes including Elementary decision tree (Evans 1997) and Decision tree with PSA.

Music

gm v1.0.2: Implements a high-level language to create music including converting your music to musical scores and audio files. It works with R Markdown, R Jupyter Notebooks, and RStudio. There vignette is available in English and in Chinese.

Networks

sfnetworks v0.5.1: Provides a tidy approach to spatial network analysis in the form of classes and functions that enable a seamless interaction between the network analysis package tidygraph and the spatial analysis package sf. There are vignettes on sf network structure, Preprocessing, Spatial joins and filters, Routing, and Spatial morphers.

valhallr v0.1.0: Implements an interface to the Valhalla routing engine’s API for turn-by-turn routing, isochrones, and origin-destination analyses. See the vignette for examples.

Science

asteRisk v0.99.4: Provides functions to calculate the positions of satellites given a known state vector. It includes implementations of the SGP4 and SDP4 simplified perturbation models to propagate orbital state vectors. See Hoots et al. (1988), Vallado et al. (2012), and Hoots et al. (2014) for background and the vignette for examples.

forImage v0.1.0: Implements a tool to measure the size of foraminifera and other unicellulars and includes functions to guide foraminiferal test biovolume calculations and cell biomass estimations. The volume function includes several microalgae models geometric adaptations based on Hillebrand et al. (1999), Sun & Liu (2003), and Vadrucci et al. (2007). See the vignette to get started.

OpenSpecy v0.9.1: Provides functions to analyze, process, identify and share Raman and (FT)IR spectra with functions to implement Savitzky-Golay smoothing in accordance with Zhao et al. (2007) and identify spectra using an onboard reference library, see Cowger et al. 2020. Analyzed spectra can be shared via Shiny App. There is a vignette.

tidypaleo v0.1.1: Provides functions with a common framework for age-depth model management, stratigraphic visualization, and common statistical transformations with a focus on stratigraphic visualization using ggplot2. There are vignettes on Age-depth Models, Nested Analyses, and Stratigraphic Diagrams.

VulnToolkit v1.1.2: Provides functions to analyze and summarize tidal data sets and to access to NOAA mean sea level data. See Hill & Anisfeld (2015) for background and the vignette for examples.

Statistics

corncob v0.2.0: Implements functions for modeling correlated count data using the beta-binomial distribution, described in Martin et al. (2020). See the vignette for an introduction.

hawkesbow v1.0.2: Implements an estimation method for Hawkes processes when count data are only observed in discrete time, using a spectral approach derived from the Bartlett spectrum. See Cheysson and Lang (2020) for background and the vignette for examples.

LMMELSM v0.1.0: Implements two-level mixed effects location scale models on multiple observed or latent outcomes, and between-group variance modeling. See Williams et al. (2020) and Hedeker et al. (2008) for background and README for an example.

mixpoissinreg v1.0.0: Provides functions to fit mixed Poisson regression models (Poisson-Inverse Gaussian or Negative-Binomial) with count data response variables. See Barreto-Souza and Simas (2016) for background. There are five vignettes on Global and Local Influence, Confidence and Prediction Intervals, MLE, Tidy Methods, and Overdispersed Count Data.

ppdiag v0.1.0: Provides a suite of diagnostic tools for univariate point processes including tools for simulating and fitting both common and more complex temporal point processes and the diagnostic tools described in Brown et al. (2002) and Wu et al. (2020). There is a vignette on Markov Modulated Point Processes and another on Diagnostic Tools.

robustlm v0.1.0: Implements a computationally efficient exponential squared loss algorithm for variable selection proposed by Wang et al.(2013). See the vignette.

smmR v1.0.2: Provides functions to estimate and simulate multi-state semi-Markov models. The methods implemented are described in Barbu & Limnios (2008) and Trevezas & Limnios (2011). The vignette contains an extended example.

spotoroo v0.1.1: Implements an algorithm to cluster satellite hot spot data spatially and temporally. See the vignette.

Utilities

clock v0.2.0: Provides a comprehensive library for date-time manipulations using a new family of orthogonal date-time classes (duration, time points, zoned-times, and calendars) that partition responsibilities so that the complexities of time zones are only considered when they are really needed. There is a Getting Started guide, as well as vignettes on FAQ, and Examples and Recipies.

crosstable v0.2.1: Provides functions to create descriptive tables for continuous and categorical variables, apply summary statistics, and create reports using rmarkdown or officer. There is an Introduction, and vignettes on Troubleshooting, Making Automatic Reports, and Selecting Variables.

pkgdepends v0.1.0: Provides functions to find recursive dependencies for R packages from various sources including CRAN, Bioconductor, and GitHub enabling users to obtain a consistent set of packages to install. See README to get started.

pkglite v0.1.1: Implements a tool, grammar, and standard to represent and exchange R package source code as text files. Converts one or more source packages to a text file and restores the package structures from the file. There are vignettes on Generating File Specifications, Representing Packages, and Compact Package Representation.

Visualization

datplot v1.0.0: Provides tools to process and prepare data for visualization and employs the concept of aoristic analysis. See aorist and the vignettes Data Preparation and Visualization and Visualizing Chronological Distribution.

ferrn v0.0.1: Implements diagnostic plots for optimization, with a focus on projection pursuit which show paths the optimizer takes in the high-dimensional space. See README for examples.

funcharts v1.0.0: Provides functional control charts for statistical process monitoring of functional data, using the methods of Capezza et al. (2020) and Centofanti et al. (2020). There are vignettes on Capezza 2020, Centofanti 2020 and on the mfd class.

gghilbertstrings v0.3.3: Provides functions to plot Hilbert curves which are used to map one dimensional data into the 2D plane. A specific use case maps a character column in a data frame into 2D space allowing visually comparing long lists of URLs, words, genes or other data that has a fixed order and position. See README for examples.

mapsf v0.1.1: Provides functions to create and integrate thematic maps including functions to design various cartographic representations such as proportional symbols, choropleth or typology maps. Look here for examples.

¹ I have used phrases like By my count and stuck to CRAN in the past, but I do not believe that I have explained what I mean. For some time now, but I believe more frequently in recent months, packages will appear as new on CRAN, only to be removed within a relatively short period of time for failing to resolve check problems. If you happen to know about these packages and search for them by name on CRAN you will receive the message:

Package XXXX was removed from the CRAN repository. Formerly available versions can be obtained from the archive. Archived on 2021-04-17 as check problems remained after update. A summary of the most recent check results can be obtained from the check results archive. Please use the canonical form https://CRAN.R-project.org/package=XXXX to link to this page.

I did not include the ten packages that were identified as being new for March when I created my list of March packages on April 10, 2021, but were removed by the time I finalized my list for this post a week later, in my total count of new CRAN packages. So, there is some instability with the notion of counting new packages in a given month.

Congratulations Class of 2020!

Sat, 16 May 2020 00:00:00 +0000

Yesterday, along with R-Ladies Founder Gabriela de Queiroz I was honored to be asked to give a short talk at an online graduation ceremony for the BSc and MSc Statistics Graduates of the class of 2020 (all R users) at the California State University, East Bay. My talk contains some statistics jargon, but I thought it might be helpful to Class of 2020 graduates in other disciplines who have made some serious effort to acquire critical thinking skills. I know that not everyone thinks a graduation ceremony is important, but it seems to me that the Class of 2020 is getting seriously short changed, and could benefit from some attention. If any readers of this post would like to address the Class of 2020, I would be happy to publish their remarks in R Views.

Address to California State East Bay BSc and MSc Statistics Graduates: May 15, 2020

Dear Fellow Alumni: Congratulations!! It is difficult enough to make that final push to graduate under normal circumstances, but you have done it in the middle of a nightmarish doomsday scenario - Well done! You have demonstrated that you have what it takes to think clearly under pressure.

Your families need you, your friends need you, your communities need you, and it is not an exaggeration to say that the world needs you. You have demonstrated some skill at reasoning about uncertainty, at a time when fearful men and women are anxious about walking out their front doors. They want bright line rules for how to avoid the virus, and they want absolute certainty that they will be safe.

“If I wear a mask will I be protected? If a stay six feet apart does that mean everything will be OK? How about three feet? Well then, can I get my toenails done?”

The role of the statistician is to help people cope with the irreducible uncertainty in the world. Irreducible uncertainty - say that phase to yourself. Let it roll around in your head, but don’t let it scare you. You know that even when you have data that you believe in, data that you have cleaned and curated yourself; when you know something about the application you are working on and you have good reasons to use an informative prior - even then there is uncertainty. There is no way to escape the bias variance trade off - no way to be absolutely certain.

You have acquired a probabilistic mindset that should help you to help others: to help them navigate the narrow path flanked by despair on one side and magical thinking on the other. When you look at the news, you can see how the nuanced explanations of the health care experts get watered down into oversimplified statements that are mostly misleading. It is now an everyday thing to see some authoritative sounding news person or public official show a forecast with enormous error bands and just ignore them. You are now in the group with some responsibility to stand up and say: “Whoa! Hold on here!

Alright, so there’s one minor problem - the world, your community, your prospective employers may not yet know that they need you. The only way to help them to see the light in you is for you to be the person you want to be. If you want to be a statistician or data scientist, start to be one right away. Pick a problem, get the data, visualize it, analyze it, write about it. Put everything on your GitHub page. Talk about it to everyone who will listen.

You don’t have to want to be a data scientist, but you do have to want to grow in some direction. Listen to yourself, trust yourself. Decide what it will take for you to flourish and go for it. Choose your friends carefully, and get involved in some cause bigger than yourself.

Go for it. And wherever you go, walk with humility, but lead the way. Use your hard earned skills to navigate the random and the unpredictable. Take some reasonable risks, but consider the consequences for others.

You can do it! Good luck to you all. Stay safe and prosper.

And, because a graduation event is a good time to remember one’s own teachers, I would like to express my gratitude to Cal State East Bay Professors, Eric Suess and Bruce Trumbo, and also to Donald Lewis, Professor of Philosophy at Cal State Dominguez Hills.

A First Look at Confidence Distributions

Tue, 05 Nov 2019 00:00:00 +0000

Using a probability distribution to characterize uncertainty is at the core of statistical inference. So, it seems natural to try to summarize the information about the parameters in statistical models with probability distributions. R. A. Fisher thought so. In fact, he expended a great deal of effort over more than thirty years, and put his professional reputation on the line trying to do so, with only limited success. Fisher’s central difficulty was that, in the Frequentist tradition to which he was committed, parameters are not random variables. They are fixed and immutable constituents of the statistical models describing the behavior of populations, which we must estimate because we generally only have access to samples from populations, not to the full populations themselves. Now Bayesians, of course, characterize parameters with probability distributions from the get-go. Parameters are given prior distributions and combined with the likelihood function generated by the data to produce posterior distributions that characterize the parameters. Fisher wanted the posterior distributions without having to assume the priors. This was a key motivating idea for his work on Fiducial probability.

A few statisticians apparently quietly worked on this program throughout the twentieth century, even though Fisher’s Fiducial ideas were mostly forgotten and not part of the mainstream statistical current. D. R. Cox (Cox (1958)), for example, pioneered the idea of constructing confidence distributions from confidence intervals, and Bradley Efron (Efron (1998)) expressed great optimism that Fisher’s work in this area would become important in the twenty-first century. (Efron’s paper is a masterpiece that summarizes a good bit of twentieth-century statistical research.)

Recently, however, there seems to have been a resurgence of Fisher’s ideas among statisticians interested in chasing the idea of an orthodox Frequentist view of parameter distributions. The 2013 paper of Xie and Singh lays out the modern theory of confidence distributions as a fundamental idea that organizes a great deal of statistical practice. In the initial Summary, the authors write:

. . .the concept of a confidence distribution subsumes and unifies a wide range of examples, from regular parametric (fiducial distribution) examples to bootstrap distributions, significance (p-value) functions, normalized likelihood functions, and, in some cases, Bayesian priors and posteriors.

Later in the paper, they go on to define a confidence distribution as:

A function H_n(·) = H_n(x, ·) on X × \(\Theta\) → [0, 1] is called a confidence distribution (CD) for a parameter \(\theta\), if
* R1) For each given x ∈ X , H_n(·) is a cumulative distribution function on \(\Theta\);
* R2) At the true parameter value \(\theta\) = \(\theta\)₀, H_n(\(\theta\)₀) ≡ H_n(x, \(\theta\)₀), as a function of the sample

The simplest example of a confidence distribution I could find that is adequate to illustrate some of the key concepts comes from the book Confidence, Likelihood, Probability: Statistical Inference with Confidence Distributions by Schweder and Hjort. On page 62, the authors point out that the distribution of the p-value for the parameter \(\theta\) describing the probability of success for a binomial trial can be considered as an approximate confidence distribution for \(\theta\). The distribution is approximate because the distribution is discrete and the “half-correction” is used to improve the approximation. Note that because the p-values follow uniform distributions under the null hypothesis, it should be clear that the assumptions of the definition above are satisfied.

Suppose Y ~ Bin(n,\(\theta\)), then

C(\(\theta\)) = P(Y > y₀) + .5 * P(Y = y₀) is a confidence distribution for \(\theta\).

To illustrate this, we consider the experiment of realizing 8 successes in 20 trials and write a short helper function.

CD <- function(theta,n=20,y0=8){
            1 - sum(dbinom(x = seq(from = 0, to = y0), size = n, prob = theta)) + 
           .5 * dbinom(x = y0, size = n, prob = theta)}

Here, we compute the CDF and plot it.

library(tidyverse)
library(highcharter)
conf_dist <-  
  tibble(theta = seq(0, 1, by = .01)) %>%  
  mutate(probability = map_dbl(theta, CD))

hchart(conf_dist, "line", hcaes(x = theta, y = probability)) %>%
hc_title(text = "Confidence Distribution for Binomial Model",
         margin = 20, align = "left",
         style = list(color = "black", useHTML = TRUE)) %>%
hc_tooltip(valueDecimals=4, valuePrefix="cum prob = ")

This is nice, but to really see how computing the confidence distribution might be useful, we compute and plot the confidence curve introduced by Birnbaum in his 1961 paper.

conf_curve <-  
  conf_dist %>%  
  mutate(confidence = 2 * abs(.5 - probability))

hchart(conf_curve, "line", hcaes(x = theta, y = confidence)) %>%
  hc_title(text = "Confidence Curve for Binomial Model",
           margin = 20, align = "left",
           style = list(color = "black", useHTML = TRUE)) %>%
  hc_tooltip(valueDecimals=4, valuePrefix="conf level = ")

Pick a point on the left branch of the curve. The y value gives you the level of confidence and the x value is the lower bound of the corresponding confidence interval. Move horizontally across to the right branch to read off the upper end of the confidence interval. So reading up and down the curve you can read off the confidence intervals for any value of confidence.

As a check, we compute the 95% confidence interval for \(\theta\) using the normal approximation to the binomial.

ub <- 8 / 20 + (1.96 / 20) * sqrt(8 * 12 / 20)
lb <- 8 / 20 - (1.96 / 20) * sqrt(8 * 12 / 20)
cat("95% CI = [" , lb , "," , ub, "]")

## 95% CI = [ 0.1853 , 0.6147 ]

If you are a Bayesian, there is a really amusing side to confidence distributions. In order to be clear that what they are doing with confidence distributions is in fact different from what Bayesians do when they choose priors, the champions of confidence distributions appeal to epistomology, the study of knowledge and justified belief. On page (xiv) of their book, Schweder and Hjort write:

The concept of confidence distribution is rather basic, but has proved difficult for statisticians to accept. The main reason is perhaps that confidence distributions represent epistemic probability obtained from the aleatory probability of the statistical model (i.e. the chance variation in nature and society), and to face both types of probability at the same time might be challenging. The traditional Bayesian deals only with subjective probability, which is epistemic when based on knowledge, and the frequentist of the Neyman-Wald school deals only with sampling variability, that is, aleatory probability.

Please indulge me while I unpack this. Aleatory probabilities are what nature and the world give us: the decay times of alpha particles, the valuable behaviors of large populations, etc. Hard-core frequentists will only allow themselves to compute aleatory probabilities. As soon as you compute a confidence distribution or even a confidence interval, you are working with epistemic probabilities: what you believe would be true under repeated sampling that may be impossible to actually carry out. But, you are justified in doing this because these epistemic probabilities are anchored in aleatory probabilities. When Bayesians base their choice of priors on rational beliefs based on plausible evidence, they are on the same epistemic footing as frequentists computing confidence intervals. When Bayesians are capricious in choosing their priors, they are not. Asking most statisticians to think about these things gives them headaches. Thus, quietly, in work that builds bridges, ends the Bayesian vs. Frequentist controversy.

Notes:
1. Schweder and Hjolt’s book is really worth owning. Not only does it offer a comprehensive account of confidence distributions and how they may be useful in practice, but it is also a good general reference on statistical inference.

If you are interested in exploring confidence distributions further, have a look at the pvaluefunctions and gmeta that are both on CRAN.