R Package on R Views

The COVID19 package, an interface to the COVID-19 Data Hub

Wed, 08 Dec 2021 00:00:00 +0000

The COVID-19 Data Hub provides a daily summary of COVID-19 cases, deaths, recovered, tests, vaccinations, and hospitalizations for 230+ countries, 760+ regions, and 12000+ administrative divisions of lower level. It includes policy measures, mobility, and geospatial data. This post presents version 3.0.0 of the COVID19 package to seamlessly import the data in R.

Why another package for COVID-19 data

Many packages now exist to retrieve COVID-19 related data from within R. As an example, covid19br retrieves case data for Brazil, covid19sf for San Francisco, covid19us for United States, covid19india for India, covid19italy for Italy, covid19swiss for Switzerland, covid19france for France, and so on. There also other packages, such as coronavirus, that retrieve national-level statistics worldwide from the Center for Systems Science and Engineering at Johns Hopkins University (JHU CCSE). However, national counts only represent a small portion of the available governmental data, and having the information scattered across many packages and different interfaces makes international comparisons of large, detailed outbreak data difficult, and prevents inferences from such data to be effective.

COVID19 is the official package created around COVID-19 Data Hub: a unified database harmonizing open governmental data around the globe at fine-grained spatial resolution. Moreover, as epidemiological data alone are typically of limited use, the database includes a set of identifiers to match the epidemiological data with exogenous indicators and geospatial information. By unifying the access to the data, this database makes it possible to study the pandemic in its global scale with high resolution, taking into account within-country variations, non pharmaceutical interventions, and environmental and exogenous variables.

In particular, this package allows you to download a large set of epidemiological variables, exogenous indicators from World Bank, mobility data from Google and Apple mobility reports, and geospatial information from Eurostat for Europe or GADM worldwide, in, literally, one line of code.

What’s new in version 3.0.0

Version 3 is a major update of COVID-19 Data Hub, which includes a great improvement in the spatial coverage, new data on vaccines, and a new set of identifiers to enable geospatial analyses. The full changelog is available here.

The large amount of data that is now available (~2GB) has led to some breaking changes in the way the data are provided. Version 3 of the COVID19 package is designed to be compatible with the latest version of COVID-19 Data Hub, and process large amount of data at speed with low memory requirements. The documentation and a quick start guide is available here.

Data coverage

The figure shows the granularity and the spatial coverage of the data as of November 27, 2021.

What’s included?

library(COVID19)  # load the package
x <- covid19()    # download the data

Refer to the documentation for the description of each variable.

colnames(x)

##  [1] "id"                                  "date"                               
##  [3] "confirmed"                           "deaths"                             
##  [5] "recovered"                           "tests"                              
##  [7] "vaccines"                            "people_vaccinated"                  
##  [9] "people_fully_vaccinated"             "hosp"                               
## [11] "icu"                                 "vent"                               
## [13] "school_closing"                      "workplace_closing"                  
## [15] "cancel_events"                       "gatherings_restrictions"            
## [17] "transport_closing"                   "stay_home_restrictions"             
## [19] "internal_movement_restrictions"      "international_movement_restrictions"
## [21] "information_campaigns"               "testing_policy"                     
## [23] "contact_tracing"                     "facial_coverings"                   
## [25] "vaccination_policy"                  "elderly_people_protection"          
## [27] "government_response_index"           "stringency_index"                   
## [29] "containment_health_index"            "economic_support_index"             
## [31] "administrative_area_level"           "administrative_area_level_1"        
## [33] "administrative_area_level_2"         "administrative_area_level_3"        
## [35] "latitude"                            "longitude"                          
## [37] "population"                          "iso_alpha_3"                        
## [39] "iso_alpha_2"                         "iso_numeric"                        
## [41] "iso_currency"                        "key_local"                          
## [43] "key_google_mobility"                 "key_apple_mobility"                 
## [45] "key_jhu_csse"                        "key_nuts"                           
## [47] "key_gadm"

Data transparency

This package applies no pre-processing to the original data, that are provided as-is. The data acquisition pipeline is open source and all the original data providers are listed here.

As an example, the following code snippet plots the fraction of confirmed cases on a given day per number of tests performed on that day in U.S. Notice that around June 2021, the fraction becomes negative. This is a known issue due to decreasing cumulative counts in the original data provider. This package applies no cleaning procedure for this kind of issues, which are typically due to changes in the data collection methodology. If the provider corrects the data retroactively, the changes are reflected in this package.

library(xts)
library(dygraphs)
x <- covid19("USA", verbose = FALSE)  # download the data
ts <- xts(x[,c("confirmed", "tests")], order.by = x$date)  # convert to an xts object
ts$ratio <- diff(ts$confirmed) / diff(ts$tests)  # compute daily ratio
dygraph(ts$ratio, main = "Daily fraction confirmed/tests in U.S.")  # plot

World Bank data

Country-level covariates by World Bank Open Data can be easily added. Refer to the table at the bottom of this page for relevant indicators. The following code snippet shows e.g., how to download the number of hospital beds for each country. Refer to the quickstart guide for more details.

x <- covid19(wb = c("hosp_beds" = "SH.MED.BEDS.ZS"), verbose = FALSE)

Mobility data

Mobility data are obtained from Google and Apple mobility reports. The following example shows how to download the data by Google. Refer to the quickstart guide for Apple’s reports and for more details.

x <- covid19(gmr = TRUE, verbose = FALSE)
colnames(x[,48:53])

## [1] "retail_and_recreation_percent_change_from_baseline"
## [2] "grocery_and_pharmacy_percent_change_from_baseline" 
## [3] "parks_percent_change_from_baseline"                
## [4] "transit_stations_percent_change_from_baseline"     
## [5] "workplaces_percent_change_from_baseline"           
## [6] "residential_percent_change_from_baseline"

Spatial data

The dataset contains NUTS codes to match the Eurostat database for Europe, and GID codes to match the GADM worldwide database. The following example shows how to access spatial data using GADM for U.S. counties. Similar maps are available worldwide for most other countries at the various granularity levels.

First, download level 3 data for U.S.

x <- covid19("USA", level = 3, verbose = FALSE)

GADM data by country can be found here. Download the geopackage for U.S. using GADM version 3.6:

url <- "https://biogeo.ucdavis.edu/data/gadm3.6/gpkg/gadm36_USA_gpkg.zip"
zip <- tempfile()
download.file(url, destfile = zip)

Unzip the geopackage:

exdir <- tempfile()
unzip(zip, exdir = exdir)

Load the sf package and list the layers:

library(sf)
file <- paste0(exdir, "/gadm36_USA.gpkg")
st_layers(file)

## Driver: GPKG 
## Available layers:
##     layer_name geometry_type features fields
## 1 gadm36_USA_2 Multi Polygon     3148     13
## 2 gadm36_USA_1 Multi Polygon       51     10
## 3 gadm36_USA_0 Multi Polygon        1      2

Read layer 2 that corresponds to U.S. counties. Note: in general, there is not a perfect correspondence between GADM layers and the granularity level from this package. It is recommended to read all the layers, and match on the corresponding GID. Read more about how key_gadm from this package is mapped to the corresponding GID here.

g <- st_read(file, layer = "gadm36_USA_2")

## Reading layer `gadm36_USA_2' from data source 
##   `/private/var/folders/w0/skxpg0h51jg72m5b01y_8n_c0000gn/T/RtmpmUy7P6/file56232c611dfd/gadm36_USA.gpkg' 
##   using driver `GPKG'
## Simple feature collection with 3148 features and 13 fields
## Geometry type: MULTIPOLYGON
## Dimension:     XY
## Bounding box:  xmin: -179.2 ymin: 18.91 xmax: 179.8 ymax: 72.69
## Geodetic CRS:  WGS 84

Subset the data to extract only the counts as of, e.g., 15 November 2021. Select only the administrative divisions inside the following bounding box for better visualization.

x <- x[
  x$date == "2021-11-15" & 
  x$latitude > 24.9493 & x$latitude < 49.5904 &
  x$longitude > -125.0011 & x$longitude < -66.9326,]

Merge COVID-19 data with the spatial data:

library(dplyr)
gx <- right_join(g, x, by = c("GID_2" = "key_gadm"))

Plot e.g., the total number of confirmed cases:

plot(gx["confirmed"], logz = TRUE)

Academic publications

See the publications that use COVID-19 Data Hub.

Cite as

Guidotti, E., Ardia, D., (2020), “COVID-19 Data Hub”, Journal of Open Source Software 5(51):2376, doi: 10.21105/joss.02376.

A BibTeX entry for LaTeX users is

@Article{,
    title = {COVID-19 Data Hub},
    year = {2020},
    doi = {10.21105/joss.02376},
    author = {Emanuele Guidotti and David Ardia},
    journal = {Journal of Open Source Software},
    volume = {5},
    number = {51},
    pages = {2376}
}

February 2021: "Top 40" New CRAN Packages

Fri, 19 Mar 2021 00:00:00 +0000

In February, two hundred forty-three new packages made it to CRAN, many of them very interesting and at least one entertaining. It was exceptionally difficult to pick the “Top 40”, but here they are, more or less, in eleven categories: Computational Methods, Data, Finance, Games, Genomics, Machine Learning, Mathematics, Medicine, Networks and Graphs, Statistics, Utilities, and Visualization. iconr in the Networks and Graphs section is a package for doing computational archaeology, a relatively new field that I hope will dig R. I also hope that sassy in the Statistics sections helps some statisticians find their way to R.

Computational Methods

blaster v1.0.3: Implements an efficient BLAST-like sequence comparison algorithm, written in C++11 and using native R data types. See Schmid et al. (2018) for background and README for an example.

rando v0.2.0: Provides random number generating functions that are much more context aware than the built-in functions. The functions are also safer, as they check for incompatible values, and reproducible.

Data

AWAPer 0.1.46: Provides catchment area weighted climate data NetCDF files from the Bureau of Meteorology Australian Water Availability Project for all of Australia. There is a vignette on Daily Area Weighted PET and Precipitation and another on Daily Point Precipitation

caRecall v0.1.0: Provides API access to the Government of Canada Vehicle Recalls Database used by the Defect Investigations and Recalls Division for vehicles, tires, and child car seats. See the vignette.

geofi v1.0.0: Provides tools for reading Finnish open geospatial data in R. There are vignettes on Datasets, Joining Attributes, Making Maps, Data Manipulation, and Color-coded Maps.

hockeystick v0.4.0: Provides easy access to essential climate change data sets for non-climate experts. Users can download the latest raw data from authoritative sources and view it via pre-defined ggplot2 charts. Data sets include atmospheric CO2, instrumental and proxy temperature records, sea levels, Arctic/Antarctic sea-ice, and Paleoclimate data. Sources include: NOAA Mauna Loa Laboratory, NASA GISTEMP, National Snow and Sea Ice Data Center, CSIRO, NOAA Laboratory for Satellite Altimetry, and Vostok Paleo carbon dioxide and temperature data. See README for examples.

votesmart v0.1.0: Implements a wrapper to the Project VoteSmart API. See the vignette.

Finance

PriceIndices v0.0.3: Provides functions to compute bilateral and multilateral indexes. For details, see: de Haan and Krsinich (2017) and Diewert and Fox (2020). The vignette offers examples.

treasuryTR v0.1.1: Generates Total Returns (TR) from bond yield data with fixed maturity (e.g. reported treasury yields) which may provide an alternative to commercial products. See Swinkels (2019) for background and the vignette for examples.

Games

pixelpuzzle v1.0.0: Implements a puzzle game that can be played in the R console. Restore the pixel art by shifting rows. Learn how to play here.

Genomics

CDSeq v1.0.8: Provides functions to estimate cell-type-specific gene expression profiles and sample-specific cell-type proportions simultaneously using bulk sequencing data. See Kang et al. (2019) for the theory and the vignette for examples.

ClusTorus v0.0.1: Provides various tools for clustering multivariate angular data on the torus including angular adaptations of usual clustering methods such as the k-means clustering, pairwise angular distances. See the vignette for examples.

dsb v0.1.0: Provides a method for normalizing and denoising protein expression data from droplet based single cell experiments. See the vignette for tutorials on how to integrate dsb with Seurat, Bioconductor and the AnnData class in Python. The preprint Mulè et al. (2020) describes the details.

Machine Learning

besridge v1.0.4: Provides functions to perform ridge regression in complex situations on high dimensional data using the primal dual active set algorithm proposed in Wen et al. (2020). Functions support regression, classification, count regression and censored regression, group variable selection and nuisance variable selection. See the vignette for examples.

ROCket v1.0.1: Provides functions for estimating receiver operating characteristic (ROC) curves and area under the curve (AUC) calculation which distinguish two types of ROC curve representations: 1) parametric curves - the true positive rate (TPR) and the false positive rate (FPR) are functions of a score parameter and 2) function curves - TPR is a function of FPR. See Gonçalves et al. (2014) and Cai & Pepe (2004) for background and README to get started.

wordpiece v1.0.2: Provides functions to apply Wordpiece tokenization to input text, given an appropriate vocabulary. The BERT tokenization conventions are used by default. See the vignette for an example.

Mathematics

fractD v0.1.0: Estimates the of fractal dimension of a black area in 2D and 3D (slices) images using the box-counting method. See Klinkenberg (1994) for background and the vignette for examples.

spacefillr v0.2.0: Generates random and quasi-random space-filling sequences including Halton, Sobol and other sequences with errors distributed as various types of jittered blue noise. See Joe and Kuo (2018), Christensen et al. (2018) and Heitz et al. (2019) for background and look here for examples.

tensorsign v0.1.0: Provides an efficient algorithm for nonparametric tensor completion via sign series. The algorithm which employs the alternating optimization approach to solve the weighted classification problem is described in Lee and Wang (2021)

Medicine

bhmbasket v0.9.1: Provides functions to evaluate basket trial designs with binary endpoints using Bayesian hierarchical models and Bayesian decision rules. See Berry et al. (2013), Neuenschwander et al. (2016) and Fisch et al. (2015) for background and the vignette for an example.

bp v1.0.1: Provides functions to aid in the analysis of blood pressure data of all forms by providing both descriptive and visualization tools for researchers. There is a vignette.

CHOIRBM v0.0.2: Provides functions for visualizing body map data collected with the Collaborative Health Outcomes Information Registry CHOIR). See the vignette.

QDiabetes v1.0-2: Calculates the risk of developing type 2 diabetes using risk prediction algorithms derived by ClinRisk. Look here for information and examples.

SteppedPower v0.1.0: Provides tools for power and sample size calculations and design diagnostics for longitudinal mixed models with a focus on stepped wedge designs using methods introduced in Hussey and Hughes (2007) and extensions discussed in Li et al. (2020). See the vignette to get started.

Networks and Graphs

bnmonitor v0.1.0. Implements sensitivity and robustness methods for Bayesian networks including methods to perform parameter variations via a variety of co-variation schemes, to compute sensitivity functions and to quantify the dissimilarity of two Bayesian networks via distances and divergences. See Chan and Darwiche (2002), Cowell et al. (2007), and Goergen and Leonell (2020) for background and README for examples.

iconr v0.1.0: Provides formal methods for studying archaeological iconographic data sets (rock-art, pottery decoration, stelae, etc.) using network and spatial analysis See Alexander (2008) and Huet (2018) for background and the vignette for examples.

MLVSBM 0.2.1: Provides functions for simulation, inference and clustering of multilevel networks using a stochastic block model framework as described in Chabert-Liddell et al. (2021). There is a tutorial.

motifr v1.0.0: Provides tools to analyze motifs(small configurations of nodes and edges) in multi-level networks (networks which combine multiple networks in one, e.g. social-ecological networks.) See The motif zoo and Baseline model comparisons.

Statistics

cfda v0.9.9: Provides functions to encode categorical data as functional data and perform basis statistical analysis. See Preda et al. (2020) for background and the vignette to get started.

cvCovEst v0.3.4: Implements an efficient cross-validated approach for covariance matrix estimation, particularly useful in high-dimensional settings. See the vignette for background and examples.

flipr v0.2.1: Implements a permutation framework point estimation, confidence intervals or hypothesis testing for multiple data types. There is a Tour of Permutation Inference, and vignettes on Alternative Hypothesis Testing, the Exactness of Permutation Tests, and Calculating p-value Functions.

ipmr v0.0.1: implements integral projection models using an expression based framework that handles density dependence and environmental stochasticity and provides tools for diagnostics, plotting, simulations, and analysis. See Easterling et al. (2000) for an in depth description of integral projection models. There is an Introduction and vignettes on Age-Size IPMS, Density Dependent IPMS, Hierarchical Notation, and Data Structures.

metapack v0.1.1: Provides functions performing Bayesian inference for meta-analytic and network meta-analytic models through Markov chain Monte Carlo algorithm. See Yao et al. (2015) for the theory, the vignette for an introduction and the online documentation.

sassy v1.0.4: Loads a collection of packages that collectively aim to make R easier for SAS® programmers. Functions bring many familiar SAS® concepts to R, including data libraries, data dictionaries, formats and format catalogs, a data step, and a traceable log. There is an Introduction, and vignettes with example Figures, Listings, and Tables, as well as a few Disclaimers which include a statement indicating that the packages were developed in the context of the pharmaceutical industry but should be generally helpful.

Utilities

gargoyle v0.0.1: Implements an event-Based framework for building Shiny apps. Instead of relying on standard Shiny reactive objects, this package allow to relying on a lighter set of triggers, so that reactive contexts can be invalidated with more control. See the vignette.

multidplyr Provides simple multicore parallelism through functions that partition a data frame across multiple worker processes. See the vignette.

quarto v0.1: Provides an interface to the Quarto markdown publishing system and allows converting R Markdown documents and Jupyter Notebooks to a variety of output formats.

var v0.0.2: Provides functions to manage, provision and use virtual machines pre-configured for R, and develop, test and build package in a clean environment. Vagrant and a provider such as Virtualbox must be installed.

Visualization

ggh4x v0.1.2.1: Extends ggplot2 facets by setting individual scales per panel, resizing panels, providing nested facets, and allowing multiple colour and fill scales per plot. See the Introduction, and the vignettes Facets, Misc, Position Guides, and Statistics.

tastypie v0.0.3: Provides functions and templates for making pie charts even though you probably shouldn’t. See the vignettes available templates and Your favorite template, and look here for examples.

terrainr v0.3.1: Provides functions to retrieve, manipulate, and visualize geospatial data, with an aim towards producing ‘3D’ landscape visualizations in the Unity 3D rendering engine. Functions are also provided for retrieving elevation data and base map tiles from the USGS National Map. There is an Introduction and a vignette on importing terrain tiles.