Flexdashboard on R Views

Onboard and Offboard Data Manipulation in Flexdashboard

Wed, 23 Jan 2019 00:00:00 +0000

Harrison Schramm is a Professional Statistician and Non-Resident Senior Fellow at the Center for Strategic and Budgetary Assessments.

The Shiny set of tools, and, by extension, Flexdashboard, give professional analysts tools to rapidly put interactive versions of their work in the hands of clients. Frequently, an end user will interact with data by either uploading or downloading a new set in its entirety (typically from a .csv or other similarly structured source), or do so ‘on the fly’ interactively, using tools like RHandsonTable. What if you want to do both at the same time? That is – what if you want to be able to change data interactively, or completely upload a new database without stopping the current instance or other ‘clunky’ things?

It turns out that you can, and the implementation is relatively straightforward. In this post, our aims are to: a. Make the reader more familiar with reactivity, particularly with respect to initializing and containerizing variables b. Show how to manage a single object that can be ‘touched’ by the uploadHandler and rHandsonTable c. Provide a worked example in Flexdashboard

We need to create an environment where a data object that may be manipulated by functions inside an app is initialized to a pre-loaded set. At runtime, the object may be downloaded to a .csv, manipulated directly in a spreadsheet-like interface, or replaced completely with an upload.

In the following sections, we highlight the working parts of the minimal example (also provided) based on the attributes of different ships in Star Wars.

Step 1: Initialization

library(flexdashboard)
library(dplyr)
library(magrittr)
library(rhandsontable)

After calling the applicable libraries (above), we load a starter dataset so that the application isn’t empty when launched. We also create a reactiveValues object called values that sets the handsontable hot to NULL. This is necessary because the handsontable object is self-referential in the sense that it is both an output and input device.

BFL = read.csv("StarterExampleData.csv")
BFL$Exclude = FALSE
values <- reactiveValues(hot = NULL)

The main trick is to hold the R dataframe in a reactive environment that responds to both the handsontable object and the upload handler. This is done by a series of nested if statements, that, in order, make the object equal:

the initialization file, if the object is empty (from the previous code chunk),
the uploaded file (from the upload handler, below),
and finally the handsontable object.

Note the unglamorous statement read.csv(input$InputBFL$datapath). This is necessary when reading a file provided by an upload handler. Simply reading the handle attached to the csv will point to the ‘box’, not what is ‘in’ the box.

This is different than the approaches advocated on many forums where the initialization is done in the rhandsontable code. That approach is perfectly valid, but does not offer the flexibility of an uploaded file.

BFLD = reactive({
  if(is.null(input$hot)){BFL}
  else if(!is.null(input$InputBFL)){read.csv(input$InputBFL$datapath)} 
  else{
  hot_to_r(input$hot)
  }
})

Step 2: File Handlers

Now that we have set up the reactive structure for the object BFLD, which can be accessed by other functions inside our code, we add the functionality for the upload and download handlers.

The download function is two separate items: the downloadButton that triggers the action, and the downloadHandler that performs the interface. In Shiny apps, the linkage is explicit. In Flexdashboard, the linkage is made by putting the handler immediately after the button. Note: The handlers do not always work in the RStudio App window; it may be necessary to ‘open in browser’

downloadButton("StarWarsDownload", label = "Star Wars Download")
downloadHandler(
  filename = function(){"Star_Wars_Download.csv"},
  content = function(file){
    write.csv(BFLD(), file)
  }
)

The upload function is a single block. Here the input File is linked to the reactive variable built in Step 1 by the handle input$InputBFL.

fileInput("InputBFL", "Choose CSV File",
                multiple = FALSE,
                accept = c("text/csv",
                         "text/comma-separated-values,text/plain",
                         ".csv"))

Step 3: Hands On Table

In the final step, we transform the stored reactive object BFLD into a handsontable object, and output. Note that because it is a reactive object, we call it as BFLD() with parentheses, and inside a ‘render’ context.

output$hot = renderRHandsontable({
  rhandsontable(BFLD(), height = 550) %>%  hot_rows()
})
rHandsontableOutput("hot")

Conclusion

We hope that this minimal example will help you use both rhandsontable and upload/download handlers in flexdashboard. There should be enough code and explanations here for simpler cases. These code chunks should help you worry less about IO handling in your applications and spend more time on graphics and analysis. Look here to see the example working.

While we did not develop it here, it seems that this same construct could be used to add a web-based data source.

Enterprise Dashboards with R Markdown

Wed, 16 May 2018 00:00:00 +0000

This is a second post in a series on enterprise dashboards. See our previous post, Enterprise-ready dashboards with Shiny Databases.

We have been living with spreadsheets for so long that most office workers think it is obvious that spreadsheets generated with programs like Microsoft Excel make it easy to understand data and communicate insights. Everyone in a business, from the newest intern to the CEO, has had some experience with spreadsheets. But using Excel as the de facto analytic standard is problematic. Relying exclusively on Excel produces environments where it is almost impossible to organize and maintain efficient operational workflows. In addition to fostering low productivity, organizations risk profits and reputations in an age where insightful analyses and process control translate to a competitive advantage. Most organizations want better control over accessing, distributing, and processing data. You can use the R programming language, along with with R Markdown reports and RStudio Connect, to build enterprise dashboards that are robust, secure, and manageable.

This Excel dashboard attempts to function as a real application by allowing its users to filter and visualize key metrics about customers. It took dozens of hours to build. The intent was to hand off maintenance to someone else, but the dashboard was so complex that the author was forced to maintain it. Every week, the author copied data from an ETL tool and pasted it into the workbook, spot checked a few cells, and then emailed the entire workbook to a distribution list. Everyone on the distribution list got a new copy in their inbox every week. There were no security controls around data management or data access. Anyone with the report could modify its contents. The update process often broke the brittle cell dependencies; or worse, discrepancies between weeks passed unnoticed. It was almost impossible to guarantee the integrity of each weekly report.

Why coding is important

Excel workbooks are hard to maintain, collaborate on, and debug because they are not reproducible. The content of every cell and the design of every chart is set without ever recording the author’s actions. There is no simple way to recreate an Excel workbook because there is no recipe (i.e., set of instructions) that describes how it was made. Because Excel workbooks lack a recipe, they tend to be hard to maintain and prone to errors. It takes care, vigilance, and subject-matter knowledge to maintain a complex Excel workbook. Even then, human errors abound and changes require a lot of effort.

A better approach is to write code. There are many reasons to start programming. When you create a recipe with code, anyone can reproduce your work (including your future self). The act of coding implicitly invites others to collaborate with you. You can systematically validate and debug your code. All of these things lead to better code over time. Coding in R has particular advantages given its vast ecosystem of packages, its vibrant community, and its powerful tool chain.

Using R Markdown

There are many tools for replacing complex Excel dashboards with R code. One of these tools is R Markdown, an open-source R package that turns your analyses into high quality documents, reports, presentations and dashboards. R Markdown documents are fully reproducible and support dozens of output formats including HTML, PDF, and Microsoft Word documents.

Here is the same Excel dashboard translated to an R Markdown report. Because this report is written in code, it is vastly simpler and easier to maintain. Like the Excel dashboard above, this R Markdown report is designed to take user inputs so that it could render custom report versions.

Many people are already aware that R Markdown reports combine narrative, code, and output in a single document. What is less commonly known is that you can generalize any R Markdown report by declaring parameters in the document header. R Markdown documents with parameters are known as parameterized reports. In the Excel dashboard users can select segment, group, and period. In a parameterized R Markdown document, you would specify these inputs with the following YAML header:

---
title: Customer Tracker Report
output: html_notebook
params:
  seg: 
    label: "Segment:"
    value: Total
    input: select
    choices: [Total, Heavy, Mainstream, Focus1, Focus2, 
              Specialty, Diverse1, Diverse2, Other, New]
  grp: 
    label: "Group:"
    value: Total
    input: select
    choices: [Total, Core, Extra]
  per: 
    label: "Period:"
    value: Week
    input: radio
    choices: [Week, YTD]
---

You can then call the parameters you declare in the YAML header from your R code chunks.

```{r}
params$segment
params$grp
params$per
```

You can render the document with different inputs by selecting knit with parameters in RStudio. This option will open a user interface that allows you to select the parameters you want.

If you want to automate the process of creating custom report versions, you can render these documents programmatically with the rmarkdown::render() function.

rmarkdown::render(
  input = "tracker-report.Rmd", 
  params = list(seg = "Focus1", grp = "Core", per = "Weekly")
)

Publishing to RStudio Connect

Managing access and permissions for an ocean of Excel files is painful. Data in Excel spreads through an organization without controls like a virus spreads through a body without disease prevention. There are better ways to secure the operation, access, and distribution of information.

RStudio Connect is a server product from RStudio that is designed for secure sharing of R content. It is on-premises software you run behind your firewall. You keep control of your data and of who has access. With RStudio Connect, you can see all your content, decide who should be able to view and collaborate on it, tune performance, schedule updates, and view logs. You can schedule your R Markdown reports to run automatically or even distribute the latest version by email.

When you publish a parameterized R Markdown report to RStudio Connect, an interface appears for selecting inputs. Viewers can create new report versions, then email themselves a copy. Collaborators can save and schedule new report versions, then email others a copy. You can even attach output files to these versions. Using parameterized R Markdown documents in RStudio Connect is a powerful way to communicate information.

You can publish content from the RStudio IDE by clicking the Publish button that looks like a blue Eye of Horus. Pressing this button will begin the publishing process. First, it creates a set of instructions for recreating your content. Second, it deploys your content bundle to the server. Third, it recreates your content on RStudio Connect. Push-button publishing has a long history of being used with RStudio. In 2012, RStudio enabled push-button publishing of R Markdown documents to RPubs. In 2014, RStudio enabled push-button publishing of Shiny apps to shinyapps.io. In 2016, RStudio enabled push-button publishing to RStudio Connect.

Adding Shiny

R Markdown documents are rendered with batch processing. That makes them ideal for automation, long running workflows, and custom report versions. However, if you want your documents to be immediately reactive to user input, then you can add a Shiny runtime. These interactive documents behave like a Shiny application in that they must be hosted. You can host interactive documents and Shiny applications with RStudio Connect. Deciding when to choose between R Markdown, interactive documents, and Shiny applications is a subject for a later post.

Summary

Reproducible code in R leads to better analysis and collaboration. You can use parameterized R Markdown reports to create complex, interactive dashboards. Hosting these dashboards securely in RStudio Connect gives you control over accessing, distributing, and processing data. You can use the R programming language, along with with R Markdown reports and RStudio Connect, to build enterprise dashboards that are robust, secure, and manageable.

Click here for source code.

Printing From Flex Dashboard

Wed, 28 Jun 2017 00:00:00 +0000

Shiny applications of all stripes (including flexdashboard with runtime Shiny) are revolutionary in that they put the power of R directly in the end user’s hands without needing to interact directly with the language. A common way end-users wish to interact with their data is via a dashboard that they can manipulate on the fly. Flexdashboard streamlines the process of turning an R-based analysis into a dashboard, so R users can create good-looking output for their analyses - and deploy this output to the web - with very little additional effort.

There are various ways to extract information from flexdashboard, but no good way to programatically return a printable version of the dashboard itself. This is partially by design - responsive HTML dashboards aren’t meant to be printed. There are well-documented workflows about how to share information between a web-centric output and a complementary report that renders to a printable format.

That said, there are some reasons to want a printable output that looks similar to a related HTML report. Snapshots are valuable for documentation/training materials, for embedding in emails or other reports, or for the all-too-common case of meetings with executives who like dashboards but want to consume paper-based reports. Plus, flexdashboards are an incredibly quick way to create visually appealing (read: pretty) summary output of a combination of outputs on a single page.

Both Harrison and Aaron¹ have struggled with trying to print flexdashboards in their respective practices. In a recent instance, Harrison was working with a client whose end users would be carrying snapshots of the dashboard into an industrial setting where mobile devices were not desired. It was important to the client to maintain a similar look between the on-screen and paper representations of the dashboard. It was also important to make generating these snapshots an integrated part of the application; it was not sufficient to suggest that the client manually take a screenshot of their browser window. This turned out to be a blessing, because (as you will see below) it allows the creation of a web-friendly ‘tabbed’ report with a companion ‘flat’ printed rendering.

Specific Task

Our goal was to create a “printable” flexdashboard with runtime Shiny. Flexdashboard produces a beautiful HTML dashboard with very little effort. Like all R Markdown documents, it also allows Shiny objects to be embedded inside of it, so that when rendered locally or on a server, it produces a reactive web application with minimal developer effort. Like many HTML files, however, it does not print well.

The flexdashboard in question also had tabsets, which make perfect sense on a computer or mobile device, but don’t translate well to printed media. When printed, tabsets communicate that there is pertinent information hidden from the viewer; better to remove the tabset in the printed version. If the information is pertinent, it should be displayed in the printed document; if it is not, it should be removed.

PhantomJS and Headless Chrome

The approach we settled on as good enough and easy enough was to use a headless browser to take a screenshot of a non-reactive HTML document. It’s not a perfect solution: a PDF would be better than a png file, headless browsers add a code dependency, and there is some administrative overhead to create the document to print. It is, however, quick to implement, easy to understand, and robust.

Example²

The mechanics of building a screen-capture-style report from flexdashboard are neither difficult nor obvious. In general, it requires the following steps:

Add a downloadButton to a flexdashboard with runtime Shiny
When the download button is invoked, temporarily save non-reactive copies of any objects of interest
Knit a static HTML version of the flexdashboard, using the objects saved in the prior step
Use webshot::webshot or decapitated::chrome_shot to capture a .png image of the static dashboard³
Deliver this version to the client using the downloadHandler function

The solution requires two separate R documents. The process is as follows:

Dynamic Flexdashboard document


---
title: "Snapshot Example"
runtime: shiny
output:
  flexdashboard::flex_dashboard:
    orientation: rows
    vertical_layout: scroll
---

This is standard YAML header for a flexdashboard, including the command runtime: shiny to create a Shiny application from an R Markdown document. Let’s examine the next chunks:

gears = mtcars$gear %>% as.factor() %>% levels()
selectizeInput("grs", "GEARS",
               gears, selected = gears[1])
renderUI({
downloadButton("downloadFile", "Download")
})

In this chunk, we have built a selectizeInput object to choose the number of gears to select, as well as a downloadButton named ‘downloadFile’. Notice that we have wrapped the button in a renderUI() context. This is standard shiny / flexdashboard code.

mpgp = reactive({
  mtcars %>% filter(gear %in% input$grs) %>% ggplot(aes(x = hp, y = mpg)) + geom_point() + geom_smooth()
  
})
renderPlot({
  mpgp()
})

For veteran shiny programmers, this block may seem redundant. We’re creating a plot called mpgp in a reactive context and then rendering it. In a ‘standard’ shiny implementation, we would simply have wrapped the plot code in the renderPlot() context. However, as you will see below, we’re going to need to reuse the object mpgp.

output$downloadFile <- downloadHandler(filename = function() { 
return(paste('Cars', '.png', sep=''))
},
    content = function(file){
    to_save <- list(
    mpgg = mpgg(),
    mpgp = mpgp()
    )
      saveRDS(to_save, "config_data.RDS")
      rmarkdown::render("ShadowCars.Rmd")
      webshot::webshot("ShadowCars.html", file = file)
                        })

This chunk contains the secret sauce. When the downloadButton is invoked, the program saves all elements to appear in the downloaded report in a file called ‘config_data.RDS’. These elements are then used as arguments to render a shadow document, ‘ShadowCars.Rmd’. Finally, webshot::webshot converts the hidden document to a .png and delivers it to the client side.

The full dashboard looks like this:

Shadow Document

The key elements of the shadow document code are presented below:

---
title: "Snapshot Example"
output:
  flexdashboard::flex_dashboard:
    orientation: rows
---

Note that our YAML for the shadow document is similar to the dashboard, except it does not include runtime: shiny

The code blocks that render the objects created in the dynamic document are straightforward to the point of triviality; they simply reference the saved objects (which are retrieved using readRDS) and render them using whatever we choose, in this case kable() for the table and the native print() for the plot.

data <- readRDS("config_data.RDS")
data$mpgg %>% kable()

data$mpgp %>% print()

And of course, while mpgp was reactive in the original dashboard, it is static in the shadow doc.

The report delivered to the client is as follows:

Conclusion

In this short post, we have shown how to create a downloadable custom report by using existing tools to create a static flexdashboard. From the .png environment, the rendered image can be converted to other formats as required.

Harrison Schramm is an Operations Research Analyst at CANA Advisors. Aaron Berg is a Customer Success Representative at RStudio.↩
To recreate this minimal example for yourself, please download the files from this gist.↩
The webshot package requires phantomJS to be installed as an external dependency, but will work on all systems. decapitated will only work on systems with versions of Chrome that can operate in headless mode.↩

Flexdashboard on R Views

Onboard and Offboard Data Manipulation in Flexdashboard

Step 1: Initialization

Step 2: File Handlers

Step 3: Hands On Table

Conclusion

Enterprise Dashboards with R Markdown

Why coding is important

Using R Markdown

Publishing to RStudio Connect

Adding Shiny

Summary

Printing From Flex Dashboard

Specific Task

PhantomJS and Headless Chrome

Example2

Dynamic Flexdashboard document

Shadow Document

Conclusion

Example²