Interview on R Views

Interview with Oscar Baruffa, Creator of the Big Book of R

Tue, 04 Jan 2022 00:00:00 +0000

Welcome to the new year! If you’re itching to improve your R skills in 2022, we have the resource for you.

We’re excited to share the Big Book of R. “Your last-ever bookmark”, the Big Book of R is an impressive collection of R-related books from a variety of subjects. Creator Oscar Baruffa first published the book in August 2020. Since then, it has grown from a list of 80 to over 200, garnering 73,000 unique visitors and 195,000 pageviews from readers around the globe.

The Book is organized by different subjects that range from introduction to R programming to big data to archeology. Its organization and search functionality make it easy for newcomers to find books related to their topic of interest.

The Big Book of R is a wonderful example of collaboration in the R Community. Oscar wrote the book using the bookdown package. Contributors can file an issue or create a pull request on Github. And of course, the book would not be possible without the authors who have written books to guide others on their R journey.

While you are preparing for 2022, we encourage you to cozy up to one of the great books in the Big Book of R.

Check out the Big Book of R
Contribute to the Github Repository
Follow Oscar on Twitter
Subscribe to the newsletter

Interview with Oscar

Hello! Could you tell us a bit about yourself, please?

I’m a South African now living in the Netherlands, working as a Data Specialist in an international development non-profit focused on sustainable trade systems. My role is basically that of a senior analytics manager and I’m the first one, so I get to have all the fun directing the development of our data pipeline! No really, it’s a lot of fun :).

I studied Mechanical Engineering in my undergraduate and masters degrees and have been dabbling in tech-related side projects and hobbies for many years.

How did you get started with the R Community?

I think it was sometime in late 2018 when I was busy learning a bit of Python and Jesse Mostipak started popping up on my Twitter feed. She made R sound quite fun so I thought I’d give it a try. After I got to the exercise in R for Data Science by Hadley Wickham and Garrett Grolemund where you’re introduced to faceting a plot, my mind was blown and I was hooked.

I then started participating in the #TidyTuesday challenge and recorded some screencasts on my “Other People’s RStats” YouTube channel. I took a bit of a leap of faith submitting a lightning talk for satRday Johannesburg in April 2019, which was being organised by Andrew Collier and Megan Beckett. I was selected, which was a welcome surprise for my little topic. They were so helpful when I was trying to figure out the pull request flow for submitting my presentation. I also attended my first-ever R workshop the day before (which Andrew presented), which was the first time I’d ever sat in a room with other people who love R. It was awesome!

While doing all of this, I was also following more and more people on Twitter who were tweeting about R, collecting bookmarks of packages, tutorials, and, of course, books. I was having a lot of fun. By early 2020, I wrote my first book with Veerle van Son called Twitter for R Programmers as a way to introduce others to the R community on the social platform. So by that point, you could say I’d become heavily invested in the community :).

What inspired you to start the Big Book of R?

I had been diligently collecting bookmarks of books as I was finding them. After about 2 years of doing so, I had an inkling that I must have a large collection. One day, I counted about 80. I compared it to other lists of books that I’d seen published and I had way more, so I figured this might be quite unique to have so many.

Having had written my own short book, I also appreciated how much effort it took and I felt there must be better usability for readers and discoverability for authors if books were all listed in one place and grouped by topic (I invented a library - haha!). I also hoped (still do) that it might encourage more writing too. I put all the books together, spent some time categorising them, and in August 2020 published it. It made quite a splash!

Tell us about the design choices you made to make the book inclusive and open to the community.

I opted to use bookdown and git as I already had a bit of experience using them for Twitter for R Programmers. It felt very “meta” and fitting to use bookdown, which had in turn been used for almost all of the books in the collection. I was hoping this format would allow others to submit books as well — and they’ve generously done so. I kept the collection as a plain text format in the hope that it would slightly lower the barrier to people submitting books, but in the end, many people just tag me in a tweet which is welcome :).

What has surprised you the most about this project?

I knew people would like this but I didn’t expect how much it would be appreciated. Every now and then, I get a message of thanks for creating and maintaining it that really makes me feel warm and fuzzy inside. I hope people also reach out to the authors and do the same. Their effort in writing these books is immense and a little bit of appreciation will make their day — guaranteed!

What also surprises me is how there’s very consistent spikes in views and a steady growth in daily visitors whenever someone else shares the Big Book of R. If you’re interested to see the analytics, I’ve made them publicly accessible.

What’s in store for the Big Book of R in 2022?

I’m sure the collection will keep growing :).

I’ve just remodeled how the content is generated. Instead of capturing the book entries directly into markdown, it’s now generated by reading the data from a Google sheet. This new setup gives me the flexibility to do things like alphabetize the books more easily, add additional fields and tags more easily, set up Twitter/LinkedIn bots to automatically post about books in the collection, set up scripts to detect book updates, etc. Basically, I want to open up more possibilities for further automation and discoverability. If anyone is building anything using this data, I’d love to hear about it.

If anyone has ideas of how to improve, please get in touch with me or submit an issue in the repo.

Do you have any ongoing or upcoming projects you’d like the R Community to know about?

If things work out, there’s a chance I’ll be a technical reviewer on two R books being worked on in 2022, so that’ll be a new experience for me that I’m looking forward to. I’m going to keep writing useful articles about R, data and data careers over on my blog (and releasing fun R-related products here and there). The best way to be notified of those is to sign up to my newsletter. I’m also looking to write some more on the topic of Project Management to build upon my other bit of work that I’m really proud of, Project Management Fundamentals for Data Analysts.

Closing question: what is your favorite R package right now?

I think I’d give that accolade to {dplyr}! I’m pretty sure that filter() and group_by() are my most-used functions. Nothing gives me greater pleasure than a good anti_join().

We at RStudio would like to thank Oscar for his contribution to RViews and the creation of a great resource for the R Community. Happy reading in 2022!

R for Quantitative Health Sciences: An Interview with Jarrod Dalton

Wed, 06 Feb 2019 00:00:00 +0000

This interview came about through researching R-based medical applications in preparation for the upcoming R/Medicine conference. When we discovered the impressive number of Shiny-based Risk Calculators developed by the Cleveland Clinic and implemented in public-facing sites, we wanted to learn more about the influence of R Language in the development of statistical science at this prominent institution. We were fortunate to have Jarrod Dalton of the Quantitative Health Sciences Department grant this interview.

Jarrod Dalton, PhD is an assistant staff scientist in the Department of Quantitative Health Sciences and an assistant professor of medicine in the Cleveland Clinic Lerner College of Medicine at Case Western Reserve University in Cleveland, Ohio. (Twitter: @daltonjarrod)

JBR: QHS has been a leader in medical statistical research for quite some time. Have recent developments in big data, data science and machine learning changed the nature of your work? Are these trends bringing new tools and new challenges to QHS projects?

JD: Yes and no. On one hand, the health care sector has been at least as dynamic over the past 10-20 years as the fields of data science and machine learning. Clinical and biological research is quite diverse, and these trends have only added to its diversity. We work on studies with p>>n, n>>p, and everything in between. The complexity of problems in biomedical research has spurred new methodological innovations by our department, such as random survival forests. Modern machine learning algorithms are amenable to certain types of problems; I’d say that the most impactful manifestations to date of machine learning in medicine are in the fields of radiology and genomics, where diagnostic and prognostic problems are well-defined.

On the other hand, there are unique challenges relating to the application of machine learning algorithms in medicine. Doctors and patients are averse to trusting a model if they don’t understand how and why the prediction is being made. Doctors often have justifiable and either unquantified or unquantifiable reasons for disbelieving predictions on the basis of other clinical information they obtain during the process of care. Predictions inform treatment decisions (or decisions not to administer treatments), and many other issues are involved with optimal clinical decision-making (e.g., physician judgment, patient preferences, cost effectiveness of different therapies, discounting rate for health events that are distant in the future, trade-offs between quality of life and longevity, issues relating to health literacy, numeracy and the communication of risk, and desired degree of participation in the decision-making process on behalf of the patient).

Much of our work has, and will continue to be, in the clinical trials space, as well as good-old biostatistical consulting. We have a department of over 100 people and publish over 400 academic research studies every year, the vast majority of which arising from traditional statistical collaboration.

JBR: Ten years ago, a typical medical statistics department might consist of a number of Ph.D. statisticians who did almost no coding supported by a legion of SAS programmers. Have open-source languages such as R and Python changed the way work gets done? Do more statisticians now do their own coding? Do you see a movement away from SAS towards open source tools? Do you see clinicians doing their own analyses with R?

JD: We have dedicated consulting teams that are embedded within some of the more research-intensive clinical specialties at our institution, such as heart and vascular, oncology, urology, anesthesiology, neurology, and orthopaedics. Another consulting team, which we call the “alpha-beta team”, is composed of statisticians who allocate their time to the smaller sub-specialties. More recently, we have been successful at establishing externally funded research labs, headed by QHS principal investigators. Each of these teams has their own way of doing things. We are supported by a number of dedicated RStudio servers, as well as a high-performance computing cluster with R and Python capabilities. On the SAS front, our institution has a high-performance Enterprise Miner environment to support both research and business intelligence. All this having been said, roughly speaking, about half of our department uses R and half uses SAS. Some of our researchers in genomics and image analytics use Python, or complex pipelines that incorporate Python, R, and other tools.

I personally have used R since 2002. I have seen the power of open-source software, with R constantly reinventing itself in a variety of ways. I can’t believe that ggplot and plyr are 10 years old. The tidyverse has changed the way I think. This is especially so for the dplyr and purrr packages, which have enabled much greater efficiency and transparency. My team has recently taken advantage of distributed database computing via a dbplyr/Teradata Warehouse stack, using electronic health data from 2.7 million patients.

More and more physician scientists are training with R. Our partner university, Case Western Reserve, has a two-course sequence on data science based in R, and that sequence is a component of several Master’s programs that many of our clinicians pursue. Some of them can code up a storm! Others know just enough to be dangerous.

JBR: The number of Shiny-based Risk Calculators implemented on your website is astounding, both in their level of sophistication and in the number of topics covered. What is your goal for this project, and how would you like the calculators to be used? Can you say something about the challenges (both medical and technical) you faced in building these calculators?

JD: The goal of the project is to inform clinicians as to our best estimates of predicted outcomes. These predictions have been shown in several studies to be more accurate than clinical judgment or crude decision trees. Ultimately, these more accurate predictions should translate into better medical decision-making, especially with regard to treatment selection. The major challenge occurs up front: working with the clinician to clearly articulate the prediction that is needed – that which would be most hopeful for prospective decision-making. Modeling usually goes very well except when the outcome of interest is very rare: those models often turn out to be not very useful clinically because they never predict a high probability for the rare outcome.

From a technical perspective, the challenge is how to make our data insights and predictive models available online. Before R shiny, our RiskCalc team had tried several web platforms and were not satisfied. We have some very sophisticated models and those platforms either do not support complex computing algorithms or require a lot of programming effort. Using R shiny makes the process of converting our models into web applications quick and easy. The next steps for our RiskCalc team are to improve the user interface and collect feedback from the clinicians.

JBR: How important is reproducible research to QHS, and what role does R play in building reproducible workflows?

We have seen a steady progression toward reproducible research practices in medicine. All clinical trial protocols must now be pre-registered, and proof of adherence to the pre-registered protocol is now a standard requirement of many of the top journals. Somewhat controversially, there has recently been a lot of discussion about a “replication crisis”, particularly in the psychological sciences (but perhaps unfairly so). In any case, the increased focus on replicability has led to an increased need for reproducible research practices.

Nik Krieger and I have recently made an R package, called projects, that is specifically designed for reproducible academic manuscript development workflows. While the projects package has other features - like the ability to develop and maintain a coauthor database, complete with institutional affiliations, or the ability to automatically generate title pages for manuscripts using the authors’ institutional affiliations - its core functionality is establishing project directories with Markdown templates corresponding to each phase of the academic research pipeline (protocol, data wrangling, analysis, and reporting). Project metadata are stored in a tibble, so that teams can prioritize and strategize directly from the R console.

JBR: Do you have any additional thoughts about the use of R in Medicine that you would like to share with our readers?

What comes to mind are current challenges in integrating R into production environments in medicine, such as the electronic health record (EHR). EHR systems are not open-source, and there are many vendors. Even for a single vendor, implementations at different institutions may look wildly different from a data perspective. Our EHR system has over 10,000 tables. There are so many challenges to implementing anything in the healthcare space. That may sound pessimistic, but I actually intend to communicate the significant number of opportunities for using R to positively influence the health of populations. We have fantastic clinical partners and champions. We’re always getting better. The work is important and rewarding.

A Mathematician's Perspective on Topological Data Analysis and R

Wed, 14 Nov 2018 00:00:00 +0000

A few years ago, when I first became aware of Topological Data Analysis (TDA), I was really excited by the possibility that the elegant theorems of Algebraic Topology could provide some new insights into the practical problems of data analysis. But time has passed, and the sober assessment of Larry Wasserman seems to describe where things stand.

TDA is an exciting area and is full of interesting ideas. But so far, it has had little impact on data analysis.

Nevertheless, TDA researchers have been quietly working the problem and at least some of them are using R (see below). Since I first read Professor Wasserman’s paper, I have been very keen on getting the perspective of a TDA researcher. So, I am delighted to present the following interview with Noah Giansiracusa, Algebraic Geometer, TDA researcher and co-author of a recent JCGS paper on a new visualization tool for persistent homology.

Q: Hello Dr. Giansiracusa. Thank you for making time for us at R Views. How did you get interested in TDA?

While doing a postdoc in pure mathematics (algebraic geometry, specifically) I, probably like many people, could not escape a feeling that crept up from time to time—particularly during the more challenging moments of research frustration—that perhaps the efforts I was putting into proving abstract theorems might have been better spent working in a more practical, applied realm of mathematics. However, pragmatic considerations made me apprehensive at that point in my career to take a sudden departure, for I finally felt like I was gaining some momentum in algebraic geometry, developing a nice network of supportive colleagues, etc., and also that I would very soon be on the tenure track job market and I knew that if I was hired (a big “if”!) it would be for a subject I had actually published in, not one I was merely curious about or had recently moved into. But around this same time I kept hearing about an exciting but possibly over-hyped topic called topological data analysis: TDA. It really seemed to be in the air at the time (this was about five years ago).

Q: Why do you think TDA took off?

I can only speak for myself, but I think there were two big reasons that TDA generated so much buzz among mathematicians at the early stages.

First, it was then, and still is impossible to escape the media blitz on “big data” and the “data revolution” and related sentiments. This is felt strongly within academic circles (our deans would love us all to be working in data it seems!) but also in the mainstream press. Yet, I think pure mathematicians often felt somewhat on the periphery of this revolution: we knew that the modern developments in data and deep learning and artificial intelligence would not be possible without the rigorous foundations our mathematical ancestors had laid, but we also knew that most of the theorems we are currently proving would in all likelihood play absolutely zero role in any of the contemporary story. TDA provided a hope for relevance, that in the end the pure mathematician would come out of the shadows of obscurity and strike a data science victory proving our ineluctable relevance and superiority in all things technical—and this hope quickly turned to hype.

I think I and many other pure mathematicians were rooting for TDA, to show the world that our work has value. We tired of telling stories of how mathematicians invented differential geometry before Einstein used it in relativity and your GPS would not be possible without this. We needed a more fresh, decisive victory in the intellectual landscape; number theory used in cryptography is great, but still too specialized: TDA had the promise of bringing us into the (big) data revolution. And so we hoped, and we hyped.

And second, from a very practical perspective, I simply did not have time to retrain myself in applied math, the usual form of applied math based heavily on differential equations, modeling, numerical analysis, etc. But TDA seemed to offer a chance to gently transition to data science mathematical relevance—instead of starting from scratch, pure mathematicians such as myself would simply need to add one more chapter to our background in topics like algebraic topology and then we’d be ready to go and could brand ourselves as useful! And if academia didn’t work out, Google would surely rather open the doors of employment to a TDA expert than to a traditional algebraic topologist (or algebraic geometer, in my case).

I think these are two of the main things that brought TDA so much early attention before it really had many real-world successes under its belt, and they are certainly what brought me to it; well, that and also an innocent curiosity to understand what TDA really is, how it works, and whether or not it does what it claims.

Q: So how did you get started?

I first dipped my toes in the TDA waters by agreeing to do a reading course with a couple undergraduates interested in the topic; then I led an undergraduate/master’s level course where we studied the basics of persistent homology, downloaded some data sets, and played around. We chose to use R for that since there are many data sets readily available, and also because we wanted to do some simple experiments like sampling points from nice objects like a sphere but then adding noise, so we knew we wanted to have a lot of statistical functions available to us and R had that plus TDA packages already. While doing this I grew to quite like R and so have stuck with it ever since. In fact, I’m now using it also on a (non-TDA) project to analyze Supreme Court voting patterns from a computational geometry perspective.

Q: Do you think TDA might become a practical tool for statisticians?

First of all, I think this is absolutely the correct way to phrase the question! A few years ago TDA seemed to have almost an adversarial nature to it, that topologists were going to do what data scientists were doing but better because fancy math and smart people were involved. So the question at the time seemed to be whether TDA would supplant other forms of data science, and this was a very unfortunate way to view things.

But, it was easy to entirely discredit TDA by saying that it makes no practical difference whether your data has the homology of a Klein bottle, or there were no real-world examples where TDA had outperformed machine learning. This type of dialogue was missing the point. As your question suggests, TDA should be viewed as a tool to be added to the quiver of data science arrows, rather than an entirely new weapon.
In fact, while this clarification moves the dialogue in a healthy direction (TDA and machine learning should work together, rather than compete with each other!) I think there’s still one further step we should take here: TDA is not really a well-defined entity. For instance, when I see topics like random decision forests, it looks very much like topology to me! (Any graph, of which a tree is an example, is a 1-dimensional simplicial complex, and actually if you look under the hood, the standard Rips complex approach in TDA builds its higher dimensional simplicial complexes out of a 1-dimensional simplicial complex, so both random forests and TDA—and most of network theory—are really rooted in the common world of graph theory.)

Another example: the 0-dimensional barcode for the Rips flavor of TDA encodes essentially the same information as hierarchical clustering. All I’m saying here is that there’s more traditional data science in TDA than one might first imagine, and there’s more topology in traditional data science than one might realize. I think this is healthy, to recognize connections like these—it helps one see a continuum of intellectual development here rather than a discrete jump from ordinary data science to fancy topological data science.

That’s a long-winded way of saying that you phrased the question well. The (less long-winded) answer to the question is: Yes! Once one sees TDA as one more tool for extracting structure and statistics from data, it is much easier to imagine it being absorbed into the mainstream. It need not outperform all previous methods or revolutionize data science, it merely needs to be, exactly as you worded it, a practical tool. Data science is replete with tools that apply in some settings and not others, work better with some data than others, reveal relevant information sometimes more than others, and TDA (whatever it is exactly) fits right into this. There certainly will be some branches of TDA that gain more traction over the years than others, but I am absolutely convinced that at least some of the methods used in TDA will be absorbed into statistical learning just as things like random decision trees and SVMs (both of which have very topological/geometric flavors to them!) have. This does not mean that every statistician needs to learn TDA, just as not every statistician needs to learn all the latest methods in deep learning.

Q: Where do you think TDA has had the most success?

Over the past few years I think the biggest strides TDA has made have been in terms of better interweaving it with other methods and disciplines—so big topics with lots of progress but still room for more have included confidence intervals, distributions of barcodes, feature selection and kernel methods in persistent homology. These are all exciting topics and healthy for the long-term development of TDA.

I think, perhaps controversially, the next step might actually be to rid ourselves of the label TDA. For one thing, TDA is very geometric and not just topological (which is to say: distances matter!). But the bigger issue for me is that we should refer to the actual tools being used (mapper, persistent homology in its various flavors, etc.) rather than lump them arbitrarily together under this common label. It took many years for statisticians to jump on the machine learning bandwagon, and part of what prevented them from doing so sooner was language; the field of statistical learning essentially translates machine learning into more familiar statistical terminology and reveals that it is just another branch of the same discipline. I suspect something similar will take place with TDA… er, I should say, with these recent topological and geometric data analysis tools.

Q: Do you think focusing on the kinds of concrete problems faced when trying to apply topological and algebraic ideas to data analysis will turn out to be a productive means of motivating mathematical research?

Yes, absolutely—and this is also a great question and a healthy way to look at things! Pure mathematicians have no particular agenda or preconceived notion of what they should and should not be studying: pure mathematics, broadly speaking, is logical exploration and development of structure and symmetry. The more intricate a structure appears to be, and the more connected to other structures we have studied, the more interested we tend to be in it. But that really is pretty much all we need to be interested—and to be happy.

So TDA provides a whole range of new questions we can ask, and new structures we can uncover, and inevitably many of these will tie back to earlier areas of pure mathematics in fascinating ways—all the while, throughout these explorations pure mathematicians likely will end up laying foundations that help provide a stable scaffolding for the adventurous data practitioners who jump into methodology before the full mathematical landscape has been revealed. So TDA absolutely will lead to new, important mathematical research: important both because we’ll uncover beautiful structures and connections, and important also because it will provide some certainty to the applied methods build on top of this.

Q: More specifically, what role might the R language play in facilitating the practice or teaching of mathematics?

Broadly speaking, I think many teachers—especially in pure mathematics—undervalue the importance of computer programming skills, though this is starting to change as pure mathematicians increasingly enjoy experimentation as a way of exploring small examples, honing intuition, and finding evidence for conjectures. While the idea of theorem-proof mathematics is certainly the staple of our discipline, it’s not the only way to understand mathematical structure. In fact, students often find mathematical material resonates with them much more strongly if they uncover a pattern by experimenting on a computer rather than just being fed it through lecture or textbooks. Concretely, if students play with something like the distribution of prime numbers, they might get excited to see the fascinating interplay between randomness and structure that emerges, and that can better prepare them to appreciate formally learning the prime number theorem in a classroom. So as things like TDA emerge, the number of pure mathematics topics that can be explored on a computer increases, and I think that’s a great thing.

Q: Where does R fit in?

Well, much of the mathematical exploration I’m referring to here is symbolic—so very precise and algebraic flavor—and R certainly has no limitations working precisely, but it’s not the main goal of the language so one likely would use a computer algebra system instead. But, one exciting thing TDA does help us see is that there’s a marvelous interface between the symbolic and numerical worlds (here represented by the topology and the statistics, respectively) and I think this is great for both teaching and research. The more common ground we find between topics that previously seemed quite disparate, the more chance we have of building meaningful interdisciplinary collaborations, the more perspectives we can provide our students to motivate and study something, and the more we see unity within mathematics. My favorite manifestation of this is that TDA is the study of the topology of discrete spaces—but discrete spaces have no non-trivial topology! What’s really going on then is that data gives us a discrete glimpse into a continuous, highly structured world, and TDA aims to restore the geometric structure lost due to sampling. In doing so one cannot, and should not, avoid statistics, so pure mathematics is brought meaningfully in contact with statistics and I absolutely love that. This means the R language finds a role in pure math where it may previously not have: topology with noise, algebra with uncertainty.

Thank you again! I think your ideas are going to inspire some R Views readers.

Editors note: here are some R packages for doing TDA:
* TDA contains tools for the statistical analysis of persistent homology and for density clustering.
* TDAmapper enables TDA using Discrete Morse Theory.
* TDAstats offers a tool set for TDA including for calculating persistent homology in a Vietoris-Rips complex.
* pterrace builds on TDA and offers a new multi-scale and parameter free summary plot for studying topological signals.

Interview with Joe Cheng

Wed, 04 Jan 2017 00:00:00 +0000

Recently, I had the opportunity to interview RStudio’s Joe Cheng. Joe, the inventor and lead developer for Shiny, was the first person that J.J. Allaire invited to join the RStudio IDE project. We talked about those early days, how Shiny got started, Joe’s background as a software developer, his take on the R language and more. What follows is an edited transcript of our conversation.

JBR: Hello Joe, thank you for being with us today. In the interview we did with J.J. last year, he said that eight years or so ago, he worked on an early version of the RStudio IDE for about nine months before he realized that he couldn’t do the whole thing by himself. About what happened next, J.J. said:

So I recruited Joe Cheng to work with me. Joe and I had worked together at a couple of previous companies. At that time I told him, Joe, I make no representation or promise that this is going to be a company. It just might be an open-source project. Is that okay with you?

Then J.J. said you replied, “That’s fine.”

Really? You just said, “That’s fine.” Did you really say that? This must have been quite a big deal for you to commit to something so uncertain. What were you thinking? Can you tell us a bit about the backstory?

Joe Cheng: I had started working for J.J.’s first company back in 1996 when I was a college freshman. I was an intern in their web development department. I never met J.J. then, but I worked closely with the first employee at Allaire, a guy name Charles Teague. Several years later, Charles kind of called me out of the blue and said he was starting another company with J.J., a very small one. This is about 2002 or 2003. I went to work for Onfolio when it was about five or six people. I got to know J.J. very well.

I’ve worked for many start-ups in my career. I really enjoy early-stage startups and by far, to me, the most important thing, more important than a business model, more important than who’s investing is whether the founder is someone that you trust and whose values align with yours. I just thought J.J. was an amazing individual and he really believed in every piece of software we ever made together. He did it for the right reasons and really did things with a lot of respect for our users and customers, and really believed in taking care of his employees. He proved that again and again while we had worked together over the years.

For me, there is nothing risky about working for any project that J.J.’s involved with. I know that whatever would happen, whether something amazing was going to happen with R, or whether it wasn’t a good bet, that if you work for J.J. really interesting things are going to happen and good things tend to follow. The other consideration was that I had been working in the startup world since I graduated from college. I don’t think that I was involuntarily out-of-work for more than a week and a half or two weeks.

The bottom line was that I thought for this particular industry at that particular moment it was OK if the opportunity was unstable. There was lots of interesting stuff happening all over the industry. If it didn’t work out I would just move on to the next thing. I didn’t join J.J. because I thought there was an amazing business model or an amazing business to be built, but because I thought the technical problems were interesting and I believed in J.J. That was really all that mattered to me at the time. But I’m pleasantly surprised that it’s worked out as well as it did.

JBR: So you worked on the idea with J.J. for a while and then I guess three years ago or so you had the idea for Shiny. In an interview you did at useR! 2014, you talked about how the JavaScript Meteor project sparked the idea for you.

Joe Cheng: Yes.

JBR: But besides the reactive programming tricks – I think that’s the word you used – what was the big idea that really excited you? Was it bringing JavaScript to the R world or designing new workflows or something else?

Joe Cheng: I guess I have a little bit of a meta answer to that. I really get excited when I can do something that not that many other people know how to do. If I can take that ability and let other people do it, that’s really exciting. Web development is not an uncommon skill. Obviously there are a lot of people doing web development, but it is not common in the R community, or at least not as common as in other programming communities.

What got me excited was that there was clearly this hunger for interactivity. People clearly wanted to be able to build interactive applications, reports and other artifacts. Just from talking to people I got the impression that they felt doing things interactively was just beyond them; far out of their reach. In 2010 or 2011, the very first time we talked to Danny Kaplan, a professor at Macalester, about the first versions out of RStudio IDE he told us: “That’s really cool and I’ll definitely use that, but what would really be nice is if I could stop hiring grad students to build the whole Java app that’s to demonstrate statistical concepts. I really want to be able to do this myself.” We heard versions of that over and over and over again.

Right from that first time I told J.J. that we could definitely could do that. Obviously, we knew how to build web applications. We knew how to build libraries. We could have built a web app framework for R. But, I argued that we shouldn’t do it until we knew the right way to do it, meaning a way to express web applications that’s actually natural to do in R. I knew this had to be more than just taking the same idioms you would do in JavaScript, in Java or Ruby, and just port those over to R. I didn’t think that would do anyone any good. We wanted to find a library API that was high-level enough that you could really focus on the intent of what you were doing, but low-level enough that there infinite possibilities of what you could build.

At the time that I made that statement, I really had no idea. I had no idea how we would solve that problem. I think those ideas percolated in my head and then finally seeing Meteor made the last pieces click. To me, it was the culmination of a lot of ideas that I’ve been building up over the course of my programming career and it happened to find a really receptive audience in the R community, so that’s been great.

JBR: Did you think Shiny would become so popular in such a short time?

Joe Cheng: I thought it had a shot at being quite popular, but I was surprised at is how deeply people got into it right away. They didn’t just see it and kind of casually start using it. Apparently, a number of people decided right away that they were going to spend hours doing this every day. Some people have been heavily investing in Shiny for like four or five years now. That was surprising to me. I knew that people would use it to build simple apps, but I was absolutely shocked how quickly people decided to really build significant things on top of it.

In a way, that was a really pleasant surprise, because it showed that we were on to something, but it also increased the pressure on us. I thought that I would have more time to build things out. I thought there would be a couple of years at least before people wanted the things that they started to build after only six months. The first couple of years after we released Shiny were really hectic. We were trying to catch up to what people wanted to do immediately with the framework.

I am also surprised at how long the growth curve has continued. Here we are four or five years after Shiny was first introduced and still every year it seems like it gets bigger and bigger. I keep thinking: “Okay, now we’ve hit peak Shiny,” but every year going to conferences you see more and more and more Shiny everywhere, which is obviously very gratifying.

JBR: How mature is Shiny now? Are there still lots of new features that you want to add or is it getting to be really close to a finished product?

Joe Cheng: I don’t know if it will ever be finished. I don’t know if that’s the nature of the kind of problem it’s trying to solve. But for quite some time now, I’ve felt like the capabilities of the framework and the features we’ve built in have far outstripped our ability to teach them effectively to the community. In that sense, Shiny is a more mature product than I think most people realize, and it has more features than most people are using. However, there are certainly some big areas where new features are needed. Right now, we are especially focused on testing and on long-running tasks. There are still a number of things that we are very interested in doing.

The other thing that’s really interesting about Shiny is that every year there’s been at least one or two big surprises where Shiny turned out to be useful. People are using Shiny in scenarios that we never envisioned. For example, the idea of gadgets and Shiny gadgets being integrated in the IDE was a new thing that started in 2015. Or another example, the first year after we released Shiny I stumbled upon a pattern of using reactive programming to do its opposite: imperative programming. It turns out that you can use Shiny to effectively mix the two styles and do reactive and imperative programming. I certainly didn’t envision that! Yes, I continue to be surprised at new vistas opening up in front of Shiny, and so far, there’s no evidence that this is slowing down.

JBR: If you could restart Shiny from scratch today, what would you change?

Joe Cheng: That’s a good question. When I began working on Shiny, some of the ideas around reactivity were not crystal clear in my head the way that they are today. I was stumbling along, going by instinct when I defined some of the early primitives, like reactive values, reactive expressions, observers. I did actually end up with the right set of primitives, but I didn’t understand their characteristics well enough to give them good names.

The biggest thing I regret is naming the reactive expressions, the functions called reactive and observers. Those names are kind of like a nod back to Meteor and some other literature I had read about reactive programming. But they don’t actually help to convey how these functions should be used. I really wish I could go back and rename the function reactive to be rx_calc and the function observer to be rx_exec. These names better reflect what the functions should be used for.

We have also been very happy with the technical bets we’ve made and how they’ve played out so far. That’s not to say someone won’t discover some critical mistake in the future, but so far this is really been kind of a charmed project. I’ve been extremely happy with how well the technology’s held up over the years.

JBR: Let’s change gears a little here and talk about R. You’ve gathered a tremendous amount of experience working with R as a developer. What’s your take on the R language from a computer scientist / software developer’s point of view? What features of R ought to inform any discussion of comparing R with other languages?

Joe Cheng: I think R is actually a pretty underrated as a language. This is probably because it has some basic features that are so foreign coming from other languages. One example is having everything vectorized. Delayed evaluation for function arguments is another.

About ten or fifteen years ago, I read a book Paul Graham wrote before his Y Combinator days, called ANSI Common Lisp. At the time, I was a Java programmer building websites and whatever, and it was an absolutely eye-opening, mind-expanding experience reading that book. As I remember it, one of his main points about why Lisp is such an amazing language is that in other languages, you build abstractions by writing functions, writing classes, and then calling them. Whereas in Lisp, it’s almost like you change the language itself to be a DSL for whatever problem you’re trying to solve. Most other languages don’t have this flexibility, certainly not Java, which at the time was my main point of comparison. It was really frustrating to me to read about these incredible ideas and this new way of solving problems – new to me, anyway – and not have the expressiveness and power in the language that I was used to and that I had access to for my day-to-day work.

I’ve really felt that way ever since then about every language that I’ve worked in. Ruby came close in some ways, but still it would not let you compute on the language in quite the same way that you could do with Lisp. Even R is not all the way there, but it is shockingly close. If you look beyond the syntax, R really is conceptually very much like Lisp in a lot of ways. One of those ways is that it makes it very, very easy to compute on the programming language itself.

When we build APIs, like for dplyr, for Shiny, we basically are not just saying here’s this little routine that you can call any time you need this kind of calculation. We are giving you a different way to express problems in code. R is just an incredible language for letting you do that. Features like formulas, or combining delayed evaluation with the substitute() function. There are many different ways you can take your standard evaluation model and turn it on it’s head to accomplish whatever it is you’re trying to accomplish. For doing reactive programming, it’s a great advantage, I think, to have a language that’s as flexible as R.

I think for day-to-day, R programmers probably don’t think about these things, but the elegant, terse syntax of dplyr and the pipe operator are possible because of how malleable a language R is and how great it is for writing DSLs in it.

Personally, one of my pet peeves during these language wars is when people say that one of the differences between say Python or Julia and R is that R is a DSL for stats, whereas these other things are general purpose languages. R is not a DSL. It’s a language for writing DSLs, which is something that’s altogether more powerful. I actually think that Julia has many of these same characteristics, but Python, even though it obviously has its own strengths, certainly doesn’t share that same level of flexibility.

JBR: Okay, so now you partially touched on this next question, but let me go and ask it anyway. To a large extent, R as it currently exists was shaped by statistician developers who are interested in a platform for computational statistics. Do you think that the work that you’re doing as you just described and with R with JavaScript features, is going to change the personality of R itself?

Joe Cheng: I think in some ways it already has. At least that’s the way a lot of people I engage with through supporting Shiny and talking at conferences see it. The way they think about expressing R code seems to have changed a bit over the last few years. Largely due to Hadley and his Tidy Manifesto, the personality of R has changed. There is more of a focus now on the consistent expression of similar problems and thinking about things from the user’s point of view. It’s not just about making calculations possible, but also thinking about how users want to consume libraries.

JBR: Should statisticians be worried about the influence of computer science and software engineering that is making its way into R?

Joe Cheng: No, I don’t think so. The kind of lessons that we have adopted so far like falling into the “pit of success” (designing a package so it’s hard to use it incorrectly) or adopting source control have all been good. I really think that Hadley especially has raised the bar for the community and changed what people aspire to in terms of what their APIs look like. Those of us who use R on a regular basis can feel the difference between the functions that have been in base forever and some of the newer functions people are writing which can be freshly installed from GitHub.

So far at least, I haven’t seen a lot of downside to the evolution we’re seeing in the way that people in the community write R code. Mostly it’s thinking more intentionally about the user experience and not so much about moving away from a stats focus or anything like that. Development is still very much focused on statisticians and on helping people get whatever they need to do done really quickly, whether interactively or through scripts. Some of this intentionality might just amount to taking an extra second to make sure that you’ve thought about your naming scheme, that you’ve thought about what things should be in separate functions, etc. Things like this are more what I would consider good practices, good hygiene. I think there is a danger in looking to other programming languages for inspiration and maybe picking up some of their bad habits, but I haven’t seen a lot of that so far.

JBR: If the R core genie could grant you a wish, what would you ask for?

Joe Cheng: I’ve gotten this question multiple times over the years, and honestly I have a really hard time with it. The question is just so contrary to the way I usually think. I just accept the operating system and the language for what they are. I just have to work within their limitations and features. I think I’ve been using R for long enough that I’m quite comfortable with the sandbox that it makes available to us.

But, if we had a time machine and could go back, the one change that I would make – well the most important change I would make – to R would be to have delayed evaluation be a feature that you opt into rather than being the default. As I said before, delayed evaluation for function arguments is really awesome and it makes things easy in R that are quite unnatural to do in other languages. But, I feel it’s a tool that you usually don’t want to use. When you want it, it’s awesome to have, but it would be nicer to have all function arguments evaluated except for those which have been annotated for lazy evaluation.

Other than that, there are some obvious things that people are looking into, like having out-of-memory data frames, but really for the most part, I don’t have the laundry list of grievances that some people do. I tend to just focus on the system we have and what cool things that we can build on top of it.

JBR: How about new technology? What’s going on in the wider world that you think is really exciting? What new technology might make its way into RStudio products?

Joe Cheng: For the last few years, I’ve definitely been very much focused on the web side of things, on web technologies. When I started Shiny, I thought the JavaScript world had reached a kind of inflection point where things were turning so quickly that they would start maturing and stabilizing. Now, looking back, it seems to have been hopelessly naïve to think that we were at some zenith of change. The pace of change has just accelerated in the JavaScript and front-end development world.

I’m actually really excited about all the tools that are available in JavaScript front-end development these days. I’m not talking about visualization libraries, I mean the underlying tools. Languages like TypeScript and ES2015 are making huge quality of life improvements over the kind of JavaScript we’ve been writing for many years. All the cool stuff coming out of Facebook with React, unidirectional data flow (UI) as a pure function of state, and that whole ecosystem is a huge step in the right direction. Personally, I think this work validates many of the ideas that Shiny was built on. It’s gratifying to see that those ideas that inspired me have taken hold and become the dominant paradigm in the JavaScript world.

I guess that’s not the most exciting answer, but it’s just like holding on for dear life trying to follow what’s going on these days. Up until now, we have focused probably 95% of our Shiny efforts on people who don’t want to extend their web apps with custom JavaScripts. But, more and more I’m talking to people in the community who maybe started with this mentality, but have just so enjoyed the process of building web applications that they’ve taken the time to learn JavaScript and are now looking how to merge technologies like React with what they can already do in Shiny. Besides just having HTML widgets, I am thinking about how can we kind of leverage some of the cool stuff that’s happening in React. How can we in a deeper way integrate React and maybe some similar technologies with Shiny?

JBR: One more question: what advice would you give to computer science grads who would like to become R developers? Any ideas on how they can shorten their learning curve or what should they do?

Joe Cheng: I had a pretty hard time learning R, because at the time a lot of the books were geared towards statisticians who were learning their first programming languages, not programmers who were coming from another direction. At the time, I found the O’Reilly book R In A Nutshell really helpful, because it kind of just laid out the facts and didn’t talk that much about the statistics: just the facts about the language and the basic libraries. But, that was seven years ago or whatever, so I don’t know if that is the best book at this point, but it certainly was helpful for me. Definitely, the second R book that a CS grad should read through is Hadley Wickham’s Advanced R.

I think the most important recommendation I would make is to go into R with a very open mind and instead of just drawing contrasts with whatever language you’re used to and dismissing R, approach it with a little bit of humility. Be open to the possibility that some things are different because statisticians with an extraordinary amount of domain knowledge made a decision that actually makes sense in the context of this domain.

I have to tell you, whenever I step out of the Shiny bubble and work on packages that have a more direct kind of statistical application, like Leaflet or d3 Heatmap, it’s really humbling to offer up an API to data scientists and statisticians and then have them come back and say, “You’ve got it all wrong.” You think you know what some of these concepts are, but maybe you have no idea of how they actually work in practice and so your tool is useless. They don’t usually say it that bluntly, but it’s very real that this domain is so specialized. It’s not intuitive. If you approach R as if you’re dealing with any other ecosystem of programmers, then I think you’re going to kind of miss the forest for the trees.

This is digressing a little bit, but have you ever heard of Chesterton’s fence?

JBR: No, I haven’t.

Joe Cheng: Okay. There’s this parable, I guess. If you come across a fence that’s going across a road in the middle of nowhere, your first instinct might be to say, “I can’t see why this fence is useful. I’m going to take it down.” The point of this parable is that if you don’t understand what the fence was for, you’re the last person that should be taking it down. Only once you can say what use this thing has, then maybe you’re in a position to make a decision about whether it should change. I think a lot of R is like that. In order to really appreciate some of the decisions that were made, you can’t come to it with a lot of preconceived notions, especially if you yourself don’t have a background in doing a lot of interactive data manipulation and analysis.

JBR: Well, thank you, Joe. This was marvelous.

Interview with J.J. Allaire

Wed, 12 Oct 2016 00:00:00 +0000

Welcome to “R Views”, the new R Community blog from RStudio. For this first post, I sat down with J.J. Allaire, RStudio’s founder and CEO, to discuss RStudio’s history, its mission and JJ’s vision for its future. In a short time, we touched on a wide range of subjects including RStudio’s business, the growth of the R language, the importance of the R Consortium to the R Community and J.J.’s advice to anyone coming to R for the first time. We hope you enjoy this “snapshot” of RStudio’s place in the R world.

JBR: How did you first become involved with R? What were you doing when R first came to your attention, and why R?

J.J.: As an undergraduate I studied economics and political science and as a result got very interested in quantitative research. As a student I worked with SAS, SPSS, and Excel. This was my first serious exposure to software and computing, which I also became quite fascinated with.As it turned out, my career ended up developing more around software than around quantitative analysis. Then, about eight years ago I was between jobs and looking for the next thing to work on. I was very much of the mindset that I wanted my work to be open source, because I wanted to create software that would be free and accessible to everyone as well as long lasting.

That’s when I learned about R. It was really wonderful to hear that there was a robust and broadly used open source statistical computing environment. I thought that this might be a great place to apply my energy. Quantitative analysis was my original interest in school, and in the meantime, I had acquired considerable skill in writing software systems. I thought I could take some of that, apply it to R, and make a meaningful contribution.

JBR: Was there any specific event that motivated you to start RStudio? When you did start, what kind of company did you envision then?

J.J.: RStudio really wasn’t started as a company. It began as an open source project that I just worked on by myself. Initially, I started working on the concept of a web based IDE for R. About nine months or so into the project I realized that it would be a really huge undertaking for me to try to build this IDE all by myself.

So, I recruited Joe Cheng to work with me. Joe and I had worked together in a couple of previous companies. At the time I told him: “Joe, I make no representation or promise that this is going to be a company. It might just be an open source project, is that okay with you?” He said, “That’s fine”. It was really just the two of us working on an open source project at the beginning. Again, there was no notion that it would have to become a company. My feeling was that if there was a company that could evolve out of it that would be fine, but that was not required.

JBR: Are you surprised about how RStudio turned out?

J.J.: I didn’t know how it was going to turn out. I definitely knew that open source software has the potential to be very widely used. I’m not surprised at all that R is a really big deal, but in no way did I presume or know whether there would be a company that you could build around R. I honestly would have been surprised by any outcome. I really didn’t know.

JBR: What is your role at RStudio? Do you run the company? Are you responsible for product development? What gets you up in the morning?

J.J.: What I principally do is write software. That’s what I love to do. I’m the CEO of RStudio but I don’t really run the company per se. I collaborate with other people who are engaged in running the company. In particular I spend a lot of time with with Tareef Kawaf who is the President of RStudio and is primarily responsible for running it.

I really spend the vast majority of my time, over eighty percent, writing software. I started off as the principal developer on RStudio IDE and then Joe and I worked on it together for a number of years. I continue to spend time on the IDE, but I’m also the principal developer on quite a few R packages including the R Markdown package. I’m the maintainer of a few other packages like htmlwidgets and the dygraphs package, and I have also contributed to several other packages like shiny and Rcpp and others.

JBR: What does a typical work day look like for you?

J.J.: On a typical work day, I look at my email from overnight and see if there’s anything urgent. I may give feedback on some pull requests or take a look at a few bugs. I probably spend about an hour just taking stock of things that are in my inbox. Then I’ll work for five or six hours, writing code for various products and packages.

After that, I may check in with RStudio’s Seattle office when those guys come online at midday, East Coast time. Then, maybe I’ll do some more coding until the end of the day. Various things come up, but I have very few meetings that I regularly attend. I try to keep things clear for working on product development.

JBR: From the outside it looks like RStudio has committed a considerable amount of resource to developing open source packages and free products. Many people wonder how RStudio can even be a business. Can you talk a little bit about RStudio’s business plan, how does RStudio make money?

J.J.: Well, first of all, the mission of the company, is to create open source software for data analysis and statistical computing. So, if you see that we’re creating a lot of open source software that should never surprise you: that is our principal mission. At the same time, in order to fulfill our mission we need to bring together as many talented people as we can. For that to happen obviously we also need to develop and grow a business around R.

I’d say the commercial business has two major facets, the first is concerned with organizations that want to deploy R at scale; scale here meaning either computing scale or just adopting R within a larger organization. For these requirements, we have enhanced, commercial versions of some of our server products, for example RStudio Server Pro and Shiny Server Pro, that have features aimed at deploying R at scale. Our open source products can typically be adopted by many, many people without buying our professional products, and that’s how we want it to be. When customers get more serious about R, or they are deploying R in a larger environment, they tend to buy our professional products. So that’s the main way we make money, those professional server products.

We also have a cloud business where we provide versions of our products over the web. For example, we have shinyapps.io for deploying Shiny applications. It’s not a big a part of our business yet, but we expect it to become more and more significant, and we also expect to have other cloud offerings in the future.

JBR: What do RStudio’s long range goals look like?

J.J.: One core set of goals is centered around creating an elegant and productive environment for using R. Mostly here I’m talking about the RStudio IDE. A really nice environment for working with R not only needs to provide the tools to make users very productive, but it also needs to surface all of the richness that is inside R so that it’s conceptually clear to people what’s possible, and then makes it easy for them to do what they want to do.

Other goals are centered on creating APIs for R that are really intuitive and work well together. That’s the tidyverse, if you will, the family of packages that Hadley and others have developed like ggplot2, dplyr, tidyr, lubridate, stringr and so on.

We have also focused quite a bit on helping people communicate what they do in R. Here, I’m talking about R Markdown which is all about creating documents and web content from R, and Shiny which is about creating interactive applications with R.

In the future, we want to continue to excel along those three axes and make all of our open-source products and packages even better. We also continue to learn more about the needs of companies who are deploying R at scale, whether it be trying to achieve high performance, ensuring robust security, or integrating seamlessly with other enterprise software systems. As we continue to learn we will continue to develop products along this axis as well. Right now, we’ve solved some of the problems, but there are a whole lot more to address, and I don’t think we’re nearly done on any of these fronts. Building up our professional server product line is a big focus as is building hosted products to let people access and use R in the cloud.

Also, we want to do a better job of educating people about what we already have. So we are trying to create a well-thought-through curriculum that people can access through different mediums including books, videos, websites and interactive self-paced tutorials. You’ll see lots more investment in education from us in the future.

JBR: So far, from what you’ve said, RStudio projects all seem to be related to producing tools to help others develop R software and do analytics work, but what would a complete R based tool chain look like?

J.J.: That’s interesting. I think what we have been talking about, the things that we’ve traditionally focused on like helping to deploy R at scale are horizontal tools. They are very, very helpful, but there’s actually another layer of tools that we haven’t talked about. In any given domain of application, you want to have a suite of tools that are specific to that domain. These are also extremely important, but we are more of a facilitator of those tools than a creators, since we’re not domain experts.

Over time, I think that the whole R community is engaged in a really big collective project to build out all the tools required to let people analyze data in the different domains. Some of the work that we are doing is very much focused on enabling this. For example, Hadley tries both to build tools, and to teach people how to build their own great tools for R. As a result, people have built some wonderful packages to make it easier to do specialized types of analyses. The work I’ve done on Rcpp is all about letting people access the highest performance capabilities within their domain whether that’s reading data really fast, or analyzing it quickly, or optimizing certain algorithms. Also, many of the tools that we’ve developed in data visualization are about providing an extensible platform to let people do custom static and interactive visualizations for different domains. So what does a complete tool chain look like? I think part of it is made up of the tools that we build, and the rest of it is made up of the tools other people build on top of our stuff.

JBR: Does sparklyr represent a new direction for RStudio, or is this just part of the tool chain you described?

J.J.: I don’t think it’s a new direction. In his keynote address at the 2014 UseR! conference in UCLA, John Chambers spoke about how the original concept for R was to create an interface language. One of the motivators was that people wanted to use the robust and high performance FORTRAN libraries for doing statistical analysis that already existed, but they wanted to use them in a flexible and interactive way. So the idea was to create a language that was really good at providing interfaces to other computing systems. In the beginning, it was interfacing to an existing body of FORTRAN code, but over time that has been extended to run just about everything. R is a very good interface language.

John gave several examples during his talk. One was Rcpp which I’ve already talked about. The other was H2O, which is a distributed computing system for doing machine learning. H2O is written in Java, not R, but the developers have provided it with a very nice R interface. I think that’s how we view just about everything that might be relevant to statisticians or data analysts. If it’s not already in R you can use R to create a fluent and expressive interface for it.

That’s how we view sparklyr. Spark provides a set of big data capabilities that people want to take advantage of, and we’re creating a rich interface in R so the tools and techniques, the working environment and the ability to communicate, things like R Markdown and Shiny, just automatically come with you when you go to work with Spark. John mentioned H2O, but R’s been doing things with distributed processing for a long time, so I think we’re just fulfilling the vision and mission of R against the latest and greatest distributing computing platforms.

JBR: What do you think about R and big data in general? Do you think what RStudio is doing is going to help overcome the perceived limitations of R?

J.J.: Again, you should understand R as a user interface, or a language that’s capable of providing very good user interfaces. Developers don’t implement core algorithms or core distributed computing primitives in R. They use R as a way of interacting with those things.Once people understand this, and they see how good R is, how they can use dplyr to interact with a Spark cluster, I think the light bulb will go off and they’ll say: “Okay, I understand where R fits now. It’s the world’s best interactive environment for data analysis and now I can use it directly against Spark”.

JBR: RStudio is a founding platinum member of the R consortium, seems from the outside really looks like a really big commitment for a small company.

J.J.: Yes

JBR: Why is the R Consortium important? What are your hopes for its future, and what do you think the R Consortium can accomplish?

J.J.: What’s really interesting about the R consortium is that it represents all of R’s stakeholders. There are the original stakeholders, the R Foundation and R Core, the creators of R. There’s a whole community of package developers and users, and then there’s the vendors who are in various ways making a business around R. The R Consortium is an alliance of all these people that says “Hey, we all have a stake in this being as good as it can be. We all have a stake in the long term integrity of the system, so let’s get together in one venue that will allow us to have a dialog about things and fund projects that can help the R Community”.

The R Consortium is a really profound and important group. It has the potential to be the nexus point for everyone in the R Community. That’s why we’re so enthusiastic about the R Consortium and why we’ve got folks within RStudio spending considerable time on it. If it works well, the R Consortium is going to provide a lot of ballast for R, provide a great forum for communicating about R, and help establish trust in R. We want to support it as much as we can.

JBR: In addition to supporting the R community it looks like RStudio packages are geared toward reproducible research and improving scientific research in general. Do you have specific goals or aspirations with respect to science itself?

J.J.: There is a broader movement in science around reproducible research that we can contribute to. Reproducible research is all about the integrity of the work you do. It’s all about the fact that people rely on scientific software to make very important decisions and to gain a clear understanding about how the world works. The integrity and the trustworthiness of the process by which that software yields its results is critical.

Our goal is to make sure that the tools for working reproducibly within the R ecosystem are excellent. In the past, people may have felt that working reproducibly was going to require more work, more effort, and thought “I don’t have time to do that”. We’re trying to make it so that reproducible work actually takes less effort. It’s sort of that pit of success idea where we provide a great tool chain for creating compelling documents which has the very pleasant side effect that everything that people do is reproducible.

Also, reproducibility is one of the reasons we focus on teaching people how to write and work with code to do data analysis. We don’t have visual abstractions for doing data analysis because those types of techniques are not reproducible, not easy to inspect, not easy to share, and not easy to automate. We really focus on code. It’s a core principle that, no matter what, we’d rather make code easier to work with than try to eliminate it.

JBR: Do you have any advice for people coming to R for the first time, either as students or as experienced statisticians and researchers?

J.J.: I would suggest that they get a copy of the R for Data Science book written by Hadley Wickham and Garrett Grolemund. The book explains the tidyverse and fundamental principles for working with data in R. There are a lot of great tools for working in R but not everybody who comes to R encounters them right away. We’re trying to change that. I would also recommend that newcomers to R take a careful look at all the tools and packages that we have developed at RStudio.

Also, when you have questions or run into problems don’t give up. There’s a lot of great activity around R on stackoverflow and other places and there’s an excellent chance you’re going to find the answers to your questions if you look carefully for them.

JBR: Do you have any thoughts on what statisticians and data scientists ought to know about computer science in general?

J.J.: That’s an interesting question. A lot of the popular philosophy about R revolves aroundBo Cowgill’s famous quote: “The best thing aboutRis that it was written by statisticians. The worst thing about R is that it was written by statisticians.“The fact is, however, that the folks who made R knew quite a bit about computing and computer science. Their goal was, and continues to be, to provide an expressive language that lets you accomplish what you want to accomplish with a minimal requirement for mastering computer science. R is trying to provide an interface and a vocabulary that allows non-computer science people to still do sophisticated things with data.

I think there will always be sets of problems that you can’t solve with a high level vocabulary. So, becoming more educated about general programming principles, doing some lower level computations or lower level text analysis or parsing and those sorts of things is useful. But, the whole point of R is to not force everyone to become a computer scientist.

JBR: Last question: where do you think R and RStudio will be in ten years?

J.J.: Currently, R is quite prominent in the universities. It’s taught to undergraduates and researchers and, more and more it’s becoming part of the core curriculum. However, in the world of practitioners, there’s still an awful lot people using proprietary software that in many cases is less capable, less flexible, less powerful than R.

I’m hopeful that over the next ten years we’ll start to see the kind of predominance R now has in universities really take hold in the world of practicing statisticians. I’m hopeful that will happen, and I’m hopeful that RStudio can be a big part of making the transition to R a positive experience. We want to continue to make R easier to learn and easier to use so that when organizations decide that they’re going to commit to R for the long term and replace their proprietary statistics software, they’ll come to an environment that is really refined, really powerful, really easy to learn, and that they can actually deploy R at scale and be successful with it. I hope that RStudio can facilitate all those things in the next ten years.

JBR: Thank you J.J.!