Stocks on R Views

Introduction to Kurtosis

Thu, 04 Jan 2018 00:00:00 +0000

Happy 2018 and welcome to our first reproducible finance post of the year! What better way to ring in a new beginning than pondering/calculating/visualizing returns distributions.

We ended 2017 by tackling skewness, and we will begin 2018 by tackling kurtosis.

Kurtosis is a measure of the degree to which portfolio returns appear in the tails of our distribution. A normal distribution has a kurtosis of 3, which follows from the fact that a normal distribution does have some of its mass in its tails. A distribution with a kurtosis greater than 3 has more returns out in its tails than the normal, and one with kurtosis less than 3 has fewer returns in its tails than the normal. That matters to investors because more bad returns out in tails means that our portfolio might be at risk of a rare but huge downside. The terminology is a bit confusing. Negative kurtosis is considered less risky because it has fewer returns out in the tails. Negative == less risky? We’re not used to that in finance.

Kurtosis is often has the word ‘excess’ appended to its description, as in ‘negative excess kurtosis’ or ‘positive excess kurtosis’. That ‘excess’ is in comparison to a normal distribution kurtosis of 3. A distribution with negative excess kurtosis equal to -1 has an actual kurtosis of 2.

Enough with the faux investopedia entry, let’s get to the calculations, R code and visualizations.

Here’s the equation for excess kurtosis. Note that we subtract 3 at the end:

\[Kurtosis=\sum_{t=1}^n (x_i-\overline{x})^4/n \bigg/ (\sum_{t=1}^n (x_i-\overline{x})^2/n)^{2}-3 \]

By way of reminder, we will be working with our usual portfolio consisting of:

+ SPY (S&P500 fund) weighted 25%
+ EFA (a non-US equities fund) weighted 25%
+ IJS (a small-cap value fund) weighted 20%
+ EEM (an emerging-mkts fund) weighted 20%
+ AGG (a bond fund) weighted 10%

Before we can calculate kurtosis, we need to find portfolio monthly returns, which was covered in this post.

Building off that previous work, we will be working with two objects of portfolio returns:

+ portfolio_returns_xts_rebalanced_monthly (an xts of monthly returns)
+ portfolio_returns_tq_rebalanced_monthly (a tibble of monthly returns)

Now we are going to test our past self’s work on skewness, and reuse that code flow to expedite the kurtosis work. The logic will remain the same, but we will call different built-in functions and different by-hand calculations.

For the xts world, we use the kurtosis() function instead of the skewness() function.

kurt_xts <-  kurtosis(portfolio_returns_xts_rebalanced_monthly$returns)

kurt_xts

## [1] 0.5267736

For tidy, we have the same piped flow and use the formula for kurtosis for our by-hand calculations. Our by-hand result is labeled with kurt_byhand, and involves quite a few parentheticals to map it back to the kurtosis equation above.

kurt_tidy <-
  portfolio_returns_tq_rebalanced_monthly %>% 
  summarise(
            kurt_builtin = kurtosis(returns),
            kurt_byhand = 
              ((sum((returns - mean(returns))^4)/length(returns))/
              ((sum((returns - mean(returns))^2)/length(returns))^2)) - 3) %>% 
  select(kurt_builtin, kurt_byhand)

Let’s confirm that we have consistent calculations.

kurt_xts

## [1] 0.5267736

kurt_tidy$kurt_builtin

## [1] 0.5267736

kurt_tidy$kurt_byhand

## [1] 0.5267736

We have consistent results from xts and the tidy built-in/by-hand worlds, and we were able to reuse our code from above to shorten the development time here. dd

Let’s do the same with the visualizations and head straight for a density plot, starting with the same portfolio_density_plot.

portfolio_density_plot <- 
  portfolio_returns_tq_rebalanced_monthly %>% 
  ggplot(aes(x = returns)) +
  stat_density(geom = "line", alpha = 1, colour = "cornflowerblue")

portfolio_density_plot

We are interested in both tails for kurtosis, so let’s shade at 2 standard deviations above and below the mean return (for our skewness work, we only shaded the negative tail).

mean <- mean(portfolio_returns_tq_rebalanced_monthly$returns)

sd_pos <-  mean + (2 * sd(portfolio_returns_tq_rebalanced_monthly$returns))

sd_neg <- mean - (2 * sd(portfolio_returns_tq_rebalanced_monthly$returns))

sd_pos_shaded_area <- 
  ggplot_build(portfolio_density_plot)$data[[1]] %>% 
  filter(x > sd_pos )

sd_neg_shaded_area <- 
  ggplot_build(portfolio_density_plot)$data[[1]] %>% 
  filter(x < sd_neg)

  portfolio_density_plot <-  
    portfolio_density_plot + 
    geom_area(data = sd_pos_shaded_area, aes(x = x, y = y), fill="pink", alpha = 0.5) +
    geom_area(data = sd_neg_shaded_area, aes(x = x, y = y), fill="pink", alpha = 0.5) +
    scale_x_continuous(breaks = scales::pretty_breaks(n = 10))
  
  portfolio_density_plot

That density chart is a good look at the mass in both tails, where we have defined ‘tail’ as being two standard deviations away from the mean. We can add a line for the mean, as did in our skewness visualization, with the following.

mean <- mean(portfolio_returns_tq_rebalanced_monthly$returns)

mean_line_data <- 
  ggplot_build(portfolio_density_plot)$data[[1]] %>% 
  filter(x <= mean)


portfolio_density_plot +
  
geom_segment(data = mean_line_data, aes(x = mean, y = 0, xend = mean, yend = density), 
               color = "black", linetype = "dotted") +
  
  annotate(geom = "text", x = mean, y = 5, label = "mean", 
           fontface = "plain", angle = 90, alpha = .8, vjust =  1.75)

Finally, we can calculate and chart the rolling kurtosis with the same logic as we did for skewness. The only difference is that here we call fun = kurtosis instead of fun = skewness.

window <- 6
rolling_kurt_xts <- na.omit(apply.rolling(portfolio_returns_xts_rebalanced_monthly, window, 
                           fun = kurtosis))

Now we pop that xts object into highcharter for a visualization.

highchart(type = "stock") %>%
  hc_title(text = "Rolling Kurt") %>%
  hc_add_series(rolling_kurt_xts, name = "Rolling kurtosis", color = "cornflowerblue") %>%
  hc_yAxis(title = list(text = "kurtosis"),
           opposite = FALSE,
           max = .03, 
           min = -.03) %>% 
  hc_navigator(enabled = FALSE) %>% 
  hc_scrollbar(enabled = FALSE)

We didn’t cover this before, but what if we wanted to use ggplot for the rolling kurtosis? We could convert that xts object to a tibble with tk_tbl() from the timetk package, and then pipe straight to ggplot.

rolling_kurt_xts %>% 
  tk_tbl(preserve_index = TRUE, rename_index = "date") %>% 
  rename(rolling_kurtosis = calcs) %>% 
  ggplot(aes(x = date, y = rolling_kurtosis)) + 
  geom_line(color = "cornflowerblue") +
  xlab("date") +
  ylab("rolling kurtosis") + 
  ggtitle("Rolling Kurtosis")

Interestingly, this portfolio has displayed slight positive rolling excess kurtosis for most of its life, except during the last half of 2015 through early 2016.

That’s all for today. Our work on kurtosis was made a lot more efficient by our work on skewness - so let’s thank our 2017 selves for constructing a reproducible and reusable code flow! See you next time.

Introduction to Skewness

Wed, 13 Dec 2017 00:00:00 +0000

In previous posts here, here, and here, we spent quite a bit of time on portfolio volatility, using the standard deviation of returns as a proxy for volatility. Today we will begin to a two-part series on additional statistics that aid our understanding of return dispersion: skewness and kurtosis. Beyond being fancy words and required vocabulary for CFA level 1, these two concepts are both important and fascinating for lovers of returns distributions. For today, we will focus on skewness.

Skewness is the degree to which returns are asymmetric around the mean. Since a normal distribution is symmetric around the mean, skewness can be taken as one measure of how returns are not distributed normally. Why does skewness matter? If portfolio returns are right, or positively, skewed, it implies numerous small negative returns and a few large positive returns. If portfolio returns are left, or negatively, skewed, it implies numerous small positive returns and few large negative returns. The phrase “large negative returns” should trigger Pavlovian sweating for investors, even if it’s preceded by a diminutive modifier like “just a few”. For a portfolio manager, a negatively skewed distribution of returns implies a portfolio at risk of rare but large losses. This makes us nervous and is a bit like saying, “I’m healthy, except for my occasional massive heart attack.”

Let’s get to it.

First, have a look at one equation for skewness:

\[Skew=\sum_{t=1}^n (x_i-\overline{x})^3/n \bigg/ (\sum_{t=1}^n (x_i-\overline{x})^2/n)^{3/2}\]

Skew has important substantive implications for risk, and is also a concept that lends itself to data visualization. In fact, I find the visualizations of skewness more illuminating than the numbers themselves (though the numbers are what matter in the end). In this section, we will cover how to calculate skewness using xts and tidyverse methods, how to calculate rolling skewness, and how to create several data visualizations as pedagogical aids. We will be working with our usual portfolio consisting of:

+ SPY (S&P500 fund) weighted 25%
+ EFA (a non-US equities fund) weighted 25%
+ IJS (a small-cap value fund) weighted 20%
+ EEM (an emerging-mkts fund) weighted 20%
+ AGG (a bond fund) weighted 10%

Before we can calculate the skewness, we need to find portfolio monthly returns, which was covered in this post.

Building off that previous work, we will be working with two objects of portfolio returns:

+ portfolio_returns_xts_rebalanced_monthly (an xts of monthly returns)
+ portfolio_returns_tq_rebalanced_monthly (a tibble of monthly returns)

Let’s begin in the xts world and make use of the skewness() function from PerformanceAnalytics.

library(PerformanceAnalytics)
skew_xts <-  skewness(portfolio_returns_xts_rebalanced_monthly$returns)

skew_xts

## [1] -0.1710568

Our portfolio is relatively balanced, and a slight negative skewness of -0.1710568 is unsurprising and unworrisome. However, that final number could be omitting important information and we will resist the temptation to stop there. For example, is that slight negative skew being caused by one very large negative monthly return? If so, what happened? Or is it caused by several medium-sized negative returns? What caused those? Were they consecutive? Are they seasonal? We need to investigate further.

Before doing so and having fun with data visualization, let’s explore the tidyverse methods and confirm consistent results.

We will make use of the same skewness() function, but because we are using a tibble, we use summarise() as well and call summarise(skew = skewness(returns). It’s not necessary, but we are also going to run this calculation by hand, the same as we have done with standard deviation. Feel free to delete the by-hand section from your code should this be ported to enterprise scripts, but keep in mind that there is a benefit to forcing ourselves and loved ones to write out equations: it emphasizes what those nice built-in functions are doing under the hood. If a client, customer or risk officer were ever to drill into our skewness calculations, it would be nice to have a super-firm grasp on the equation.

library(tidyverse)
library(tidyquant)
skew_tidy <-
  portfolio_returns_tq_rebalanced_monthly %>% 
  summarise(skew_builtin = skewness(returns),
            skew_byhand = 
              (sum((returns - mean(returns))^3)/length(returns))/
              ((sum((returns - mean(returns))^2)/length(returns)))^(3/2)) %>% 
  select(skew_builtin, skew_byhand)

Let’s confirm that we have consistent calculations.

skew_xts

## [1] -0.1710568

skew_tidy$skew_builtin

## [1] -0.1710568

skew_tidy$skew_byhand

## [1] -0.1710568

The results are consistent using xts and our tidyverse, by-hand methods. Again, though, that singular number -0.1710568 does not fully illuminate the riskiness or distribution of this portfolio. To dig deeper, let’s first visualize the density of returns with stat_density from ggplot2.

portfolio_density_plot <- 
  portfolio_returns_tq_rebalanced_monthly %>% 
  ggplot(aes(x = returns)) +
  stat_density(geom = "line", alpha = 1, colour = "cornflowerblue")

portfolio_density_plot

The slight negative skew is a bit more evident here. It would be nice to shade the area that falls below some threshold again, and let’s go with the mean return. To do that, let’s create an object called shaded_area using ggplot_build(portfolio_density_plot)$data[[1]] %>% filter(x < mean(portfolio_returns_tq_rebalanced_monthly$returns)). That snippet will take our original ggplot object and create a new object filtered for x values less than mean return. Then we use geom_area to add the shaded area to portfolio_density_plot.

shaded_area_data <- 
  ggplot_build(portfolio_density_plot)$data[[1]] %>% 
  filter(x < mean(portfolio_returns_tq_rebalanced_monthly$returns))

portfolio_density_plot_shaded <- 
  portfolio_density_plot + 
  geom_area(data = shaded_area_data, aes(x = x, y = y), fill="pink", alpha = 0.5)

portfolio_density_plot_shaded

The shaded area highlights the mass of returns that fall below the mean. Let’s add a vertical line at the mean and median, and some explanatory labels. This will help to emphasize that negative skew indicates a mean less than the median.

First, create variables for mean and median so that we can add a vertical line.

median <- median(portfolio_returns_tq_rebalanced_monthly$returns)
mean <- mean(portfolio_returns_tq_rebalanced_monthly$returns)

We want the vertical lines to just touch the density plot so we once again use a call to ggplot_build(portfolio_density_plot)$data[[1]].

median_line_data <- 
  ggplot_build(portfolio_density_plot)$data[[1]] %>% 
  filter(x <= median)

Now we can start adding aesthetics to the latest iteration of our graph, which is stored in the object portfolio_density_plot_shaded.

portfolio_density_plot_shaded +
  
  geom_segment(aes(x = 0, y = 1.9, xend = -.045, yend = 1.9),
               arrow = arrow(length = unit(0.5, "cm")), size = .05) +
  
  annotate(geom = "text", x = -.02, y = .1, label = "returns <= mean", 
           fontface = "plain", alpha = .8, vjust =  -1) + 
  
  geom_segment(data = shaded_area_data, aes(x = mean, y = 0, xend = mean, yend = density), 
               color = "red", linetype = "dotted") +
  
  annotate(geom = "text", x = mean, y = 5, label = "mean", color = "red", 
           fontface = "plain", angle = 90, alpha = .8, vjust =  -1.75) +
  
  geom_segment(data = median_line_data, aes(x = median, y = 0, xend = median, yend = density), 
               color = "black", linetype = "dotted") +
  
  annotate(geom = "text", x = median, y = 5, label = "median", 
           fontface = "plain", angle = 90, alpha = .8, vjust =  1.75) +
  
  
  ggtitle("Density Plot Illustrating Skewness")

We added quite a bit to the chart, possibly too much, but it’s better to be over-inclusive now to test different variants. We can delete any of those features when using this chart later, or refer back to these lines of code should we ever want to reuse some of the aesthetics.

At this point, we have calculated the skewness of this portfolio throughout its history, and done so using three methods. We have also created an explanatory visualization.

Similar to the portfolio standard deviation, though, our work is not complete until we look at rolling skewness. Perhaps the first two years of the portfolio were positive skewed, and last two were negative skewed but the overall skewness is slightly negative. We would like to understand how the skewness has changed over time, and in different economic and market regimes. To do so, we calculate and visualize the rolling skewness over time.

In the xts world, calculating rolling skewness is almost identical to calculating rolling standard deviation, except we call the skewness() function instead of StdDev(). Since this is a rolling calculation, we need a window of time for each skewness; here, we will use a six-month window.

window <- 6
rolling_skew_xts <- na.omit(rollapply(portfolio_returns_xts_rebalanced_monthly, window, 
                           function(x) skewness(x)))

Now we pop that xts object into highcharter for a visualization. Let’s make sure our y-axis range is large enough to capture the nature of the rolling skewness fluctuations by setting the range to between 3 and -3 with hc_yAxis(..., max = 3, min = -3). I find that if we keep the range from 1 to -1, it makes most rolling skews look like a roller coaster.

library(highcharter)
highchart(type = "stock") %>%
  hc_title(text = "Rolling") %>%
  hc_add_series(rolling_skew_xts, name = "Rolling skewness", color = "cornflowerblue") %>%
  hc_yAxis(title = list(text = "skewness"),
           opposite = FALSE,
           max = 3, 
           min = -3) %>% 
  hc_navigator(enabled = FALSE) %>% 
  hc_scrollbar(enabled = FALSE)

For completeness of methods, we can calculate rolling skewness in a tibble and then use ggplot.

We will make use of rollapply() from within tq_mutate in tidyquant.

rolling_skew_tidy <- 
  portfolio_returns_tq_rebalanced_monthly %>% 
  tq_mutate(select = returns, 
            mutate_fun = rollapply,
            width      = window,
            FUN        = skewness,
            col_rename = "skew")

rolling_skew_tidy is ready for ggplot. ggplot is not purpose-built for time series plotting, but we can set aes(x = date, y = skew) to make the x-axis our date values.

library(scales)
theme_update(plot.title = element_text(hjust = 0.5))
rolling_skew_tidy %>% 
  ggplot(aes(x = date, y = skew)) +
  geom_line(color = "cornflowerblue") +  
  ggtitle("Rolling Skew with ggplot") +
  ylab(paste("Rolling", window, "month skewness", sep = " ")) + 
  scale_y_continuous(limits = c(-3, 3), breaks = pretty_breaks(n = 8)) + 
  scale_x_date(breaks = pretty_breaks(n = 8))

The rolling charts are quite illuminating and show that the six-month-interval skewness has been positive for about half the lifetime of this portfolio. Today, the overall skewness is negative, but the rolling skewness in mid-2016 was positive and greater than 1. It took a huge plunge starting at the end of 2016, and the lowest reading was -1.65 in March of 2017, most likely caused by one or two very large negative returns when the market was worried about the US election. We can see those worries start to abate as the rolling skewness becomes more positive throughout 2017.

That’s all for today. Thanks for reading and see you next time when we tackle kurtosis.