{"id":2132,"date":"2021-10-25T03:28:14","date_gmt":"2021-10-25T03:28:14","guid":{"rendered":"http:\/\/optimumsportsperformance.com\/blog\/?p=2132"},"modified":"2021-10-25T13:54:23","modified_gmt":"2021-10-25T13:54:23","slug":"r-tips-tricks-recreating-within-column-iteration-as-you-would-do-in-excel","status":"publish","type":"post","link":"https:\/\/optimumsportsperformance.com\/blog\/r-tips-tricks-recreating-within-column-iteration-as-you-would-do-in-excel\/","title":{"rendered":"R Tips &#038; Tricks: Recreating within column iteration as you would do in excel"},"content":{"rendered":"<p>One of the easiest things to do in excel is within column iteration. What I mean by this is you create a new column where the starting value is a 0 or a value that occurs in a different column and then all of the following values within that column depend on the value preceding it.<\/p>\n<p>For example, in the below table we can see that we have a value for each corresponding ID. The New Value is calculated as the most recent observation of <strong>Value + lag(New Value) &#8211; 2.<\/strong> This is true for all observations except the first observation, which simply takes <strong>Value <\/strong>of the first ID observation. So, in ID 2, we get:\u00a0<strong><em>New Value = 7 + 4 &#8211; 2 = 9 <\/em><\/strong>and in ID 3 we get: <em><strong>New Value = 3 + 9 &#8211; 2 = 10.<\/strong><\/em><\/p>\n<p><a href=\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.18.47-PM.png\"> <img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-2133\" src=\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.18.47-PM.png\" alt=\"\" width=\"517\" height=\"168\" srcset=\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.18.47-PM.png 782w, https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.18.47-PM-300x97.png 300w, https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.18.47-PM-768x249.png 768w, https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.18.47-PM-624x203.png 624w\" sizes=\"auto, (max-width: 517px) 100vw, 517px\" \/><\/a><br \/>\nThis type of function is pretty common in excel but it can be a little tricky in R. I&#8217;ve been meaning to do a blog about this after a few questions that I&#8217;ve gotten and <span style=\"color: #0000ff;\"><strong><a style=\"color: #0000ff;\" href=\"https:\/\/twitter.com\/aaronzpearson\">Aaron Pearson<\/a><\/strong><\/span> reminded me about it last night, so let&#8217;s try and tackle it.<\/p>\n<p><span style=\"text-decoration: underline;\"><strong>Creating Data<\/strong><\/span><\/p>\n<p>We will create two fake data sets:<\/p>\n<ul>\n<li>Data set 1 will be a larger data set with multiple subjects.<\/li>\n<li>Data set 2 will only be one subject, a smaller data set for us to first get an understanding of what we are doing before trying to perform the function over multiple people.<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<pre class=\"brush: r; title: ; notranslate\" title=\"\">\r\n\r\nlibrary(tidyverse)\r\n\r\n## simulate data\r\nset.seed(1)\r\nsubject &lt;- rep(LETTERS&#x5B;1:3], each = 50)\r\nday &lt;- rep(1:50, times = 3)\r\nvalue &lt;- c(\r\n  round(rnorm(n = 20, mean = 120, sd = 40), 2),\r\n  round(rnorm(n = 10, mean = 150, sd = 20), 2),\r\n  round(rnorm(n = 20, mean = 110, sd = 30), 2),\r\n  round(rnorm(n = 20, mean = 120, sd = 40), 2),\r\n  round(rnorm(n = 10, mean = 150, sd = 20), 2),\r\n  round(rnorm(n = 20, mean = 110, sd = 30), 2),\r\n  round(rnorm(n = 20, mean = 120, sd = 40), 2),\r\n  round(rnorm(n = 10, mean = 150, sd = 20), 2),\r\n  round(rnorm(n = 20, mean = 110, sd = 30), 2))\r\n\r\ndf_1 &lt;- data.frame(subject, day, value) df_1 %&gt;% head()\r\n\r\n### Create a data frame of one subject for a simple example\r\ndf_2 &lt;- df_1 %&gt;%\r\n  filter(subject == &quot;A&quot;)\r\n\r\n<\/pre>\n<p><a href=\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.29.26-PM-1.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-2137\" src=\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.29.26-PM-1.png\" alt=\"\" width=\"216\" height=\"210\" \/><\/a><\/p>\n<p><span style=\"text-decoration: underline;\"><strong>Exponentially Weighted Moving Average (EWMA)<\/strong><\/span><\/p>\n<p>We will apply an exponentially weighted moving average to the data as this type of equation requires within column aggregation.<\/p>\n<p>EWMA is calculated as:<\/p>\n<p><em><strong>EWMA_t = lamda*x_t + (1 &#8211; lamda) * Z_t-1<\/strong><\/em><\/p>\n<p>Where:<\/p>\n<ul>\n<li>EWMA_t = the exponentially weighted moving average value at time <em>t<\/em><\/li>\n<li>Lamda = the weighting factor<\/li>\n<li>x_t = the most recent observation<\/li>\n<li>Z_t-1 = the lag of the EWMA value<\/li>\n<\/ul>\n<p><span style=\"text-decoration: underline;\"><strong>accumulate()<\/strong><\/span><\/p>\n<p>Within {<strong>tidyverse<\/strong>} we will use the <strong>accumulate()<\/strong> function, which allows us to create this type of within column aggregation. The function takes a few key arguments:<\/p>\n<ul>\n<li>First we need to pass the function the name of the column of data with our observations over time<\/li>\n<li><strong>.y<\/strong> which represents the value of our most recent observation<\/li>\n<li><strong>.f<\/strong> which is the function that we want to apply to our within column aggregation (in this example we will use the EWMA equation)<\/li>\n<li><strong>.x<\/strong> which is going to provide us with the lagged value within the new column we are creating<\/li>\n<\/ul>\n<p>Here is what it looks like in our smaller data set, <em><strong>df_2<\/strong><\/em><\/p>\n<p>&nbsp;<\/p>\n<pre class=\"brush: r; title: ; notranslate\" title=\"\">\r\ndf_2 &lt;- df_2 %&gt;%\r\n  mutate(ewma = accumulate(value, ~ lamda * .y + (1 - lamda) * .x))\r\n\r\n<\/pre>\n<p>Here, we are using <strong>mutate()<\/strong> to create a new column called <strong>ewma. <\/strong>We used <strong>accumulate()<\/strong> and passed it the <strong>value<\/strong> column, which is the column of our data that has our observations and our function for calculating ewma, which follows the tilde.<\/p>\n<p>Within this function we see <strong>.y, <\/strong>the most recent observation and <strong>.x<\/strong>, the lag ewma value. By default, the first row of the new <strong>ewma<\/strong> column will be the first observation in the <strong>value<\/strong> row. Here is what the first few rows of the data look like:<\/p>\n<p><a href=\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.44.25-PM.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-2138\" src=\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.44.25-PM.png\" alt=\"\" width=\"358\" height=\"237\" srcset=\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.44.25-PM.png 420w, https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.44.25-PM-300x199.png 300w\" sizes=\"auto, (max-width: 358px) 100vw, 358px\" \/><\/a><\/p>\n<p>Now that new column has been created we can visualize the observed values and the EWMA values:<\/p>\n<p><a href=\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.47.55-PM.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-2139\" src=\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.47.55-PM-1024x790.png\" alt=\"\" width=\"491\" height=\"379\" srcset=\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.47.55-PM-1024x790.png 1024w, https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.47.55-PM-300x231.png 300w, https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.47.55-PM-768x593.png 768w, https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.47.55-PM-624x482.png 624w, https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.47.55-PM.png 1664w\" sizes=\"auto, (max-width: 491px) 100vw, 491px\" \/><\/a><\/p>\n<p><span style=\"text-decoration: underline;\"><strong>Applying the approach to all of the subjects<\/strong><\/span><\/p>\n<p>To apply this approach to all of the subjects in our data we simply need to use the <strong>group_by()<\/strong> function to tell R that we want to have the algorithm start over whenever it encounters a new subject ID.<\/p>\n<pre class=\"brush: r; title: ; notranslate\" title=\"\">\r\n\r\ndf_1 &lt;- df_1 %&gt;%\r\n  group_by(subject) %&gt;%\r\n  mutate(ewma = accumulate(value, ~ lamda * .y + (1 - lamda) * .x))\r\n\r\n<\/pre>\n<p>And then we can plot the outcome:<\/p>\n<p><a href=\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.50.37-PM.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-large wp-image-2140\" src=\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.50.37-PM-1024x795.png\" alt=\"\" width=\"625\" height=\"485\" srcset=\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.50.37-PM-1024x795.png 1024w, https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.50.37-PM-300x233.png 300w, https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.50.37-PM-768x596.png 768w, https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.50.37-PM-624x484.png 624w, https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.50.37-PM.png 1660w\" sizes=\"auto, (max-width: 625px) 100vw, 625px\" \/><\/a><\/p>\n<p>Pretty easy!<\/p>\n<p><span style=\"text-decoration: underline;\"><strong>What if we want the start value to be 0 (or something else) instead of the first observation?<\/strong><\/span><\/p>\n<p>This is a quick fix within the <strong>accumulate() <\/strong>function by using the <strong>.init<\/strong> argument and simply passing it whatever value you want the new column to begin with. What you need to be aware of when you do this, however, is that this argument will add an additional observation to the vector of data and thus we need to remove the last row of the data set to ensure that {<strong>tidyverse<\/strong>} can perform the operation without giving you an error. To accomplish this, when I pass the <strong>value <\/strong>column to the function I add a bracket and then minus 1 of the total count, <strong>n()<\/strong>, of observations in that column.<\/p>\n<pre class=\"brush: r; title: ; notranslate\" title=\"\">\r\ndf_2 %&gt;%\r\n  mutate(ewma = accumulate(value&#x5B;-n()], ~ lamda * .y + (1 - lamda) * .x, .init = 0)) %&gt;%\r\n  head()\r\n<\/pre>\n<p><a href=\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.56.03-PM.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-2141\" src=\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.56.03-PM.png\" alt=\"\" width=\"331\" height=\"188\" srcset=\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.56.03-PM.png 436w, https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2021\/10\/Screen-Shot-2021-10-24-at-8.56.03-PM-300x171.png 300w\" sizes=\"auto, (max-width: 331px) 100vw, 331px\" \/><\/a><\/p>\n<p>Now we see that the first value in <strong>ewma<\/strong> is 0 instead of 94.94, which of course changes all of the values following it since the equation is using the lagged <strong>ewma <\/strong>value (<strong>.x<\/strong>).<\/p>\n<p>For the complete code, please see my <span style=\"color: #0000ff;\"><strong><a style=\"color: #0000ff;\" href=\"https:\/\/github.com\/pw2\/R-Tips-Tricks\/blob\/master\/recreate%20excel%20data%20iteration%20within%20column.R\">GitHub Page<\/a><\/strong><\/span>.<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>One of the easiest things to do in excel is within column iteration. What I mean by this is you create a new column where the starting value is a 0 or a value that occurs in a different column and then all of the following values within that column depend on the value preceding [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[45],"tags":[],"class_list":["post-2132","post","type-post","status-publish","format-standard","hentry","category-r-tips-tricks"],"_links":{"self":[{"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/posts\/2132","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/comments?post=2132"}],"version-history":[{"count":4,"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/posts\/2132\/revisions"}],"predecessor-version":[{"id":2143,"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/posts\/2132\/revisions\/2143"}],"wp:attachment":[{"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/media?parent=2132"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/categories?post=2132"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/tags?post=2132"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}