|
| 1 | +--- |
| 2 | +pre: <b>10/24. </b> |
| 3 | +title: "Iteration" |
| 4 | +weight: 20 |
| 5 | +summary: "Iterate through elements and give our function superpowers." |
| 6 | +format: |
| 7 | + hugo: |
| 8 | + toc: true |
| 9 | + output-file: "_index.en.md" |
| 10 | + reference-links: true |
| 11 | + code-link: true |
| 12 | + |
| 13 | +--- |
| 14 | + |
| 15 | +```{r, include=FALSE} |
| 16 | +answers = TRUE |
| 17 | +``` |
| 18 | + |
| 19 | +## Overview |
| 20 | + |
| 21 | +Iteration allows us to tell R to work on a whole sets of things at once: multiple files, multiple columns, multiple whatever. This can save quite a bit of time. It also lets us to work on problems with dependence, where the decisions of each step depends on the result of the previous step. |
| 22 | + |
| 23 | +For our worksheet today, we are going to be solving some annoyances of the past. I am going to walk you though modifying our `pet_split()` function from last week's functions worksheet to make it even more generalizable. |
| 24 | + |
| 25 | +## The Data |
| 26 | + |
| 27 | +We are going to be using class survey data for lab today. Please load it in using the following: |
| 28 | + |
| 29 | +```{r} |
| 30 | +survey = read.csv("https://raw.githubusercontent.com/Intro-to-Data-Science-Template/intro_to_data_science_reader/main/content/class_worksheets/4_r_rstudio/data/survey_data.csv") |
| 31 | +``` |
| 32 | + |
| 33 | +## A Refresher on our `pet_split()` Function |
| 34 | + |
| 35 | +Recall from last week that our `pet_split()` function looked at the `pets` column in our `survey` dataframe, and tidy-ed up the column so that instead of having a single character with multiple pets in it, we had a dataframe with `TRUE` and `FALSE` for each pet type, along with "other." You can see the finished function below: |
| 36 | + |
| 37 | +```{r} |
| 38 | +pet_split = function(pet_vector) { |
| 39 | + |
| 40 | + # make new dataframe for output |
| 41 | + pet_output = data.frame( |
| 42 | + "id" = 1:length(pet_vector), |
| 43 | + "dog" = NA, |
| 44 | + "cat" = NA, |
| 45 | + "fish" = NA, |
| 46 | + "bird" = NA, |
| 47 | + "reptile" = NA, |
| 48 | + "rock" = NA, |
| 49 | + "none" = NA, |
| 50 | + "other" = NA) |
| 51 | + |
| 52 | + # get a binary for each known pet type |
| 53 | + pet_output$dog = grepl(pattern = "dog", x = pet_vector, ignore.case = TRUE) |
| 54 | + pet_output$cat = grepl(pattern = "cat", x = pet_vector, ignore.case = TRUE) |
| 55 | + pet_output$fish = grepl(pattern = "fish", x = pet_vector, ignore.case = TRUE) |
| 56 | + pet_output$bird = grepl(pattern = "bird", x = pet_vector, ignore.case = TRUE) |
| 57 | + pet_output$reptile = grepl(pattern = "reptile", x = pet_vector, ignore.case = TRUE) |
| 58 | + pet_output$rock = grepl(pattern = "rock", x = pet_vector, ignore.case = TRUE) |
| 59 | + pet_output$none = grepl(pattern = "none", x = pet_vector, ignore.case = TRUE) |
| 60 | + |
| 61 | + # remove all known pets and clean remaining text |
| 62 | + pet_vector = gsub(pattern = "dog", pet_vector, replacement = "", ignore.case = TRUE) |
| 63 | + pet_vector = gsub(pattern = "cat", pet_vector, replacement = "", ignore.case = TRUE) |
| 64 | + pet_vector = gsub(pattern = "fish", pet_vector, replacement = "", ignore.case = TRUE) |
| 65 | + pet_vector = gsub(pattern = "bird", pet_vector, replacement = "", ignore.case = TRUE) |
| 66 | + pet_vector = gsub(pattern = "reptile", pet_vector, replacement = "", ignore.case = TRUE) |
| 67 | + pet_vector = gsub(pattern = "rock", pet_vector, replacement = "", ignore.case = TRUE) |
| 68 | + pet_vector = gsub(pattern = "none", pet_vector, replacement = "", ignore.case = TRUE) |
| 69 | + pet_vector = gsub(pattern = ",", pet_vector, replacement = "", ignore.case = TRUE) |
| 70 | + pet_vector = trimws(pet_vector) |
| 71 | + |
| 72 | + # Fill in "other" |
| 73 | + pet_output$other = pet_vector |
| 74 | + # Turn blanks into NAs |
| 75 | + pet_output[pet_output$other == "", "other"] = NA |
| 76 | + |
| 77 | + # return |
| 78 | + return(pet_output) |
| 79 | +} |
| 80 | +
|
| 81 | +pet_split(pet_vector = survey$pets) |
| 82 | +``` |
| 83 | + |
| 84 | +Now, that is cool, but we can make it better. Specifically, if we look at our survey dataframe, we have the exact same problem in our `<DRINK>_days` columns and the `recreation` column. By the end of this worksheet, our `pet_split()` function will work on *any* column with comma separated values. |
| 85 | + |
| 86 | +## Break it Down |
| 87 | + |
| 88 | +Our first step is going to be writing the code to accomplish what we want, then we can package it as a function. I've gutted our `pet_split()` function below. We will be starting from that base and working to make it so that we never explicitly call for anything in our code. For example, instead of coding all of the possibilities of pet (dog, cat, fish, bird, reptile, rock, none) inside the function itself, we want to write our code such that it can accept *any* list of possibilities as an argument and work from that. |
| 89 | + |
| 90 | +```{r} |
| 91 | +# set up a psudo argument |
| 92 | +pet_vector = survey$pets |
| 93 | +
|
| 94 | +# make new dataframe for output |
| 95 | +pet_output = data.frame( |
| 96 | +"id" = 1:length(pet_vector), |
| 97 | +"dog" = NA, |
| 98 | +"cat" = NA, |
| 99 | +"fish" = NA, |
| 100 | +"bird" = NA, |
| 101 | +"reptile" = NA, |
| 102 | +"rock" = NA, |
| 103 | +"none" = NA, |
| 104 | +"other" = NA) |
| 105 | +
|
| 106 | +# get a binary for each known pet type |
| 107 | +pet_output$dog = grepl(pattern = "dog", x = pet_vector, ignore.case = TRUE) |
| 108 | +pet_output$cat = grepl(pattern = "cat", x = pet_vector, ignore.case = TRUE) |
| 109 | +pet_output$fish = grepl(pattern = "fish", x = pet_vector, ignore.case = TRUE) |
| 110 | +pet_output$bird = grepl(pattern = "bird", x = pet_vector, ignore.case = TRUE) |
| 111 | +pet_output$reptile = grepl(pattern = "reptile", x = pet_vector, ignore.case = TRUE) |
| 112 | +pet_output$rock = grepl(pattern = "rock", x = pet_vector, ignore.case = TRUE) |
| 113 | +pet_output$none = grepl(pattern = "none", x = pet_vector, ignore.case = TRUE) |
| 114 | +
|
| 115 | +# remove all known pets and clean remaining text |
| 116 | +pet_vector = gsub(pattern = "dog", pet_vector, replacement = "", ignore.case = TRUE) |
| 117 | +pet_vector = gsub(pattern = "cat", pet_vector, replacement = "", ignore.case = TRUE) |
| 118 | +pet_vector = gsub(pattern = "fish", pet_vector, replacement = "", ignore.case = TRUE) |
| 119 | +pet_vector = gsub(pattern = "bird", pet_vector, replacement = "", ignore.case = TRUE) |
| 120 | +pet_vector = gsub(pattern = "reptile", pet_vector, replacement = "", ignore.case = TRUE) |
| 121 | +pet_vector = gsub(pattern = "rock", pet_vector, replacement = "", ignore.case = TRUE) |
| 122 | +pet_vector = gsub(pattern = "none", pet_vector, replacement = "", ignore.case = TRUE) |
| 123 | +pet_vector = gsub(pattern = ",", pet_vector, replacement = "", ignore.case = TRUE) |
| 124 | +pet_vector = trimws(pet_vector) |
| 125 | +
|
| 126 | +# Fill in "other" |
| 127 | +pet_output$other = pet_vector |
| 128 | +# Turn blanks into NAs |
| 129 | +pet_output[pet_output$other == "", "other"] = NA |
| 130 | +``` |
| 131 | + |
| 132 | +### Arbitrary Output |
| 133 | + |
| 134 | +The first step of our current code is to create a dataframe for our outputs. We still want to do that, but without us defining each possibility ourselves inside the function. Instead, we will provide a vector of possibilities, and have R iterate through those to make our columns. We can use a `for()` loop for that. |
| 135 | + |
| 136 | +First, we'll create a vector of our possibilities, in this case our pets: |
| 137 | + |
| 138 | +```{r} |
| 139 | +possible_columns = c("dog", "cat", "fish", "bird", "reptile", "rock", "none") |
| 140 | +``` |
| 141 | + |
| 142 | +Next, we need code to iterate through those possibilities, and create a dataframe from them. We'll start with making what we know, a column for IDs which has as many rows as our intended input, `pet_vector` from above. Next, we will iterate through all possible options, and make a new column for each. Here I iterate through our `possible_columns` vector, and for each element (`option` in the loop) I create a column of `NA`s. |
| 143 | + |
| 144 | +```{r} |
| 145 | +# make a base dataframe with rows for each of our cases. |
| 146 | +pet_output = data.frame( |
| 147 | + "id" = 1:length(pet_vector) |
| 148 | + ) |
| 149 | +
|
| 150 | +# iterate through all optinos and create a column with NAs for it |
| 151 | +for(option in possible_columns){ |
| 152 | + |
| 153 | + # make a new column with a character version of each possible option. |
| 154 | + pet_output[, as.character(option)] = NA |
| 155 | + |
| 156 | +} |
| 157 | +``` |
| 158 | + |
| 159 | +If we look at out output now, it is *exactly* the same as if we made each column ourselves, but now it is done by providing a vector of options. *And we can change those options to whatever we want*. This will come in handy later. |
| 160 | + |
| 161 | +### Test for Each Option |
| 162 | + |
| 163 | +Our next step, as before, is to test for each possible option and fill in the respective columns. We will use iteration here as well. |
| 164 | + |
| 165 | +::: {.question} |
| 166 | +Using the same principle as above, iterate over each option in `possible_columns` and use `grepl()` to test if the pet appeared in that case. Fill the respective columns. *Make sure that `pet_vector` and `possible_columns` are reset to normal before you try!* |
| 167 | +::: |
| 168 | + |
| 169 | +```{r, include=answers, results='asis', echo=FALSE} |
| 170 | +cat(" |
| 171 | +::: {.answer} |
| 172 | +
|
| 173 | +The following code will iterate through `possible_columns` and replace the pattern `grepl()` is looking for with each option. It will test for that option, and save the results in the corresponding column. |
| 174 | +
|
| 175 | +\``` |
| 176 | +for(option in possible_columns){ |
| 177 | + |
| 178 | + # fill dataframe iterativly. |
| 179 | + pet_output[ , option] = grepl(option, pet_vector, ignore.case = TRUE) |
| 180 | + |
| 181 | +} |
| 182 | +\``` |
| 183 | +::: |
| 184 | +") |
| 185 | +``` |
| 186 | + |
| 187 | +### Remove the Known Options to Find "Other" |
| 188 | + |
| 189 | +Once we have our "knowns" taken care of, we can work on the others. The process is nearly identical, just swap `grepl()` with `gsub()` and apply it to `pet_vector` like before. |
| 190 | + |
| 191 | +::: {.question} |
| 192 | +Iterate over each option in `possible_columns` and use `gsub()` to remove all of our known possibilities (and commas) from `pet_vector`. You can then use `trimsws()` to remove the extra spaces. Assign the remaining values to the "other" column of `pet_output`. |
| 193 | +::: |
| 194 | + |
| 195 | +```{r, include=answers, results='asis', echo=FALSE} |
| 196 | +cat(" |
| 197 | +::: {.answer} |
| 198 | +
|
| 199 | +The following will remove all known possibilities, clean the remainder, and assign it to the 'other' column. |
| 200 | +
|
| 201 | +\``` |
| 202 | +for(option in possible_columns){ |
| 203 | + |
| 204 | + # remove all known options |
| 205 | + pet_vector = gsub(pattern = option, pet_vector, replacement = '', ignore.case = TRUE) |
| 206 | + |
| 207 | +} |
| 208 | +
|
| 209 | +# clear commas and whitespace |
| 210 | +pet_vector = gsub(pattern = ',', pet_vector, replacement = '', ignore.case = TRUE) |
| 211 | +pet_vector = trimws(pet_vector) |
| 212 | +
|
| 213 | +# Fill in 'other' |
| 214 | +pet_output$other = pet_vector |
| 215 | +# Turn blanks into NAs |
| 216 | +pet_output[pet_output$other == '', 'other'] = NA |
| 217 | +\``` |
| 218 | +::: |
| 219 | +") |
| 220 | +``` |
| 221 | + |
| 222 | +### Turn it Back into a Function |
| 223 | + |
| 224 | +If we look at our code all together now, it looks like the following. If we run it, it will return the exact same thing as our old `pet_split()` function, but instead of each option being hand-coded by us, it knows how to work with any given vector of options and create our desired output. |
| 225 | + |
| 226 | +```{r} |
| 227 | +# make dummy argument |
| 228 | +pet_vector = survey$pets |
| 229 | +
|
| 230 | +# set all known options |
| 231 | +possible_columns = c("dog", "cat", "fish", "bird", "reptile", "rock", "none") |
| 232 | +
|
| 233 | +# make a base dataframe with rows for each of our cases. |
| 234 | +pet_output = data.frame( |
| 235 | + "id" = 1:length(pet_vector) |
| 236 | + ) |
| 237 | +
|
| 238 | +# iterate through all options and create a column with NAs for it |
| 239 | +for(option in possible_columns){ |
| 240 | + |
| 241 | + # make a new column with a character version of each possible option. |
| 242 | + pet_output[, as.character(option)] = NA |
| 243 | + |
| 244 | +} |
| 245 | +
|
| 246 | +# fill output df |
| 247 | +for(option in possible_columns){ |
| 248 | + |
| 249 | + # fill dataframe iterativly. |
| 250 | + pet_output[ , option] = grepl(option, pet_vector, ignore.case = TRUE) |
| 251 | + |
| 252 | +} |
| 253 | +
|
| 254 | +# clear all know options |
| 255 | +for(option in possible_columns){ |
| 256 | + |
| 257 | + # remove all known options |
| 258 | + pet_vector = gsub(pattern = option, pet_vector, replacement = '', ignore.case = TRUE) |
| 259 | + |
| 260 | +} |
| 261 | +
|
| 262 | +# clear commas and whitespace |
| 263 | +pet_vector = gsub(pattern = ',', pet_vector, replacement = '', ignore.case = TRUE) |
| 264 | +pet_vector = trimws(pet_vector) |
| 265 | +
|
| 266 | +# Fill in 'other' |
| 267 | +pet_output$other = pet_vector |
| 268 | +# Turn blanks into NAs |
| 269 | +pet_output[pet_output$other == '' & !is.na(pet_output$other), 'other'] = NA |
| 270 | +``` |
| 271 | + |
| 272 | +If we turn it into a function, it will look like this: |
| 273 | + |
| 274 | +```{r} |
| 275 | +pet_split = function(pet_vector, possible_columns){ |
| 276 | + |
| 277 | + # make a base dataframe with rows for each of our cases. |
| 278 | + pet_output = data.frame( |
| 279 | + "id" = 1:length(pet_vector) |
| 280 | + ) |
| 281 | + |
| 282 | + # iterate through all options and create a column with NAs for it |
| 283 | + for(option in possible_columns){ |
| 284 | + |
| 285 | + # make a new column with a character version of each possible option. |
| 286 | + pet_output[, as.character(option)] = NA |
| 287 | + |
| 288 | + } |
| 289 | + |
| 290 | + # fill output df |
| 291 | + for(option in possible_columns){ |
| 292 | + |
| 293 | + # fill dataframe iterativly. |
| 294 | + pet_output[ , option] = grepl(option, pet_vector, ignore.case = TRUE) |
| 295 | + |
| 296 | + } |
| 297 | + |
| 298 | + # clear all know options |
| 299 | + for(option in possible_columns){ |
| 300 | + |
| 301 | + # remove all known options |
| 302 | + pet_vector = gsub(pattern = option, pet_vector, replacement = '', ignore.case = TRUE) |
| 303 | + |
| 304 | + } |
| 305 | + |
| 306 | + # clear commas and whitespace |
| 307 | + pet_vector = gsub(pattern = ',', pet_vector, replacement = '', ignore.case = TRUE) |
| 308 | + pet_vector = trimws(pet_vector) |
| 309 | + |
| 310 | + # Fill in 'other' |
| 311 | + pet_output$other = pet_vector |
| 312 | + # Turn blanks into NAs |
| 313 | + pet_output[pet_output$other == "" & !is.na(pet_output$other), 'other'] = NA |
| 314 | + |
| 315 | + # return output |
| 316 | + return(pet_output) |
| 317 | +} |
| 318 | +
|
| 319 | +pet_split(pet_vector = survey$pets, |
| 320 | + possible_columns = c("dog", "cat", "fish", "bird", "reptile", "rock", "none")) |
| 321 | +``` |
| 322 | + |
| 323 | +That's pretty cool, but what do you think would happen if we tried it on another column? |
| 324 | + |
| 325 | +```{r} |
| 326 | +pet_split(pet_vector = survey$tea_days, |
| 327 | + possible_columns = c("monday", "tuesday", "wednesday", "thursday", "friday", "saturday", "sunday")) |
| 328 | +``` |
| 329 | + |
| 330 | +While it now has a bit of an odd name, our function can now work on *any* column! It is hard to express how big of a deal that is. We now have a single general tool that can adapt itself to several situations. The input is *arbitrary*, as long as it is formatted the same way (values separated by commas), we can put *anything* through this function and get a nice tidy dataframe back. A whole new universe of possibilities just opened. |
| 331 | + |
| 332 | +::: {.question} |
| 333 | +Try our function on some other columns in the `survey` dataframe! |
| 334 | +::: |
| 335 | + |
| 336 | + |
| 337 | + |
| 338 | + |
0 commit comments