Model Tuning

class: title-slide, center

# Model Tuning

## Tidy Data Science with the Tidyverse and Tidymodels

### W. Jake Thompson

#### [https://tidyds-2021.wjakethompson.com](https://tidyds-2021.wjakethompson.com) &#183; [https://bit.ly/tidyds-2021](https://bit.ly/tidyds-2021)

.footer-license[*Tidy Data Science with the Tidyverse and Tidymodels* is licensed under a [Creative Commons Attribution 4.0 International License](https://creativecommons.org/licenses/by/4.0/).]

<div style = "position:fixed; visibility: hidden">
  `$$\require{color}\definecolor{blue}{rgb}{0, 0.623529411764706, 0.717647058823529}$$`
  `$$\require{color}\definecolor{light_blue}{rgb}{0.0392156862745098, 0.870588235294118, 1}$$`
  `$$\require{color}\definecolor{yellow}{rgb}{0.996078431372549, 0.843137254901961, 0.4}$$`
  `$$\require{color}\definecolor{dark_yellow}{rgb}{0.635294117647059, 0.47843137254902, 0.00392156862745098}$$`
  `$$\require{color}\definecolor{pink}{rgb}{0.796078431372549, 0.16078431372549, 0.482352941176471}$$`
  `$$\require{color}\definecolor{light_pink}{rgb}{1, 0.552941176470588, 0.776470588235294}$$`
  `$$\require{color}\definecolor{grey}{rgb}{0.411764705882353, 0.403921568627451, 0.450980392156863}$$`
</div>
  
<script type="text/x-mathjax-config">
  MathJax.Hub.Config({
    TeX: {
      Macros: {
        blue: ["{\\color{blue}{#1}}", 1],
        light_blue: ["{\\color{light_blue}{#1}}", 1],
        yellow: ["{\\color{yellow}{#1}}", 1],
        dark_yellow: ["{\\color{dark_yellow}{#1}}", 1],
        pink: ["{\\color{pink}{#1}}", 1],
        light_pink: ["{\\color{light_pink}{#1}}", 1],
        grey: ["{\\color{grey}{#1}}", 1]
      },
      loader: {load: ['[tex]/color']},
      tex: {packages: {'[+]': ['color']}}
    }
  });
</script>

---
class: your-turn

# Your Turn 0

.big[
* Open the R Notebook **materials/exercises/14-tuning.Rmd**
* Run the setup chunk
]

---
<div class="hex-book">
  <a href="https://tune.tidymodels.org">
    <img class="hex" src="images/hex/tune.png">
  </a>
  <a href="https://www.tmwr.org/tuning.html">
    <img class="book" src="images/books/tmwr-tuning.png">
  </a>
</div>

---
class: center middle inverse

# KNN

---
# `nearest_neighbor()`

Specifies a model that uses K Nearest Neighbors

```r
nearest_neighbor(neighbors = 1)
```

k = .display[neighbors] (PLURAL)

.footnote[regression and classification modes]

---
class: your-turn

# Your Turn 1

Here's a new recipe (also in your .Rmd)...

```r
normalize_rec <-
  recipe(Sale_Price ~ ., data = ames) %>% 
    step_novel(all_nominal()) %>% 
    step_dummy(all_nominal()) %>% 
    step_zv(all_predictors()) %>% 
    step_center(all_predictors()) %>% 
    step_scale(all_predictors())
```

---
class: your-turn

# Your Turn 1

...and a new model. Can you tell what type of model this is?

```r
knn5_spec <- 
  nearest_neighbor(neighbors = 5) %>% 
  set_engine("kknn") %>% 
  set_mode("regression")
```

---
class: your-turn

# Your Turn 1

Combine the recipe and model into a new workflow named `knn_wf`. Fit the workflow to `cv_folds` and collect its RMSE.

---
class: your-turn

```r
knn5_wf <-
  workflow() %>%
  add_recipe(normalize_rec) %>%
  add_model(knn5_spec)

knn5_wf %>%
  fit_resamples(resamples = cv_folds) %>%
  collect_metrics()
#> # A tibble: 2 x 6
#>   .metric .estimator      mean     n    std_err .config             
#>   <chr>   <chr>          <dbl> <int>      <dbl> <chr>               
#> 1 rmse    standard   37191.       10 1130.      Preprocessor1_Model1
#> 2 rsq     standard       0.786    10    0.00971 Preprocessor1_Model1
```

---
class: your-turn

# Your Turn 2

Repeat the process in Your Turn 1 with a similar workflow that uses `neighbors = 10`. Does the RMSE change?

---
class: your-turn

```r
knn10_spec <- nearest_neighbor(neighbors = 10) %>% 
    set_engine("kknn") %>% 
    set_mode("regression")

knn10_wf <- 
  knn5_wf %>% 
  update_model(knn10_spec)

knn10_wf %>%
  fit_resamples(resamples = cv_folds) %>% 
  collect_metrics()
#> # A tibble: 2 x 6
#>   .metric .estimator      mean     n   std_err .config             
#>   <chr>   <chr>          <dbl> <int>     <dbl> <chr>               
#> 1 rmse    standard   35817.       10 972.      Preprocessor1_Model1
#> 2 rsq     standard       0.806    10   0.00869 Preprocessor1_Model1
```

---
class: pop-quiz

# Pop quiz!

How can you find the best value of `neighbors`/`k`?

Compare all the separate values/models

---
class: center middle inverse

# `tune_grid()`

---
# tune

Functions for fitting and tuning models

<https://tune.tidymodels.org/>

---
# `tune()`

A placeholder for hyper-parameters to be "tuned"

```r
nearest_neighbor(neighbors = tune())
```

---
# `tune_grid()`