Skip to contents

Filter Categorical Predictors using Contingency Tables

Source: R/step_select_xtab.R
step_select_xtab.Rd

`step_select_xtab` creates a *specification* of a recipe step that will filter predictors using their relationship with the outcome as measured using statistical tests for association.

Usage

step_select_xtab(
  recipe,
  ...,
  outcome,
  role = "predictor",
  trained = FALSE,
  top_p = NA,
  threshold = NA,
  cutoff = NA,
  exact = FALSE,
  fdr = TRUE,
  exclude = NULL,
  skip = FALSE,
  id = recipes::rand_id("select_xtab")
)

# S3 method for step_select_xtab
tidy(x, ...)

Arguments

recipe

A recipe object. The step will be added to the sequence of operations for this recipe.

...

One or more selector functions to choose which predictors are affected by the step. See [selections()] for more details. For the `tidy` method, these are not currently used.

outcome

A single character string that specifies a single categorical variable to be used as the class.

role

For model terms created by this step, what analysis role should they be assigned?. By default, the function assumes that resulting distances will be used as predictors in a model.

trained

A logical to indicate if the quantities for preprocessing have been estimated.

top_p

An integer that will be used to select the predictors with the smallest p/FDR values. A value of `NA` implies that this criterion will be ignored.

threshold

A numeric value between 0 and 1 representing the percentile of best scoring features to select. For example `threshold = 0.9` will retain only predictors with scores in the top 90th percentile and a smaller threshold will select more features. Note that `top_p` and `threshold` are mutually exclusive but either can be used in conjunction with `cutoff` to select the top-ranked features and those that have filter scores that are larger than the cutoff value.

cutoff

A numeric value, in p-value/FDR units, where predictors with _smaller_ than the threshold will be retained. A value of `NA` implies that this criterion will be ignored.

exact

Should an exact test be used?

fdr

Should false discovery rates (FDR) be used instead of p-values?

exclude

A character vector of predictor names that will be removed from the data. This will be set when `prep()` is used on the recipe and should not be set by the user.

skip

A logical. Should the step be skipped when the recipe is baked by bake.recipe()? While all operations are baked when prep.recipe() is run, some operations may not be able to be conducted on new data (e.g. processing the outcome variable(s)). Care should be taken when using skip = TRUE as it may affect the computations for subsequent operations.

id

A character string that is unique to this step to identify it.

x

A `step_select_xtab` object.

Value

An updated version of `recipe` with the new step added to the sequence of existing steps (if any). For the `tidy` method, a tibble with a `terms` column for which predictors were removed.

Details

The recipe will stop if all of `top_p`, `threshold` and `cutoff` are left unspecified. If both are used, they are combined via 'or'.

The Benjamini-Hochberg FDR correction is used (see [stats::p.adjust()]).

Warnings from [stats::chisq.test()] and [stats::fisher.test()] are suppressed.

Examples

data(attrition, package = "modeldata")

rec <-
  recipe(Attrition ~ ., data = attrition) %>%
  step_select_xtab(all_nominal(), -all_outcomes(), outcome = "Attrition",
                   top_p = 1, cutoff = 0.001, exact = TRUE) %>%
  prep()

rec %>% juice(all_nominal(), -all_outcomes()) %>% names()
#> [1] "BusinessTravel"          "EnvironmentSatisfaction"
#> [3] "JobInvolvement"          "MaritalStatus"          
#> [5] "OverTime"               

tidy(rec, number = 1)
#> # A tibble: 9 × 2
#>   terms                    id               
#>   <chr>                    <chr>            
#> 1 JobSatisfaction          select_xtab_u05VQ
#> 2 WorkLifeBalance          select_xtab_u05VQ
#> 3 Department               select_xtab_u05VQ
#> 4 RelationshipSatisfaction select_xtab_u05VQ
#> 5 Gender                   select_xtab_u05VQ
#> 6 Education                select_xtab_u05VQ
#> 7 PerformanceRating        select_xtab_u05VQ
#> 8 EducationField           select_xtab_u05VQ
#> 9 JobRole                  select_xtab_u05VQ