Feature selection step using the Relief algorithm — step_select

Relief-based algorithms use nearest neighbors of randomly sampled observations (without replacement) to derive feature weights/scores that describe the relevance of each feature to the target variable. The feature weights represent the differences between the normalized feature values from each randomly sampled observation and a neighboring observation. If the neighboring observation's class is the same as the sampled observation (termed a 'hit') but the feature values are different, then this reduces the score on the basis that widely varying feature values for the same class are not desirable. Conversely, if a neighboring observation's class is different from the sampled observation (termed a 'miss') and the feature values are different, then this increases the score on the basis that observations of different classes are widely separated by their feature values. The feature weights / scores range from -1 (worst) to +1 (best).

Usage

step_select_relief(
  recipe,
  ...,
  outcome = NULL,
  role = NA,
  trained = FALSE,
  top_p = NA,
  threshold = NA,
  cutoff = NA,
  neighbors = 5,
  sample_size = 10,
  exclude = NULL,
  scores = NULL,
  skip = FALSE,
  id = recipes::rand_id("select_relief")
)

# S3 method for step_select_relief
tidy(x, ...)

Arguments

recipe: A recipe object. The step will be added to the sequence of operations for this recipe.
...: One or more selector functions to choose which variables are affected by the step. See selections() for more details. For the tidy method, these are not currently used.
outcome: A character string with the name of the response variable to use to evaluate information gain value against the predictors.
role: Not used by this step since no new variables are created.
trained: A logical to indicate if the quantities for preprocessing have been estimated.
top_p: An integer with the number of best scoring features to select.
threshold: A numeric value between 0 and 1 representing the percentile of best scoring features to select. For example `threshold = 0.9` will retain only predictors with scores in the top 90th percentile and a smaller threshold will select more features. Note that `top_p` and `threshold` are mutually exclusive but either can be used in conjunction with `cutoff` to select the top-ranked features and those that have filter scores that are larger than the cutoff value.
cutoff: A numeric value where predictors with _larger_ absolute filter scores than the cutoff will be retained. A value of `NA` implies that this criterion will be ignored.
neighbors: An integer with the number of neighbors for find for each sampled instance. Default is 5.
sample_size: An integer with the number of instances to sample. Default is 10.
exclude: A character vector of predictor names that will be removed from the data. This will be set when `prep()` is used on the recipe and should not be set by the user.
scores: A tibble with 'variable' and 'scores' columns containing the names of the variables and their information gain scores. This parameter is only produced after the recipe has been trained.
skip: A logical. Should the step be skipped when the recipe is baked by bake.recipe()? While all operations are baked when prep.recipe() is run, some operations may not be able to be conducted on new data (e.g. processing the outcome variable(s)). Care should be taken when using skip = TRUE as it may affect the computations for subsequent operations.
id: A character string that is unique to this step to identify it.
x: A `step_select_relief` object.

Value

A step_select_relief object.

Details

`step_select_relief` creates a *specification* of a recipe step that selects a subset of predictors based on the scores of the relief algorithm. This step requires the FSinR package to be installed. The top `top_p` scoring features, or features whose scores occur in the top percentile `threshold` will be retained as new predictors.

The recipe will stop if all of `top_p`, `threshold` and `cutoff` are left unspecified.

Examples

library(recipes)

data(cells, package = "modeldata")

rec <-
 recipe(class ~ ., data = cells[, -1]) %>%
 step_select_relief(
   all_predictors(),
   outcome = "class",
   top_p = 10
 )

prepped <- prep(rec)

new_data <- bake(prepped, new_data = NULL)
prepped
#> 
#> ── Recipe ──────────────────────────────────────────────────────────────────────
#> 
#> ── Inputs 
#> Number of variables by role
#> outcome:    1
#> predictor: 56
#> 
#> ── Training information 
#> Training data contained 2019 data points and no incomplete rows.
#> 
#> ── Operations 
#> Relief feature selection (46 excluded)