MARS (Multivariate Adaptive Regression Splines)
MARS is an extension of CART that uses splines to model non-linear relationships between variables. MARS is a regression algorithm that uses a technique called forward stepwise selection to construct a piecewise linear model. A piecewise linear model is a model where the output variable is a linear function of the input variables, but the slope of the linear function can change at different points in the input space. The sites where piecewise linear functions (basis functions) connect are known as knots.Based on the distribution of the data and the requirement to capture non-linearities, MARS automatically chooses and positions knots
Basis Functions: Basis functions, or piecewise linear functions, are used by MARS to represent the relationship between predictors and the response variable. Simple linear functions that are defined across a particular range of a predictor variable make up each basis function.
In MARS, a basis function is described as:
[Tex]h(x) = \Bigg \{ x – t \;\; if \; x>t \\ t-x \;\; if x \leq t \Bigg\} [/Tex]
Where , x is a predictor variable and t is the knot function.
Knot Function: The sites where piecewise linear functions (basis functions) connect are known as knots. Based on the distribution of the data and the requirement to capture non-linearities, MARS automatically chooses and positions knots
MARS starts by constructing a model with a single piece. The algorithm then uses forward stepwise selection to add new pieces to the model. At each step, the algorithm adds the piece that reduces the residual sum of squares the most. The algorithm continues adding pieces until the model reaches a specified level of complexity. MARS can be used to model complex relationships between variables. It is particularly useful for modeling complex relationships in data.
Decision Tree Algorithms
Decision trees are a type of machine-learning algorithm that can be used for both classification and regression tasks. They work by learning simple decision rules inferred from the data features. These rules can then be used to predict the value of the target variable for new data samples.
Decision trees are represented as tree structures, where each internal node represents a feature, each branch represents a decision rule, and each leaf node represents a prediction. The algorithm works by recursively splitting the data into smaller and smaller subsets based on the feature values. At each node, the algorithm chooses the feature that best splits the data into groups with different target values.
Table of Content
- Understanding Decision Trees
- Components of a Decision Tree
- Working of the Decision Tree Algorithm
- Understanding the Key Mathematical Concepts Behind Decision Trees
- Types of Decision Tree Algorithms
- ID3 (Iterative Dichotomiser 3)
- C4.5
- CART (Classification and Regression Trees)
- CHAID (Chi-Square Automatic Interaction Detection)
- MARS (Multivariate Adaptive Regression Splines)
- Implementation of Decision Tree Algorithms
Contact Us