4️⃣Random Forest

Random Forest is a popular and powerful ensemble learning method for classification and regression problems. It is a collection of decision trees, where each tree is grown on a randomly selected subset of the data. The idea behind using multiple decision trees is that they will make different decisions based on different parts of the data, and by combining their predictions we can get a more accurate and stable result.

In Random Forest, each tree is built using a random subset of the data and a random subset of features, and the final prediction is made by averaging the predictions of all the trees. This technique helps to reduce the overfitting problem that can occur with a single decision tree.

Random Forest can be used for both classification and regression problems, and it is known for its ability to handle high dimensional data and missing values. It is also robust to outliers and can be used for feature selection. It is widely used in many industries, such as banking, finance, and healthcare

Analogy: An analogy for Random Forest is to think of a group of people trying to predict the outcome of a coin flip. Each person has their own decision-making process for guessing heads or tails, but together as a group, they are more likely to make a correct prediction than just one person alone.

Example

In this example, we first generate some synthetic data using the make_classification function from the sklearn.datasets module. Next, we create an instance of the RandomForestClassifier class with 100 trees and a random seed of 0. We then fit the model to the data using the fit method and make predictions on new data using the predict method.

You can adjust the number of trees in the forest by changing the n_estimators parameter. The more trees you use, the more robust the model will be, but it will also take longer to train. Other parameters such as max_depth and min_samples_split can also be adjusted.

Random Forest is a powerful algorithm and it's widely used in many industry problem, as it can handle high dimensional data and also it can handle missing values and outliers. However, it's worth noting that a random forest model can be overfitting and it can be resolved by tuning the parameters or by using the techniques like cross-validation, pruning and regularization.

Python Code

Machine-Learning/Chapter_2_Random_Forest.ipynb at main · ariesharry/Machine-LearningGitHub

from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import make_classification

# Generate some synthetic data
X, y = make_classification(n_features=4, n_informative=2,
                           n_redundant=0, random_state=0)

# Create the model
clf = RandomForestClassifier(n_estimators=100, random_state=0)

# Train the model
clf.fit(X, y)

X_new, _ = make_classification(n_features=4, n_informative=2,
                           n_redundant=0, random_state=1)
# Predict on new data
y_pred = clf.predict(X_new)
print(y_pred)

Output

[1 0 1 1 1 0 1 1 1 1 1 0 1 1 1 1 1 1 0 0 1 1 1 1 1 1 0 1 0 1 1 1 0 1 1 1 0
 0 0 1 0 1 1 1 0 0 1 1 1 0 1 1 1 1 1 0 0 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 0 1
 0 1 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 0 0 1 0 1 1]

In this example, we first generate some synthetic data using the make_classification function from the sklearn.datasets module. Next, we create an instance of the RandomForestClassifier class with 100 trees and a random seed of 0. We then fit the model to the data using the fit method and make predictions on new data using the predict method.

You can adjust the number of trees in the forest by changing the n_estimators parameter. The more trees you use, the more robust the model will be, but it will also take longer to train. Other parameters such as max_depth and min_samples_split can also be adjusted.

It's also worth noting that you can use RandomForestRegressor if you want to use random forest for regression problem.

from sklearn.ensemble import RandomForestRegressor

# Create the model
clf = RandomForestRegressor(n_estimators=100, random_state=0)

# Train the model
clf.fit(X, y)

# Predict on new data
y_pred = clf.predict(X_new)
print(y_pred)

Output

[0.74 0.01 1.   1.   1.   0.03 0.99 0.98 0.98 0.84 1.   0.42 1.   0.84
 0.82 1.   1.   1.   0.43 0.29 1.   0.83 1.   1.   0.98 0.94 0.42 0.98
 0.01 0.85 1.   0.89 0.03 0.98 0.61 0.83 0.01 0.05 0.02 0.99 0.   0.9
 0.98 0.73 0.48 0.27 0.99 0.91 0.98 0.01 1.   0.97 0.98 0.85 0.95 0.01
 0.02 1.   1.   1.   0.   0.01 1.   1.   0.69 0.53 0.96 1.   0.98 0.63
 1.   0.84 0.02 0.96 0.03 0.86 0.   1.   0.57 0.51 0.97 0.25 0.37 0.04
 0.02 0.86 0.81 0.98 0.84 0.01 0.04 0.   0.01 1.   0.   0.45 0.61 0.01
 1.   0.98]

Please note that this is just a basic example to show you how to use the Random Forest classifier and regressor, in real-world problems you need to preprocess and clean the data, and also evaluate your model using different evaluation metric and methods.

Reference:

https://deepai.org/machine-learning-glossary-and-terms/random-forest