Perform variance inflation factor calculation, and optionally plot correlation matrix
+
Note that your dataframe should only have numerical features to perform VIF
+
+
+
+
+
Type
+
Default
+
Details
+
+
+
+
+
df
+
pd.DataFrame
+
+
dataframe to plot
+
+
+
plot_corr
+
bool
+
False
+
to plot the correlation matrix
+
+
+
figsize
+
tuple
+
(10, 10)
+
Matplotlib figsize
+
+
+
+
1. Why you should use VIF: to detect multicollinearity (more than 2 columns)
+
+
Compute variance inflation factor
+
The VIF is variance inflation factor the ratio of the variance of βˆj when fitting the full model (with other features) divided by the variance of βˆj if fit on its own
+
+
Min(VIF) = 1 (no collinearity)
+
VIF >5 or >10 means high collinearity
+
+
+
2. How to calculating VIF: Set the suspected collinearity feature (e.g. X1) as label, and try to predict X1 using a regression model and other features
+
3. What to do with high collinearity:
+
+
Drop one of them
+
Combine them to create a new feature
+
Perform an analysis designed for highly correlated variables, such as principal components analysis or partial least squares regression.
Cramer’s V measures association between two nominal variables.
+
Cramer’s V lies between 0 and 1 (inclusive). - 0 indicates that the two variables are not linked by any relation. - 1 indicates that there exists a strong association between the two variables.
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
/home/quan/anaconda3/envs/dev/lib/python3.10/site-packages/sklearn/base.py:439: UserWarning: X does not have valid feature names, but LinearRegression was fitted with feature names
+
+
+
+
+
+
+
+
+
+
+
from sklearn.model_selection import train_test_split
/home/quan/anaconda3/envs/dev/lib/python3.10/site-packages/sklearn/base.py:439: UserWarning: X does not have valid feature names, but LinearRegression was fitted with feature names
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
/home/quan/anaconda3/envs/dev/lib/python3.10/site-packages/sklearn/base.py:439: UserWarning: X does not have valid feature names, but DecisionTreeClassifier was fitted with feature names
/home/quan/anaconda3/envs/dev/lib/python3.10/site-packages/sklearn/base.py:439: UserWarning: X does not have valid feature names, but DecisionTreeClassifier was fitted with feature names
Plot 2D graph of metric value for each pair of hyperparameters
+
+
+
+
+
+
+
+
+
+
+
Type
+
Default
+
Details
+
+
+
+
+
search_cv
+
dict
+
+
A dict with keys as column headers and values as columns. Typically an attribute (cv_results_) of GridSearchCV or RandomizedSearchCV
+
+
+
param1
+
str
+
+
Name of the first hyperparameter
+
+
+
param2
+
str
+
+
Name of the second hyperparameter
+
+
+
scoring
+
str
+
f1_macro
+
Metric name
+
+
+
log_param1
+
bool
+
False
+
To log the first hyperparameter
+
+
+
log_param2
+
bool
+
False
+
To log the second hyperparameter
+
+
+
figsize
+
tuple
+
(20, 10)
+
Matplotlib figsize
+
+
+
min_hm
+
NoneType
+
None
+
Minimum value for the metric to show
+
+
+
max_hm
+
NoneType
+
None
+
Maximum value of the metric to show
+
+
+
higher_is_better
+
bool
+
True
+
Set if high metric is better
+
+
+
+
+
dt = RandomForestClassifier(random_state=42)
+param_grid={
+'n_estimators': np.arange(2,20),
+'min_samples_leaf': np.arange(1,80),
+}
+# Note: in order to use params_2D_heatmap, you should set scoring to a list, and set refit to False
+clf = RandomizedSearchCV(dt,param_grid,n_iter=100,
+ scoring=['f1_macro'],n_jobs=-1,
+ cv=5,verbose=1,random_state=42,refit=False)
+clf.fit(df_num.drop('Survived',axis=1),df_num['Survived'])
+
+
Fitting 5 folds for each of 100 candidates, totalling 500 fits
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
/tmp/ipykernel_4395/2399879344.py:14: MatplotlibDeprecationWarning: The get_cmap function was deprecated in Matplotlib 3.7 and will be removed two minor releases later. Use ``matplotlib.colormaps[name]`` or ``matplotlib.colormaps.get_cmap(obj)`` instead.
/tmp/ipykernel_4395/2399879344.py:14: MatplotlibDeprecationWarning: The get_cmap function was deprecated in Matplotlib 3.7 and will be removed two minor releases later. Use ``matplotlib.colormaps[name]`` or ``matplotlib.colormaps.get_cmap(obj)`` instead.
+
+
+
+
+
+
+
+
+
+
+
dt = RandomForestClassifier(random_state=42)
+param_grid={
+'n_estimators': np.arange(2,20),
+'min_samples_leaf': np.arange(1,80),
+'max_features': [0.3,0.4,0.5,0.6,0.7,0.8,0.9,1],
+}
+# Note: in order to use params_2D_heatmap, you should set scoring to a list, and set refit to False
+clf = RandomizedSearchCV(dt,param_grid,n_iter=100,
+ scoring=['f1_macro'],n_jobs=-1,
+ cv=5,verbose=1,random_state=42,refit=False)
+clf.fit(df_num.drop('Survived',axis=1),df_num['Survived'])
+
+
Fitting 5 folds for each of 100 candidates, totalling 500 fits
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
To better showcase the interpretation of Partial Dependency Plot, we will reuse the Titanic dataset, but now the independent variable (the one we need to predict) will be Pclass (3 classes to predict)
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
+ This module contains several Python function for simple data preprocessing for ML, such as handling missing values, minmax scaling, and one hot encoding
+
+ This should give you an idea on how this library works
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
In this tutorial, we will use the Pima Indian dataset https://www.kaggle.com/datasets/uciml/pima-indians-diabetes-database. The objective of the dataset is to diagnostically predict whether or not a patient has diabetes, based on certain diagnostic measurements included in the dataset.
+
+
from that_ml_library.data_preprocess import*
+from that_ml_library.chart_plotting import*
+from that_ml_library.ml_helpers import*
For this dataset, we only need to perform scaling on numerical features, as there’s no categorical features to process. Note that scaling of numerical data is required for linear model such as linear regression, and not required for tree model, though it doesn’t hurt the performance of such tree model. For simplicity, we will scale the numerical features for all the models we are going to run below.
For Blood Pressure and Skin Thickness, a value of 0 does not make sense, so we will consider these as missing values. The strategy for filling in the missing values will be ‘median’
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
Interpretation of coefficients in logistic regression: For a coefficient β1, given a one unit increase in one of the variables (say X1), the odds of X (p(X)/1-p(X)) will be multiplied by e^β1.
+
In this case, we can tell from coefficient of Glucose that: the odd of having diabetes will be multiplied by ~40 if Glucose level is increased by 1
This is the validation curve for hyperparameter max depth, which can be used to prune a decision tree.
+
For max depth, we can observe the overfitting behavior when max depth is too high, which means the tree is able to make unlimited splits to the extreme of classifying each of the training data points correctly, thus failing to generalize. On the other hand, a small depth is enough to achieve a good CV score (from depth 1 to 5). This is a telling sign that some features have strong signals to predict diabetes than others.
Based on the learning curve for the best tuned decision tree, more data is better for the model to learn, as training score seems to convert, and CV score still slowly increases. However, as the gap between CV score and train score is closing, adding lots of data might not be beneficial. For the scalability of the model, the relationship between training time and fit time is linear, which makes sense as more data allows tree to grow more, which takes more time to fit and predict.
/home/quan/anaconda3/envs/dev/lib/python3.10/site-packages/sklearn/base.py:439: UserWarning: X does not have valid feature names, but DecisionTreeClassifier was fitted with feature names
+
+
+
+
+
+
+
+
+
+
Glucose alone can make CV F1 score to reach 0.704, which is not far from the CV score of the best tuned model. With domain knowledge, we can confirm that high glucose is the number one indicator for diabetes, because diabetes patients are unable to effectively use insulin to break down glucose.
By looking at the 3D map of these three hyperparameters, we might need to exclude max_features of 1 and min_samples_leaf that is more than 80. In fact, max_features of 0.8 and 0.9 shows the most promising set of hyperparameters, thus we can now simplied our search space to this:
Insulin is not an impact feature for this dataset. Digging deeper on the source of the data, there’s a description that said: “This population has minimal European admixture, and their diabetes appears to be exclusively type 2 diabetes, with no evidence of the autoimmunity characteristic of type 1 diabetes”. Type 2 diabetes patients can still produce insulin, but the body is unable to use it. This information about insulin can be useful later if we want to perform feature selection, in order to boost this model’s performance
pdp_numerical_only(final_model,X_train_processed,num_features,class_names=['no diabetes','diabetes'],
+ nrows=4,ncols=2)
+# y_class is ignored in binary classification or classical regression settings.
+
+
+
+
+
+
+
+
+
For Partial Dependency Plot, Glucose, BMI and Age show the most changes compared to other features. This matches the feature importances plot earlier.
+
+
plot_ice_pair(final_model,X_train_processed,pair_features=['Glucose','Age'],class_idx=1,figsize=(10,4))
+# class_idx is ignored in binary classification or classical regression settings.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/images/tree_depth_3_PID.png b/images/tree_depth_3_PID.png
new file mode 100644
index 0000000..f852834
Binary files /dev/null and b/images/tree_depth_3_PID.png differ
diff --git a/images/tree_depth_3_titanic.png b/images/tree_depth_3_titanic.png
new file mode 100644
index 0000000..f345674
Binary files /dev/null and b/images/tree_depth_3_titanic.png differ
diff --git a/index.html b/index.html
new file mode 100644
index 0000000..7459a6b
--- /dev/null
+++ b/index.html
@@ -0,0 +1,637 @@
+
+
+
+
+
+
+
+
+
+
+that-ml-library
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
that-ml-library
+
+
+
+
+ A useful package for EDA and quick ML model building
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Install
+
pip install that_ml_library
+
For tree visualization function (plot_tree_dtreeviz or plot_tree_sklearn), you also need to install graphviz. Please follow the instruction here
This library should only be utilized solely for developing a proof of concept or prototype for your machine learning model with your specific dataset, with the aim of evaluating the model’s performance and interpretability. For deployment in a production environment, opt for a more organized methodology, such as https://scikit-learn.org/stable/modules/compose.html#pipeline
