jutadmin – Page 2 – Joy Uche Okafor

The Wisconsin Breast Cancer dataset can be found here. The Multiple Linear Regression, Polynomial Regression, Support Vector Regression, Decision Tree Regression and Random Forest Regression models were tried on the dataset. The model performances were evaluated using the R2 coefficient and the best model was selected. R2 is a goodness of fit parameter usually between 0 and 1. The closer R2 is to 1, the better the model. Multiple Linear Regression(MLR) #Import the Libraries import numpy as np import matplotlib.pyplot as plt import pandas as pd # Import the dataset dataset = pd.read_csv('Breast_cancer_data.csv') X = dataset.iloc[:, :-1].values y = dataset.iloc[:, -1].values #Split dataset into training and test set from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0) # Train the MLR model on the training set from sklearn.linear_model import LinearRegression regressor = LinearRegression() regressor.fit(X_train, y_train) Output: LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None, normalize=False) #Predicting the test results y_pred = regressor.predict(X_test) np.set_printoptions(precision=2) print(np.concatenate((y_pred.reshape(len(y_pred),1), y_test.reshape(len(y_test),1)),1)) output: [[ 0.47 0. ] [ 0.65 1. ] [ 0.87 1. ] [ 0.89 1. ] [ 0.83 1. ] [ 0.84 1. ] [ 0.89 1. ] [ 0.94 1. ] [ 1.32 1. ] [ 1.09 1. ] [ 0.53 1. ] [ 0.5 1. ] [ 1.19 1. ] [ 0.58 1. ] [ 0.49 1. ] [ 0.38 0. ] [ 1.06 1. ] [-0.01 0. ] [ 0.03 0. ] [-0.06 0. ] [ 0.69 0. ] [ 0.25 0. ] [ 0.69 1. ] [ 0.92 1. ] [ 0.11 0. ] [ 1.11 1. ] [ 1.29 1. ] [ 0.28 0. ] [ 0.89 1. ] [ 0.23 0. ] [ 1. 1. ] [ 0.35 0. ] [ 0.7 1. ] [ 0.31 0. ] [ 1.31 1. ] [ 0.49 0. ] [ 0.87 1. ] [ 0.39 0. ] [ 0.69 1. ] [ 0.33 0. ] [ 0.42 0. ] [ 1.37 1. ] [ 0.5 0. ] [ 1.11 1. ] [ 0.73 1. ] [-0.06 0. ] [ 1.38 1. ] [ 0.64 1. ] [ 0.87 1. ] [ 0.16 0. ] [-0.17 0. ] [ 0.24 0. ] [ 0.25 0. ] [ 0.99 1. ] [ 0.83 1. ] [ 1.02 1. ] [ 0.94 1. ] [ 0.73 1. ] [ 0.93 1. ] [-0.05 0. ] [ 0.34 0. ] [ 0.5 0. ] [ 1.23 1. ] [ 0.84 1. ] [ 0.14 0. ] [ 0.69 1. ] [-0.46 0. ] [-0.18 0. ] [ 0.1 0. ] [ 1.01 1. ] [ 0.61 1. ] [ 0.13 0. ] [ 0.94 1. ] [ 0.56 0. ] [ 0.13 0. ] [ 0.71 1. ] [ 1.31 1. ] [ 0.6 1. ] [ 1.11 1. ] [ 1.21 1. ] [ 0.27 0. ] [-0.58 0. ] [ 0.14 0. ] [ 1.03 1. ] [ 0.49 0. ] [ 0.92 1. ] [ 1.05 1. ] [ 1.41 1. ] [ 0.31 0. ] [-0.48 0. ] [ 1.23 1. ] [ 0.63 0. ] [ 0.53 1. ] [ 0.02 0. ] [ 1.01 1. ] [ 0.99 1. ] [ 0.32 0. ] [ 0.86 1. ] [ 0.95 1. ] [ 0.82 1. ] [ 1.21 1. ] [ 0.9 1. ] [ 0.66 1. ] [ 0.37 1. ] [ 0.27 0. ] [ 0.69 1. ] [-0.18 0. ] [ 0.5 1. ] [ 0.66 0. ] [ 0.13 0. ] [ 0.66 1. ] [-0.04 0. ] [ 0.14 0. ] [ 0.52 1. ]] #Evaluating the model performance from sklearn.metrics import r2_score r2_score(y_test, y_pred) Output: 0.6259734883274091 Polynomial Regression # Import the libraries import numpy as np import matplotlib.pyplot as plt import pandas as pd #Import the dataset dataset = pd.read_csv('Breast_cancer_data.csv') X = dataset.iloc[:, :-1].values y = dataset.iloc[:, -1].values #Split the dataset into Training and Test set from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0) # Train the Polynomial Regression Model on the Training set from sklearn.preprocessing import PolynomialFeatures from sklearn.linear_model import LinearRegression poly_reg = PolynomialFeatures(degree = 2) X_poly = poly_reg.fit_transform(X_train) regressor = LinearRegression() regressor.fit(X_poly, y_train) output: LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None, normalize=False) # Predict the test results y_pred = regressor.predict(poly_reg.transform(X_test)) np.set_printoptions(precision=2) print(np.concatenate((y_pred.reshape(len(y_pred),1), y_test.reshape(len(y_test),1)),1)) output: [[ 0.38 0. ] [ 0.68 1. ] [ 0.95 1. ] [ 0.9 1. ] [ 0.89 1. ] [ 0.91 1. ] [ 1.19 1. ] [ 0.99 1. ] [ 1.09 1. ] [ 0.99 1. ] [ 0.54 1. ] [ 0.48 1. ] [ 1.07 1. ] [ 0.44 1. ] [ 0.57 1. ] [ 0.44 0. ] [ 0.84 1. ] [-0.09 0. ] [-0.07 0. ] [-0.11 0. ] [ 0.61 0. ] [ 0.11 0. ] [ 0.9 1. ] [ 0.97 1. ] [ 0.11 0. ] [ 1.21 1. ] [ 1.17 1. ] [ 0.11 0. ] [ 0.97 1. ] [ 0.06 0. ] [ 1.14 1. ] [-0.03 0. ] [ 0.77 1. ] [ 0.19 0. ] [ 1.16 1. ] [ 0.46 0. ] [ 0.92 1. ] [ 0.06 0. ] [ 0.79 1. ] [ 0.21 0. ] [ 0.36 0. ] [ 0.88 1. ] [ 0.37 0. ] [ 1.07 1. ] [ 0.84 1. ] [ 0.02 0. ] [ 0.96 1. ] [ 0.66 1. ] [ 0.96 1. ] [ 0.03 0. ] [ 0.07 0. ] [ 0.2 0. ] [ 0.03 0. ] [ 1.01 1. ] [ 0.94 1. ] [ 1.06 1. ] [ 1.17 1. ] [ 0.93 1. ] [ 0.95 1. ] [-0.26 0. ] [ 0.19 0. ] [ 0.35 0. ] [ 1.12 1. ] [ 0.92 1. ] [ 0.1 0. ] [ 0.85 1. ] [-0.42 0. ] [-0.11 0. ] [-0.02 0. ] [ 1.02 1. ] [ 0.65 1. ] [-0.12 0. ] [ 1.03 1. ] [ 0.52 0. ] [ 0.02 0. ] [ 0.76 1. ] [ 0.96 1. ] [ 0.64 1. ] [ 1.07 1. ] [ 0.98 1. ] [ 0.25 0. ] [-0.88 0. ] [-0.12 0. ] [ 1.05 1. ] [ 0.48 0. ] [ 1.16 1. ] [ 1.15 1. ] [ 0.84 1. ] [ 0.14 0. ] [-0.38 0. ] [ 1.11 1. ] [ 0.6 0. ] [ 0.55 1. ] [ 0.2 0. ] [ 1.01 1. ] [ 0.92 1. ] [ 0.01 0. ] [ 0.93 1. ] [ 0.99 1. ] [ 0.89 1. ] [ 0.72 1. ] [ 0.97 1. ] [ 0.64 1. ] [ 0.39 1. ] [ 0.08 0. ] [ 0.98 1. ] [-0.65 0. ] [ 0.5 1. ] [ 0.58 0. ] [-0.01 0. ] [ 0.77 1. ] [-0.05 0. ] [-0.02 0. ] [ 0.56 1. ]] # Evaluating the model performance from sklearn.metrics import r2_score r2_score(y_test, y_pred) output: 0.714466598440052 Random Forest Regression (RFR) Repeat steps 1-3 above # Train the random forest regression model on the whole dataset from sklearn.ensemble import RandomForestRegressor regressor = RandomForestRegressor(n_estimators = 10, random_state = 0) regressor.fit(X_train, y_train) output: RandomForestRegressor(bootstrap=True, ccp_alpha=0.0, criterion='mse', max_depth=None, max_features='auto', max_leaf_nodes=None, max_samples=None, min_impurity_decrease=0.0, min_impurity_split=None, min_samples_leaf=1, min_samples_split=2, min_weight_fraction_leaf=0.0, n_estimators=10, n_jobs=None, oob_score=False, random_state=0, verbose=0, warm_start=False) # Predict the test results y_pred = regressor.predict(X_test) np.set_printoptions(precision=2) print(np.concatenate((y_pred.reshape(len(y_pred),1), y_test.reshape(len(y_test),1)),1)) output: [[0.2 0. ] [1. 1. ] [1. 1. ] [1. 1. ] [1. 1. ] [1. 1. ] [1. 1. ] [1. 1. ] [1. 1. ] [1. 1. ] [0.9 1. ] [1. 1. ] [1. 1. ] [0.1 1. ] [1. 1. ] [0.3 0. ] [0.5 1. ] [0. 0. ] [0. 0. ] [0. 0. ] [0.9 0. ] [0. 0. ] [0.9 1. ] [1. 1. ] [0.2 0. ] [1. 1. ] [1. 1. ] [0. 0. ] [1. 1. ] [0. 0. ] [1. 1. ] [0.1 0. ] [0.9 1. ] [0.4 0. ] [1. 1. ] [0.7 0. ] [1. 1. ] [0.1 0. ] [1. 1. ] [0. 0. ] [0.5 0. ] [1. 1. ] [0.3 0. ] [1. 1. ] [1. 1. ] [0. 0. ] [1. 1. ] [0.9 1. ] [1. 1. ] [0. 0. ] [0. 0. ] [0. 0. ] [0. 0. ] [1. 1. ] [1. 1. ] [1. 1. ] [1. 1. ] [1. 1. ] [1. 1. ] [0. 0. ] [0. 0. ] [0.1 0. ] [1. 1. ] [1. 1. ] [0. 0. ] [1. 1. ] [0. 0. ] [0. 0. ] [0. 0. ] [1. 1. ] [0.5 1. ] [0. 0. ] [1. 1. ] [1. 0. ] [0. 0. ] [1. 1. ] [1. 1. ] [1. 1. ] [1. 1. ] [1. 1. ] [0. 0. ] [0. 0. ] [0.3 0. ] [1. 1. ] [1. 0. ] [1. 1. ] [0.9 1. ] [1. 1. ] [0. 0. ] [0. 0. ] [1. 1. ] [0.5 0. ] [0.7 1. ] [0. 0. ] [0.9 1. ] [1. 1. ] [0. 0. ] [1. 1. ] [1. 1. ] [0.7 1. ] [1. 1. ] [1. 1. ] [0.9 1. ] [0. 1. ] [0. 0. ] [0.7 1. ] [0. 0. ] [0.2 1. ] [0.5 0. ] [0. 0. ] [0.7 1. ] [0. 0. ] [0. 0. ] [0.6 1. ]] # Model performance evaluation from sklearn.metrics import r2_score r2_score(y_test, y_pred) output: 0.7056779930136551 Decision Tree Regression (DTR) Repeat steps 1-3 above # Train the decision tree regression model on the training set from sklearn.tree import DecisionTreeRegressor regressor = DecisionTreeRegressor(random_state = 0) regressor.fit(X_train, y_train) output: DecisionTreeRegressor(ccp_alpha=0.0, criterion='mse', max_depth=None, max_features=None, max_leaf_nodes=None, min_impurity_decrease=0.0, min_impurity_split=None, min_samples_leaf=1, min_samples_split=2, min_weight_fraction_leaf=0.0, presort='deprecated', random_state=0, splitter='best') # Predict the test results y_pred = regressor.predict(X_test) np.set_printoptions(precision=2) print(np.concatenate((y_pred.reshape(len(y_pred),1), y_test.reshape(len(y_test),1)),1)) output: [[0. 0.] [1. 1.] [1. 1.] [1. 1.] [1. 1.] [1. 1.] [0. 1.] [1. 1.] [1. 1.] [1. 1.] [1. 1.] [1. 1.] [1. 1.] [0. 1.] [1. 1.] [1. 0.] [0. 1.] [0. 0.] [0. 0.] [0. 0.] [1. 0.] [0. 0.] [1. 1.] [1. 1.] [0. 0.] [1. 1.] [1. 1.] [0. 0.] [1. 1.] [0. 0.] [1. 1.] [0. 0.] [0. 1.] [0. 0.] [1. 1.] [0. 0.] [1. 1.] [0. 0.] [1. 1.] [0. 0.] [1. 0.] [1. 1.] [0. 0.] [1. 1.] [0. 1.] [0. 0.] [1. 1.] [1. 1.] [1. 1.] [0. 0.] [0. 0.] [0. 0.] [0. 0.] [1. 1.] [1. 1.] [1. 1.] [1. 1.] [1. 1.] [1. 1.] [0. 0.] [0. 0.] [0. 0.] [1. 1.] [1. 1.] [0. 0.] [1. 1.] [0. 0.] [0. 0.] [0. 0.] [1. 1.] [0. 1.] [0. 0.] [1. 1.] [1. 0.] [0. 0.] [1. 1.] [1. 1.] [1. 1.] [1. 1.] [1. 1.] [0. 0.] [0. 0.] [0. 0.] [1. 1.] [1. 0.] [1. 1.] [1. 1.] [1. 1.] [0. 0.] [0. 0.] [1. 1.] [0. 0.] [1. 1.] [0. 0.] [1. 1.] [1. 1.] [0. 0.] [1. 1.] [1. 1.] [1. 1.] [1. 1.] [1. 1.] [1. 1.] [0. 1.] [0. 0.] [1. 1.] [0. 0.] [0. 1.] [0. 0.] [0. 0.] [1. 1.] [0. 0.] [0. 0.] [0. 1.]] #Model performance evaluation from sklearn.metrics import r2_score r2_score(y_test, y_pred) output: 0.49317243569387104 Support Vector Regression (SVR) Repeat step 1 above #Import the dataset dataset = pd.read_csv('Breast_cancer_data.csv') X = dataset.iloc[:, :-1].values y = dataset.iloc[:, -1].values y = y.reshape(len(y),1) # Split data into train and test sets from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0) #Feature scaling from sklearn.preprocessing import StandardScaler sc_X = StandardScaler() sc_y = StandardScaler() X_train = sc_X.fit_transform(X_train) y_train = sc_y.fit_transform(y_train) #Train the SVR model on the training set from sklearn.svm import SVR regressor = SVR(kernel = 'rbf') regressor.fit(X_train, y_train) output: SVR(C=1.0, cache_size=200, coef0=0.0, degree=3, epsilon=0.1, gamma='scale', kernel='rbf', max_iter=-1, shrinking=True, tol=0.001, verbose=False) #Predicting the test results y_pred = sc_y.inverse_transform(regressor.predict(sc_X.transform(X_test))) np.set_printoptions(precision=2) print(np.concatenate((y_pred.reshape(len(y_pred),1), y_test.reshape(len(y_test),1)),1)) output: [[ 4.09e-01 0.00e+00] [ 8.40e-01 1.00e+00] [ 1.04e+00 1.00e+00] [ 9.56e-01 1.00e+00] [ 9.91e-01 1.00e+00] [ 1.02e+00 1.00e+00] [ 1.05e+00 1.00e+00] [ 1.05e+00 1.00e+00] [ 9.41e-01 1.00e+00] [ 1.00e+00 1.00e+00] [ 8.40e-01 1.00e+00] [ 4.55e-01 1.00e+00] [ 9.77e-01 1.00e+00] [ 3.91e-01 1.00e+00] [ 6.21e-01 1.00e+00] [ 5.37e-01 0.00e+00] [ 1.01e+00 1.00e+00] [ 8.56e-03 0.00e+00] [ 2.87e-02 0.00e+00] [ 3.42e-02 0.00e+00] [ 8.67e-01 0.00e+00] [ 6.42e-02 0.00e+00] [ 9.75e-01 1.00e+00] [ 1.05e+00 1.00e+00] [ 4.10e-03 0.00e+00] [ 9.70e-01 1.00e+00] [ 9.34e-01 1.00e+00] [ 1.22e-01 0.00e+00] [ 1.04e+00 1.00e+00] [-6.41e-02 0.00e+00] [ 1.03e+00 1.00e+00] [ 8.64e-04 0.00e+00] [ 9.20e-01 1.00e+00] [-3.60e-02 0.00e+00] [ 9.54e-01 1.00e+00] [ 3.89e-01 0.00e+00] [ 1.04e+00 1.00e+00] [-2.01e-03 0.00e+00] [ 8.25e-01 1.00e+00] [ 4.86e-02 0.00e+00] [ 6.78e-01 0.00e+00] [ 9.44e-01 1.00e+00] [ 3.30e-01 0.00e+00] [ 1.00e+00 1.00e+00] [ 9.74e-01 1.00e+00] [-4.28e-03 0.00e+00] [ 9.41e-01 1.00e+00] [ 6.77e-01 1.00e+00] [ 1.03e+00 1.00e+00] [ 4.82e-02 0.00e+00] [ 1.81e-02 0.00e+00] [ 2.24e-02 0.00e+00] [-6.66e-02 0.00e+00] [ 1.04e+00 1.00e+00] [ 1.03e+00 1.00e+00] [ 1.03e+00 1.00e+00] [ 1.03e+00 1.00e+00] [ 1.08e+00 1.00e+00] [ 1.05e+00 1.00e+00] [ 1.58e-02 0.00e+00] [ 7.41e-02 0.00e+00] [ 2.42e-01 0.00e+00] [ 9.40e-01 1.00e+00] [ 1.03e+00 1.00e+00] [-7.70e-02 0.00e+00] [ 1.01e+00 1.00e+00] [-1.71e-02 0.00e+00] [ 2.28e-01 0.00e+00] [ 1.65e-03 0.00e+00] [ 1.04e+00 1.00e+00] [ 7.87e-01 1.00e+00] [-4.95e-02 0.00e+00] [ 1.05e+00 1.00e+00] [ 5.54e-01 0.00e+00] [-1.71e-02 0.00e+00] [ 9.57e-01 1.00e+00] [ 9.56e-01 1.00e+00] [ 7.25e-01 1.00e+00] [ 1.00e+00 1.00e+00] [ 9.87e-01 1.00e+00] [ 2.62e-01 0.00e+00] [ 6.49e-02 0.00e+00] [-8.02e-02 0.00e+00] [ 1.03e+00 1.00e+00] [ 5.86e-01 0.00e+00] [ 1.05e+00 1.00e+00] [ 9.93e-01 1.00e+00] [ 9.28e-01 1.00e+00] [-8.39e-02 0.00e+00] [ 3.60e-02 0.00e+00] [ 9.70e-01 1.00e+00] [ 7.68e-01 0.00e+00] [ 5.21e-01 1.00e+00] [ 1.66e-01 0.00e+00] [ 1.04e+00 1.00e+00] [ 1.05e+00 1.00e+00] [-4.20e-02 0.00e+00] [ 1.05e+00 1.00e+00] [ 1.05e+00 1.00e+00] [ 1.01e+00 1.00e+00] [ 9.54e-01 1.00e+00] [ 1.05e+00 1.00e+00] [ 6.85e-01 1.00e+00] [ 3.63e-01 1.00e+00] [-1.97e-02 0.00e+00] [ 5.18e-01 1.00e+00] [ 7.57e-02 0.00e+00] [ 5.03e-01 1.00e+00] [ 6.45e-01 0.00e+00] [-5.32e-02 0.00e+00] [ 8.48e-01 1.00e+00] [ 1.84e-02 0.00e+00] [-1.92e-02 0.00e+00] [ 6.24e-01 1.00e+00]] #Model performance evaluation from sklearn.metrics import r2_score r2_score(y_test, y_pred) output: 0.7661368518523295 From the model performance evaluations of all the models, the best model is the SVR model with an R2 of 0.766