Linear Models - Scikit Learn

By Salerno | March 14, 2020

1. Linear Models

The target value is expected to be a linear combination of the features.

1.1. Ordinary Least Squares (OLS)

The OLS is a optimization math technique that aim to find the better adjustment for a set data and try to minimize the residual sum of squares between the observed targets in the dataset and the targets predicted by the linear approximation.

``````
from sklearn import linear_model

reg = linear_model.LinearRegression()
reg.fit([[0, 0], [1, 1], [2, 2]], [0, 1, 2])``````
``## LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None, normalize=False)``
``````reg.coef_
``````
``## array([0.5, 0.5])``

2. Linear Regression Example - Diabetes Dataset

``````
# Code source: Jaques Grobler

import matplotlib.pyplot as plt
import numpy as np
from sklearn import datasets, linear_model
from sklearn.metrics import mean_squared_error, r2_score

# Use only one feature
diabetes_X = diabetes_X[:, np.newaxis, 2]

# Split the data into training/testing sets
diabetes_X_train = diabetes_X[:-20]
diabetes_X_test = diabetes_X[-20:]

# Split the targets into training/testing sets
diabetes_y_train = diabetes_y[:-20]
diabetes_y_test = diabetes_y[-20:]

# Create linear regression object
regr = linear_model.LinearRegression()

# Train the model using the training sets
regr.fit(diabetes_X_train, diabetes_y_train)

# Make predictions using the testing set``````
``## LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None, normalize=False)``
``````diabetes_y_pred = regr.predict(diabetes_X_test)

# The coefficients
print('Coefficients: \n', regr.coef_)
# The mean squared error``````
``````## Coefficients:
##  [938.23786125]``````
``````print('Mean squared error: %.2f'
% mean_squared_error(diabetes_y_test, diabetes_y_pred))
# The coefficient of determination: 1 is perfect prediction``````
``## Mean squared error: 2548.07``
``````print('Coefficient of determination: %.2f'
% r2_score(diabetes_y_test, diabetes_y_pred))

# Plot outputs``````
``## Coefficient of determination: 0.47``
``````plt.scatter(diabetes_X_test, diabetes_y_test,  color='black')
plt.plot(diabetes_X_test, diabetes_y_pred, color='blue', linewidth=3)

plt.xticks(())``````
``## ([], <a list of 0 Text xticklabel objects>)``
``plt.yticks(())``
``## ([], <a list of 0 Text yticklabel objects>)``
``plt.show()``