75 mins lesson duration•10 mins read
4: Predictive Modeling with Scikit-Learn
Features vs Labels, train-test splitting, fitting Linear Regression, and checking MSE / R2.
The Machine Learning Paradigm
Traditional programming writes instructions to turn input data into outcomes. Machine learning feeds input data and outcomes into a learning algorithm to extract the rules/model.
Core Math Concepts
- Loss Function: Measures how far off our predictions are from the actual values. In Linear Regression, we minimize Mean Squared Error (MSE):
MSE = \frac{1}{N} \sum_{i=1}^{N} (y_i - \hat{y}_i)^2
2. **Gradient Descent**: Optimizes the weights to reduce the loss function iteratively.
#### The ML Workflow
- **Train-Test Split**: Divide data (typically 80/20) to ensure the model is evaluated on unseen data, preventing overfitting.
- **Feature Scaling**: Adjust numerical ranges so features contribute equally.
- **Validation**: Calculate metrics like $R^2$ (coefficient of determination) to evaluate performance.
Interactive Lesson Code Snippet
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score
from sklearn.model_selection import train_test_split
import numpy as np
# Generate synthetic linear data (y = 3x + 5 + noise)
np.random.seed(42)
X = 2 * np.random.rand(100, 1)
y = 5 + 3 * X + np.random.randn(100, 1)
# Split data into train/test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Train the model
model = LinearRegression()
model.fit(X_train, y_train)
# Predict on test data
y_pred = model.predict(X_test)
# Calculate Evaluation Metrics
mse = mean_squared_error(y_test, y_pred)
r2 = r2_score(y_test, y_pred)
print("Trained Model weights:")
print(f"Slope (w1): {model.coef_[0][0]:.4f} (Ideal: 3.00)")
print(f"Intercept (b): {model.intercept_[0]:.4f} (Ideal: 5.00)")
print("\nModel Evaluation Metrics on Test Set:")
print(f"Mean Squared Error (MSE): {mse:.4f}")
print(f"R-squared Score (R2): {r2:.4f}")Language: python
Lesson Code (Python)
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score
from sklearn.model_selection import train_test_split
import numpy as np
# Generate synthetic linear data (y = 3x + 5 + noise)
np.random.seed(42)
X = 2 * np.random.rand(100, 1)
y = 5 + 3 * X + np.random.randn(100, 1)
# Split data into train/test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Train the model
model = LinearRegression()
model.fit(X_train, y_train)
# Predict on test data
y_pred = model.predict(X_test)
# Calculate Evaluation Metrics
mse = mean_squared_error(y_test, y_pred)
r2 = r2_score(y_test, y_pred)
print("Trained Model weights:")
print(f"Slope (w1): {model.coef_[0][0]:.4f} (Ideal: 3.00)")
print(f"Intercept (b): {model.intercept_[0]:.4f} (Ideal: 5.00)")
print("\nModel Evaluation Metrics on Test Set:")
print(f"Mean Squared Error (MSE): {mse:.4f}")
print(f"R-squared Score (R2): {r2:.4f}")Console Output
Trained Model weights:
Slope (w1): 2.8228 (Ideal: 3.00)
Intercept (b): 5.2151 (Ideal: 5.00)
Model Evaluation Metrics on Test Set:
Mean Squared Error (MSE): 0.6537
R-squared Score (R2): 0.8072Test Your Knowledge
Instant feedbackQuick Check: Supervised Learning
Why do we split data into train and test sets?
Up next · Continue learning
Developing Intelligent Agentic Data Pipelines
Tool-calling patterns, ReAct architecture, safety sandboxing, and orchestrating analytical agents.
12 mins read90 mins
Start next lesson