Skip to main content

Logistic Regression with scikit-learn

This binary example uses scikit-learn's bundled breast-cancer dataset. It is a teaching dataset, not a clinical decision system. We stratify the split and keep standardization inside a pipeline so the scaler sees training data only.

from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import (
    average_precision_score,
    classification_report,
    confusion_matrix,
    roc_auc_score,
)
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

X, y = load_breast_cancer(return_X_y=True, as_frame=True)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, stratify=y, random_state=42
)

model = make_pipeline(
    StandardScaler(),
    LogisticRegression(max_iter=1_000),
)
model.fit(X_train, y_train)
y_score = model.predict_proba(X_test)[:, 1]
y_pred = (y_score >= 0.5).astype(int)

print(confusion_matrix(y_test, y_pred))
print(classification_report(y_test, y_pred, digits=3))
print(f"ROC AUC: {roc_auc_score(y_test, y_score):.3f}")
print(f"Average precision: {average_precision_score(y_test, y_score):.3f}")

Read scores and decisions separately​

predict_proba returns class probabilities in the order given by classes_; the second column above corresponds to class 1. ROC AUC and average precision evaluate ranking across thresholds. The 0.5 cutoff is only a default decision rule. Choose a deployment threshold on validation data using the relative costs of false positives and false negatives, then report performance once on the untouched test set.

Accuracy can hide minority-class failures. Report the confusion matrix plus class-specific precision and recall. If probabilities drive decisions, also check calibration; discrimination and calibration are different properties.

Interpret the model​

The log-odds are linear in the transformed features. A coefficient is a change in log-odds for a one-unit increase of its input while other included inputs are held fixed. With standardized features, one unit means one training-set standard deviation. Coefficients describe conditional association, not causation, and correlation among features can make them unstable.

For multiclass data, scikit-learn's non-liblinear solvers optimize a multinomial loss. Solver and penalty compatibility is version-sensitive, so consult the API when switching to L1 or elastic-net regularization.