Logistic Regression with PyTorch

In PyTorch, binary logistic regression is one linear layer trained with log loss. Keep the model output as an unrestricted logit and use BCEWithLogitsLoss, which combines sigmoid and binary cross-entropy more stably than applying sigmoid before BCELoss.

import numpy as np
import torch
from sklearn.datasets import make_classification
from sklearn.metrics import accuracy_score, roc_auc_score
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

torch.manual_seed(42)
torch.set_num_threads(1)

X, y = make_classification(
    n_samples=320,
    n_features=6,
    n_informative=4,
    n_redundant=0,
    class_sep=1.4,
    random_state=42,
)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, stratify=y, random_state=42
)
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

X_train_t = torch.tensor(X_train, dtype=torch.float32)
y_train_t = torch.tensor(y_train[:, None], dtype=torch.float32)
X_test_t = torch.tensor(X_test, dtype=torch.float32)

model = torch.nn.Linear(X_train.shape[1], 1)
loss_fn = torch.nn.BCEWithLogitsLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.05)

model.train()
for _ in range(200):
    optimizer.zero_grad()
    loss = loss_fn(model(X_train_t), y_train_t)
    loss.backward()
    optimizer.step()

model.eval()
with torch.no_grad():
    logits = model(X_test_t).squeeze(1)
    probabilities = torch.sigmoid(logits).numpy()

predictions = (probabilities >= 0.5).astype(int)
assert np.isfinite(probabilities).all()
print(f"Accuracy: {accuracy_score(y_test, predictions):.3f}")
print(f"ROC AUC: {roc_auc_score(y_test, probabilities):.3f}")

The loss consumes logits, while sigmoid is applied only for probability-like outputs at evaluation time. A threshold converts scores into actions; select it on validation data, not on the test set. Scaling is learned from the training partition and must be persisted with the model.

PyTorch provides optimization primitives, not the solver/penalty matrix, cross-validation utilities, or coefficient-inference tools of a classical linear-model library. Use it when this linear classifier belongs inside a larger neural architecture or custom training objective.