Skip to main content

Naive Bayes and PyTorch

PyTorch does not ship a classical Naive Bayes estimator. Naive Bayes training estimates class priors and class-conditional distribution parameters; it is not normally an autograd optimization problem.

Why the custom tensor classifier was removed​

The earlier example implemented a partial count model with dense tensors and labeled it as a PyTorch Naive Bayes classifier. That approach obscured several important requirements:

  • multinomial feature likelihoods use smoothed event counts and require non-negative inputs;
  • Bernoulli likelihoods include both feature presence and absence;
  • Gaussian likelihoods estimate a mean and variance for each feature within each class;
  • implementations work in log space to avoid numerical underflow;
  • sparse document matrices should not be densified casually;
  • priors, smoothing, unseen categories, sample weights, and incremental fitting need explicit semantics.

A few tensor operations do not supply those estimator guarantees. Use scikit-learn's tested variant that matches the data-generating assumptions.

Where PyTorch does fit​

PyTorch may learn an upstream neural representation or an end-to-end classifier. If a neural encoder supplies features to Naive Bayes, freeze the representation and fit every preprocessing step and likelihood parameter on training data only. Be cautious: dense learned embeddings are often correlated and signed, so MultinomialNB is usually not an appropriate likelihood.

If the goal is a differentiable probabilistic model, define that model and its likelihood explicitly rather than calling it classical Naive Bayes. A different objective and dependency structure imply a different model.

Follow the scikit-learn example for a count-vectorized text baseline. Evaluate discrimination and calibration separately, preserve sparse inputs, and compare against a regularized linear baseline under identical splits.