On the Hardness of Approximating Distributions with Probabilistic Circuits

Abstract

A fundamental challenge in probabilistic modeling is balancing expressivity and tractable inference. Probabilistic circuits (PCs) aim to directly address this tradeoff by imposing structural constraints that guarantee efficient inference of certain queries while maintaining expressivity. Since inference complexity on PCs depends on circuit size, understanding the size bounds across circuit families is key to characterizing the tradeoff between tractability and expressive efficiency. However, expressive efficiency is often studied through exact representations, where exactly encoding distributions while enforcing various structural properties often incurs exponential size blow-ups. Thus, we pose the following question: can we avoid such size blow-ups by allowing some small approximation error? We first show that approximating an arbitrary distribution with bounded f-divergence is NP-hard for any model that can tractably compute marginals. We then prove an exponential size gap for approximation between the class of decomposable PCs and additionally deterministic PCs.

Citation

@inproceedings{LelandTPM25,
  author    = {Leland, John and Choi, YooJung},
  title     = {On the Hardness of Approximating Distributions with Probabilistic Circuits},
  booktitle = {The 8th Workshop on Tractable Probabilistic Modeling (TPM)},
  month     = {July},
  year      = {2025},
}