Online Structure Learning of Probabilistic Circuits

Abstract

Probabilistic circuits (PCs) are a class of probabilistic models that guarantee exact and efficient inference of various probabilistic queries by enforcing certain structural properties. Despite recent advances in learning PCs, these methods are mostly limited to offline batch learning. In particular, while tractable inference of many queries requires determinism, to the best of our knowledge, there are no online structure learners for deterministic PCs. To bridge this gap, we introduce an online incremental structure learning framework for deterministic and structured decomposable PCs. We use distribution-preserving splits to dynamically increase the expressivity of the PC structure and merge, when necessary, to manage size blowup, both based on tractable heuristics and closed-form parameter updates using the circuit properties. We pair this with a tunable decay factor to systematically adapt to concept drift. Empirical evaluations on density estimation benchmark datasets show that our method achieves stable training through non-static data streams and offers robust adaptation via parameter decay.