Learning under constraints: inference, selection and validation with maximum entropy models

Abstract

Maximum entropy models provide a principled framework for statistical inference in complex systems by constraining the information known about a system while leaving all other aspects maximally unconstrained. This thesis investigates the use of maximum entropy models in model selection, hypothesis testing, and statistical validation, with particular emphasis on the distinction between their canonical and microcanonical formulations. Although these formulations are often equivalent in classical settings, this equivalence can break down when the number of constraints grows with system size, as is common in complex systems. The thesis develops statistical methods to address these challenges from complementary perspectives. It compares canonical and microcanonical models using the Minimum Description Length framework and develops e-value-based approaches for hypothesis testing between maximum entropy models. E-values are a recent and promising alternative to traditional p-values, particularly suited to settings where evidence needs to be accumulated over time or across experiments. The thesis further establishes a principled connection between e-values and MDL, showing how the distributions used to compute description lengths can naturally yield e-values with desirable statistical properties. These methods are also investigated in the context of contingency tables and sequential inference. Finally, maximum entropy models are employed as null models to statistically validate triadic interactions in resting-state fMRI data, introducing a new non-linear model for assessing such interactions between brain regions. Together, these contributions provide methodological tools for statistical inference with maximum entropy models and highlight the importance of the choice of reference model in shaping statistical results.