We live in an age of information, but, when confronted with the question of what information is, it becomes hard to define it in a way that differentiates it from data. There is a subjective quality to how epistemic agents, thinking beings, interact with data and judge its ability to be informative. This subject-object relationship between an epistemic agent and the ontological substrate they think about and interact with is at the heart of model building and the endeavor which we call physics. Intimately interwoven with one another, epistemology, Bayesian inference, and information theory shed light on the efficacy of Statistical Mechanics and the role of information in physical systems. This thesis explores the theoretical foundations and phenomenology of the information-entropic measure known as Configurational Entropy (CE). Though a full theory is still under development, CE has so far revealed a rich informational narrative describing phenomena involving both scale and instability. Applications involving both phase transitions, and solitonic and stellar configurations in field theories are described. Furthermore, the epistemological foundation upon which the scaffolding of this measure is erected is discussed, unveiling the general framework necessary for both specialization and generalization of its use.