A Self-Adaptive Penalty Method for Integrating Prior Knowledge Constraints into Neural ODEs

Source abstract

Abstract The continuous dynamics of natural systems has been effectively modelled using Neural Ordinary Differential Equations (Neural ODEs). However, for accurate and meaningful predictions, it is crucial that the models follow the underlying rules or laws that govern these systems. In this work, we propose a self-adaptive penalty algorithm for Neural ODEs to enable modelling of constrained natural systems. The proposed self-adaptive penalty function dynamically adjusts the penalty parameters during training, allowing for a more flexible and efficient optimization process. Furthermore, by explicitly incorporating prior knowledge through constraints, the method enhances the interpretability of Neural ODE-based models, making them better suited for applications involving physical, biological, or other natural processes. The proposed approach is validated through the modelling of three natural systems: population growth, chemical reaction dynamics, and the motion of a damped harmonic oscillator. The numerical experiments and a comparison with other penalty Neural ODE approaches and vanilla Neural ODE, demonstrate the effectiveness of the proposed self-adaptive penalty algorithm for Neural ODEs in modelling constrained natural systems. Moreover, the self-adaptive penalty approach provides more accurate and robust models with reliable and meaningful predictions. The code to replicate the experiments presented in this work is available at https://github.com/CeciliaCoelho/PriorKnowledgeNeuralODE .