Random Projection Neural Networks of Best Approximation: Convergence Theory and Practical Applications

Source abstract

Abstract. We investigate the concept of best approximation for feedforward neural networks (FNNs) and explore their convergence properties through the lens of random projection neural networks (RPNNs). RPNNs have predetermined and permanently fixed internal weights and biases, offering computational efficiency. We demonstrate that there exists a choice of external weights, for any family of such RPNNs, with nonpolynomial infinitely differentiable activation functions, that exhibit an exponential convergence rate when approximating any infinitely differentiable function. For illustration purposes, we test the proposed RPNN-based function approximation, with parsimoniously chosen basis functions, across five benchmark function approximation problems. Results show that RPNNs achieve performance comparable to established methods, such as Legendre polynomials, highlighting their potential for efficient and accurate function approximation.