Development of Artificial Neural Networks with Adaptive and Trainable Activation Functions Applied to Continuous Time Series Analisis
Stochastic differential equations, Neural networks, Adaptive and trainable activation functions, NeuroDiffEq.
The use of artificial neural networks (ANNs) to solve complex mathematical problems, such as ordinary differential equations (ODEs) and partial differential equations (PDEs), has gained prominence in recent years. In particular, ANNs with adaptive and trainable activation functions have shown promise by automatically adjusting their properties during training, improving both convergence and accuracy. However, the effectiveness of these networks in stochastic differential equations (SDEs) is not yet fully understood. This research investigates the performance of adaptive activation functions, focusing on the Universal Activation Function (UAF), and compares it with GLN-Mish and GLN-ReLU. To this end, four important SDEs in physics and finance were solved, including the Langevin equation and the Cox-Ingersoll-Ross equation, as well as the Brownian motion equation and an exponential equation, using tools from the NeuroDiffEq library in Python. For the Langevin equation, for example, the ANN was able to converge to the analytical solution with few code adjustments, i.e., few training epochs, while for the Cox-Ingersoll-Ross equation, it initially showed some convergence difficulties. However, the neural network was still able to solve the equation, even with the interference of the stochastic term. The results show that, although the activation functions have initial convergence difficulties, they eventually approximate the analytical solution efficiently. Additionally, the analysis of Mean Squared Error (MSE) values and the behavior of the confidence interval reinforces the effectiveness of ANNs with adaptive and trainable activation functions. The ability of these activation functions to dynamically adjust to the specific demands of complex problems offers new perspectives for future research, highlighting their potential not only in solving differential equations but also in a wide range of scientific and financial applications.