Application of transformer-type neural networks in tasks of estimating a set of technological parameters based on video recording data

Source abstract

The effectiveness of using a transformer-based neural network model is investigated for estimating technological parameters of the wood material transportation process based on video recordings of the end surface of transported stacks. This task is formulated as a regression problem. A model variant based on the transformer neural network architecture is proposed, exhibiting enhanced stability and efficiency. In particular, the combination of the GELU (Gaussian Error Linear Unit) activation function and additional fully connected layers improved the smoothness and width of the loss function landscape, thereby increasing the model’s generalization capability and stabilizing the training process. A comparison of loss function landscapes and an analysis of their impact on training efficiency were conducted for the proposed and alternative transformer architectures using a given set of photographic data. The quality metric R2 of the proposed solution ranged between 0.9 and 0.95. A comparison of the effectiveness of the proposed transformer architecture with alternative methods for solving the problem was also performed. The developed method is applied in real technological processes for controlling the transportation of wood materials.