Speaker
Description
With the capability of modern computers for processing large amounts of data, machine learning (ML) is being more and more applied in computational science. Many of the ML learning methods aim to accurately predict the whole solution vector of dynamical systems via data compression, such as convolutional autoencoder. These works focus on predicting only the quantities of interests (QoIs) or outputs using ML without data compression.
Transformer models have been proposed to overcome the difficulties of recurrent neural networks, such as LSTM, for long-term prediction. Many transformer models have been suggested for time series forecast. Most of the transformer models are applied to predict daily life activities, such as customer electricity usage, traffic road occupancy rate, etc. In this work, we explore the promising performance of a transformer model: temporal fusion transformer (TFT) on predicting outputs of parametric dynamical systems. The outputs of such systems vary not only with physical parameters but also with external time-varying input signals. Accurately catching the dynamics of such systems is challenging. We have adapted and extended the original TFT for single output prediction to a multiple-output TFT that is able to predict multiple outputs. The multiple-output transformer generalizes the interpretability of the original TFT model. The generalized interpretable attention weight matrix explores not only the temporal correlations in the sequence, but also the interactions between the multiple outputs.
