Tensor time delay embedding extension for multivariate time series analysis

DenisTikhonov1✉Emailtihonov.denis.m@gmail.com

VadimStrijov1Emailstrijov@forecsys.ru

Forecsys LLC

Abstract

This paper aims to construct a new dimensionality reduction method that uses time series analysis approaches, multilinear algebra, and dynamical system reconstruction theory.The proposed multilinear method combines time delay embedding and tensor as a multilinear map to a low-dimensional space.It prevents the loss of nonlinear higher-order information between various time series and allows the selection of time series components that are recognized as noise in a single case.The results show that the method allows for a better reconstruction of the original attractor from an incomplete set of variables. A computational experiment was carried out on the Lorenz attractor, and the accelerometer of a mobile device was measured using two classes of human movements.The accuracy of the reconstructed attractor is tested to determine the ability to forecast an unused time series from the dynamic system under study.

Keywords

time series

dynamical system

multilinear algebra

dimensionality reduction

Denis Tikhonov and Vadim Strijov: These authors contributed equally to this work.

Introduction

The theory of dynamical systems is a mathematical discipline that has close intersections with various research areas such as mechanics, chaos theory, and time series analysis.It investigates physical systems from a mathematical point of view and tries to build some abstract structures for studying and predicting future states Arnold1998dynmical, Benner2015Survey,Broomhead1989Time and for more specific purposes with continuum dynamics like fluid mechanics\cite{Brunton2020Machine}, human activity recognition motrenko2015extracting,ignatov2016human,grabovoy2020quasi, economical modeling \cite{gandolfo1971economic} and others.

Traditional methods for system reconstruction are based on physical principles, conservation laws, and empirical modeling Kevrekidis2003Equation,Sugihara2012Detecting,Ye2015Equation.An alternative method is to reconstruct equations and dynamical systems based solely on a fixed number of time series data obtained from an experiment with the usage of previous values in time.This approach is called time delay embedding (TDE) and first describe in Packard1980.It allows to move from a scalar value of a time series at a point in time to a vector representation which is called phase space.With theoretical results from Takens's theorem \cite{Takens1981Dynamical} time delay embedding fully reconstructs an unknown dynamical system or makes a diffeomorphisms to the original system without matching geometric shape.

The delay embedding method itself has many options such as uniform delay embedding, method characteristic lengths \cite{Cellucci2003Comparative}, autocorrelation and minimum mutual information \cite{Bradley2015Chaos}, empirical non-uniform methods, reduced autoregressive models JUDD1998273 and topological methods \cite{Tan_2023}.A combination of TDE vectorization and machine learning techniques results in various approaches for the discovery of dynamical systems \cite{Crutchfield1987EquationsOM}.Its include nonlinear regression \cite{Voss1999Amplitude}, artificial neural networks GONZALEZGARCIA1998S965, normal form identification \cite{Majda2009normal}, nonlinear spectral analysis \cite{Giannakis2012Nonlinear}, modeling emergent behavior \cite{Roberts2013Model} and automated refinement and inference of dynamics Schmidt2011Automated, Daniels2015Automated.

Previously mentioned methods have two disadvantages: the first one is a large dimension size of the initial phase space, the second one is the loss of cross information in the multidimensional time series case.First, as the dimension of the phase space increases, the distances between the points of the trajectory tend to the constant value.That makes distances uninformative and unstable due to the curse of dimensionality \cite{PowellWarrenB2011Adps}.It assumes that a more stable and robust model is constructed in a low dimensional subspace.The most common method for dimensionality reduction is the principal component analysis (PCA) \cite{Broomhead1986Extracting}.Second, traditional methods and graph models cause a loss of higher-order or cross-component information due to the separate use of each multivariate time series \cite{Wolf2016Advantages}.The papers KRUPPA20175610, Chen_2019, chen2022stabilitymultilineardynamicalsystems study tensor forecasting models for time series with multilinear algebra approaches.The methods have extensions to multivariate time series closely related to autoencoders \cite{Lusch_2018}.

The key idea is to combine the classical dimensionality reduction technique with the tensor approach.On the one hand, the tensor as a multilinear map reduces the dimension of initial phase spaces.On the other hand, it combines several time series in a simple form.This map prevents the loss of higher-order information between several time series and selects time series components that are recognized as noise in one time series case.This method also preserves the diffeomorphism between dynamical systems.Thus, the proposed method is essentially a method for a feature engineering.

The key contributions of the paper are the application of the previously proposed dynamic system model and extension to the time delay embedding and multivariate times series case.The computational experiment explores walking and jogging.The experiment was performed on data obtained from a mobile device's accelerometer Malekzadeh2019.The proposed method is tested for the forecasting accuracy of an unused time series from the dynamic system under study.The main conclusions about the accuracy and validity of the approach are the same as the conclusions in chen2022stabilitymultilineardynamicalsystems, Chen2021Multilinear.

The paper is organized into three main sections. In section 2, multilinear dynamical systems with time delay embedding are introduced.The multilinear map method can effectively reconstruct an attractor of the dynamical system.In section 4, a tensor preliminaries review includes notations and various tensor products.In section 8 experiment results with numerical examples are presented.Section \ref{сonclusion} draws some conclusions and plans for future work.

Multilinear dynamical system

Time delay embedding

Time delay embedding augments the scalar time series

$s_t$

into a higher dimension through the construction of delay vector

$\mathbf{x}_t$

given as

$\mathbf{x}_t = [x_t, x_{t-\tau}, ..., x_{t-(n-1)\tau}]$

The embedding parameters

$\tau$

is delay lag and

$n$

is embedding dimension.According to Taken's theorem, only one variable with time delays reconstructs a dynamical system.A periodical times series and reconstructed attractor is shown in Figure (1).This augmentation with previous measures is called the trajectory matrix.Resulting trajectory matrix $\mathbf{S}$ of a time series $\mathbf{s}$ is defined as

$\mathbf{X} = \begin{bmatrix} x_1 & \dots & x_n \\ x_2 & \dots & x_{n+1} \\ \vdots &\ddots& \vdots \\ x_{k} & \dots & x_{N} \end{bmatrix}^{\mathsf{T}} = \begin{bmatrix} \mathbf{x}_1, \mathbf{x}_2, \dots, \mathbf{x}_k \end{bmatrix}, \quad k = N - n + 1,$

_time_delay

where

$n$

is the width of the window,

$N$

is the lengths of the time series and

$\tau$

is equal to 1.

The original phase space from time delay embedding in Eq.(\ref{eq_time_delay}) has a high dimension.Thus, principal component analysis (PCA) is often used to reduce the dimensionality of the original phase space, by transforming an initial set of variables into a smaller one that is called a subspace.

$\mathbf{S} = \mathbf{W}^{\mathsf{T}} \mathbf{X} = \begin{bmatrix} \mathbf{s}_1, \mathbf{s}_2, \dots, \mathbf{s}_k \end{bmatrix}, \quad \mathbf{s}_i\in \mathbb{R}^p, \,$

_basic_linear_map

where

$\mathbf{W}$

is the transformation matrix of the PCA algorithm.The number of selected components is

$p$

, corresponding to the largest eigenvalues.

Fig. 1

Segment of time series (left) and phase trajectory with PCA in 3d (right) with methods shown in Eq. (\ref{eq_basic_linear_map}).

Low-dimensional representation in phase space allows to use of more robust and simpler models and applications.

Tensor representation of time series

Tensors are multidimensional generalizations of matrices.The number of dimensions is the order of a tensor. Each dimension is called a mode.A vector

$\mathbf{v} \in \mathbb{R}^{n}$

has one mode, row, a matrix

$\mathbf{M} \in\mathbb{R}^{n\times n}$

has two modes, rows and columns, a

$N$

-th order tensor

$\mathbf{\mathsf{A}}\in \mathbb{R}^{n \times n \times \dots \times n}$

has $N$ modes.

The

$n$

-mode multiplication of tensor

$\mathbf{\mathsf{A}} \in \mathbb{R}^{I_1 \times \dots \times I_N}$

and matrix

$\mathbf{M} \in \mathbb{R}^{J \times I_n}$

is defined by

$\mathbf{\mathsf{C}} \in \mathbb{R}^{I_1 \times \dots \times I_{n-1} \times J \times I_{n+1} \times \dots \times I_N}$

with the elements:

$\mathbf{\mathsf{C}} = \mathbf{\mathsf{A}} \times_n^2 \mathbf{M} = \mathbf{\mathsf{A}} \times_n \mathbf{M}, \quad c_{i_1, ..., i_{n-1}, j, i_{n+1}, ..., i_N} = \sum_{i_n = 1}^{I_n} a_{i_1, ..., i_n, ..., i_N} m_{j,i_n}.$

_n_mode_matrix

In case of tensor

$\mathbf{\mathsf{A}} \in \mathbb{R}^{I_1 \times \dots \times I_N}$

and vector

$\mathbf{v}\in \mathbb{R}^{I_n}n$

-mode multiplication gives

$\mathbf{\mathsf{C}} \in \mathbb{R}^{I_1 \times \dots \times I_{n-1} \times I_{n+1} \times \dots \times I_N}$

$\mathbf{\mathsf{C}} = \mathbf{\mathsf{A}} \times_n^1 \mathbf{v} = \mathbf{\mathsf{A}} \times_n \mathbf{v}, \quad c_{i_1, ..., i_{n-1}, i_{n+1}, ..., i_N} = \sum_{i_n = 1}^{I_n} a_{i_1, ..., i_n, ..., i_N} v_{i_n}$

_n_mode_vector

Formally, mode products for a matrix Eq.(\ref{eq_n_mode_matrix}) and a vector Eq.(\ref{eq_n_mode_vector}) are the same operations, but in this paper, for simplicity, only one notation Eq.(\ref{eq_n_mode_matrix}) is used for both operations.It is implied that in the case of a matrix the second mode is used, in the case of a vector only one first mode of the vector is used.

Multilinear dynamical system for multivariate time series

This paper discusses the topic of a dynamic system, which is given by

$\mathbf{v}_{t+1} = \mathbf{\mathsf{A}} \times_1 \mathbf{v}_{t} \times_2 \mathbf{v}_{t} \times_3 ... \times_{k-1} \mathbf{v}_{t},$

_dynamic_law

where

$\mathbf{\mathsf{A}} \in \mathbb{R}^{n \times n \times...\times n}$

is a multilinear map, and

$\mathbf{v} \in \mathbb{R}^{n}$

is the state variable. It is assumed that the tensor

$\mathbf{\mathsf{A}}$

has multilinear properties in the sense of the definition of algebraic multilinearity.

The vectors of the state variables are the values of some measured quantities at time

$t$

.It is assumed that these quantities completely describe the state of the dynamic system.In the case of a mathematical pendulum, these quantities are velocity and acceleration.With certain restrictions, it is possible to completely reconstruct dynamically using only these variables.

This paper proposes to construct a map into a low-dimensional subspace, i.e. dimensionality reduction, instead of reconstructing the dynamics itself, as some evolution rule of a system.The evolution rule is a function that describes what future states follow from the current state of the dynamical system.

This map is used in further models for anomaly detection, classification, and signal phase extraction (in the case of periodic time series).Thus, Eq.(\ref{eq_dynamic_law}) is modified as

$\mathbf{s}_{t} = \mathbf{\mathsf{A}} \times_1 \mathbf{x}_{t} \times_2 \mathbf{x}_{t} \times_3 ... \times_{k-1} \mathbf{x}_{t},$

_dim_red_model

where

$\mathbf{x}_t = [x_{t},x_{t-1},...,x_{t-n}]$

is a vector from time series

$\mathbf{x}$

with

$n$

delays,

$\mathbf{S}_t \in \mathbb{R}^{p}$

is a vector with

$p\ll n$

that represent system in its phase space.

In the case of Eq.(\ref{eq_dim_red_model}) only univariate time series is used.It has an extension to the case of multivariate time series.

To simplify the theory and to clarify it connections with computational experiment, let multivariate time series have three types of measurements, that come from a triaxial accelerometer. Let

$\mathbf{x}, \mathbf{y}, \mathbf{z}$

be the time series of acceleration along x,y,z axes.A signal from each axis separately restores the attractor of the dynamic system according to Taken’s theorem using time delay embedding as Eq.(\ref{eq_time_delay}).There are maps between each variable

$\mathbf{X} = \mathbf{I}^{\mathsf{T}} \mathbf{X}, \quad \mathbf{X} = \mathbf{W}^{\mathsf{T}}_{\text{y}} \mathbf{Y}, \quad \mathbf{X} = \mathbf{W}^{\mathsf{T}}_{\text{z}} \mathbf{Z},$

_linear_maps

where

$\mathbf{X}, \mathbf{Y}, \mathbf{Z}$

are trajectory matrices in initial phase space,

$\mathbf{W}^{\mathsf{T}}_{\text{y}}, \mathbf{W}^{\mathsf{T}}_{\text{z}}$

are the transformation matrices,

$\mathbf{I}$

is an identity matrix.Thus, the multilinear model is modified as follows:

$\mathbf{s}_{t} = \mathbf{\mathsf{A}} \times_1 (\mathbf{I}^{\mathsf{T}}\mathbf{x}_{t}) \times_2 (\mathbf{W}^{\mathsf{T}}_{\text{y}} \mathbf{y}_{t}) \times_3 (\mathbf{W}^{\mathsf{T}}_{\text{z}} \mathbf{z}_{t}) = \hat{\mathbf{\mathsf{A}}} \times_1 \mathbf{x}_{t} \times_2 \mathbf{y}_{t} \times_3 \mathbf{z}_{t},$

_dim_red_model_2

where

$\hat{\mathbf{\mathsf{A}}} = \mathbf{\mathsf{A}} \times_1 \mathbf{I}^{\mathsf{T}} \times_2 \mathbf{W}^{\mathsf{T}}_{\text{y}} \times_3 \mathbf{W}^{\mathsf{T}}_{\text{z}}$

is modified dynamic tensor,

$\mathbf{x}_{t}, \mathbf{y}_{t}, \mathbf{z}_{t}$

are state variable vectors from each axis at time

$t$

.Thus, in a shorter form, the equation is transformed into Eq.(\ref{eq_dim_red_model_final})

$\mathbf{s}_{t} = \mathbf{\mathsf{\hat{A}}} \times_1 \mathbf{x}_{t} \times_2 \mathbf{y}_{t} \times_3 \mathbf{z}_{t}.$

_dim_red_model_final

The graphical representation of the Penrose notation of the proposed method is shown in Figure (2).

Fig. 2

Tensor dynamical system in graphical notation in Eq. (\ref{eq_dim_red_model_final}).

The tensor

$\hat{\mathbf{\mathsf{A}}}$

allows to select not only the main components, as in the case of PCA for univariate time series, but filters them according to multilinear dependencies with other time series.In this way, additional information from a set of time series allows us to select components that would not have been identified from the noise or would have been of lesser significance in the case of independent analysis.

Alternative view on tensor representation

The resulting mapping is alternatively represented in the classical linear algebra notations. Let the mapping function is

$f_{A}: \mathbf{X} \times \mathbf{Y} \times \mathbf{X} \longrightarrow \mathbf{S}$

where

$\mathbf{X}, \mathbf{Y}, \mathbf{Z} \in \mathbb{R}^{n}$

and

$\mathbf{S} \in \mathbb{R}^{p}$

are vector spaces,

$n$

is embedding dimension,

$p$

is dimension of resulting space.There are a basis

$\{\mathbf{e}_{k,1}, \dots ,\mathbf{e}_{k,n}\},$

for each

$\mathbf{X}, \mathbf{Y}, \mathbf{Z}$

and a basis

$\{\mathbf{b}_{i,1}, \dots ,\mathbf{b}_{i,p}\}$

for

$\mathbf{S}$

. Thus tensor

$\mathbf{\mathsf{A}}$

is a collection of scalars values as

$f_{A}(\mathbf{e}_{\text{x} j_1}, \mathbf{e}_{\text{y} j_2} ,\mathbf{e}_{\text{z} j_3}) = \mathbf{\mathsf{A}}_{1,j_1,j_2,j_3} \mathbf{b}_{1} + ... + \mathbf{\mathsf{A}}_{p,j_1,j_2,j_3} \mathbf{b}_{p},$

where

$1\leq j_k \leq n, 1\leq k \leq 3$

It determines the multilinear function

$f_{A}$

for

$\mathbf{x}_{t}, \mathbf{y}_{t}, \mathbf{z}_{t}$

$f_{A}(\mathbf{x}_{t}, \mathbf{y}_{t}, \mathbf{z}_{t}) = \sum_{j = 1}^{p} \sum_{i_1 = 1}^{n}\sum_{i_2 = 1}^{n}\sum_{i_3 = 1}^{n} \mathbf{\mathsf{A}}_{j,i_1, i_2, i_3} \cdot x_{i_1,t} y_{i_2,t} z_{i_3,t} \cdot \mathbf{b}_{p}.$

The proposed method is essentially a feature engineering technique.It combines linear dimensionality reduction methods and the tensor approach with nonlinear aggregation.The tensor itself contains the weights of the models mapping the original phase spaces from signal sources.

Dimension size problem

In TDE the reconstruction of dynamical systems is possible if the lag is taken at least equal to

$2q+1$

with

$q$

the dimension of the manifold on which the dynamical system is defined.It is not clear how dimension $q$ is estimated.

In the case of periodic or quasi-periodic time series with a non-chaotic structure, the system will return to the same state at certain moments.Thus, at time

$t, t+T, t+2T...$

, where

$T$

--- dominant period of the system, all points correspond to the same area in the phase space.If two points of the phase trajectory with significantly different times are in the same area of the phase space, then it is called an intersection.In other words, the nearest neighbors in phase space are the nearest neighbors in time.An example of intersection is shown in Figure (3).

Fig. 3

A phase trajectory of aggregated X and Y components with three intersections (red) in the space of insufficient dimension. Other visual intersections appear due to the viewing angle.

In this way, it is possible to select the minimum dimension of a dynamic system based on two criteria:

1)the appearance of self-intersections,

2)the slowdown of the growth of the target metric with an increase in the dimension of the space.

However, the problem of choosing the dimension of the phase space is beyond the scope of the current work.

Experimental results and discussion

The Lorenz system

This example uses the Lorenz attractor to analyze reconstructed phase spaces. A scheme of experiment is shown in Figure (4).

Fig. 4

Schematic of the embedding process and the relationship between its reconstruction.

The variables under study are defined by a system of differential equations

$\left\{\begin{aligned} \frac{d x}{d t} &= \sigma (y-x), \\ \frac{d y}{d t} &= x(r-z) + y,\\ \frac{d z}{d t} &= xy - bz\\ \end{aligned}\right.$

_siroriginal

with the following parameters

$\sigma=10$

$r=28$

$b=8/3$

The result phase trajectory has the form shown in Fig. 4.It shows the reconstruction scheme and various state spaces.For comparison, an attractor is shown, that is obtained by the time delay embedding method.

Fig. 5

Phase trajectory of Lorenz linear system reconstructed with PCA (left) and tensor dynamical system (TDS) approach (right).

As shown in Figure (5), additional information in the multilinear model reconstructs the shape of the phase trajectory similar to PCA.Both methods qualitatively restore the petals, maintaining repeating dynamics in two different modes of the original attractor.This result is obtained due to the noise-free time series and a sufficient length of history in each methods.

Human movement dataset

The purpose of the computational experiment is to analyze the quality of attractor reconstruction and compare it with the PCA as a basic linear approach for real data.

Fig. 6

Experiment scheme on real data.

The experiment is performed on data obtained from the accelerometer of a mobile device Malekzadeh2019.This dataset includes time-series data generated by accelerometer and gyroscope sensors.It is collected with an iPhone 6s kept in the participant's front pocket using SensingKit.All data is collected at the 50Hz sample rate.A total of 24 participants of various genders, ages, weights, and heights performed six activities in the same environment and conditions: downstairs, upstairs, walking, jogging, sitting, and standing.For this experiment only walking or jogging is chosen.

Fig. 7

Average coefficient of determination (R2) and root mean square deviation (rMSE) between predicted and true value of Z axis for walking over 24 participants.

begin{table}[!htbp]\centering\caption{Average coefficient of determination (R2) and root mean square deviation (rMSE) between predicted and true value of Z axis for \textbf{walking} over 24 participants}\label{tb_r2_mse_walking}\scriptsize\begin{tabular}{l|p{1cm}p{1cm}|p{1cm}p{1cm}|p{1cm}p{1cm}|p{1cm}p{1cm}}\toprule & \multicolumn{2}{p{2cm}}{Only X axis} & \multicolumn{2}{p{2cm}}{Only Y axis} & \multicolumn{2}{p{2cm}}{Tensor X,Y} & \multicolumn{2}{p{2cm}}{RSS of X,Y} \\ & R2 & rMSE & R2 & rMSE & R2 & rMSE & R2 & rMSE \\Dim & & & & & & & & \\\midrule3 & 0.23 & 0.91 & 0.34 & 0.88 & 0.17 & 0.90 & 0.30 & 0.90 \\7 & 0.40 & 0.79 & 0.46 & 0.80 & 0.34 & 0.83 & 0.44 & 0.85 \\15 & 0.59 & 0.69 & 0.60 & 0.68 & 0.60 & 0.67 & 0.57 & 0.73 \\20 & 0.63 & 0.66 & 0.65 & 0.67 & 0.68 & 0.61 & 0.62 & 0.69 \\25 & 0.66 & 0.60 & 0.68 & 0.66 & 0.73 &0.54 & 0.65 & 0.68 \\\bottomrule\end{tabular}\end{table}

The main idea is to restore the attractor of the system using four different methods and then to forecast a new unknown component with linear mapping; for this case, it is values of the Z axis of the accelerometer.

For a basic simple model, PCA with a single time series is chosen.This method is chosen for a correct comparison without taking into account the influence of the model architecture.In particular, a simple two layer autoencoder with a large number of parameters effectively restores the attractor.In our case, this is PCA with X and Y components separately,

$\mathbf{S}_{1} = \mathbf{W}_1^{\mathsf{T}} \mathbf{X}, \quad \mathbf{S}_{2} = \mathbf{W}_2^{\mathsf{T}} \mathbf{Y},$

where

$\mathbf{W}_1, \mathbf{W}_2$

are the transformation matrices of the PCA algorithm,

$\mathbf{S}_1, \mathbf{S}_2$

are resulting low-dimensional representation.

The alternative approach is to aggregate the initial X and Y time series as root sum squares (RSS) as

$\mathbf{X}_{\text{RSS}} = (\mathbf{X}^{\circ 2} + \mathbf{Y}^{\circ 2})^{\circ 1/2}, \quad \mathbf{S}_{3} = \mathbf{W}_2^{\mathsf{T}} \mathbf{X}_{\text{RSS}},$

where

$\mathbf{W}_3$

are the transformation matrices of the PCA algorithm,

$\mathbf{S}_3$

are resulting low-dimensional representation,

$\mathbf{S}^{\circ 2}$

is the element-wise power ( known as the Hadamard power).

For correct comparison, Z-component is excluded from the model.Thus, the tensor approach is modified as follows

$f_{A}: \mathbf{X} \times \mathbf{Y} \longrightarrow \mathbf{S},\quad \mathbf{s}_{t} = \hat{\mathbf{\mathsf{A}}} \times_1 \mathbf{x}_{t} \times_2 \mathbf{y}_{t},$

where only

$x$

and

$y$

components of the time series are used to restore the attractor of the system.

Fig. 8

Average coefficient of determination (R2) and root mean square deviation (rMSE) between predicted and true value of Z axis for jogging over 24 participants.

The inverse mapping of time series into an original space is made by using the multivariate regression model as

$\mathbf{\hat{Z}}_{i} = \mathbf{B}_i^{\mathsf{T}} \mathbf{S}_{i} , \quad \mathbf{B}_i = (\mathbf{S}_i^{\mathsf{T}} \mathbf{S}_i)^{-1}\mathbf{S}_i^{\mathsf{T}} \mathbf{Z}_{i}$

where

$i$

is model index,

$\mathbf{B}_i$

are coefficient matrices.

In this experiment, the results obtained with four approaches were compared in terms of forecasting accuracy.All four cases have an equal number of time points and window length (i.e.

$n = 70$

in Eq.(\ref{eq_time_delay})).The data is included from all 24 participants in jogging and walking.Time series without any activity type changes are selected, i.e. there are no stairs, climbs, or turns in the walking route.

begin{table}[!htbp]\centering\caption{Average coefficient of determination (R2) and root mean square deviation (rMSE) between predicted and true value of Z axis for \textbf{jogging} over 24 participants}\label{tb_r2_mse_jogging}\scriptsize\begin{tabular}{l|p{1cm}p{1cm}|p{1cm}p{1cm}|p{1cm}p{1cm}|p{1cm}p{1cm}}\toprule & \multicolumn{2}{p{2cm}}{Only X axis} & \multicolumn{2}{p{2cm}}{Only Y axis} & \multicolumn{2}{p{2cm}}{Tensor X,Y} & \multicolumn{2}{p{2cm}}{RSS of X,Y} \\ & R2 & rMSE & R2 & rMSE & R2 & rMSE & R2 & rMSE \\Dim & & & & & & & & \\\midrule3 & 0.16 & 0.92 & 0.17 & 0.92 & 0.17 & 0.92 & 0.17 & 0.91 \\7 & 0.28 & 0.90 & 0.27 & 0.90 & 0.28 & 0.85 & 0.29 & 0.89 \\15 & 0.40 & 0.82 & 0.41 & 0.85 & 0.46 & 0.82 & 0.41 & 0.85 \\20 & 0.44 & 0.81 & 0.45 & 0.82 & 0.51 & 0.78 & 0.45 & 0.83 \\25 & 0.46 & 0.81 & 0.47 & 0.81 & 0.55 & 0.73 & 0.47 & 0.81 \\\bottomrule\end{tabular}\end{table}

Figure (8) shows the attractor dimension and rMSE/R2 graph for jogging and Figure (7) for walking.This indicates that the proposed method has comparable metrics to classical approaches.Table (2) and Table(1) show the average metric values for all participants.The metrics have high values up to 0.55 and 0.73 explained variance.For the large values of the dimension of the attractor space, the quality of the proposed approach is better than that of similar ones.

Thus, on several real time series it was shown that in the case of a linear dependence, the proposed method allows to obtain more interpretable results and reduces the number of intersections. In the case of clearly nonlinear dependences, the result becomes complex and non-robust.

Conclusion and future work

This paper solves the problem of dimensionality reduction for the phase reconstruction of multivariate time series. The work results in the generalization tensor dynamical system in the case of multivariate time series. This article improves the case which investigates a tensor dynamic system with a univariate time series. The proposed method retains the required properties and reproduces the type of the original attractor with a high accuracy in linear case.

The computational experiment was performed on the Lorenz attractor and accelerometer human motion data. The classical linear approaches and the proposed method were compared.

There are three main directions for future work. The first is to take into account nonlinear relationships through autoencoders and nonlinear activation functions. The second is to increase computational efficiency with a more complex approach which will use not all available components, but those with the highest correlation in the multivariate time series.The third is to optimize the construction of the tensor representation due to the exponential growth of the number of parameters in the case of a larger number of time series. This optimization will be important in analyzing such higher-order dynamical systems for various applications.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Funding Declaration

There are no research grants from funding agencies or research support by organizations that may gain or lose financially through publication of this manuscript.

Data availability statement

The data is publicly available at \\ https://github.com/mmalekzadeh/motion-sense

The code of the computational experiment is available at \\ https://github.com/Denis-Tihonov/TensorDynamic

bibliography{sn-bibliography}

Author Contribution

Denis Tikhonov: Dveloped the computational model and performed computational experiments.Vadim Strijov: Conceptualized the study and edited the manuscript.

Data Availability

The data is publicly available at https://github.com/mmalekzadeh/motion-sense, Mobile Sensor Data Anonymization.

References:

Kevin Judd and Alistair Mees (1998) Embedding as a modeling problem. Physica D: Nonlinear Phenomena 120(3): 273-286 https://doi.org/10.1016/S0167-2789(98)00089-X

Bradley, Elizabeth and Kantz, Holger (2015) Nonlinear time-series analysis revisited. Chaos: An Interdisciplinary Journal of Nonlinear Science 25: https://doi.org/10.1063/1.4917289, 03

Cellucci, C. and Albano, Alfonso and Rapp, Paul (2003) Comparative study of embedding methods. Physical review. E, Statistical, nonlinear, and soft matter physics 67: 066210 https://doi.org/10.1103/PhysRevE.67.066210, 07

Broomhead, David and King, Gregory (1986) Extracting qualitative dynamics from experimental data. Physica D 20: 217-236. Physica D Nonlinear Phenomena 20: 217-236 https://doi.org/10.1016/0167-2789(86)90031-X, 06

Gandolfo, Giancarlo (1971) Economic dynamics: Methods and models. Elsevier, 16

Takens, Floris Detecting strange attractors in turbulence. In: Rand, David and Young, Lai-Sang (Eds.) Dynamical Systems and Turbulence, Warwick 1980, Lecture Notes in Mathematics, 1981, 898, Springer-Verlag, 366--381, Berlin Heidelberg

Brunton, Steven and Noack, Bernd and Koumoutsakos, Petros (2020) Machine Learning for Fluid Mechanics. Annual Review of Fluid Mechanics 52: https://doi.org/10.1146/annurev-fluid-010719-060214, 01

Tan, Eugene and Algar, Shannon and Corr êa, D ébora and Small, Michael and Stemler, Thomas and Walker, David (2023) Selecting embedding delays: An overview of embedding techniques and a new method using persistent homology. Chaos: An Interdisciplinary Journal of Nonlinear Science 33(3) https://doi.org/10.1063/5.0137223, March, AIP Publishing, 1089-7682

Lusch, Bethany and Kutz, J. Nathan and Brunton, Steven L. (2018) Deep learning for universal linear embeddings of nonlinear dynamics. Nature Communications 9(1) https://doi.org/10.1038/s41467-018-07210-0, November, Springer Science and Business Media LLC, 2041-1723

Chen, Can and Surana, Amit and Bloch, Anthony and Rajapakse, Indika (2021) Multilinear Control Systems Theory. SIAM Journal on Control and Optimization 59: 749-776 https://doi.org/10.1137/19M1262589, 02

Daniels, Bryan and Nemenman, Ilya (2015) Automated adaptive inference of phenomenological dynamical models. Nature communications 6: 8133 https://doi.org/10.1038/ncomms9133, 08

Schmidt, Michael and Vallabhajosyula, Ravishankar and Jenkins, Jerry and Hood, Jonathan and Soni, Abhishek and Wikswo, John and Lipson, Hod (2011) Automated refinement and inference of analytical models for metabolic networks. Physical biology 8: 055011 https://doi.org/10.1088/1478-3975/8/5/055011, 08

Roberts, A. (2023) Model Emergent Dynamics in Complex Systems. Society for Industrial and Applied Mathematics, 10.1137/1.9781611973563, 978-1-61197-355-6, , 06

Giannakis, Dimitrios and Majda, Andrew (2012) Nonlinear Laplacian spectral analysis for time series with intermittency and low-frequency variability. Proceedings of the National Academy of Sciences of the United States of America 109: 2222-7 https://doi.org/10.1073/pnas.1118984109, 02

Majda, Andrew and Franzke, Christian and Crommelin, Daan (2009) Normal forms for reduced stochastic climate models. Proceedings of the National Academy of Sciences of the United States of America 106: 3649-53 https://doi.org/10.1073/pnas.0900173106, 03

Ye, Hao and Beamish, R.J. and Glaser, Sarah and Grant, Sue and Hsieh, Chih-hao and Richards, Laura and Schnute, Jon and Sugihara, George (2015) Equation-free mechanistic ecosystem forecasting using empirical dynamic modeling. Proceedings of the National Academy of Sciences of the United States of America 112 https://doi.org/10.1073/pnas.1417063112, 03

Sugihara, George and May, Robert and Ye, Hao and Hsieh, Chih-hao and Deyle, Ethan and Fogarty, Michael and Munch, Stephan (2012) Detecting Causality in Complex Ecosystems. Science (New York, N.Y.) 338: https://doi.org/10.1126/science.1227079, 09

Voss, Henning and Kolodner, Paul and Abel, Markus and Kurths, Juergen (1999) Amplitude Equations from Spatiotemporal Binary-Fluid Convection Data. Physical Review Letters 83: https://doi.org/10.1103/PhysRevLett.83.3422, 10

R. Gonz ález-Garc ía and R. Rico-Mart ínez and I.G. Kevrekidis (1998) Identification of distributed parameter systems: A neural net based approach. Computers & Chemical Engineering 22: S965-S968 https://doi.org/10.1016/S0098-1354(98)00191-4, European Symposium on Computer Aided Process Engineering-8

Kevrekidis, Ioannis and Gear, C. and Hyman, James and Kevrekidid, Panagiotis and Runborg, Olof and Theodoropoulos, Constantinos (2003) Equation-Free, Coarse-Grained Multiscale Computation: Enabling Mocroscopic Simulators to Perform System-Level Analysis. Communications in Mathematical Sciences 1: https://doi.org/10.4310/CMS.2003.v1.n4.a5, 01

James P. Crutchfield and Bruce S. McNamara (1987) Equations of Motion from a Data Series. Complex Syst. 1https://api.semanticscholar.org/CorpusID:14493184

D. Broomhead and R. Jones (1989) Time-series analysis. Proceedings of the Royal Society of London A: Mathematical, Physical and Engineering Sciences 423: 103 –121 https://doi.org/10.1098/rspa.1989.0044, The Royal Society

Benner, Peter and Gugercin, Serkan and Willcox, Karen (2015) A Survey of Projection-Based Model Reduction Methods for Parametric Dynamical Systems. SIAM Review 57(4): 483-531 https://doi.org/10.1137/130932715

L. Arnold (1998) Random Dynamical Systems. Springer Monographs in Mathematics, Springer Berlin, Heidelberg, 10.1007/978-3-662-12878-7

Brunton, Steven L. and Proctor, Joshua L. and Kutz, J. Nathan (2016) Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the National Academy of Sciences 113(15): 3932 –3937 https://doi.org/10.1073/pnas.1517384113, March, Proceedings of the National Academy of Sciences, 1091-6490

Wolf, Michael M. and Klinvex, Alicia M. and Dunlavy, Daniel M. (2016) Advantages to modeling relational data using hypergraphs versus graphs. 10.1109/HPEC.2016.7761624, Laplace equations;Electronic mail;Data models;Computational modeling;Analytical models;Sparse matrices;Software, 1-7, , , 2016 IEEE High Performance Extreme Computing Conference (HPEC)

Can Chen. On the Stability of Multilinear Dynamical Systems. 10.48550/arXiv.2105.01041, math.OC, arXiv, 2105.01041, 2022

Packard, N. H. and Crutchfield, J. P. and Farmer, J. D. and Shaw, R. S. (1980) Geometry from a Time Series. Phys. Rev. Lett. 45: 712--716 https://doi.org/10.1103/PhysRevLett.45.712, American Physical Society, Sep, 0, 9

Kai Kruppa (2017) Comparison of Tensor Decomposition Methods for Simulation of Multilinear Time-Invariant Systems with the MTI Toolbox. IFAC-PapersOnLine 50(1): 5610-5615 https://doi.org/10.1016/j.ifacol.2017.08.1107, 2405-8963, 20th IFAC World Congress

Chen, Can and Surana, Amit and Bloch, Anthony and Rajapakse, Indika (2019) Multilinear Time Invariant System Theory. Society for Industrial and Applied Mathematics, 10.1137/1.9781611975758.18, 978-1-61197-575-8, 118-125, 1, 06

Ignatov, Andrey D and Strijov, Vadim V (2016) Human activity recognition using quasiperiodic time series collected from a single tri-axial accelerometer. Multimedia tools and applications 75(12): 7257--7270 https://doi.org/10.1007/s11042-015-2643-0, Springer

Motrenko, Anastasia and Strijov, Vadim (2015) Extracting fundamental periods to segment biomedical signals. IEEE journal of biomedical and health informatics 20(6): 1466--1476 https://doi.org/10.1007/s11042-015-2643-0, IEEE

Grabovoy, AV and Strijov, VV (2020) Quasi-Periodic Time Series Clustering for Human Activity Recognition. Lobachevskii Journal of Mathematics 41(3): 333--339 https://doi.org/10.1134/S1995080220030075, Springer

Usmanova, KR and Zhuravlev, Yu I and Rudakov, KV and Strijov, VV (2020) Approximation of Quasiperiodic Signal Phase Trajectory Using Directional Regression. Moscow University Computational Mathematics and Cybernetics 44(4): 196--202 https://doi.org/10.3103/S0278641920040068, Springer

Powell, Warren B (2011) Approximate dynamic programming: solving the curses of dimensionality, second edition. John Wiley & Sons, Hoboken, N.J, 10.1002/9781118029176, Wiley series in probability and statistics, eng, Dynamic programming, 047060445X, 2nd ed.

Malekzadeh, Mohammad and Clegg, Richard G. and Cavallaro, Andrea and Haddadi, Hamed (2019) Mobile Sensor Data Anonymization. ACM, New York, NY, USA, adversarial training, deep learning, edge computing, sensor data privacy, time series analysis, 3310068, 10.1145/3302505.3310068, 10, 49--58, Montreal, Quebec, Canada, 978-1-4503-6283-2, IoTDI '19, Proceedings of the International Conference on Internet of Things Design and Implementation

Bengio, Yoshua and LeCun, Yann Scaling Learning Algorithms Towards {AI}. Large Scale Kernel Machines, 2007, MIT Press

Hinton, Geoffrey E. and Osindero, Simon and Teh, Yee Whye (2006) A Fast Learning Algorithm for Deep Belief Nets. Neural Computation 18: 1527--1554

Goodfellow, Ian and Bengio, Yoshua and Courville, Aaron and Bengio, Yoshua (2016) Deep learning. MIT Press, 1

Additional Files

Additional file 3

Additional file 4

Additional file 5