% \documentclass[fleqn,10pt]{wlscirep}
\documentclass{wlscirep}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{ulem}
\usepackage{multirow} % multirow 패키지 추가
\usepackage{graphicx} % resizebox 사용을 위한 패키지
\title{Multi-task Learning for Estimation of Remote PPG and Respiration Signals with Complex valued Convolutional Neural Network}

\author[1,+]{Junghwan Lee}
\author[1,+]{YuSang Nam}
\author[1,+]{Jihwan Won}
\author[1]{Suhwan Baek}
\author[1]{Donggyu Sim}
\author[2,*]{Ryanghee Sohn}
\author[1,*]{Cheolsoo Park}

\affil[1]{The Department of Computer Engineering, Kwangwoon University, Seoul, 01899, Republic of Korea} 
\affil[2]{The Emma Healthcare Corporation, Sujeong-gu, Seongnam-si, 13135, Republic of Korea}

\affil[*]{chief@emmahc.com, parkcheolsoo@kw.ac.kr}

\affil[+]{these authors contributed equally to this work}

\keywords{Complex-valued neural network, Multi-task Learning, Remote photoplethysmography}

\begin{abstract}
Remote and continuous biometric signal monitoring has become increasingly crucial for the prompt diagnosis of physiological disorders. However, traditional contact sensors might pose the risk of virus spread and cause discomfort, thereby impeding the continuous monitoring process. Furthermore, the enhancement of diagnostic performance using deep neural networks necessitates the use of large models, which could be a burden when developing embedded edge devices. Thus, we propose a multitask learning model to estimate the remote photoplethysmogram (PPG) and respiratory rate simultaneously based on facial videos using complex-valued neural networks. The RGB channel images are obtained from a region of interest of the facial video streams and a complex-numbered dataset is constructed. The multitask learning model designed for the complex domain can yield a small network architecture by reducing the number of parameters, which is advantageous for small embedded devices. Using a public dataset of face video streams from multiple participants, the proposed multitask learning model could simultaneously learn the remote PPG and respiratory rate with higher performance and a smaller structure compared with conventional real-valued neural networks. These results validate the potential of the proposed model for the accurate and efficient remote monitoring of physiological disorders.
\end{abstract}
\begin{document}

\flushbottom
\maketitle
% * <john.hammersley@gmail.com> 2015-02-09T12:07:31.197Z:
%
%  Click the title above to edit the author information and abstract
%
\thispagestyle{empty}

% \noindent Please note: Abbreviations should be introduced at the first mention in the main text – no abbreviations lists. Suggested structure of main text (not enforced) is provided below.

\section*{Introduction}

Photoplethysmography (PPG) is an optical technique that was first introduced in the 1930s \cite{b1}. PPG is used extensively to monitor various vital signs, such as the pulse rate, respiratory rate, and blood oxygenation, and becomes popular owing to its noninvasiveness. In a PPG sensor, red or green LED light is transmitted to the skin, and partially absorbed and reflected. Subsequently, the change in the absorbed/reflected light is measured using a photodetector, which generates a PPG signal. PPG detects optical absorption variations in human skin in response to changes in the blood volume during the cardiac cycle. It is commonly used to monitor the heart rate (HR) through a contact device. However, as the application of contact devices requires specific medical instruments and professional attention, it is not suitable for the mass monitoring of a large group of subjects \cite{b2, a7}. To address these issues, camera-based PPG measurement, which is known as remote PPG (rPPG), has been developed. rPPG is a remote PPG method measuring changes in the optical blood flow based on an image, allowing for the collection of a biosignal using only ambient light as the light source, without the need for a separate light source from the PPG sensor device. rPPG could detect minute changes in the appearance of human skin and has been demonstrated for the non-contact case monitoring such as epidemic diseases, skin damage, or neonates \cite{b3}.

Remote health monitoring technology has become critical for managing patients with COVID-19 during the recent pandemic situation \cite{b4}. The Korea Centers for Disease Control and Prevention recommends the use of remote health strategies as far as possible to reduce the risk of COVID-19 transmission in medical environments. Remote health monitoring technology is based on communication systems such as mobile phones and online health portals \cite{a6}. Specifically, a mobile phone camera could be used to measure physiological signals from users based on facial video streams to estimate their HR and respiratory rate \cite{b5, a55}. The remote measurement of vital signs is a safe means of monitoring contagious diseases such as COVID-19, which could be transmitted through personal contact. For these purpose, rPPG technology has improved significantly in recent years, with various applications in telemedicine \cite{b7}. Several studies have successfully measured multiple physiological signals, including the HR, respiratory rate, blood oxygen saturation, pulse rate variability, blood perfusion, pulse transit time, blood pressure, atrial fibrillation, and systolic and diastolic peaks, using rPPG \cite{b8}. 

Previous studies have investigated camera-based methods for measuring vital signs. For example, researchers developed a method to estimate the HR from videos by tracking the average brightness of a region of interest (ROI) on the face of the subject. This method calculates the first-order derivative of the signal, applies a low-pass filter, and conducts autoregressive spectral analysis to identify the dominant frequencies that correspond to the HR. However, the average HR could only be estimated over a 30s period \cite{b9}. In another study, the HR and respiratory rate were estimated from videos by spatially averaging the red, green, and blue channels in a manually selected ROI, which was followed by frequency spectrum analysis after band-pass filtering. It has been found that the green channel has the strongest plethysmography signal \cite{b9}. In addition, a highly sensitive thermal camera has also been applied to estimate the HR with fast Fourier transform on the signal from a rectangular ROI \cite{b10}. 

Another study presented a method for estimating the pulse rate by tracking multiple features on the head and performing principal component analysis (PCA) on their trajectories. However, a major drawback of this method is that strong head movements could result in erroneous measurements. A method was recently developed to estimate the average pulse rate by performing PCA on the red, green, and blue channel signals obtained from a manually selected ROI on the face of the subject over 30 s. This method could also estimate physiological parameters such as the respiratory rate and blood oxygen saturation using color channel signals. Previous approaches have typically involved the definition of an ROI for signal acquisition, which could not be adaptively updated to create a more intuitive and practical scenario. In recent studies, deep learning models have been actively applied to analyze biological signals through its end-to-end architecture. PPG prediction was also performed by training a deep learning algorithm, such as convolutional neural networks (CNN), using facial images \cite{b10}. The forehead and cheeks are commonly selected as ROIs for PPG estimation since the difference between the two regions could provide valuable information regarding the time difference of the blood flow from the heart \cite{b10}. 

Real number-based algorithms for rPPG face significant limitations, such as heightened sensitivity to noise like motion artifacts and ambient light interference, which are common in real-world applications \cite{c1, c2}. These algorithms often struggle to separate noise from true physiological signals, leading to inaccurate measurements due to their reliance on the amplitude information alone, neglecting the phase information that could enhance the signal processing performance. Consequently, they fail to capture the full complexity of physiological signals, reducing their effectiveness in clinical and daily life scenarios. To address these issues, we propose a complex-valued algorithm that incorporates both amplitude and phase information, allowing for better noise filtration and signal isolation. This dual representation, supported by the signal processing models like Fourier and Hilbert transforms, improves the robustness, versatility, and the accuracy of rPPG systems across various conditions, representing a significant advancement in remote photoplethysmography.

$추가된 단락$
Despite the recent advancements in rPPG research, most existing studies have been conducted under controlled laboratory environments with limited experimental conditions. These models often struggle to perform reliably in real-world scenarios due to their sensitivity to external factors such as variations in the ambient light, motion artifacts, and differences in skin tones. Moreover, most of the previous approaches rely on the real-valued neural networks, which primarily focus on the amplitude-based signal processing with neglecting the phase information. This limitation reduces their ability to effectively model the complex relationships among multiple channel signals, leading to suboptimal performance in practical applications \cite{revision1, revision2}. While some recent studies have attempted to incorporate advanced neural network architectures, they still lack the ability to fully leverage the interdependencies among the multivariate components, limiting their robustness in dynamic environments \cite{revision2}. To address these challenges, our study introduces a complex-valued deep learning framework that captures both amplitude and phase information, enabling more effective feature extraction and noise filtration  \cite{revision3}. Additionally, unlike the previous works focusing on the restricted laboratory settings, we conduct extensive experiments under diverse real-world conditions, including the varying illumination levels, flickering environments, and a broader range of skin tones. With validating our approach across multiple datasets and conditions, we demonstrate its superior adaptability and robustness compared with the conventional real-valued deep neural network models. Our research not only advances the theoretical foundation of the complex-domain neural networks in rPPG estimation but also provides a practical and scalable solution for real-world health monitoring applications.

$추가된 단락$
Building upon this, our study integrates the existing rPPG estimation model with the complex-domain neural networks, and pioneers the application of complex-valued computations specifically for the rPPG-based heart rate estimation. While the complex-valued neural networks have been explored in other domains, their application to the rPPG estimation remains have not been tried. By leveraging both amplitude and phase components of the input signals, our approach enhances the performance of the rPPG signal extraction, effectively mitigating the noise and artifacts that traditional real-valued models struggle with. To rigorously validate the effectiveness of this approach, we conducted extensive experiments across various real-world conditions. Our study uniquely examines the impact of varying illumination levels, flickering environments, and skin tone differences using our experimental and publicly available datasets. These comprehensive evaluations demonstrate the robustness of our method in practical scenarios, marking a significant step forward in the rPPG research. By introducing this theoretical framework alongside the empirical validation, we establish a new paradigm to improve the rPPG estimation accuracy and reliability in real-world applications.

Complex-valued algorithms have been developed to analyze the relationship between two channel data more effectively. Falzon et al. proposed a complex-valued common spatial patterns (CSP) algorithm known as the analytic signal-based CSP to discriminate between different mental tasks \cite{b10}. Another complex version of the CSP, known as CCSP, uses a covariance matrix to maintain the power-sum information of the real and imaginary parts of the complex-valued data \cite{b10}. Furthermore, deep neural networks has been also developed in the complex domain to take an advantage of its multichannel data process efficiency \cite{b17}. In this study, two regions of the face are simultaneously analyzed through a complex-valued deep learning model to process them efficiently and accurately. In particular, the complex-valued CNN could reduce the number of weights in the networks and improve the performance with dealing with multichannel data simultaneously compared with the real-valued original CNN \cite{b17}. Additionally, neural networks in the complex domain could enhance the speed of backpropagation, which is the fundamental learning method of deep learning models, by 2 to 3 times, and generally require only half of the necessary factors, such as weights and thresholds \cite{b11}. 

Most artificial neural networks (ANNs) and machine learning models are traditionally implemented using the real numbers. These approach enables the natural modeling of the signal processing operations such as filter rotation in the processes of the multiplication and convolution. However, with the complex numbers, these models could enable more effective learning of various transformations and patterns \cite{b31} . Complex-valued neural networks allow input neurons with similar phases to be structurally synchronized and interact with neurons having different phases, thus providing a richer and more flexible modeling framework. This structural synchronization and phase interaction facilitate the modeling of more complex patterns and transformations. In the context of rPPG, utilizing the complex numbers could improve the robustness and accuracy of the algorithms with better handling noise and capturing the full complexity of physiological signals through both amplitude and phase information. The enhanced filtering capabilities and more comprehensive signal representation of the complex-valued algorithms provide significant advantages over traditional real number-based methods.

Multi-task learning (MTL) is a training approach that involves the simultaneous learning and prediction of two or more tasks using a shared layer \cite{b16}. In this manner, the trained information could be shared, which would help to improve the model performance for each task since the useful information for one task could positively influence other tasks. Furthermore, the simultaneous prediction of multiple tasks could create a stronger generalized model with less overfitting \cite{a3}. The overall number of model weights could be reduced with creating a single model for multiple tasks, which would be more suitable for the implementation of mobile devices. In particular, when deep neural networks are designed to predict physiological information in a mobile healthcare system, MTL could make an important role for efficiently handling multiple tasks with its limited resource. 

This study proposes a complex-valued MTL algorithm that simultaneously processes video streams to extract PPG and respiratory signals together from facial videos. The MTL model has complex-valued data as an input, consisting of two facial areas in the real and imginery parts, which are processed simultaneously in a complex neural network architecture.

Traditional real-valued neural networks face challenges in processing complex signals, which can be problematic when handling rPPG signals that are sensitive to environmental changes such as varying lighting conditions and user movements. This sensitivity to motion artifacts limits the effectiveness of real-valued RGB-based rPPG algorithms. To overcome these limitations, our study implemented a multi-input model using complex-valued neural networks. Based on the previous studies indicating that the areas of the forehead and cheeks provide significant information to produce rPPG, we focus on these two regions of interest (ROI) areas \cite{r1}. With training a multi-input complex-valued neural network using the complex-valued input consisting of the images from the forehead and cheeks, we aim to better learn the highly correlated common physiological information from the input areas, surpassing the limitations of conventional real-valued RGB channel-based remote PPG algorithms.

This complex-valued process enables a more efficient and accurate extraction of PPG and respiratory signals compared with the conventional real-valued single task learning models, which is demonstrated using a public dataset of face video streams from multiple participants.

\section{Algorithms}

\subsection{Multi-Task Learning (MTL)}
The standard procedure of a machine learning algorithm includes a learning process for one task at a time. Large problems are typically broken down into smaller and relatively independent subproblems that are learned separately and subsequently combined \cite{b13}. MTL is an alternative method for improving the model generalization by incorporating domain knowledge from two or more tasks, as illustrated in Fig. 1. MTL learns multiple tasks simultaneously by using their shared representations, which could improve the learning of other tasks [13]. MTL facilitates inductive transfer between tasks, with the aim of leveraging additional information sources to enhance the learning performance of the current task [15]. Inductive transfer could lead to faster learning, more interpretable models, and efficient model generalization, while reducing the number of training examples or fixing the training set \cite{b13}.

MTL has been applied extensively in various domains, such as computer vision, natural language processing, and medical image analysis. Ruder et al. provided an overview of deep neural networks in MTL \cite{b15}, and Meyerson et al. proposed a high-level classification of existing deep MTL methods, highlighting the effectiveness of MTL based on the parallel ordering assumption \cite{b15}. Hard parameter sharing, which is depicted in Fig. 1, is the most commonly used architecture for MTL. It includes multiple task-specific output layers and hidden layers that are often shared across tasks. Although MTL models aim to learn informative features for multiple tasks simultaneously, it is challenging for the shared layers to update the optimal weights for entirely different tasks. However, when two similar tasks such as the PPG and respiratory rate are learned, an MTL model can achieve generalized parameters for multiple tasks using their common features \cite{b14}.

The MTL architecture has also been used to simultaneously estimate various biological signals, such as the HR and respiratory rate \cite{b15}. One study used an MTL model with hard parameter-sharing networks to learn and predict the PPG and respiratory rate simultaneously.
 In one study, an MTL model with hard parameter sharing networks was used to learn and predict the PPG and respiratory rate simultaneously. The goal was to reduce the parameters and prevent overfitting by considering the two tasks in a single model. MTL enables knowledge sharing while simultaneously addressing multiple related tasks \cite{b12}. The sharing of knowledge among tasks in MTL could improve the predictive performance of each task. As mentioned earlier, the simultaneous learning of multiple tasks enables useful information from both the PPG and respiratory rate tasks to be combined, resulting in improved performance compared with a single-task learning model. In this study, we propose a complex-valued MTL model to enhance the precision of contactless cardiopulmonary measurements. By designing the MTL in complex domain, a single complex-valued model combining two separate real-valued models could be trained, thereby reducing the number of parameters. The simultaneous learning of related tasks allows a well-learned representation to be shared, which could improve the model learning performance.

\begin{figure}[htb!]
    \centerline{\includegraphics[width=0.5\columnwidth]{IMAGE/FIGURE2_.png}}
    \caption{In MTL neural networks, the input data is first fed into a shared layer, where the learning process takes place. Thereafter, each output is generated in task-specific layers.}
    \label{fig1}
\end{figure}

Although a deep learning model has traditionally been based on real numbers, complex numbers are often used to express real-world phenomena, and to analyze mathematical and geometric relationships such as signal amplification and phases. Researchers have developed complex-valued neural networks (CVNNs) and complex backpropagation (CBP) algorithms to process complex-valued data directly. These network properties have been studied extensively \cite{b5}, \cite{b10}, and CVNNs have been demonstrated to be powerful in applications such as adaptive radar image \cite{b16} and optical image processing \cite{b20}. Recent studies have applied CVNNs to real-valued classification problems, and complex numbers have been found to have a better potential and expressive ability to create new neural structures \cite{a4}. In this study, we aimed to predict both the PPG and respiratory rate simultaneously using a complex-valued multi-task CNN model with the real-valued images from a forehead and cheeks as inputs. 

    
\subsection{Complex-Valued Neural Network (CVNN)}

The concept of complex numbers was accepted as a geometric representation by Caspar Bessel in 1799. However, it was later discovered by Karl Friedrich Gauss and gained wider recognition, eventually becoming an important mathematical extension. Interestingly, the concept of the geometric representation of complex numbers also appeared in “Alzebractatus” by John Wallace in 1685 \cite{b19}. Consequently, complex numbers have been used extensively to analyze mathematical and geometric relationships by expressing them as real-world phenomena, and have been recognized as advantageous in various fields.


\begin{figure}[htb!]
    \centerline{\includegraphics[width=0.8\columnwidth]{IMAGE/Figure3__.png}}
    \caption{CVNN model in complex domain. The weighted inputs X are summed, and an activation function converts a complex-valued internal state into a corresponding real-valued output y. }
    \label{fig1}
\end{figure}

The complex-valued neural network algorithms such as CVNNs and CBP have been developed to handle the complex values in artificial neural networks effectively. As illustrated in Fig. 2, CVNN \cite{b18} processed the complex-valued input in a similar structure to the conventional real-valued neural networks (RVNNs) model. However, owing to the complex-valued nature of CVNNs, it is necessary to map the real-valued input to the complex domain. Once the mapping procedure has been completed, the neurons in the CVNN process the complex-valued data in a similar manner to the conventional neuron model, in which all parameters and variables are complex-valued, and computations are performed based on the complex algebra.

The backpropagation algorithm is used to train CVNNs, which exhibits faster convergence in the complex domain compared with the RVNNs, as reported in \cite{b20}. The study found that CVNNs trained using the backpropagation algorithm achieved lower validation errors with fewer training epochs than RVNNs on the datasets of cancer, card, diabetes, glass, and soybean \cite{b20}. This suggests that CVNNs converge faster in learning than RVNNs, as only half of the network parameters such as weights and thresholds are required for CVNNs to solve the same problem \cite{b19}. 


\subsection{Complex-Valued Convolutional Neural Network (CV-CNN)}


In this study, we implemented a convolutional neural network (CNN) architecture for complex-valued facial image inputs, where the real part contains the forehead image pixel data and the imaginary part the cheek image pixel one, using CVNNs. As shown in Fig. 3, the structure of a complex-valued CNN (CV-CNN) follows the basic architecture of the conventional real-valued CNN (RV-CNN), where CV-CNN extend the procedure of RV-CNN operation in the complex domain. However, as illustrated in Fig. 4, the key difference between RV-CNN and CV-CNN is the computational process in the kernel and input. In CV-CNN, the complex arithmetic is used and the kernel weights for the convolution procedure are implemented with the complex values. Both the convolution kernel and the input are complex-valued, and the only distinction from RV-CNN is the multiplication of the complex numbers.

\begin{figure}[htb!]
    \centerline{\includegraphics[width=\columnwidth]{IMAGE/Figure8____.png}}
    \caption{Complex-valued convolutional neural networks (CV-CNNs) architecture}
    \label{fig1}
\end{figure}


%\begin{equation}
%    W = A + \textit{i}B
%\end{equation}

%\begin{equation}
%    Z = X + \textit{i}Y
%\end{equation}

When a complex matrix is convoluted with kernel \textit{W = A + iB}, the output corresponding to the input \textit{Z = X + iY} is the dot product of the input and kernel like Eq.(1) 

\begin{equation}
 Z \cdot W = \left (X\cdot A - Y\cdot B \right) + \mathit{i}\left( X\cdot B + Y\cdot A \right)
\end{equation}

$추가된 단락$
To achieve the same computation as the complex-valued CNN in Figure 4(a), the real-valued CNN in Figure 4(b) processes real and imaginary components separately using independent convolution operations. This requires additional element-wise operations and concatenation to approximate the complex-valued representation. \cite{b15}.

\begin{equation}
    \left [ X\cdot A - Y\cdot B, X\cdot B - Y\cdot A \right ]
\end{equation}

$추가된 단락$
As depicted in Fig 4, the convolutional layer performs the operation of multiplying the input patch with the kernel. Real-valued convolution layers, compared to the complex-valued convolution layers, necessitate a greater number of input patches and kernels to accommodate the computational requirements. Furthermore, to perform the same operations as the complex-valued convolutional layer, an additional structure with trainable parameters is required in the real-valued case. Since the complex-valued convolutions inherently process both the real and imaginary components simultaneously within a single model, the real-valued networks need to employ two separate convolutional operations—one for the real part and another for the imaginary part—leading to an effective doubling of network parameters and computational cost. This additional structure is necessary to capture the relationships between the two components, which are naturally preserved in the complex-valued convolutions. Consequently, the real-valued CNNs not only require more computational resources but also struggle to maintain the phase relationships between two signal components, making them less efficient in extracting the physiological information.

For better understanding of the complex convolution, we examine the corresponding complex-valued optimization problem in this study, where the optimization problem of the complex-valued network is looked into. Here, $|Z \cdot W|$ represents the complex numbered objective function from Eg. (1), to be maximized \cite{b20}.

\begin{equation}
\underset{\|W\|=1}{\arg\max} \left| Z \cdot W \right|
\end{equation}
where the complex variable $Z$ and $W$ can be expressed as Euler's form in Eg. (4)

\begin{equation}
\forall i, j \quad Z_{ij} = r_{ij} e^{\imath \theta_{ij}}, \quad W_{ij} = t_{ij} e^{\imath \phi_{ij}}
\end{equation}

In the context of our algorithm, the variables 
$Z_{\mathit{ij}}$ and $W_{\mathit{ij}}$ play critical roles in the model. Specifically, $Z_{\mathit{ij}}$ represents the input to the model, while $W_{\mathit{ij}}$ denotes the learned weights of the model.

The input variable $Z_{\mathit{ij}}$ is a complex number that can be expressed as $r_{ij} e^{\imath \theta_{ij}}$. Here, $r_{\mathit{ij}}$ is the magnitude or modulus of $Z_{\mathit{ij}}$, which is a non-negative real number indicating the distance of $Z_{\mathit{ij}}$ from the origin in the complex plane, effectively capturing the size of the input. The angle $\theta_{\mathit{ij}}$, known as the argument or phase, is measured in radians and represents the orientation of $Z_{\mathit{ij}}$ in the complex plane by specifying the angle between the positive real axis and the line from the origin to $Z_{\mathit{ij}}$.

\begin{figure}[htb!]
    \centering
    \includegraphics[width=\textwidth, height=0.9\textheight]{IMAGE/Figure4_____ (2).png}
    \caption{Comparison of the Complex-valued and Real-valued Convolution Neural Networks}
    \label{fig1}
\end{figure}

On the other hand, $W_{\mathit{ij}}$ corresponds to the weights learned by the model during the training process. This weight is also a complex number, expressed as $t_{ij} e^{\imath \phi_{ij}}$. The term $t_{ij} e^{\imath \phi_{ij}}$ is the magnitude or modulus of $W_{\mathit{ij}}$, representing a non-negative real number that quantifies the size of the weight by indicating its distance from the origin in the complex plane. The phase $\phi_{ij}$, similar to the phase of $Z_{\mathit{ij}}$, is the angle in radians between the positive real axis and the line from the origin to $W_{\mathit{ij}}$, representing the direction of the weight in the complex domain.
  
Based on Eg. (4), the maximization problem is derived in Eg. (5).

\begin{equation}
\underset{W}{\arg\max} \left| \sum_{ij} r_{ij} t_{ij} e^{i (\theta_{ij} + \phi_{ij})} \right|
\end{equation}

For the geometric interpretation of the objective function, we will consider each complex number in $Z$ and $W$ as a 2-dimensional vector. Using Euler's formula, each complex number can be expressed in terms of its magnitude and phase. In this perspective, when multiplying $W_{\mathit{ij}}$ and $Z_{\mathit{ij}}$, the angle $\theta_{\mathit{ij}}$ and $\phi_{\mathit{ij}}$ rotates. The sum of the rotated vectors reaches its maximum magnitude when they all have the same phase and their magnitudes are accumulated.
 The complex number Z can be expressed in a polar form as following Eq. (6), where $r$ represents the magnitude and $\theta$ represents the phase.

\begin{equation}
    Z = a + ib = re^{i\theta}
\end{equation}
where $r$ and $\theta$ are determined in Eg. (7) and (8).

\begin{equation}
r = \sqrt{a^2 + b^2}
\end{equation}

\begin{equation}
\theta = \tan^{-1}\left(\frac{b}{a}\right)
\end{equation}

In the process of calculating this $r$, we perform a series of calculation on the sum of the squares of the two input components in the networks. This method is designed to maximize the amount of relevant information that can be derived from these two inputs. Through maximizing this, we enhance the performance to capture and learn the underlying physiological information to be estimated from the two facial image patches. 

In addition, during the process of calculating $\theta$, it allows for better learning of the common physiological information, which has high correlation, by matching the phases of physiological signal components coming from the two input areas. High correlation indicates that the inputs share common physiological information, which is crucial for models to effectively understand and learn physiological signals.

This process not only helps in extracting the most significant features from the input data but also ensures that the predictive model is provided with the richest possible set of information, thereby improving its accuracy and effectiveness in the rPPG estimation task.

$추가된 단락 $
Furthermore, the process of aligning the phases of physiological signals from two input regions allows the network to emphasize the coherent and synchronized features that are fundamental to rPPG estimation. Since physiological signals such as blood volume pulse (BVP) exhibit periodic characteristics, ensuring that the phase components are properly matched between different facial regions enables the network to reinforce periodic patterns while suppressing non-physiological variations caused by external factors such as motion artifacts and lighting changes.

By leveraging this phase-matching mechanism, the model is able to identify and extract the shared periodic structure present in both input patches, effectively reducing noise sensitivity. This is particularly important in dynamic environments where uncontrolled motion or illumination fluctuations may introduce phase misalignment in traditional real-valued approaches. The complex-domain processing mitigates this issue by explicitly capturing phase correlation, ensuring that the learned representations maintain physiological consistency across different spatial locations.

Moreover, this phase-aware approach facilitates more effective signal fusion from multiple facial regions, enhancing the robustness of the extracted rPPG features. Since different facial areas may exhibit varying signal intensities due to skin tone, blood perfusion, and external conditions, the ability to synchronize and combine the most reliable components from each region leads to a more stable and accurate physiological signal estimation. Ultimately, this contributes to the overall improvement of the model’s generalizability and effectiveness, making it more adaptable to real-world biometric monitoring scenarios.

%The complex-valued convolutional layer calculates the output pixel by taking the dot product of the input patch and the kernel. In contrast, the real-valued convolutional layer with two channeled input, output, and kernels convolves with one kernel to yield one channel. It's important to note that the equivalence between real and complex networks does not hold in non-affine layers.
%In activation function and pooling layers, which operate on one channel, the equivalent real-valued layers would need to operate on two channels, which is not the case. In these layers, the complex network can be seen as connecting the channels, rather than the weights \cite{b15}.

\subsection{Complex Backpropagation for CV-CNN}
The common way to train neural networks is to minimize its loss function $\ell(W)$, which is conducted using the iterative gradient descent algorithm. Starting with an initial value for the weight $W$, at each iteration, the weights are updated by taking a step in the direction opposite to the gradient of the loss function \cite{e27}.
In cases where complex-valued weights are applied, the loss function is based on the real-values but depends on the complex-valued weights. Such functions might not be differentiable in the networks, and the steepest descent algorithm could not be processed using the standard gradient. To address this, the Wirtinger derivatives, are applied which are suitable for handling the functions consisting of the complex variables \cite{e27}.
We use the following notations for the real-valued neural networks. Given the loss function $\ell(W)$, its gradient with respect to the weight matrix  $W$ is denoted by $\frac{\partial \ell}{\partial W}$. The gradient $\frac{\partial \ell}{\partial W}$ is a matrix where each element $[i,j]$ is given by:

\begin{equation}
\left[\frac{\partial \ell}{\partial W}\right]_{[i, j]}=\frac{\partial \ell}{\partial W[i, j]}
\end{equation}


Let's consider the $n^{th}$ layer of a complex-valued network with input, weights, and output denoted by $Z_{n}$, $W_{n}$, and $Z_{n+1}$ respectively. Their real and imaginary parts are denoted by:

\begin{equation}
    Z_n=X_n+i Y_n, \quad W_n=A_n+i B_n
\end{equation}

The derivatives of the loss $\ell$ with respect to the input are denoted by:
\begin{equation}
\delta_n=\delta_n^R+i \delta_n^I=\frac{\partial \ell}{\partial X_n}+i \frac{\partial \ell}{\partial Y_n}
\end{equation}



The backpropagation process is the canducted by derivatives with respect to the weights:
\begin{equation}
\frac{\partial \ell}{\partial A_n}+i \frac{\partial \ell}{\partial B_n}
\end{equation}

When dealing with complex-valued weights $W=A+iB$, in gradient descent, it is suggested to take the step in the direction of $-\left(\frac{\partial \ell}{\partial A}+i \frac{\partial \ell}{\partial B}\right)$.
When calculating $\delta_n$ at each layer, the Jacobian matrix is used to propagate the gradient from the previous layer to the current layer. This allows for the effective calculation of the gradient of the loss function with respect to each weight in the network. The complex input $Z_n$ is separated into its real part $X_n$ and imaginary part $Y_n$ . Similarly, the output $Z_{n+1}$ is also separated into $X_{n+1}$ and $Y_{n+1}$. Since $Z_{n+1}$ is a function of $Z_n$ , the Jacobian matrix is constructed as follows:

\begin{equation}
J=\left[\begin{array}{ll}
\frac{\partial X_{n+1}}{\partial X_n} & \frac{\partial X_{n+1}}{\partial Y_n} \\
\frac{\partial Y_{n+1}}{\partial X_n} & \frac{\partial Y_{n+1}}{\partial Y_n}
\end{array}\right]
\end{equation}
Corresponding Eq. (11), when calculating the gradient of the loss function $l$, the complex gradient is separated into its real and imaginary parts. Using the Jacobian matrix at each layer, the gradient is propagated to the next layer as follows:

\begin{equation}
\delta_{n-1}=J^T \delta_n
\end{equation}
where $J^T$ is the transpose of the Jacobian matrix. The gradient of the loss function with respect to the weights of each layer $W_n=A_n+i B_n$ is calculated. The calculated gradient is then used to update the weights through the optimization algorithms such as the gradient descent.

The backpropagation algorithm is composed of two passes, forward and backward.
\begin{enumerate}
    \item Forward Pass: The input values of each layer, $Z_{n}$, are computed based on the network's architecture from the first layer to the final $N^{th}$ layer.
    \item Backward Pass: In the backward pass, the final layer’s gradient $\delta_N$ is calculated, and the $\delta_N$ and $\frac{\partial \ell}{\partial A_n}+i \frac{\partial \ell}{\partial B_n}$ are computed for every $n^{th}$ layer in reverse order, based on the chain rule. The final output is the concatenation of $\frac{\partial \ell}{\partial A_n}+i \frac{\partial \ell}{\partial B_n}$ for all layers.
\end{enumerate}

In this section, We present how to derive $\delta_N$ and $\frac{\partial \ell}{\partial A_n}+i \frac{\partial \ell}{\partial B_n}$ for each layer, given $\delta_{N+1}$. Most of the following computations are described in \cite{e25, e26}.


\subsection{Attention Module}
It is crucial to focus on the pixels that contain physiological signals to avoid unwanted noise amplification. To address this issue, we propose the addition of the attention module in CV-CNN. The distribution of physiological information from the human skin is not spatially and temporally uniform. Therefore, the proposed AttentionMask model aims to assign high weights to significant regions in the face image for the extraction of the physiological information in the middle of the complex convolution process \cite{a5}. Through the attention module, the separation of signal sources could be improved, which helps the multitask mechanism extract the shared common information between the PPG and respiratory signals \cite{b15}.

$추가된 단락$
In the proposed model, the attention module plays a critical role in ensuring that the extracted physiological signals retain the most relevant information from different facial regions. By integrating attention weights, the module dynamically emphasizes pixels with stronger physiological signals while suppressing less informative regions, effectively incorporating both spatial and temporal features \cite{b15}. This is particularly important for ensuring signal continuity between the cheek and forehead regions, which are known to contain the high quality of the rPPG signals due to strong blood perfusion \cite{b15}. Without the attention mechanism, certain facial regions with low physiological information might may introduce artifacts or weaken the extracted waveform.

The attention mechanism selectively enhances the most informative pixels by computing a spatial weight map \( A \). Given an input feature map \( X \) extracted from a facial region, the attention weight is computed as:

\begin{equation}
A = \sigma(W_f * X + b)
\end{equation}
where:
\begin{itemize}
    \item \( W_f \) is a learnable filter that determines the importance of each pixel to estimate rPPG,
    \item \( b \) is a bias term, and
    \item \( \sigma \) represents an activation function (typically a sigmoid function to constrain the weights between 0 and 1).
\end{itemize}
The final weighted feature map \( X' \) is obtained through element-wise multiplication:

\begin{equation}
X' = A \odot X
\end{equation}
where \( \odot \) represents an element-wise multiplication. This formulation ensures that pixels in the regions with strong physiological information receive higher attention weight, while those in less significant areas are down-weighted. As a result, the network learns to prioritize the most reliable physiological signal sources while reducing the interference from noise-prone regions such as shadows, occlusions, or low-blood-perfusion areas.  

By incorporating the attention module, the proposed model achieves the following advantages:
\begin{enumerate}
    \item \textbf{Selective Signal Enhancement} – The network dynamically focuses on the pixel areas with high rPPG information, such as the cheeks and forehead, maximizing the rPPG signal quality.
    \item \textbf{Robustness Against Noise and Artifacts} – Background and less significant regions (e.g., hair, shadows) receive lower attention weights, preventing them from interfering with the extracted rPPG signal.
    \item \textbf{Improved Multi-Region Consistency} – The integration of signals from different facial regions ensures that the trained representation preserves common physiological features across spatially separated areas, leading to a more stable and noise-resilient rPPG estimation.
\end{enumerate}

Overall, the attention module could ensures that the model extracts high-quality physiological signals while maintaining robustness across varying environmental conditions, significantly improving the reliability of the rPPG-based health monitoring.  

\subsection{Complex valued Multi-task learning Convolutional neural network}
\begin{figure*}[!t]
    \centerline{\includegraphics[width=\textwidth]{IMAGE/Figure5.png}}
    \caption{The flow diagram of the CVMT model architecture.}
    \label{fig1}
\end{figure*}

Figure 5 illustrates the network structure of the CVMT model. It consists of two stream modules, each comprising 5 layers of complex-valued convolutional filters. The input signal is transformed into a complex number by assigning the data from two facial regions to the real and imaginary parts, respectively. The features extracted from the two stream modules in Figure 6 are merged using an attention mask. The combined results are then used to predict PPG and RR in the Dense Layer, which is the final layer of the model. In the case of a single task, prediction is made by modifying only one output from the Dense Layer.

Multi-task learning (MTL) is an essential component of our model, enabling the simultaneous estimation of rPPG and respiratory signals. Traditional \textit{hard} MTL methods employ a shared backbone network with task-specific heads, but they often suffer from imbalanced optimization and negative transfer effects when tasks compete for shared resources. To mitigate these issues, we incorporate a \textbf{complex-valued neural network (CV-CNN)}, which enhances information representation by leveraging the intrinsic relationships between physiological signals.

\subsubsection{Differences from Conventional Hard Multi-Task Learning}

Conventional hard MTL assumes that all tasks equally benefit from the shared feature representations. However, when tasks exhibit the different levels of complexity, this approach could lead to the negative transfer, where improvements in one task degrade the performance of the other \cite{b15}. Moreover, balancing the optimization of different tasks is a challenge, as it requires careful tuning of the loss function weights \cite{b16}. However, the proposed complex-valued multi-task learning algorithm differs from the conventional hard MTL in the following aspects:

\begin{enumerate}
    \item \textbf{Complex-Valued Input Representation:} Instead of treating inputs as real-valued tensors, the proposed approach models each input as a component of a complex number \( X_c = X_r + j X_i \), where \( X_r \) and \( X_i \) correspond to the real and imaginary components. This formulation allows the model to learn \textbf{two separate sets of weights for the real and imaginary parts}, enabling richer feature extraction than that of the real-valued tensor.
    \item \textbf{Complex Convolution for Multi-Task Learning:} When performing a convolution on the complex-valued inputs, the output can be expressed as:
    \begin{equation}
        Y_c = (X_r \cdot W_r - X_i \cdot W_i) + j(X_r \cdot W_i + X_i \cdot W_r)
    \end{equation}
    where \( W_r \) and \( W_i \) are the real and imaginary parts of the network’s trained weights. This formulation ensures that each channel captures distinct yet interrelated information, preserving the shared and task-specific components of the physiological signals.
    \item \textbf{Cross-Task Correlation Learning:} The complex-valued representation enables the extraction of both \textbf{amplitude and phase information}, capturing the interdependencies between two physiological compnents. This helps improve feature extraction and reduces the negative transfer between the rPPG and respiratory estimation tasks.
\end{enumerate}

\subsubsection{Loss Function Balance and Task Prioritization}
$추가된 단락$
A fundamental challenge in MTL is the balance of the contributions of different task losses. In the existing studies, both PPG and respiratory signals have been successfully extracted, with observing that the respiratory information is inherently embedded within the PPG signal \cite{revision4}. Their intrinsic relationship suggests that leveraging the shared features between these two physiological signals could enhance the rPPG estimation accuracy. To effectively exploit this interdependence, we integrate the attention module and Multi-Task Learning framework, ensuring that the networks are trained to extract the most relevant features to both rPPG and respiratory information while maintaining the task-specific distinctions. The attention module dynamically assigns higher weights to regions containing enough physiological information, refining the extracted features and reducing interference from irrelevant areas. Meanwhile, the MTL approach optimally shares the representations of both rPPG and respiratory information, preventing redundant computations while each task benefits from the common underlying features. This synergy allows the model to achieve more robust and accurate estimation of both signals, ultimately improving the reliability of the physiological signal monitoring in real-world applications.

Given the two tasks—rPPG (\(\mathcal{L}_{rPPG}\)) and respiration signals estimation (\(\mathcal{L}_{resp}\))—we define the total loss as:

\begin{equation}
\mathcal{L}_{total} = \lambda_1 \mathcal{L}_{rPPG} + \lambda_2 \mathcal{L}_{resp}
\end{equation}
where \(\lambda_1\) and \(\lambda_2\) are task-specific weighting factors. Instead of using fixed values, we dynamically adjust these weights using homoscedastic uncertainty-based weighting \cite{b15}:

\begin{equation}
\mathcal{L}_{total} = \frac{1}{2\sigma_1^2} \mathcal{L}_{rPPG} + \frac{1}{2\sigma_2^2} \mathcal{L}_{resp} + \log \sigma_1 + \log \sigma_2
\end{equation}
where \(\sigma_1\) and \(\sigma_2\) represent the uncertainty of each task. This formulation allows tasks with higher uncertainty to be weighted less, ensuring balanced optimization.

\subsubsection{Prevention of Negative Transfer Between Tasks}

Negative transfer arises when shared features interfere with task-specific learning, leading to degraded performance \cite{b13, b14}. To prevent this issue, we introduce the following strategies:

\begin{itemize}
    \item \textbf{Complex-Valued Weight Sharing:} Unlike the conventional real-valued models, the complex-valued representations enable feature learning that is both distinct for each task yet inherently connected. This reduces task interference while preserving shared knowledge.
    \item \textbf{Gradient Normalization:} We apply GradNorm to normalize the task gradients, preventing one task from dominating the whole learning process.
    \item \textbf{Task-Specific Feature Decoupling:} Figure 1 illustrates our approach, where instead of fully sharing the backbone network, we adopt a partially shared structure. This design ensures that each task retains its independent task-specific features while still leveraging shared representations for common information. With maintaining the separate feature extraction pathways for different tasks, the model effectively balances the shared learning with each task specialization, leading to improved overall performance.

\end{itemize}


\section{Materials and Methods}
\subsection{Dataset}


\subsubsection{COHFACE Dataset}
A total of 40 individuals from the COHFACE dataset were enrolled in this study \cite{cohface}. The COHFACE dataset comprises participants with diverse skin tones, including 31 Caucasians, 7 Asians, and 2 African American individuals. The image data were captured using a Logitech HDC525 camera, and the Biograph Infiniti software from Thought Technologies was used to measure the PPG and respiratory rates. Each video had a duration of one minute with a frame rate of 20 FPS. The regions of the forehead and cheeks were extracted from 160 videos, and then divided into training data (1 to 96), validation data (97 to 127), and test data (128 to 160) for the model training and evaluation \cite{b22}. Additionally, to test the robustness of the proposed algorithm's performance across different skin tones, we evaluated the model's performance using data that included participants with various skin tones. As depicted in Fig. 6, we used two parts on a face as inputs for the model to learn the pixel amplitude differences across time. The rationale behind this approach was that there could be significant time-varying information relating to the blood flow from the heart, as reflected in the differential changes that are observed in these two face regions \cite{b22}.

\begin{figure}[htb!]
\centerline{\includegraphics[width=\columnwidth]{IMAGE/FIGURE_R2_2.PNG}}
\caption{Examples from the COHFACE dataset (top) and the experimental dataset (bottom)}
\label{fig1}
\end{figure}

\subsubsection{Experiment Dataset}

The rPPG signal is highly vulnerable to environmental noise. To address this, we utilized an additional lighting device to acquire video data under varying conditions between 200 and 1200 lux, and the presence or absence of the light flickering.

The 650 Lux environment was set to represent the average illuminance typically found in daily life without additional lighting. Experiments were also conducted in a 200 Lux environment without additional light, and a 1200 Lux environment representing a brighter condition with an additional light in a daily life. Since the fluorescent light causes flickering, which is a significant noise factor affecting the rPPG performance, experiments were conducted in each illuminance environment to compare the presence and absence of the flickering.

Each experimenter collected the facial video data in the same environment using a webcam (Logitech C920) to compare the primary factors affecting rPPG. All subjects had East Asian skin tones, and each recorded video had a duration of 5 minutes with a frame rate of 30 FPS. All subjects gave their informed consent for inclusion before their participation. The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the ethics committee of Kwangwoon University Institutional Review Board (7001546-202400329-HR(SB)-003-01). 

$추가된 섹션$
The experiments of this study utiilzed both the COHFACE and custom experimental datasets. While the COHFACE dataset includes participants with various skin tones and provides a certain degree of diversity, its small scale raises concerns regarding the generalization of the proposed model. To overcome this limitation, We supplemented the COHFACE dataset with our experimental data to address the lack of the Asian participants. Our custom dataset includes a larger sample size, specifically incorporating East Asian skin tones to improve the diversity in the skin tone analysis.

To provide more details on the dataset, Table \ref{table:dataset_info} summarizes the sample sizes and environmental conditions considered in both datasets. The custom dataset consists of 16 participants, where each recorded a 5-minute video at 30 FPS, ensuring a high temporal resolution for the rPPG estimation. Furthermore, the dataset was captured under various environmental conditions, including changes in lighting, motion levels, and background settings, to evaluate the robustness of the proposed method.  

\begin{table}[h]
    \centering
    \caption{Dataset Sample Size and Experimental Conditions}
    \label{table:dataset_info}
    \begin{tabular}{|c|c|c|c|c|}
        \hline
        Dataset & Number of Subjects & Skin Tone & Video Length & FPS \\
        \hline
        COHFACE & 40 & Diverse & 1 min & 20 FPS \\
        Custom Dataset & 16 & East Asian & 5 min & 30 FPS \\
        \hline
    \end{tabular}
\end{table}



Additionally, as shown in Table \ref{table:env_conditions}, the custom dataset includes experiments conducted under various environmental conditions to enhance the model robustness. These conditions cover the various lighting settings (e.g., natural, indoor and flickering lights), motion levels (e.g., static, mild and heavy motions), and background variations (e.g., reflective surfaces and occlusions).  
\begin{table}[h]
    \centering
    \caption{Environmental Variables Considered in the Custom Dataset}
    \label{table:env_conditions}
    \begin{tabular}{|c|c|}
        \hline
        Environmental Variable & Conditions Considered \\
        \hline
        Lighting Conditions & High-Lux, Natural, Low-Lux / Flickering \\
        Motion Artifacts & Static\\
        Background color & White \\
        Camera Settings & 30 FPS without Auto White Balance \\
        \hline
    \end{tabular}
\end{table}



\subsection{Data processing}
The input images were resized to 128 × 128 pixels, and the RGB values were separated before performing the convolution operations \cite{b22}. We obtained the complex inputs by cropping two regions (\textit{x}, \textit{y}) from the input image and putting them in the real and imaginery parts of the complex numbers like Eq.(15). 

As illustrated in Fig. 5, we applied the complex-valued multitask (CVMT) model to predict the rPPG and respiratory rates simultaneously, where the complex-version of the MTL model was employed, thereby leveraging the shared weighted values between the two tasks. The model parameters for the model are listed in Table \uppercase\expandafter{3}. We used a stochastic gradient descent (SGD) optimizer to update the network weights and applied a decay rate to mitigate overfitting \cite{b22}.

\subsection{Software and Packages Used in This Study}

The experiments in this study were conducted using Python 3.9 and various libraries essential for data processing, model training, and evaluation. The specific versions of the software and libraries used are as follows:

\begin{itemize}
    \item \textbf{Python 3.9} (\url{https://www.python.org/})
    \item \textbf{TensorFlow 2.10.0} (\url{https://www.tensorflow.org/}) – Including Keras
    \item \textbf{NumPy 1.21.0} (\url{https://numpy.org/})
    \item \textbf{Pandas 1.3.0} (\url{https://pandas.pydata.org/})
    \item \textbf{Matplotlib 3.4.3} (\url{https://matplotlib.org/})
    \item \textbf{SciPy 1.7.0} (\url{https://scipy.org/})
    \item \textbf{Pillow (PIL) 8.2.0} (\url{https://python-pillow.org/})
    \item \textbf{CVNN (Complex-Valued Neural Networks) 1.0.1} (\url{https://pypi.org/project/cvnn/})
\end{itemize}

These libraries were used for data preprocessing, visualization, and neural network model implementation.



% \begin{equation}
% z = x + \mathit{i}y \left\{\begin{matrix} x, y \in R \end{matrix}\right
% \end{equation}

\begin{equation}
z = x + \mathit{i} y \begin{pmatrix} x, y \in \mathbb{R} \end{pmatrix}
\end{equation}


In this study, we suggest complex inputs by cropping two regions from the input image using complex numbers. Our goal was to predict photoplethysmography (PPG), respiration Signals, and two additional values. The input images were resized to 128x128, and RGB values were separated prior to performing convolution operations. We used the SGD (Stochastic Gradient Descent) optimizer to update the network weights, and applied a decay rate to mitigate overfitting \cite{b22}.

% \begin{table}[htb!]
% \caption{MTL-CVMT MODEL PARAMETERS}
% \begin{center}
% \begin{tabular}{|cc|}
% \hline
% \multicolumn{1}{|c|}{\textbf{Hyperparameter}} & \textbf{Value}  \\ \hline
% \multicolumn{2}{|c|}{{\ul Optimization}}                        \\ \hline
% \multicolumn{1}{|c|}{Learning rate}           & 0.01            \\ \hline
% \multicolumn{1}{|c|}{Number of epochs}        & Until loss 0.01 \\ \hline
% \multicolumn{1}{|c|}{Optimizer}               & Complex SGD     \\ \hline
% \multicolumn{2}{|c|}{{\ul Input}}                               \\ \hline
% \multicolumn{1}{|c|}{Input shape}             & 128*128*3       \\ \hline
% \end{tabular}%
% }
% \end{center}
% \end{table}

\begin{table}[htb!]
\caption{\textbf{MTL-CVMT MODEL PARAMETERS}}
\centering
\begin{tabular}{|l|c|}
\hline
\textbf{Hyperparameter} & \textbf{Value} \\ \hline
\multicolumn{2}{|l|}{\textbf{Optimization}} \\ \hline
Learning rate & 0.01 \\ \hline
Number of epochs & Until loss 0.01 \\ \hline
Optimizer & Complex SGD \\ \hline
\multicolumn{2}{|l|}{\textbf{Input}} \\ \hline
Input shape & 128*128*3 \\ \hline
\end{tabular}
\label{table:mtl_cvmt}
\end{table}


% \begin{table}[htb!]
% \caption{MTL-CVMT MODEL PARAMETERS}
% \centering
% \begin{tabular}{|c|c|}
% \hline
% \multicolumn{1}{|c|}{\textbf{Hyperparameter}} & \multicolumn{1}{c|}{\textbf{Value}} \\ \hline
% \multicolumn{2}{|c|}{Optimization} \\ \hline
% Learning rate & 0.01 \\ \hline
% Number of epochs & Until loss 0.01 \\ \hline
% Optimizer & Complex SGD \\ \hline
% \multicolumn{2}{|c|}{\uline{Input}} \\ \hline
% Input shape & 128*128*3 \\ \hline
% \end{tabular}
% \end{table}


\subsection{Hyperparameter Optimization}
% In this study, we optimized the hyperparameters of the proposed model using the BOHB (Bayesian optimization with HyperBand) algorithm. BOHB combines the advantages of the Bayesian optimization and HyperBand, providing an efficient and robust hyperparameter search method \cite{b29}. Bayesian optimization builds a probabilistic model of the objective function and uses it to select promising hyperparameter configurations, while HyperBand is a bandit-based method that allocates resources (e.g., computational budget) to different configurations based on their performance. By integrating these two approaches, BOHB could balance the exploration of the hyperparameter space with the exploitation of the best configurations found so far. This enables BOHB to efficiently identify the optimal hyperparameter settings that maximize the performance of the model within the given resource constraints \cite{b29}.

The BOHB method combines the advantages of Bayesian optimization (BO) and HyperBand (HB) while addressing their respective drawbacks \cite{b30}. Although BO is better at finding the optimal solution compared with other methods, it has the disadvantages of being time-consuming and not being able to be parallelized. On the other hand, HB has the advantage of being parallelizable and less time-consuming due to its use of random sampling. However, it is not as effective at finding the optimal solution \cite{b30}. Therefore, BOHB leverages the strengths of both algorithms using a surrogate model and an acquisition function to aid in finding the optimal solution, while employing HyperBand for the evaluations. This combination brings together the best aspects of both methods.

In this study, Bayesian Optimization with HyperBand (BOHB) algorithms were used to optimize the hyperparameters of the proposed model. By combining Bayesian optimization with the advantages of hyperbands, BOHB provides an efficient and robust hyperparameter search method \cite{b29}. Bayesian optimization builds probability models of objective functions and uses them to select promising hyperparameter configurations, while hyperbands are bandit-based methods that allocate resources (e.g., computational budgets) to different configurations depending on their performance. By incorporating these two approaches, BOHB can balance the exploration of hyperparameter space with exploiting the best configurations found so far. This allows BOHB to efficiently identify the optimal hyperparameter setting that maximizes the model's performance within a given resource constraint \cite{b29}.

\section{Results and Discussion}

To evaluate the single-task performance of the proposed model, a single-task model that separately predicts rPPG and RR was designed as a complex version and performance analysis was performed. In order to achieve this target, we developed a CV-CNN model that utilizes cheek and cheek images to generate complex inputs representing the real and imaginary parts, and predicts a specific biometric information. Additionally, to compare the single-task performance of the CV-CNN model, we also designed CNN, TS-CAN, and Siamese network models. Subsequently, benchmarking tests were conducted for each task of predicting rPPG and RR. 


For benchmarking the prediction performance, we employed MAE, RMSE, and $R^2$ as evaluation metrics. Table \uppercase\expandafter{4} presents the single-task performance of rPPG and RR. Among all algorithms, CV-CNN achieved the second-best MAE value in the rPPG prediction task and yielded the best RMSE index. Moreover, in the prediction task of RR, it outperformed all other models and exhibited the best overall performance.
Furthermore, when comparing the proposed model with the state-of-the-art models such as TS-CAN and EfficientPhys-C for the rPPG estimation, our model demonstrated superior performance. Specifically, it achieved lower MAE and RMSE values, indicating more accurate and reliable estimations. These results highlight the efficacy of the propoesd approach in improving the accuracy of the rPPG estimation, setting a new benchmark in the field.

% \begin{table}[htb!]
% \centering % 표를 가운데 정렬
% \caption{PERFORMANCE OF SINGLE-TASK MODELS TO ESTIMATE REMOTE PPG AND RESPIRATION RATE}
% \label{table}
% \begin{tabular}{|c|ccc|ccc|}
% \hline
%                 &                             & rPPG                         &                      &                             & RR                          &                      \\ \cline{2-7} 
%                 & \multicolumn{1}{c|}{MAE}    & \multicolumn{1}{c|}{RMSE}   & R^2 & \multicolumn{1}{c|}{MAE}    & \multicolumn{1}{c|}{RMSE}   & R^2 \\ \hline
% CV-CNN          & \multicolumn{1}{c|}{0.8408} & \multicolumn{1}{c|}{0.8412} & 0.3                  & \multicolumn{1}{c|}{0.6771} & \multicolumn{1}{c|}{1.0006} & 0.3                  \\ \hline
% Siamese \cite{b23} & \multicolumn{1}{c|}{0.8807} & \multicolumn{1}{c|}{1.0008} & 0.3                  & \multicolumn{1}{c|}{0.7778} & \multicolumn{1}{c|}{1.0008} & 0.3                  \\ \hline
% CNN \cite{b10}             & \multicolumn{1}{c|}{0.8477} & \multicolumn{1}{c|}{1.1265} & 0.3                  & \multicolumn{1}{c|}{0.7778} & \multicolumn{1}{c|}{1.0008} & 0.3                  \\ \hline
% TS-CAN \cite{b24}          & \multicolumn{1}{c|}{0.8317} & \multicolumn{1}{c|}{1.0083} & 0.3                  & \multicolumn{1}{c|}{0.8652} & \multicolumn{1}{c|}{1.1086} & 0.2                  \\ \hline
% EfficientPhys-C \cite{b32}          & \multicolumn{1}{c|}{0.8490} & \multicolumn{1}{c|}{1.0130} & 0.3                  & \multicolumn{1}{c|}{0.8930} & \multicolumn{1}{c|}{1.2400} & 0.3                 \\ \hline
% \end{tabular}
% \end{table}

\begin{table}[htb!]
\centering % 표를 가운데 정렬
\caption{PERFORMANCE OF SINGLE-TASK MODELS TO ESTIMATE REMOTE PPG AND RESPIRATION Signals}
\label{table}
\begin{tabular}{|c|ccc|ccc|}
\hline
                &                             & rPPG signal                       &                      &                             & Respiration signal                          &                      \\ \cline{2-7} 
                & \multicolumn{1}{c|}{MAE}    & \multicolumn{1}{c|}{RMSE}   & $R^2$ & \multicolumn{1}{c|}{MAE}    & \multicolumn{1}{c|}{RMSE}   & $R^2$ \\ \hline
CV-CNN          & \multicolumn{1}{c|}{0.8408} & \multicolumn{1}{c|}{0.8412} & 0.3                  & \multicolumn{1}{c|}{0.6771} & \multicolumn{1}{c|}{1.0006} & 0.3                  \\ \hline
Siamese \cite{b23} & \multicolumn{1}{c|}{0.8807} & \multicolumn{1}{c|}{1.0008} & 0.3                  & \multicolumn{1}{c|}{0.7778} & \multicolumn{1}{c|}{1.0008} & 0.3                  \\ \hline
CNN \cite{b10}             & \multicolumn{1}{c|}{0.8477} & \multicolumn{1}{c|}{1.1265} & 0.3                  & \multicolumn{1}{c|}{0.7778} & \multicolumn{1}{c|}{1.0008} & 0.3                  \\ \hline
TS-CAN \cite{b24}          & \multicolumn{1}{c|}{0.8317} & \multicolumn{1}{c|}{1.0083} & 0.3                  & \multicolumn{1}{c|}{0.8652} & \multicolumn{1}{c|}{1.1086} & 0.2                  \\ \hline
EfficientPhys-C \cite{b32}          & \multicolumn{1}{c|}{0.8490} & \multicolumn{1}{c|}{1.0130} & 0.3                  & \multicolumn{1}{c|}{0.8930} & \multicolumn{1}{c|}{1.2400} & 0.3                 \\ \hline
\end{tabular}
\end{table}



In this study, we utilized the same open dataset to compare the performance of the proposed model against the conventional rPPG models, including those based on the signal processing approaches. Since the parameters were designed specifically for the existing rPPG signals, the benchmarking test was conducted solely using the rPPG signals. The performance of the benchmarked algorithms, as indicated by the predictive performance of rPPG in Table \uppercase\expandafter{5}, showed that the proposed CVMT exhibited the lowest MAE. Although our proposed Complex valued CNN model also showed a lower MAE value compared with the signal processing-based algorithms, it exhibited a higher MAE value compared to the multi-task learning-based CVMT model. This demonstrates that our proposed model is effective for the rPPG estimation and highlights the effectiveness of the multi-task learning.


% \begin{table}[htb!]
%  \caption{PERFORMANCE OF THE SINGLE TASK MODELS FOR THE ESTIMATION OF REMOTE PPG}
%     \begin{center}
%     \begin{tabular}{|c|ccc|}
%     \hline
%     \multicolumn{1}{|l|}{} & \multicolumn{3}{c|}{rPPG}                                                                       \\ \cline{2-4} 
%     \textbf{}              & \multicolumn{1}{c|}{\textbf{MAE}} & \multicolumn{1}{l|}{RMSE} & \multicolumn{1}{l|}{R^2} \\ \hline
%     Complex valued CNN (our model)    & \multicolumn{1}{r|}{0.8408}       & \multicolumn{1}{r|}{0.8412} & 0.3                           \\ \hline
%     CVMT (our model)       & \multicolumn{1}{r|}{0.6771}       & \multicolumn{1}{r|}{1.0006} & 0.3                           \\ \hline
%     GREEN \cite{b25}                  & \multicolumn{1}{c|}{0.8470}       & \multicolumn{1}{c|}{1.0017} & 0.35                          \\ \hline
%     OMIT \cite{b26}                  & \multicolumn{1}{c|}{0.8460}       & \multicolumn{1}{c|}{1.0004} & 0.27                          \\ \hline
%     LGI  \cite{b27}                  & \multicolumn{1}{c|}{0.8449}       & \multicolumn{1}{c|}{1.0003} & 0.26                          \\ \hline
%     CHROM \cite{b28}                 & \multicolumn{1}{c|}{0.8502}       & \multicolumn{1}{c|}{1.0071} & 0.14                          \\ \hline
%     \end{tabular}
%     \end{center}
%     \label{table:1}
% \end{table}

\begin{table}[htb!]
 \caption{PERFORMANCE OF THE SINGLE TASK MODELS FOR THE ESTIMATION OF REMOTE PPG}
    \begin{center}
    \begin{tabular}{|c|ccc|}
    \hline
    \multicolumn{1}{|l|}{} & \multicolumn{3}{c|}{rPPG signal}                                                                       \\ \cline{2-4} 
    \textbf{}              & \multicolumn{1}{c|}{\textbf{MAE}} & \multicolumn{1}{l|}{RMSE} & \multicolumn{1}{l|}{$R^2$} \\ \hline
    Complex valued CNN (our model)    & \multicolumn{1}{r|}{0.8408}       & \multicolumn{1}{r|}{0.8412} & 0.3                           \\ \hline
    CVMT (our model)       & \multicolumn{1}{r|}{0.6771}       & \multicolumn{1}{r|}{1.0006} & 0.3                           \\ \hline
    GREEN \cite{b25}                  & \multicolumn{1}{c|}{0.8470}       & \multicolumn{1}{c|}{1.0017} & 0.35                          \\ \hline
    OMIT \cite{b26}                  & \multicolumn{1}{c|}{0.8460}       & \multicolumn{1}{c|}{1.0004} & 0.27                          \\ \hline
    LGI  \cite{b27}                  & \multicolumn{1}{c|}{0.8449}       & \multicolumn{1}{c|}{1.0003} & 0.26                          \\ \hline
    CHROM \cite{b28}                 & \multicolumn{1}{c|}{0.8502}       & \multicolumn{1}{c|}{1.0071} & 0.14                          \\ \hline
    \end{tabular}
    \end{center}
    \label{table:1}
\end{table}



Our proposed model, the Complex Valued Multitask Learning model (CVMT), was specifically designed to simultaneously predict rPPG and RR. It was designed as a multi-task model that effectively conducts both tasks. The corresponding performance is shown in Table \uppercase\expandafter{6} with the predictive performance of rPPG and RR using metrics such as MAE, RMSE, and $R^2$.
Additionally, the table includes information regarding the number of parameters for a comparative analysis of model size. Despite the lowest number of parameters, our proposed model exhibited the superior performance on both the rPPG and RR tasks. Notably, it outperformed the MT-TSCAN model, a multi-task learning model designed to reduce the complexity of the existing CNN model, even with a 10\% reduction in parameters. The performance indicators of our proposed model surpassed those of MT-TSCAN, demonstrating its effectiveness and efficiency. 
In addition, we compared our proposed model with the EfficientPhys model, a state-of-the-art rPPG prediction model, in a multi-task prediction setting. While the EfficientPhys model utilized approximately 20\% more parameters, it was not designed with a primary focus on model size reduction. This comparison further underscores the efficiency of our proposed model, which achieved superior predictive performance with the least number of parameters. Our model's ability to deliver high accuracy with minimal computational overhead validates its efficiency and effectiveness, establishing it as a highly efficient solution for rPPG and RR prediction.

Furthermore, the propoesd multitask learning model exhibited smaller MAE values compared with those with the single-task models for both rPPG and RR, as indicated in Table \uppercase\expandafter{4}. This suggests that the multitask learning model efficiently analyzes the two-channel input information as complex numbers, leading to enhanced prediction performance. It could be inferred that the utilization of the complex-number representations significantly contributes to improving the accuracy and effectiveness of the model. Through the multi-task learning model and complex domain process, it becomes feasible to achieve the objective of designing an efficient and lightweight model that could be deployed in resource-limited environments like edge devices. Through this approach, it is possible to implement a long-term health monitoring sensor system. This advancement opens up opportunities for continuous and real-time health monitoring in various settings, thereby contributing to improved healthcare outcomes.

% \begin{table}[htb!]
% \centering % 표를 중앙으로 정렬
% \caption{PERFORMANCE OF MULTI-TASK MODELS FOR THE ESTIMATION OF REMOTE PPG AND RESPIRATION RATE}
% \resizebox{0.8\columnwidth}{!}{% % 표를 컬럼 너비의 80%로 조정
% \label{table}
% \begin{tabular}{|c|ccc|ccc|c|}
% \hline
% \multirow{}{}{} & \multicolumn{3}{c|}{rPPG}                                        & \multicolumn{3}{c|}{RR}                                         &           \\ \cline{2-8} 
%                   & \multicolumn{1}{c|}{MAE}    & \multicolumn{1}{c|}{RMSE}   & R^2  & \multicolumn{1}{c|}{MAE}    & \multicolumn{1}{c|}{RMSE}   & R^2  & Param     \\ \hline
% CVMT              & \multicolumn{1}{c|}{0.6771} & \multicolumn{1}{c|}{1.0006} & 0.3 & \multicolumn{1}{c|}{0.6777} & \multicolumn{1}{c|}{1.0006} & 0.3 & 2,097,862 \\ \hline
% MT-CNN \cite{b15}            & \multicolumn{1}{c|}{0.7778} & \multicolumn{1}{c|}{1.0008} & 0.3 & \multicolumn{1}{c|}{0.7778} & \multicolumn{1}{c|}{1.0008} & 0.3 & 2,296,036 \\ \hline
% MT-EfficientPhys \cite{b32} \cite{b32}            & \multicolumn{1}{c|}{0.7013} & \multicolumn{1}{c|}{1.0015} & 0.3 & \multicolumn{1}{c|}{0.7108} & \multicolumn{1}{c|}{1.0004} & 0.2 & 2,565,026 \\ \hline
% \end{tabular}
% }
% \end{table}

\begin{table}[htb!]
\centering % 표를 중앙으로 정렬
\caption{PERFORMANCE OF MULTI-TASK MODELS FOR THE ESTIMATION OF REMOTE PPG AND RESPIRATION Signal}
\resizebox{0.8\columnwidth}{!}{% 표를 컬럼 너비의 80%로 조정
\begin{tabular}{|c|ccc|ccc|c|}
\hline
\multirow{2}{*}{} & \multicolumn{3}{c|}{rPPG Signal}                                        & \multicolumn{3}{c|}{Respiration signal}                                         &           \\ \cline{2-8} 
                  & \multicolumn{1}{c|}{MAE}    & \multicolumn{1}{c|}{RMSE}   & \multicolumn{1}{c|}{$R^2$}  & \multicolumn{1}{c|}{MAE}    & \multicolumn{1}{c|}{RMSE}   & \multicolumn{1}{c|}{$R^2$}  & Param     \\ \hline
CVMT              & \multicolumn{1}{c|}{0.6771} & \multicolumn{1}{c|}{1.0006} & 0.3 & \multicolumn{1}{c|}{0.6777} & \multicolumn{1}{c|}{1.0006} & 0.3 & 2,097,862 \\ \hline
MT-CNN \cite{b15}            & \multicolumn{1}{c|}{0.7778} & \multicolumn{1}{c|}{1.0008} & 0.3 & \multicolumn{1}{c|}{0.7778} & \multicolumn{1}{c|}{1.0008} & 0.3 & 2,296,036 \\ \hline
MT-EfficientPhys \cite{b32}            & \multicolumn{1}{c|}{0.7013} & \multicolumn{1}{c|}{1.0015} & 0.3 & \multicolumn{1}{c|}{0.7108} & \multicolumn{1}{c|}{1.0004} & 0.2 & 2,565,026 \\ \hline
\end{tabular}
}
\end{table}

To verify that our proposed method has potential applicability and is sufficiently effective in a clinically realistic environment for humans, we conducted an experiment with controlled environmental variables such as illumination and flickering that could occur in real-world settings. Specifically, we examined the effects of lighting conditions and the presence or absence of flickering on model performance. The terms w/f and wo/f in the tables denote with flickering and without flickering, respectively, to reflect these conditions. As shown in Tables \uppercase\expandafter{7} and \uppercase\expandafter{8}, the results consistently demonstrated favorable outcomes at 650 Lux, which was designed to closely mimic everyday lighting conditions. The Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) metrics at 650 Lux showed the best performance, indicating the robustness and effectiveness of the proposed method under realistic circumstances. Our proposed model achieved a Mean Absolute Error (MAE) of 1.3775 under conditions with flickering at 650 Lux. Additionally, we observed the lowest Root Mean Square Error (RMSE) of 0.7937 under conditions without flickering at the same lighting level. Both scenarios at 650 Lux, with and without flickering, showed significantly better results compared to other conditions, demonstrating superior performance and robustness of the model under these realistic lighting settings.

% \begin{table}[htb!]
% \centering % 표를 중앙 정렬
% \caption{PERFORMANCE OF MAE UNDER DIFFERENT LIGHTING AND FLICKERING CONDITIONS}
% \resizebox{0.8\columnwidth}{!}{%
% \begin{tabular}{|c|cc|cc|cc|}
% \hline
% \multirow{}{}{} & \multicolumn{2}{c|}{200 Lux}                              & \multicolumn{2}{c|}{650 Lux}                              & \multicolumn{2}{c|}{1200 Lux}                             \\ \cline{2-7} 
%                   & \multicolumn{1}{c|}{w/f} & wo/f & \multicolumn{1}{c|}{w/f} & wo/f & \multicolumn{1}{c|}{w/f} & wo/f \\ \hline
% CVMT              & \multicolumn{1}{c|}{X}               & 1.5703             & \multicolumn{1}{c|}{1.3775}          & 1.4886             & \multicolumn{1}{c|}{1.2701}          & 1.3923             \\ \hline
% MT-CNN \cite{b15}              & \multicolumn{1}{c|}{X}               & 1.5705             & \multicolumn{1}{c|}{1.8632}          & 1.8300             & \multicolumn{1}{c|}{1.2197}          & 1.8740             \\ \hline
% \end{tabular}%
% }
% \end{table}

\begin{table}[htb!]
\centering % 표를 중앙 정렬
\caption{PERFORMANCE OF MAE UNDER DIFFERENT LIGHTING AND FLICKERING CONDITIONS}
\small % 글씨 크기 줄이기
\resizebox{0.6\columnwidth}{!}{%
\begin{tabular}{|c|cc|cc|cc|}
\hline
\multirow{2}{*}{} & \multicolumn{2}{c|}{200 Lux}   & \multicolumn{2}{c|}{650 Lux}   & \multicolumn{2}{c|}{1200 Lux}  \\ \cline{2-7} 
                  & \multicolumn{1}{c|}{w/f} & wo/f & \multicolumn{1}{c|}{w/f} & wo/f & \multicolumn{1}{c|}{w/f} & wo/f \\ \hline
CVMT              & \multicolumn{1}{c|}{X}   & 1.5703 & \multicolumn{1}{c|}{1.3775} & 1.4886 & \multicolumn{1}{c|}{1.2701} & 1.3923 \\ \hline
MT-CNN \cite{b15} & \multicolumn{1}{c|}{X}   & 1.5705 & \multicolumn{1}{c|}{1.8632} & 1.8300 & \multicolumn{1}{c|}{1.2197} & 1.8740 \\ \hline
\end{tabular}%
}
\end{table}



\begin{table}[htb!]
\caption{PERFORMANCE OF RMSE UNDER DIFFERENT LIGHTING AND FLICKERING CONDITIONS}
\centering
\resizebox{0.6\columnwidth}{!}{%
\begin{tabular}{|c|cc|cc|cc|}
\hline
\multirow{2}{*}{} & \multicolumn{2}{c|}{200 Lux}                              & \multicolumn{2}{c|}{650 Lux}                              & \multicolumn{2}{c|}{1200 Lux}                             \\ \cline{2-7} 
                  & \multicolumn{1}{c|}{w/f} & wo/f & \multicolumn{1}{c|}{w/f} & wo/f & \multicolumn{1}{c|}{w/f} & wo/f \\ \hline
CVMT              & \multicolumn{1}{c|}{X}               & 0.8807             & \multicolumn{1}{c|}{0.7937}          &0.7835             & \multicolumn{1}{c|}{0.8502}          & 0.8915             \\ \hline
MT-CNN \cite{b15}              & \multicolumn{1}{c|}{X}               & 0.8675             & \multicolumn{1}{c|}{0.7976}          & 0.8414             & \multicolumn{1}{c|}{0.8882}          & 0.8898             \\ \hline
\end{tabular}%
}
\end{table}

\begin{table}[htb!]
\caption{Effect of Motion Artifacts on Remote PPG Performance}
\centering
\small
\resizebox{0.4\columnwidth}{!}{%
\begin{tabular}{|c|c|c|c|}
\hline
                 & MAE    & RMSE   & $R^2$ \\ \hline
CVMT             & 0.9967 & 1.0027 & 0.27                 \\ \hline
TS-CAN \cite{b24}           & 1.0044 & 1.1165 & 0.25                 \\ \hline
EfficientPhys-C \cite{b32} & 0.8946 & 1.0031 & 0.27                 \\ \hline
CHROM \cite{b28}            & 0.9993 & 1.0044 & 0.26                 \\ \hline
LGI   \cite{b27}           & 1.0059 & 1.1205 & 0.26                 \\ \hline
\end{tabular}%
}
\end{table}

Table \uppercase\expandafter{9} shows the comparing results of the impact of the motion artifacts on the performance of various algorithms in rPPG measurements. The motion noise assumes small movements in daily life excluding intense movements, which is a  walking condition. All compared algorithms included their respective learning methods for the motion noise issue in rPPG. From the RMSE values in the above table, we can see that the proposed model's value of 1.0027 is the best result compared to other models. Compared to EfficientPhys-C, which has a separate module for motion noise, our proposed method shows similar MAE and slightly better RMSE values. This indicates that our proposed method can effectively learn information for biometric signals even from videos with motion noise.

%The EfficientPhys algorithm, which used a separate module for the motion noise, showed the best results. However, the proposed algorithm outperformed the other deep learning-based methods such as TS-CAN and signal processing algorithms like CHROM and LGI.


\begin{table}[htb!]
\caption{COHFACE Caucasian Group Remote PPG Performance}
\centering
\resizebox{0.6\columnwidth}{!}{%
\begin{tabular}{|c|ccc|ccc|}
\hline
\multirow{2}{*}{} & \multicolumn{3}{c|}{rPPG signal}                                                        & \multicolumn{3}{c|}{Respiration signal}                                                          \\ \cline{2-7} 
                  & \multicolumn{1}{c|}{MAE}    & \multicolumn{1}{c|}{RMSE}   & $R^2$ & \multicolumn{1}{c|}{MAE}    & \multicolumn{1}{c|}{RMSE}   & $R^2$ \\ \hline
CVMT              & \multicolumn{1}{c|}{0.6880} & \multicolumn{1}{c|}{1.0011} & 0.3                  & \multicolumn{1}{c|}{0.6877} & \multicolumn{1}{c|}{1.0010} & 0.3                  \\ \hline
MT-CNN \cite{b15}           & \multicolumn{1}{c|}{0.7668} & \multicolumn{1}{c|}{1.0009} & 0.3                  & \multicolumn{1}{c|}{0.7888} & \multicolumn{1}{c|}{1.0009} & 0.3                  \\ \hline
MT-EfficientPhys \cite{b32}  & \multicolumn{1}{c|}{0.7011} & \multicolumn{1}{c|}{1.0015} & 0.2                  & \multicolumn{1}{c|}{0.7108} & \multicolumn{1}{c|}{1.0004} & 0.2                  \\ \hline
\end{tabular}%
}
\end{table}

\begin{table}[htb!]
\caption{COHFACE Asian Group Remote PPG Performance}
\centering
\resizebox{0.6\columnwidth}{!}{%
\begin{tabular}{|c|ccc|ccc|}
\hline
\multirow{2}{*}{} & \multicolumn{3}{c|}{rPPG signal}                                                        & \multicolumn{3}{c|}{Respiration signal}                                                          \\ \cline{2-7} 
                  & \multicolumn{1}{c|}{MAE}    & \multicolumn{1}{c|}{RMSE}   & $R^2$ & \multicolumn{1}{c|}{MAE}    & \multicolumn{1}{c|}{RMSE}   & $R^2$ \\ \hline
CVMT              & \multicolumn{1}{c|}{0.8577} & \multicolumn{1}{c|}{1.0084} & 0.3                  & \multicolumn{1}{c|}{0.8514} & \multicolumn{1}{c|}{1.0044} & 0.3                  \\ \hline
MT-CNN \cite{b15}           & \multicolumn{1}{c|}{0.8885} & \multicolumn{1}{c|}{1.0361} & 0.3                  & \multicolumn{1}{c|}{0.8638} & \multicolumn{1}{c|}{1.0146} & 0.3                  \\ \hline
MT-EfficientPhys \cite{b32}  & \multicolumn{1}{c|}{0.8687} & \multicolumn{1}{c|}{1.0172} & 0.2                  & \multicolumn{1}{c|}{0.8557} & \multicolumn{1}{c|}{1.0077} & 0.2                  \\ \hline
\end{tabular}%
}
\end{table}

\begin{table}[htb!]
\caption{COHFACE African American Group Remote PPG Performance}
\centering
\resizebox{0.6\columnwidth}{!}{%
\begin{tabular}{|c|ccc|ccc|}
\hline
\multirow{2}{*}{} & \multicolumn{3}{c|}{rPPG signal}                                                        & \multicolumn{3}{c|}{Respiration signal}                                                          \\ \cline{2-7} 
                  & \multicolumn{1}{c|}{MAE}    & \multicolumn{1}{c|}{RMSE}   & $R^2$ & \multicolumn{1}{c|}{MAE}    & \multicolumn{1}{c|}{RMSE}   & $R^2$ \\ \hline
CVMT              & \multicolumn{1}{c|}{0.9270} & \multicolumn{1}{c|}{1.0619} & 0.3                  & \multicolumn{1}{c|}{0.8834} & \multicolumn{1}{c|}{1.0355} & 0.3                  \\ \hline
MT-CNN \cite{b15}           & \multicolumn{1}{c|}{0.9375} & \multicolumn{1}{c|}{1.0909} & 0.3                  & \multicolumn{1}{c|}{0.9280} & \multicolumn{1}{c|}{1.0785} & 0.3                  \\ \hline
MT-EfficientPhys \cite{b32}  & \multicolumn{1}{c|}{0.9287} & \multicolumn{1}{c|}{1.0772} & 0.2                  & \multicolumn{1}{c|}{0.8836} & \multicolumn{1}{c|}{1.0380} & 0.2                  \\ \hline
\end{tabular}%
}
\end{table}


Table \uppercase\expandafter{10} shows the rPPG measurement results for the subjects with light skin tones. This dataset demonstrated almost identical performance compared with the Table \uppercase\expandafter{10, 11}. Our proposed model achieved the best results among the compared ones, with the rPPG's MAE of 0.6880 and the RR's MAE of 0.6877. The estimation of the rPPG measurements for the subjects with light skin tones were highly accurate, showing robustness against lighting and movement noise. These consistent result aligns with the previous results in Table \uppercase\expandafter{6}.
Table \uppercase\expandafter{11} includes the rPPG measurement results for the subjects with medium skin tones from the COHFACE dataset. The proposed model achieved the best results among the compared models with the rPPG's MAE of 0.8577 and the RR's MAE of 0.8514. The proposed algorithm also demonstrated robustness for the medium skin tone subjects, maintaining high prediction accuracy. The experiment using the COHFACE dataset ensures the reliability and accuracy of the rPPG estimation model.
Table \uppercase\expandafter{12} presents the rPPG measurement results for the subjects with dark skin tones. Our proposed model achieved the best results, with the rPPG's MAE of 0.9270 and the RR's MAE of 0.8834, although the performance was slightly worse compared with those of the other skin tones dataset. This issue has been frequently observed in previous studies as well, where the PPG signal reflectance is lower for darker skin tones, leading to degraded signal quality \cite{black1}. These results indicate the need for additional improvements in the estimation algorithms or sensor technology for subjects with dark skin tones.

To demonstrate the model compression and computational efficiency, we conducted the calculation of Floating Point Operations per Second (Flops) and Multiple-Accumulate (MAC) for the estimation of the computational complexities. By employing these metrics, we were able to assess the efficiency of the multitask learning algorithm in terms of the model size reduction and computational performance, providing the evidence of the improved efficiency and effectiveness of our proposed approach. Floating Point Operations per Second (FLOPS) quantifies the number of floating-point operations executed per second in a computer system as an insight of the computational capability \cite{a8}. On the other hand, the multiple-accumulate (MAC) index reflects the computational load involved in multiplying weights with input sums during the neural network learning process and accumulating the results \cite{a9}. It serves as an indicator to evaluate the memory access requirements and computational intensity of the model. These metrics enable us to assess the computational efficiency and resource utilization in a system. 

As can be seen in Table \uppercase\expandafter{13}, the proposed model yields the lowest number of parameters, FLOPs, and MAC compared with the other conventional models, indicating the least amount of computation. Note that frequent memory access could lead to slower processing, and significantly increasing power consumption, impacting the battery life. By minimizing the computation and memory access, the proposed model offers potential advantages in terms of the computational efficiency, reduced power consumption, and improved battery life. Our findings demonstrate the usefulness of the proposed model in real-time processing tasks, such as rPPG and RR prediction, within the edge device environment. This is attributed to not only the small number of parameters but also the fact that our model has the lowest FLOPs and MAC index. By minimizing the computational requirements, the proposed model proves to be advantageous for an efficient and effective real-time processing in resource-constrained edge devices.

$추가된 단락$
To further validate the efficiency of the proposed model, we compare its computational complexity with existing approaches, including the Siamese network, CNN, and EfficientPhys-C. As shown in Table \uppercase\expandafter{13}, the proposed model has a parameter count of 20.09 M, which is lower than the Siamese networks (23.6 M) and CNN (26.6 M), while being slightly higher than EfficientPhys-C (19.14 M). However, our model significantly reduces FLOPs and MAC, indicating superior computational efficiency. 
Our model achieves 8.74 G FLOPs, which is 36\% lower than the Siamese networks (13.68 G) and 52\% lower than CNN (18.3 G). This indicates that our model requires significantly fewer floating-point operations, making it computationally lightweight. The Multiply-Accumulate (MAC) operations in our model are 4.37 G, substantially lower than the Siamese networks (6.84 G) and CNN (9.15 G). Since MAC operations directly impact power consumption, reducing MAC enhances energy efficiency and extends battery life for edge devices.

These results demonstrate that the proposed model maintains high performance while requiring fewer computations, making it well-suited for real-time physiological signal processing tasks in resource-constrained environments. Compared to conventional models, our model's optimized architecture reduces unnecessary operations, leading to faster inference speed and lower energy consumption, which is critical for wearable and edge-device applications.
In summary, our computational complexity analysis confirms that the proposed model achieves a favorable balance between the estimation accuracy and efficiency, ensuring reliable performance while minimizing the resource consumption. This makes it a practical choice for its deployment in the real-world healthcare applications where the real-time signal processing and power efficiency are crucial.


\begin{table}[htb!]
\caption{COMPARISONS OF COMPUTATIONAL COMPLEXITY}
\label{table}
\centering
\resizebox{0.8\columnwidth}{!}{%
\begin{tabular}{|c|c|c|c|c|}
\hline
           & Proposed Model & Siamese network \cite{b14} & CNN \cite{b15} & EffcientPhys-C \cite{b23} \\ \hline
Parameters & 20.09M    & 23.60M &   26.60M    & 19.14M          \\ \hline
FLOPs      & 8.74G     & 13.68G &   18.30G    & 10.89G          \\ \hline
MAC        & 4.37G     & 6.84G  &   9.15G    & 5.45G          \\ \hline
\end{tabular}%
}
\end{table}


Fig. 7 presents the signal waveforms of the predicted rPPG and RR generated by our proposed CVMT model. Notably, the predicted rPPG and respiration waveforms exhibit strikingly similar patterns to the original signals. Specifically, the predicted respiratory rate demonstrates a notably higher concordance with the original signal compared to rPPG. The correlation coefficients of the predicted signals were found to be 0.76 and 0.85 for PPG and RR, indicating significant predictive capability. While the shape and position of the peak point in the PPG waveform might not align perfectly with the original signal, the peak intervals are consistent with those of 'True' signals, resulting in accurate prediction of the heart rate.

\begin{figure}[!t]
\centerline{\includegraphics[width=0.6\columnwidth]{IMAGE/FIGURE67_.png}}
\caption{The predicted PPG and Respiration waveforms are illustrated in the figures, and they show similar patterns. The results indicate that the proposed CVMTS model is able to generate performance similar to that of the Traditional network, but with a smaller number of parameters. This effect is demonstrated in terms of model weight reduction.}
\label{fig1}
\end{figure}

\section{Discussion}
$추가된 내용 있음$
Existing deep learning-based rPPG models primarily rely on real-valued representations, limiting their ability to capture the intricate relationships between signal components. This often results in poor robustness against noise from motion artifacts, ambient lighting changes, and flickering effects. In contrast, our complex-valued neural network effectively utilizes both amplitude and phase information, enhancing noise resilience and improving signal extraction by modeling the inherent interactions between these two components.

The COVID-19 pandemic has further underscored the need for continuous and non-contact biometric monitoring, shifting healthcare paradigms away from traditional hospital-based treatment toward preventive and remote health management. While wearable biometric measurement devices have been widely used, they present challenges such as cost, wearability constraints, and the need for continuous usage conditions. To address these limitations, non-contact bio-signal measurement techniques like remote photoplethysmography (rPPG) have gained significant attention for applications ranging from stress monitoring to drowsy driving detection and infant health tracking. By leveraging rPPG, we aim to provide a more accessible and user-friendly solution for long-term health monitoring.

Experimental validation in challenging real-world conditions
experimental results further validate the advantages of our approach. Under varying lighting conditions and flickering environments, conventional models suffered from performance degradation due to their reliance on the amplitude alone. Our approach, however, demonstrated higher accuracy and stability, leveraging phase information to distinguish between true physiological signals and noise. Additionally, the tests across diverse skin tones showed that the complex representation mitigates bias seen in traditional models, improving generalizability.

Moreover, conventional bio-signal measurement methods typically rely on single-task models, requiring dedicated architectures for each physiological parameter. This results in increased computational costs and inefficiencies in real-world applications. To overcome these limitations, we introduced a multi-task learning framework using complex-valued convolutional neural networks (CV-CNN), which enable simultaneous prediction of multiple bio-signals. Multi-task learning provides several benefits, including robustness, improved generalization, and reduced sensitivity to overfitting. Our approach effectively utilizes the complex domain, treating two images as a single variable, allowing for efficient parameter sharing and faster training.

The results in Table \uppercase\expandafter{4} and \uppercase\expandafter{6} demonstrate that our CV-CNN multi-task model achieves comparable or superior performance to the Siamese Network, which employs weight-sharing mechanisms to learn the similarity and difference between input channels. Similar to weight sharing in the Siamese architecture, the kernel-sharing system in the complex domain contributes to model efficiency. Notably, when both algorithms were tasked with simultaneously predicting multiple physiological signals, our CVMT model outperformed the Siamese network with improved performance while using fewer parameters. This finding highlights the potential of complex-valued multi-task learning models for biomedical applications, particularly in resource-limited environments such as edge devices.

$추가된 섹션$
With integrating the multi-task learning and complex-valued neural networks, this study provides an efficient and scalable approach for the real-time and continuous health monitoring. The ability to reduce the model complexity while maintaining high accuracy is particularly valuable for the long-term health monitoring systems, where the computational efficiency and robustness are essential. Future research would focus on further optimizing this framework for mobile or wearable applications, refining the network architectures to improve the computational efficiency, and exploring its applicability to a wider range of physiological information beyond the heart and respiratory signals.

The proposed method demonstrates superior performance in the rPPG estimation and computational efficiency, several limitations need to be addressed to further improve its robustness and applicability. Our model has been evaluated across the various lighting conditions and different skin tones, demonstrating its robustness against certain external factors. However, real-world applications involve significantly noisier environments, including severe illumination variations, dynamic shadows, and extreme facial occlusions. The generalization ability of the proposed model under such highly unstructured conditions has yet to be fully validated. Future work will apply the adaptive noise filtering methods and self-supervised learning strategies to improve the model generalization in the diverse real-world environments.

Although our experiments utilize both publicly available and proprietary datasets, the diversity of training data is still a limiting factor. Most of existing rPPG datasets are collected in controlled laboratory conditions, which might not fully represent the variations encountered in practical applications. Expanding the dataset to include a broader range of demographic groups, movement conditions, and environmental factors will be essential to ensure that the model could be effectively generalized for the real-world deployment. Future research will explore data augmentation methods, synthetic data generation, and semi-supervised learning approaches to overcome these dataset limitations.

Despite the use of the complex-domain neural networks, which improve the robustness against certain types of noise, the method still exhibits some sensitivity to the rapid head movements and facial deformations. Motion artifacts introduce significant challenges in non-contact physiological signal monitoring, as they could distort the extracted rPPG signals. To mitigate these issues, future work will cover the integration of the optical flow-based motion compensation mechanisms and attention-driven artifact removal modules to enhance the signal stability during the heavy movement scenarios. Additionally, the investigaton of the temporal filtering and recurrent architectures could further improve the model’s resilience to motion noise.

By tackling these challenges, the proposed method would evolve into a highly robust and scalable solution for the real-time rPPG monitoring in dynamic and  real-world environments.


\section{Conclusion}

We have proposed the use of complex-valued neural networks to design an MTL model that can simultaneously estimate the PPG and respiratory rate from facial video streams. We used complex-valued neural networks to learn from two areas of the facial image, specifically the cheek and forehead regions, which were selected as ROIs. By converting the real-valued inputs from the cheek and forehead ROIs into complex-valued data and training them together, we could extract physiological information more efficiently and accurately than when using real-valued approaches. This approach enabled us to design and implement an MTL model that can yield two similar health information types (PPG and respiratory rate) from a single facial image with a reduced number of parameters while improving the performance. 
Despite our efforts to validate the remote PPG algorithm in both controlled environments and realistic environments similar to the daily life conditions, it remains highly sensitive to surrounding conditions. The algorithm is significantly affected by lighting conditions, flickering and motion noise. This sensitivity represents a limitation of this study and suggests a clear direction for future research. Improving the robustness of the algorithm against these factors is essential for its practical application. Future studies should focus on developing techniques to mitigate these environmental influences and enhance the algorithm's performance in real-world scenarios.


\section*{Data availability}
The data used in this study were obtained from two sources. First, the publicly available COHFACE dataset, which can be accessed at https://www.idiap.ch/en/dataset/cohface. Second, additional experimental data collected by the authors are available upon reasonable request. Researchers interested in accessing this dataset may contact the corresponding author [7001546-202301201-HR(SB)-011-03] with a detailed request, outlining the intended use of the data.


\bibliography{sample}

% \noindent LaTeX formats citations and references automatically using the bibliography records in your .bib file, which you can edit via the project menu. Use the cite command for an inline citation, e.g.  \cite{Hao:gidmaps:2014}.

% For data citations of datasets uploaded to e.g. \emph{figshare}, please use the \verb|howpublished| option in the bib entry to specify the platform and the link, as in the \verb|Hao:gidmaps:2014| example in the sample bibliography file.

\section*{Acknowledgements (not compulsory)}

This work was supported by Institute of Information \& communications Technology Planning \& Evaluation (IITP) grant funded by the Korea government(MSIT) (No.RS-2023-00229800, Development of quantitative emotion-sensibility evaluation model of non-face-to-face environment users and commercialization of interactive digital contents application) and This work was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) (No.RS-2023-00244176) and This study was approved by the Institutional Review Board (IRB) of Kwangwoon University (7001546-202301201-HR(SB)-011-03)


% \section*{Author contributions statement}

% Must include all authors, identified by initials, for example:
% A.A. conceived the experiment(s),  A.A. and B.A. conducted the experiment(s), C.A. and D.A. analysed the results.  All authors reviewed the manuscript. 

% \section*{Additional information}

% To include, in this order: \textbf{Accession codes} (where applicable); \textbf{Competing interests} (mandatory statement). 

% The corresponding author is responsible for submitting a \href{http://www.nature.com/srep/policies/index.html#competing}{competing interests statement} on behalf of all authors of the paper. This statement must be included in the submitted article file.

% \begin{figure}[ht]
% \centering
% \includegraphics[width=\linewidth]{stream}
% \caption{Legend (350 words max). Example legend text.}
% \label{fig:stream}
% \end{figure}

% \begin{table}[ht]
% \centering
% \begin{tabular}{|l|l|l|}
% \hline
% Condition & n & p \\
% \hline
% A & 5 & 0.1 \\
% \hline
% B & 10 & 0.01 \\
% \hline
% \end{tabular}
% \caption{\label{tab:example}Legend (350 words max). Example legend text.}
% \end{table}

% Figures and tables can be referenced in LaTeX using the ref command, e.g. Figure \ref{fig:stream} and Table \ref{tab:example}.

\end{document}