Multimodal recommendation, which integrates multi-source information (e.g., item images and text), enriches item representations and enhances the modeling of user preferences, thereby emerging as a key direction for improving rec\-om\-men\-da\-tion performance. Researchers have focused on the effective extraction and fusion mechanisms of multimodal features, which have yielded a series of representative achievements. Some studies embed single-modal features into traditional recommendation frameworks: for instance, VBPR \cite{bib8} embeds visual features into a matrix factorization framework, while ConvMF \cite{bib9} combines convolutional neural networks (CNNs) for text feature extraction with probabilistic matrix factorization. Others achieve fusion based on graph structures: MMGCN bib10, for example, constructs a user-item bipartite graph for each modality, and MEGCF bib11 integrates semantic entities in multimodal data into user-item interaction graphs and uses graph convolutional networks (GCNs) for information propagation. Additionally, some methods achieve fusion through disentangled representation learning or attention mechanisms. For example, DMRL bib12 disentangles the various factors influencing user preferences within each modality and employs a multimodal attention network to model user-specific factor weights across modalities. The aforementioned algorithms perform information fusion from multiple perspectives, improving the recommendation performance of systems to a certain extent. However, in multimodal recommendation, multi-source modal information may contain noise; this noise impairs the accuracy of feature extraction and the effectiveness of fusion, thereby reducing recommendation performance. Consequently, denoising tech\-nol\-o\-gy has become a critical component in enhancing the performance of multimodal recommendation systems.

The problem of data noise remains a critical challenge that constrains system performance. As recommendation scenarios grow more complex and data sources more diverse, effective denoising techniques are essential for enhancing the accuracy and robustness of recommender systems.

Research on denoising has been conducted across various recommendation domains. To address noise in self-attention sequential recommenders, Chen et al. bib13 proposed Rec-Denoiser, which employs differentiable masking and Jacobian regularization to mitigate the impact of noisy items. For session-based sequential recommendation, where noise arises from dynamic user intentions and behavioral uncertainty, Zhang et al. bib14 introduced a Dynamic Intention-Aware Iterative Denoising Network (DIDN) to construct dynamic item embeddings and explicitly filter noisy clicks. Further focusing on session recommendation, Feng et al. bib15 proposed a GNN with Global Noise Filtering (GNN-GNF) framework, which utilizes both item-level and session-level filtering modules to remove irrelevant data while capturing local and global user interests via GNNs. Similarly, Luo et al. bib16 designed a Dual-Perspective Denoising Model (DPDM) to identify noisy items and reconstruct noisy graph structures, addressing the inability of traditional methods to capture true user intention transitions.Recent research has confirmed the advantages of multimodal alignment in MRSs bib17,bib18,bib19. Currently, the mainstream alignment methods in this field mostly adopt item-level alignment based on contrastive learning. For example, SLM-Rec bib17 and MIRCO bib19 achieve their goals by aligning content across different modalities. BM3 bib18 achieves the alignment of behavior representations with multimodal content. They achieve this by bringing closer the representations learned under different perspectives of the same item, while pushing away the representations of different items bib17,bib18,bib19. In the more specific context of multimodal recommendation, which is most relevant to our work, denoising requires handling noise across multiple modalities. Xu et al. bib20 tackled three key issues: multimodal content noise, user feedback noise, and their misalignment. Their DA-MRS system reduces multimodal noise by constructing modality-specific item-item semantic and behavioral graphs. To solve the problems of strong graph structure dependence and excessive interaction noise in multimodal recommendation, Xu et al. bib21 proposed a Collaborative Denoising Graph Contrastive Learning (CDGCL) framework. It uses a modality-aware contrastive learning module to capture inter-modal and intra-modal collaborative relationships, and a multi-strategy denoising module to filter irrelevant interactions, thereby improving recommendation performance. Yuan et al. bib22 proposed a Contrastive Learning-based Cross-Modal Feature Alignment and Fusion Model (CLAM). It uses Item ID embeddings as anchors for indirect contrastive learning to align cross-modal features, and combines multi-task learning to address information loss or noise introduction during feature alignment, as well as insufficient modal fusion caused by over-reliance on interaction data.

While these existing methods have made significant progress in denoising, they often lack targeted mechanisms for handling fine-grained, local noise (e.g., in specific image regions or words), which is crucial for precise cross-modal alignment. To bridge this gap, this study proposes hierarchical alignment: through phrase-image region alignment, it compresses token-level text features and calculates similarity, thereby reducing unnecessary computations and laying the foundation for screening more relevant image regions in subsequent steps. For token-image region alignment, a multi-head attention mechanism is adopted to aggregate image region features, preserving fine-grained semantic interaction information. Additionally, a confidence-weighted contrastive loss module is designed: it calculates the contrastive loss via confidence evaluation and positive-negative sample pairs, and optimizes alignment parameters in a backpropagation manner, ultimately enhancing the discriminative ability of modal features.

The goal of a recommendation system is to provide users with personalized item recommendations. Let denote the user set, and denote the item set. The user-item interaction matrix is , where represents the number of users and represents the number of items. The matrix element indicates whether there is an interaction (such as click, purchase, etc.) between user and item . If there is an interaction, ; otherwise, . The normalized adjacency matrix of the user-item interaction graph is . The multimodal features of each item are extracted through pre-trained models, where represents the feature matrix of text, and represents the feature matrix of images, with being the embedding dimension.

Incorporating multimodal features offers significant advantages for enhancing the performance of recommendation systems, especially when processing different types of information. First, the embedding representations of users and items are updated through propagation via multi-layer graph convolutions:

Among them, , denote the embedding representations of user and item at the -th layer. and represent the neighborhood sets of user and item , respectively. is a symmetric normalization term.

For each word in a sentence, we use Transformer bib23,bib24 to extract Token features and obtain the text Token embedding matrix , , where is the feature vector of the -th Token. For the input image, non-overlapping patches of pixels are adopted to generate 256 image Patches, and each Patch serves as an independent visual unit. Then, the pre-trained ViT-B/16 is used to extract Patch features, and the image Patch embedding matrix is obtained, where is the feature vector of the -th Patch, and is the embedding dimension. Then, for the text and image feature maps, KNN adjacency matrices and are constructed respectively, and features are extracted via graph convolution:

Where and denote the feature matrices after graph convolution, is the ReLU activation function, is a diagonal matrix, and is a learnable weight matrix used to map features from the -th layer to the -th layer.

Then, self-attention calculation is performed on the high-level text features extracted via graph convolution to capture the dependency relationships between feature vectors. The specific formula is as follows:

Where , all from the features of the same modality. Then, the residual coefficient is used to fuse the original features and the self-attention output to perform preliminary denoising on text features:

where is a trainable parameter. This method not only retains the valid information in the original features but also combines the noise suppression effect of the self-attention mechanism. Similarly, the preliminarily denoised image features can be obtained.

Most existing studies on multimodal denoising recommendation are based on cross-modal attention mechanisms. Current methods achieve alignment via full attention, which leads to high computational complexity and overlooks the hierarchical nature of semantics. In this section, hierarchical cross-modal alignment is adopted to accurately match the semantics between different modalities, thereby reducing the noise interference caused by insufficient modal alignment.

In multimodal denoising recommendation, traditional con\-tras\-tive learning exhibits insufficient discriminative learning for denoised features, making it difficult to distinguish noise residues from valid semantics. How to design a denoising-oriented contrastive learning strategy to enhance the discrim\-in\-abil\-ity of modal features has become a key factor for the model to more accurately capture user preferences in noisy scenarios. As shown in Figure 2, this paper adopts a confidence-weighted contrastive loss to optimize features and improve their discriminability.

The aligned text features and image features are fused. Weights are assigned to the text features and image features respectively, and then the fused features are obtained by weighted summation. The specific formula is as follows:

where represents the weight, obtained through model training.

Finally, the predicted rating of the user for the item is predicted by the inner product of the fused features and the user embedding, as follows:

Experiments were conducted on three commonly used Amazon datasets: Baby, Sports, and Clothing. These datasets belong to different categories, and significant differences may exist in product forms and user preferences, this helps test the robustness and generalization ability of the model under various scenarios. Composed of visual and textual features, these datasets can better demonstrate the model's capability in multimodality. In the experiments, the datasets were split into training, validation, and test sets at a ratio of 8:1:1. Detailed statistical information of the three datasets is presented in Table 1.

begin{table}[h]\centering\caption{The Statistics of Datasets}\label{tab:dataset_stats}\begin{tabular}{|c|c|c|c|c|}\hlineDataset & Users & Items & Interactions & Sparsity \\\hlineBaby & 19445 & 7050 & 139110 & 99.88% \\\hlineSports & 35598 & 18357 & 256308 & 99.95% \\\hlineClothing & 39387 & 23033 & 237488 & 99.97% \\\hline\end{tabular}\end{table}

We adopted three widely used evaluation metrics: Recall@K, Precision@K and Normalized Discounted Cumulative Gain (NDCG@K). K is set to 20. Recall@K represents the percentage of a user's rated items that appear in the top K recommended items. Precision@K measures how many of the top K recommended items are actually items that users are interested in, and it is used to demonstrate the accuracy of recommendations. NDCG@K is an evaluation metric for ranking results and is used to assess the accuracy of ranking.

To evaluate the performance of MDR-CHCA, we conducted a comparative analysis with eight advanced recommendation models.

The operating system, GPU, and experimental environment used for the model and algorithms in this study are as follows: the operating system is Windows 10, the GPU is an Nvidia RTX 3090, and the experimental environment is built based on Python 3.8 and PyTorch 1.12.0. After repeated experiments, the hyperparameters used in the model are set as follows: the Top- (number of retained image regions per phrase) is set to 10; the Top- (number of retained fine-grained connections per word) is set to 5; the temperature hyperparameter is set to 0.5; the Xavier method is adopted for parameter initialization; the optimizer is Adam with a learning rate of 0.001; the embedding size is set to 64; and the batch size for dataset training is set to 2048.

begin{table}[htbp] \centering \setlength{\tabcolsep}{2pt} \caption{The performance of different methods}\label{tab:performance} \begin{tabular}{l *{9}{c}} \hline \multirow{2}{*}{Model} & \multicolumn{3}{c}{Baby} & \multicolumn{3}{c}{Sports} & \multicolumn{3}{c}{Clothing} \\ \cmidrule(lr){2-4} \cmidrule(lr){5-7} \cmidrule(lr){8-10} & R@20 & NDCG@20 & P@20 & R@20 & NDCG@20 & P@20 & R@20 & NDCG@20 & P@20 \\ \hline VBPR & 0.0592 & 0.0234 & 0.0032 & 0.0719 & 0.0325 & 0.0039 & 0.0433 & 0.0185 & 0.0022 \\ MMGCN & 0.0622 & 0.0270 & 0.0033 & 0.0634 & 0.0270 & 0.0035 & 0.0426 & 0.0184 & 0.0021 \\ LATTICE & 0.0837 & 0.0367 & 0.0046 & 0.0928 & 0.0419 & 0.0050 & 0.0710 & 0.0316 & 0.0036 \\ GRCN & 0.0799 & 0.0348 & 0.0044 & 0.0856 & 0.0387 & 0.0047 & 0.0650 & 0.0283 & 0.0034 \\ SLMRec & 0.0878 & 0.0393 & 0.0047 & 0.1042 & 0.0468 & 0.0057 & 0.0728 & 0.0325 & 0.0038 \\ BM3 & 0.0864 & 0.0378 & 0.0047 & 0.0980 & 0.0442 & 0.0054 & 0.0662 & 0.0304 & 0.0035 \\ MICRO & 0.0882 & 0.0398 & 0.0047 & 0.0997 & 0.0462 & 0.0052 & 0.0803 & 0.0361 & 0.0041 \\ CDGCL & 0.0946 & 0.0409 & 0.0050 & 0.1061 & 0.0483 & 0.0053 & 0.0853 & 0.0392 & 0.0044 \\ MDR-CHCA & 0.0970 & 0.0422 & 0.0053 & 0.1093 & 0.0491 & 0.0055 & 0.0908 & 0.0413 & 0.0047 \\ Improve% & 2.5% & 3.2% & 6.0% & 3.0% & 1.6% & 3.8% & 6.4% & 5.4% & 6.4% \\ \hline \end{tabular}\end{table}

The experimental results of comparing our method with advanced baseline models are presented in the table. As observed from Table 2, the following conclusions can be drawn:

Compared to baseline models, our proposed method achieves superior performance across all metrics on three datasets. Specifically, relative to the current strongest baseline model CDGCL, MDR-CHCA achieves average gains of 3.9%, 2.8%, and 6.1% on the Baby, Sports, and Clothing datasets, respectively. This demonstrates that MDR-CHCA effectively captures user preferences to deliver more accurate recommendation results.In-depth analysis of baseline models reveals that traditional multimodal recommendation models like VBPR and MMGCN exhibit limited performance due to the absence of effective denoising mechanisms. SLM-Rec and BM3 enhance model robustness to some extent through self-supervised learning techniques, but they primarily focus on item-level representation learning and fail to effectively address fine-grained cross-modal semantic alignment challenges. MICRO achieves significant performance gains by constructing a multimodal latent item graph and conducting contrastive learning across dual spaces, establishing itself as a strong baseline. However, MICRO still exhibits limitations in fine-grained semantic matching. CDGCL, an advanced model specifically designed for multimodal denoising, employs modality-aware contrastive learning and multi-strategy denoising modules to achieve outstanding performance across three datasets, positioning it as the current state-of-the-art baseline. However, CDGCL primarily focuses on denoising at the graph structural level, failing to fully exploit the hierarchical semantic information within each modality. In contrast, MDR-CHCA emphasizes cross-modal semantic alignment and intra-modal feature optimization. By adopting a hierarchical approach, it enables fine-grained matching of data across different modalities. Compared to CDGCL, MDR-CHCA maintains robust denoising capabilities while achieving consistent improvements across all metrics through its refined semantic alignment mechanism. This enhancement is particularly pronounced on the semantically complex Clothing dataset. Furthermore, traditional models' contrastive losses (e.g., VBPR's Bayesian loss, MMGCN's standard contrastive loss) uniformly weight positive and negative samples, struggling to distinguish high-quality samples from noise interference. MDR-CHCA, however, assigns greater weight to high-confidence positive samples, enabling the model to focus on user preference information embedded in high-quality samples. Simultaneously, it performs targeted optimization on negative samples by filtering out high-noise negatives through confidence assessment. This addresses the issue of baseline models failing to strengthen high-quality samples due to uniform sampling. These innovations drive MDR-CHCA's progress in multimodal recommendation prediction.

Translated with DeepL.com (free version)

This paper conduct an in-depth study on the key components of the model by comparing it with two variants, aiming to examine the impact of the main components on the final performance. The two variants are defined as follows:

From the results in Figure 3, the following conclusions can be drawn: Both modules have a certain impact on the model. After removing the hierarchical alignment module, the model performance decreases significantly, which indicates that hierarchical alignment can significantly improve the efficiency of semantic matching and suppress noise interference. As observed from MDR-CHCA-B, screening high-quality samples through dynamic thresholds can effectively enhance feature discriminability in noisy scenarios. Overall, the three metrics of the MDR-CHCA-A model are significantly lower than those of MDR-CHCA-B, which demonstrates that the hierarchical cross-modal alignment module has a more significant impact on the model performance.

In this study, we focus on investigating the impact of hyperparameters on the performance of the recommendation model, including (the number of retained regions for phrase-image region alignment) and the temperature parameter .

To comprehensively evaluate the effects of and , we adjusted the values of these two hyperparameters while keeping other settings unchanged. By plotting the scores of Recall@20 and NDCG@20 on the datasets, we could clearly observe their specific impacts on model performance.

Figure 4 presents the results of adjusting the number of retained regions on the three datasets, where was adjusted within the range of [5, 10, 15, 20]. We found that: when , the model achieved very poor results, which may be attributed to insufficient semantic coverage leading to performance degradation; when the model reached the optimal performance; thereafter, as further increased, the model performance decreased instead. This is because an excessively large causes the model to retain more low-quality regions, thereby affecting cross-modal alignment.

Furthermore, we conducted a detailed investigation on the temperature parameter , covering multiple values of [0.1, 0.3, 0.5, 0.7]. As can be clearly observed from Figure 5, the model achieved the best performance when was set to 0.5. Deviation from this value resulted in a decline in model performance, which proves that , as a temperature parameter, requires precise tuning to balance the "discriminability" and "generalization" of the confidence distribution.