MSAFNet: a novel approach to facial expression recognition in embodied AI systems

2.1. FER based on traditional approaches

Early FER works mainly on hand-crafted features and traditional machine-learning classification methods. The hand-crafted features can be divided into appearance-based features and geometric features. The commonly used appearance-based features include LBP ^[17], Gabor wavelets ^[5], and histogram of oriented gradients (HOG) ^[18]. Geometric features are obtained by measuring the relative position of significant features, such as eyes, nose, and mouth ^[19], ^[20]. Moreover, support vector machine (SVM) ^[21] is the most common and effective machine learning classification algorithm. Ghimire et al. proposed a FER method using a combination of appearance and geometric features with SVM classification^[21]. Although the traditional methods have a good performance on in-the-lab FER datasets, the performance on in-the-wild FER datasets is significantly degraded. It can be ascribed that lighting, noise, and other factors can easily affect hand-crafted features. Moreover, the method provides better results in facial expression datasets.

2.2. FER based on CNN models

Compared with traditional machine learning methods, deep neural networks, especially CNN, can learn directly from the input reducing the dependence on pre-processing. With the rapid development of deep learning, many deep neural networks such as AlexNet ^[22], VGG ^[23], and ResNet ^[24] are widely used in FER tasks and have shown good performance. Wu et al. proposed FER-CHC with cross-hierarchy contrast to enhance CNN-based models by critical feature exploitation^[25]. Teng et al. designed typical facial expression network (TFEN) combining dual 2D/3D CNNs for robust video FER across four benchmarks^[26]. Zhao et al. developed a cross-modality attention CNN (CM-CNN) that fused grayscale, LBP, and depth features via hierarchical attention mechanisms, effectively addressing illumination/pose variations and enhancing recognition of subtle expressions^[27]. Cai et al. introduced probabilistic attribute tree CNN (PAT-CNN) addressing identity-induced intra-class variations through probabilistic attribute modeling^[28]. Liu et al. combined CNN-extracted facial features with GCN-modeled high-aggregation subgraphs (HASs) to boost recognition robustness^[29].

2.3. FER based on attention mechanism

Attention mechanism has been widely applied in FER tasks out of their effectiveness in focusing the network on useful regions relevant to expression recognition. Zhang et al. proposed a cross-fusion dual-attention network with three innovations: grouped dual-attention for multi-scale refinement, adaptive C2 activation mitigating computational bottlenecks, and distillation-residual closed-loop framework enhancing feature purity^[30]. Li et al. developed SPWFA-SE combining Slide-Patch/Whole-Face attention with SE blocks to jointly capture local details and global contexts, improving FER accuracy^[31]. Zhang et al. designed lightweight GSDNet using gradual self-distillation for inter-layer knowledge transfer and ACAM with learnable coefficients for adaptive enhancement^[32]. Tao et al. introduced a hierarchical attention network integrating local-global gradient features via multi-context aggregation, employing attention gates to amplify discriminative regions^[14]. Chen et al. introduced a hierarchical attention network integrating local-global gradient features via multi-context aggregation, employing attention gates to amplify discriminative regions^[33].

2.4. FER based on visual transformer

Transformers ^[34] have been widely used in natural language processing (NLP) tasks and have shown significant performance. They are good at capturing the long-distance relation between words by their self-attention mechanism. Inspired by the success of transformers, Dosovitsliy et al. proposed Vit^[35], a pure transformer, applied to image patches on classification tasks and has shown significant performance in the field of computing vision, such as object detection ^[36], object tracking ^[37], and instance segmentation ^[38]. Visual transformers are also applied to FER by some researchers. Ma et al. introduced a transformer-augmented network (TAN) combining intra-patch transformers (position-disentangled attention) and inter-patch transformers to capture local features and cross-region dependencies, integrated with online label correction for noise reduction^[39]. Zhang et al. developed transformer-based multimodal emotional perception (T-MEP) with triple transformers for audio/image/text features, aligning multimodal semantics through fused self/cross-attention in visual latent space^[40]. Liu et al. proposed patch attention convolutional vision transformer (PACVT) that also extracts local and global features, but uses simple add operation to combine different features^[41]. Different from them, our method can obtain rich emotional information from local and global features without a cropping strategy, and employs a learnable way to integrate the relation of local and global features. It is effective to recognize facial expression images.

3.1. Overview

As shown in Figure 2, our proposed MSAFNet consists of three components, including the LFEM, the GFEM, and the GLFM. The LFEM introduces a CNN (ResNet) as the backbone to extract local features. Specifically, an original facial expression image is fed into the LFEM and the GFEM as input. Concurrently, a MSA block is embedded into the local module to direct the model to focus more on regions that are crucial for expression recognition. The GFEM also converts the original facial image to different tokens with positional information by a linear embedding layer. Then the tokens are fed to the transformer encoder, which can extract global features of the original image. Finally, the GLFM compatibly models the relationship between local and global features, and the output features are utilized for FER.

Figure 2. The overview of our proposed method for FER. The proposed method consists of three components, including the LFEM, the GFEM and GLFM. The images and labels are from RAF-DB. FER: Facial expression recognition; LFEM: local feature extraction module; GFEM: global feature extraction module; GLFM: global-local feature fusion module.

3.2. LFEM

Inspired by Res2Net ^[42], we design a MSA block. In this module, we manipulate ResNet18 ^[24] as the backbone from given facial images for the LFEM, and the average pool and fully connected layer are removed after the Conv5 block. To optimize the salient information in local regions, this module designs a MSA block which is embedded after the Conv5 block.

As shown in Figure 3, the feature maps extracted after the Conv5 block are fed into the attention block. After a $$1\times 1$$ convolution, the feature maps $$X1\in R^{C1\times H1\times W1}$$ are obtained, where $$C1$$, $$H1$$, and $$W1$$ represent the number of channels, width, and height of the feature map after the convolution operation, respectively. The module splits the feature maps $$X1$$ into $$s$$ groups feature map subsets, denoted by $$x_i$$, where $$i\in \{0,1,2,\cdots ,s-1\}$$. The spatial size of each group feature map subset is the same as the input feature maps $$X$$, while the number of channels is $$C{1}^{{}^{\prime }}=C1/s$$. The $$i-th$$ group feature map subset import $$x_i\in R^{C{1}^{{}^{\prime }}\times H1\times W1}$$, $$i\in \{0,1,2,\cdots ,s-1\}$$. Each $$x_i$$ is processed by a corresponding $$3\times 3$$ convolution designated by $$f_i(\cdot )$$, and the output is denoted by $$y_i\in R^{C{1}^{{}^{\prime }}\times H1\times W1}$$. According to the module, the input and output have the same dimension. When $$i\ge 1$$, the $$i-th$$ group feature subset is computed with the output of $$y_{i-1}$$ and then fed as the input of $$f_i(\cdot )$$. Thus, $$y_i$$ can be written as:

Figure 3. The details of MSA block. MSA: Multi-scale attention; GAP: global average polling; FC: fully connected layer; Sig: sigmoid function.

When each group feature subset $$\{x_i,0\le i\le s-1\}$$ goes through $$3\times 3$$ convolution, the output $$\{y_i,1\le i\le s-1\}$$ can acquire a larger receptive field than $$\{y_i,j\le i\}$$. After that, each $$y_i$$ can contain characteristic information of feature subsets with different receptive field scales and different scales, thus obtaining multi-scale spatial information. Different sizes of $$s$$ can learn different information, and larger $$s$$ may get richer scale information. This module sets the size of $$s$$ as 4, which was carefully chosen through comprehensive ablation studies to balance computational complexity and model performance. This systematic analysis justifies our design choice of $$s=4$$ as the optimal configuration that achieves superior accuracy while maintaining reasonable computational demands.

Following the multi-scale spatial attention information, we subsequently compute attention weights along channel dimensions. By using global average pooling, each output $$y_i$$ from a convolution operation for each group feature subset $$x_i$$ is condensed into a vector. Then, we employ two fully connected layers to model the channel correlations. In neural networks, activation functions are primarily used to introduce non-linearity, enabling the network to learn complex patterns. Therefore, we use a sigmoid activation function to normalize the output, which can obtain the channel attention weight of each group feature subset $$F_i\in R^{C{1}^{{}^{\prime }}\times 1\times 1}$$. It can be defined as:

where $$\sigma$$ denotes the sigmoid function, which normalizes the output as a range of [0, 1], effectively transforming the output into a probability value. Additionally, $$\rho$$ denotes the ReLU activation function, a typical non-linear function, defined as $f(x)=max(0,x)$, which maps the input signal to the feature space; $$W_1$$ and $$W_2$$ denote the FC operation; $$F_i$$ denotes the channel attention weight of different group feature subsets. Furthermore, it splices the attention weights to acquire the final MSA weights $$F\in R^{C1\times 1\times 1}$$:

Finally, we acquire the output $$X_{local}$$ by multiplying feature maps $$X$$ with the MSA weights $$F$$.

Different from Res2Net, our MSA not only captures the multi-scale information from feature map subsets, but also calculates channel information and aggregates all information from feature map subsets, which can make the attention information richer. It considers both spatial semantic information and channel semantic information and effectively combines information from both spatial and channel dimensions. This design, leveraging self-attention, reduces redundancy, accelerates training, and improves convergence. By emphasizing comprehensive feature fusion and diverse interactions, our MSA ensures more efficient information flow, better addressing gradient vanishing and enhancing training stability in deep architectures.

3.3. GFEM

Given the significant performance of vision transformers (ViT) ^[35] in numerous computer vision tasks, several transformer architectures have been widely adopted. Transformer architecture is good at capturing the long-distance dependencies between pixels. Therefore, the GFEM utilizes a transformer architecture following Mixformer ^[43]. As shown in Figure 2, the GFEM combines local-window self-attention (W-MSA) with depth-wise convolution in a parallel design, providing complementary information for each branch. The ViT branch employs W-MSA to model global facial semantics: non-overlapping patches undergo dynamic correlation analysis. Channel interaction (CLout) enhances critical regions via element-wise recalibration. The CNN branch extracts localized details via depthwise convolution, with spatial interaction (SLout) amplifying discriminative regions. Fused features combine both streams channel-wise, processed by the MIX function for robust classification. This dual-stream design enables complementary modeling: W-MSA captures spatial dependencies, while CNN optimizes channel-wise features, achieving multi-scale representation through hierarchical fusion. The channel interaction contains a global average pooling layer and two 1$$\times$$1 convolution layers with BN layer and GELU activation function. And a sigmoid function is used to generate attention. At last, the channel interaction is applied to the value in W-MSA. The spatial interaction involves two 1×1 convolution layers with BN and GELU, followed by a sigmoid function used to generate attention.

Where $$C{I}_{in}$$, $$C{I}_{out}$$ represent the input and output of channel interaction; $$S{I}_{in}$$, $$S{I}_{out}$$ represent the input and output of spatial interaction; $$GELU$$ denotes activation function; $$\sigma$$ denotes the sigmoid function.

Based on the parallel design, the mix transformer block can be formulated as follows:

where $$CONCAT$$ represents a function that mixes the feature between W-MSA and depth-wise convolution. $$Conv$$ denotes depth-wise convolution; $$LN$$ represents layer normalization; $${\hat{z}}^{l+1}$$ and $$z^{l+1}$$ denote the output features of the $$CONCAT$$ and the $$MLP$$, respectively.

3.4. GLFM

In this field, GLFM effectively integrates features from the LFEM and the GFEM. The primary objective is to combine local and global features to capture richer spatial and channel information, thereby enhancing FER performance. Compared with traditional neural networks that separately process local and global features, our model enables the complementarity between local and global features. As shown in Figure 4, this approach allows for a more accurate description of both the details and overall characteristics of facial expressions. Given local feature maps $$X_{local}\in R^{C1\times H1\times W1}$$ extraction from the LFEM, where $$C1,H1,W1$$ are the channel dimension, height and width, and global feature maps $$X_{global}\in R^{C2\times L2}$$ extracted from the GFEM, where $$C2,D2$$ are channel dimensions and token numbers. The local feature maps $$X_{local}\in R^{C1\times H1\times W1}$$ are rearrange to $$X_{local}\in R^{C1\times L1}$$ where $$L1=H1\times W1$$. This rearrangement of local feature maps is an essential step in the process. By reshaping the feature maps, local and global information can be effectively fused within the same dimensional space, thereby optimizing subsequent operations. This paper presents the first method to fuse local feature maps $$X_{local}$$ and global feature maps $$X_{global}$$ via an element-wise summation for information interaction:

Figure 4. The details of GLFM. MLP denotes multilayer perceptron, $$\oplus$$ denotes element-wise summation, and $$\otimes$$ denotes element-wise multiplication. GLFM: Global-local feature fusion module.

Where $$X_{fusion}\in R^{C3\times L3}$$ is the interactive features, and $$\oplus$$ denotes element-wise summation. For purpose of performing information interaction better, the interaction features $$X_{fusion}$$ are transposed into $$x_{fusion}^T\in R^{C3\times L3}$$ and fed into MLP block including two fully connected layers and a non-linearity layer that can c the information on the token level. Employing this method can acquire the feature $$x_{fusion1}^T\in R^{C3\times L3}$$ and transpose it into $$X_{fusion1}\in R^{C3\times L3}$$. And then the features $$X_{fusion1}$$ are fed into two different MLP blocks to interact the information on the channel level. The details can be defined as follows:

where $$LN$$ denotes layer normalization, which is the process of normalizing the output of a specific layer in a neural network; it helps maintain stability during training by preventing difficulties caused by large differences in the outputs of different layers. This can reduce training time, improve stability, and enhance convergence. Additionally, $$GELU$$ denotes the GELU activation function, which is similar to ReLU but with smoother output for small negative values. It helps mitigate issues such as gradient explosion or vanishing gradients during training and is commonly used in deep learning to transform the model's output. $$W_3,W_4,W_5,W_6,W_7$$ and $$W_8$$ denote the fully connected operation. And we can get the last fusion features as:

where $$\otimes$$ denotes element-wise multiplication, and $$\oplus$$ denotes element-wise summation.

4.1. Datasets

RAF-DB^[7] is a real-world FER dataset containing facial images downloaded from the Internet. The facial images are labeled with seven classes of basic expressions or 12 classes of compound expressions by 40 trained human coders. In our experiment, the proposed method only utilizes seven basic expressions, including anger, disgust, fear, happiness, sadness, surprise, and neutral. It involves 12, 271 images for training and 3, 069 images for testing.

FER2013^[44] is collected from the Internet that was first for ICML 2013 Challenges in representation learning. It contains 385, 887 facial images collected by Google search engine with 28, 709 images for training, 3, 589 images for validating, and 3, 589 images for testing. All images are grayscale with the size of $$48\times 48$$ pixels. The facial images are also labeled with seven basic expression labels, including anger, disgust, fear, happiness, sadness, surprise, and neutral.

FERPlus^[9] is extended from FER2013 dataset which was for ICML 2013 Challenges. It also contains 28, 709 training, 3, 589 validation, and 3, 589 testing images. All images are $$48\times 48$$ pixels. Different from FER2013 dataset, each image in FERPlus is relabeled by ten annotators and FERPlus adds three categories, including contempt, unknown, and not a face. Therefore, FERPlus dataset has better quality labels than FER2013. The proposed algorithm reports overall sample accuracy on the test set with eight categories.

Occlusion and pose variant datasets are test subsets of RAF-DB and FERPlus collected by the work ^[45]. These datasets are Occlusion-RAF-DB, Pose-RAF-DB, Occlusion-FERPPlus, and Pose-FERPlus, and the facial expression images are manually annotated with various occlusion and variant poses. The pose datasets can be divided into two types with poses larger than 30 degrees and larger than 45 degrees.

4.2. Implementation details

In the experiments, the multitask cascaded convolutional network (MTCNN) ^[46] is employed for face detection in images. The detected faces were subsequently cropped and further downsized to 256$$\times$$256 pixels. MTCNN was chosen due to its superior accuracy and speed. Its cascaded network structure enables high-precision detection and effectively handles challenging scenarios, including varying illumination, pose variations, and occlusions. Furthermore, the multitask learning framework of MTCNN allows for simultaneous face localization and keypoint detection, thereby enhancing the efficiency of subsequent processing tasks. To prevent over-fitting, we randomly cropped the facial photos to 224$$\times$$224 pixels during training. We also randomly flip the data during data augmentation. The ResNet18 is pre-trained on MS-Celeb-1M face recognition dataset ^[47] similarly to region attention networks (RAN) ^[45] and self-cure networks (SCN) ^[48]. In the experiments, the model is trained with stochastic gradient descent (SGD) optimizer algorithm. The momentum is 0.9, the weight decay is 0.0005 and the batch size is set to 128. The learning rate is initialized as 0.01 and it decays to 0.9 every five epochs after 30 epochs. And we train the model for 150 epochs in total. Our method is implemented with Pytorch toolbox on GeForce RTX 2080Ti GPU platform.

4.3. Comparison with state-of-the-arts

This section compares the proposed approach MSAFNet with several state-of-the-art methods on RAF-DB, FERPlus, FER2013, Occlusion-RAF-DB, Pose-RAF-DB, Occlusion-FERPlus, and Pose-FERPlus. MSAFNet consistently achieves high accuracy and demonstrates stable performance across these benchmarks. Notably, it exhibits strong generalization capabilities, particularly in complex scenarios involving diverse facial expressions and emotion categories.

4.3.1. Results on RAF-DB

Comparison results with other state-of-the-art methods on RAF-DB in recent years with seven emotion categories are shown in Table 1. Multi-scale and local attention network (MA-Net) ^[52] utilized global and local features to address the issues both occlusion and pose variation and got an accuracy of 88.40%. Adaptive multilayer perceptual attention network (AMP-Net) ^[15] uses different fine-grained features to extract global, local and salient features and obtained recognition accuracy of 89.25% on RAF-DB dataset. As shown in Table 1, our proposed method MSAFNet obtains the recognition accuracy of 89.77% on RAF-DB and achieves 1.66% and 0.81% improvement compared with the MA-Net ^[52] and AMP-Net ^[15], respectively. Compared to the visual transformers with feature fusion (VTFF) ^[54] which used transformers and attention selective fusion, our method has 1.92% improvement. PACVT ^[41] can also extract local and global features with attention weights and ViT. Compared with PACVT, our method achieves a higher accuracy by about 1.85%. In the confusion matrix shown in Figure 5, happiness expression has the highest recognition accuracy, while fear and disgust expression has poor performance. This disparity primarily arises from three factors: data scarcity, inter-class similarity and feature subtlety.

Table 1

Comparison with other methods on RAF-DB dataset

Methods	Year	Accuracy (%)
The bold format is used to indicate the best (highest) accuracy. gACNN: Region attention mechanism; RAN: region attention networks; SCN: self-cure networks; OADN: occlusion-adaptive deep network; DACL: deep attentive center loss; MA-Net: multi-scale and local attention network; FDRL: feature decomposition and reconstruction learning; VTFF: visual transformers with feature fusion; AMP-Net: adaptive multilayer perceptual attention network; ADDL: adaptive deep disturbance-disentangled learning; PACVT: patch attention convolutional vision transformer; DBFN: dual-branch fusion network; MSAFNet: multi-scale attention and convolution-transformer fusion network.
gACNN[49]	2018	85.07
RAN[45]	2020	86.90
SCN[48]	2020	87.01
OADN[50]	2020	87.16
DACL[51]	2021	87.78
MA-Net[52]	2021	88.40
FDRL[53]	2021	89.47
VTFF[54]	2021	88.14
AMP-Net[15]	2022	89.25
ADDL[55]	2022	89.34
PACVT[41]	2023	88.21
GSDNet[32]	2024	90.91
DBFN[56]	2024	87.65
MSAFNet(ours)	2025	90.06

Figure 5. The confusion matrices of MSAFNet on the RAF-DB datase.

4.3.2. Results on FER2013

Table 2 shows the results that our method compares with state-of-the-art methods on FER2013 dataset. Our MSAFNet obtains an accuracy of 73.25% on FER2013 dataset, which is competitive with other advanced methods. The confusion matrix in Figure 6 shows that fear expression is the poorest to recognize and happiness has the highest recognition rate.

Table 2

Comparison with other methods on FER2013 dataset

Methods	Year	Accuracy (%)
The bold format is used to indicate the best (highest) accuracy. SHCNN: Shallow convolutional neural network; CNN: convolution neural network; AWHFL: adaptive weighting of handcrafted feature losses; LBAN-IL: local binary attention network with instance loss; MSAFNet: multi-scale attention and convolution-transformer fusion network.
SHCNN[57]	2019	69.10
Pre-trained CNN[11]	2019	71.14
AWHFL[58]	2019	72.67
FreNet[59]	2020	64.41
LBAN-IL[60]	2021	73.11
MSAFNet(ours)	2025	73.25

Figure 6. The confusion matrices of MSAFNet on the FER2013 dataset.

4.3.3. Results on FERPlus

Comparison results with other state-of-the-art methods on FERPlus are shown in Table 3. The recognition accuracy on FERPlus has been considerably improved when compared to the FER2013 dataset since FERPlus has been relabeled and non-face images have been removed. As shown in Table 3, our MSAFNet obtains the recognition accuracy of 89.82%. Compared to VTFF ^[54] and PACVT ^[41] which also utilized transformer architecture, our method achieves 1.01% and 1.1% improvement. The results of the confusion matrix on FERPlus are shown in Figure 7. The confusion matrix shows that happiness, neutral, and surprise have better performance than other expressions, and contempt, disgust, and fear have poor performance. The reason for these results may be that contempt, disgust, and fear lack enough data compared to other expressions.

Table 3

Comparison with other methods on FERPlus dataset

Methods	Year	Accuracy (%)
The bold format is used to indicate the best (highest) accuracy. CSLD: Crowd-sourced label distribution; VGG: visual geometry group networks; SHCNN: shallow convolutional neural network; RAN: region attention networks; SCN: self-cure networks; VTFF: visual transformers with feature fusion; PACVT: patch attention convolutional vision transformer; GSDNet: gradual self distillation network; CBAM-4CNN: convolutional block attention module with convolutional neural network; MSAFNet: multi-scale attention and convolution-transformer fusion network.
CSLD [9]	2016	83.85
ResNet+VGG [61]	2017	87.40
SHCNN [57]	2019	86.54
RAN [45]	2020	88.55
RAN-VGG [45]	2021	89.16
SCN [48]	2020	88.01
VTFF [54]	2021	88.81
PACVT [41]	2023	88.72
GSDNet[32]	2024	90.32
CBAM-4CNN[62]	2024	87.75
MSAFNet(ours)	2025	89.82

Figure 7. The confusion matrices of MSAFNet on the FERPlus dataset.

4.4. Ablation analysis

To evaluate the effectiveness of our method, we perform a series of ablation studies on RAF-DB dataset. In the experiments, we evaluate the impact of the proposed components, the impact of different fusion methods, and the impact of different attention methods, respectively.

4.5. Complexity analysis

We compare the number of parameters (params) and floating point operations (FLOPs) of our method with other methods, as shown in Table 9. We can see that the params and FLOPs of our method are only 23.42 M and 3.60 G. The parameters and FLOPs of our method are significantly lower than those of MA-Net ^[52] and AMP-Net ^[15]. These results demonstrate that our MSAFNet has lower complexity and achieves better performance than other methods.

4.6. Visualization

In this section, in order to better validate the performance of MSA, we utilize gradient-weighted class activation mapping (Grad-CAM) ^[66] to visualize SE, CBAM, ECA, and our MSA respectively. As shown in Figure 8, LFEM generates highly localized activations focusing on fine-grained facial components, while GFEM produces broader activation patterns capturing holistic facial structure. This contrast validates the complementary roles of LFEM in micro-feature extraction and GFEM in macro-context modeling. As shown in Figure 9, our MSA enables the network to better focus on the key areas, such as the eyes, nose, and mouth. For facial occlusion or variant poses, MSA can still focus on eyes, nose, and mouth regions, and other attention methods only pay attention to eyes, nose, or mouth regions. The results can further illustrate that our MSA can capture the important information of the regions related to FER, verifying the effectiveness of our method.

Figure 8. The CAM of LFEM and GFEM. The images and labels are from FER2013 and RAF-DB. CAM: Class activation mapping; LFEM: local feature extraction module; GFEM: global feature extraction module.

Figure 9. Visualization results. The CAM of MSA is compared with other attention methods. The images and labels are from FER2013 and RAF-DB. CAM: Class activation mapping; MSA: multi-scale attention.

Authors' contributions

Made substantial contributions to conception and design of the study and performed data analysis and interpretation: He, H.; Liao, R.; Li, Y.

Performed data acquisition and provided administrative, technical, and material support: He, H.

Availability of data and materials

The datasets used in this study are sourced from publicly available datasets, including RAF-DB, FER2013, and FERPlus. These datasets can be accessed at: RAF-DB: http://www.whdeng.cn/RAF/model1.html; FER2013: https://www.kaggle.com/datasets/msambare/fer2013; FERPlus: https://www.kaggle.com/datasets/debanga/facial-expression-recognition-ferplus. For proprietary or additional datasets used in this study, access requests can be made by contacting the corresponding author at hehuifang@gdep.edu.cn. The code used in this research is available at https://github.com/SCNU-RISLAB/MSAFNet or can be obtained upon request.

Financial support and sponsorship

This work was supported in part by the Guangdong Association of Higher Education under the "14th Five-Year" Plan for Higher Education Research Projects, grant number 24GYB148.

Conflicts of interest

All authors declared that there are no conflicts of interest.

Ethical approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Copyright

© The Author(s) 2025.