Open-set fault diagnosis for solar insecticidal lamp IoT systems using adaptive prototypes

2.1. Problem formulation and overview

We consider an open-set diagnosis problem in which each input sample is a multivariate temporal window

where L and F denote the window length and the number of signal channels, respectively. Let the known class set be denoted by $$ \mathcal{C} $$ = {1, …, K}. During training, only samples from $$ \mathcal{C} $$ are available, whereas validation and test sets may additionally contain samples from unseen unknown classes. The goal is to assign a test sample to either a known class or the unknown category^[25]. This setting differs from generalized zero-shot fault diagnosis methods that attempt to recognize unseen classes by exploiting auxiliary semantic or fault-related information. In the present open-set setting, unknown fault classes are unavailable during model fitting and threshold calibration, and the main objective is reliable unknown-fault rejection rather than explicit unseen-class recognition.

As shown in Figure 1, the proposed framework contains three stages: discriminative representation learning, uncertainty-aware prototype matching, and class-adaptive conformal rejection. First, a temporal encoder maps each signal window to a latent feature vector. Second, one prototype is constructed for each known class from the proper training split, and class-wise feature dispersion is used to define UAPD. Third, the CACT strategy calibrates one rejection threshold per known class by using calibration samples from known classes only. In this way, the final decision rule jointly accounts for class-dependent feature geometry and class-dependent acceptance boundaries.

Figure 1. Overview of the proposed open-set diagnosis framework. Raw multichannel electrical signal records are first divided into training, validation, and test subsets, and sliding-window segmentation and standardization are then performed within each subset. A temporal encoder learns latent representations, which are used for prototype construction, uncertainty-aware distance estimation, and class-adaptive open-set rejection. GELU: Gaussian error linear unit.

2.2. Data preprocessing

Following the preprocessing pipeline in Figure 1A, the raw multichannel electrical signal records associated with system operation were first divided into training, validation, and test sets at a ratio of 7:2:1 before window segmentation. The sliding-window operation was then performed separately within each subset, using a window length of 16 and a stride of 4. Therefore, although adjacent windows within the same subset overlap by 75%, no overlapping windows derived from the same acquisition record were shared across the training, validation, and test sets. The generated window samples were standardized before model training, and the normalization parameters were computed using only the training set and then applied unchanged to the validation and test sets. In the implementation, the known-class training data were further divided into a proper training split $$ \mathcal{D} $$_tr and a calibration split $$ \mathcal{D} $$_cal, while the open-set validation and test sets were denoted by $$ \mathcal{D} $$_val and $$ \mathcal{D} $$_te, respectively^[26].

For a known training sample (x_i, y_i) with y_i ∈ $$ \mathcal{C} $$, the encoder produces a latent feature

where f_θ(·) denotes the temporal encoder and d is the latent feature dimension. In what follows, we use prototype to refer to the class mean used for open-set inference, and center to refer to the learnable center parameter used only in the training loss. For consistency, class-specific quantities are indexed by the generic class index c ∈ $$ \mathcal{C} $$ throughout the manuscript; for example, the inference prototype is denoted as P_c ∈ ℝ^d rather than by a fixed class-specific notation such as P₂.

2.3. Temporal encoder and prototype-oriented feature learning

The temporal encoder consists of three stacked 1D convolutional blocks, each followed by batch normalization and Gaussian error linear unit (GELU) activation^[27]. Dropout is used for regularization, and adaptive average pooling aggregates the temporal dimension into a fixed-length representation. A linear projection layer then maps the pooled representation to a d-dimensional latent space. In the implementation, d = 64.

To learn a feature space that is both discriminative and compact, we train the encoder jointly with a linear classifier head using three complementary objectives:

where $$ \mathcal{L} $$_CE is the standard cross-entropy loss over known classes, $$ \mathcal{L} $$_center is the center loss, and $$ \mathcal{L} $$_sc is the supervised contrastive loss.

The center loss is defined as

where $$ \mu_{y_i} $$ ∈ ℝ^d is the learnable center of class y_i. This term encourages intra-class compactness, whereas the supervised contrastive term further increases separation between known classes^[28]. It should be noted that the learnable center μ_c is used only during representation learning and is optimized as a parameter of the center-loss term. It is not directly used as the prototype for open-set inference. Instead, the center loss indirectly shapes the latent-space geometry by encouraging features from the same class to concentrate around their corresponding training center. This more compact and structured feature geometry subsequently affects the empirical class prototypes and class-wise variances estimated after training. In our implementation, λ_c = 0.03 and λ_sc = 0.10.

After training, the classifier head is discarded, and the latent space is summarized by one prototype per known class:

where N_c is the number of proper training samples belonging to class c. Unlike the learnable center μ_c, the prototype P_c is a post-hoc empirical class mean computed from the trained encoder features after representation learning is complete. Thus, P_c reflects the final feature distribution induced by the joint training objective, including the effect of center loss, cross-entropy loss, and supervised contrastive loss. The same post-training features are also used to estimate the class-wise variance vector σ_c², so the variance normalization in UAPD is coupled to the learned feature geometry rather than to the learnable center parameters themselves. These prototypes serve as the anchors for the subsequent open-set decision process.

2.4. UAPD

A plain Euclidean distance assumes that all known classes have comparable feature spread in the latent space. This assumption is often too restrictive for open-set diagnosis, because some classes naturally form compact clusters while others remain more dispersed due to operating variability or residual inter-class ambiguity. A uniform metric therefore tends to over-penalize dispersed classes and under-penalize deviations from compact classes^[29].

To address this issue, we estimate a class-wise variance vector for each prototype:

Based on P_c and σ_c², we define the UAPD between a test feature z and class c as

where ϵ > 0 is a small constant for numerical stability.

This metric can be interpreted as a diagonal Mahalanobis-like prototype distance. Dimensions with low class-wise dispersion contribute more strongly to the distance, whereas naturally noisy dimensions are down-weighted. Consequently, compact classes receive tighter effective boundaries, while broader classes are not unfairly rejected simply because their latent distribution is intrinsically more spread out. The diagonal formulation was adopted as a deliberate design choice to keep the open-set decision rule lightweight, interpretable, and directly aligned with the class-wise uncertainty estimated for each latent feature dimension. A full-covariance Mahalanobis distance can additionally model correlations among latent dimensions, but it requires estimating and inverting one covariance matrix for each known class. This may introduce numerical difficulties when the covariance matrix is ill-conditioned or when strong correlations among latent dimensions make the inverse unstable. It also increases the statistical burden because a full covariance matrix contains many more parameters than a diagonal variance vector, which can make the estimated distance sensitive to the empirical feature distribution of each class. Therefore, we used the diagonal variance vector to preserve the main class-dependent scale information while avoiding the additional numerical and statistical complexity of full-covariance estimation.

Unlike a conventional Mahalanobis-distance classifier, UAPD is not used as an independent post-hoc statistical classifier. Instead, it is embedded in the learned prototype space and uses class-wise latent feature dispersion estimated from the trained encoder features to normalize prototype matching. In this sense, UAPD serves as a class-adaptive prototype-matching metric for open-set diagnosis, rather than as a standalone covariance-based classifier.

2.5. CACT

Although UAPD provides a more faithful distance measure, open-set diagnosis still requires a rejection rule to determine whether the nearest known prototype is sufficiently reliable. A single global threshold is often inadequate because different classes may exhibit markedly different latent-space geometry even after feature learning. We introduce CACT, which calibrates one rejection threshold for each known class by using calibration samples from known classes only^[30]. Unlike ordinary class-wise thresholding, the thresholds in CACT are obtained from conformal nonconformity scores rather than manually tuned using unknown samples or an auxiliary unknown set. This design is consistent with the target deployment condition, where real unknown faults are not available during threshold calibration.

The validity of the CACT thresholds relies on the exchangeability assumption of conformal prediction. Specifically, the calibration samples and future deployment samples from the same known class are expected to be generated from the same underlying distribution, or at least to remain approximately exchangeable after preprocessing and feature extraction. Under this condition, the class-specific conformal quantile provides a controlled known-class rejection risk at the prescribed significance level. If substantial distribution shifts occur, such as seasonal variation, sensor drift, component aging, or abrupt changes in operating conditions, the exchangeability assumption may be weakened and the calibrated thresholds may require online updating or recalibration.

For each calibration sample z_i^cal from class c, we compute its class-consistent nonconformity score

The threshold for class c is then defined as the upper conformal quantile of the corresponding calibration scores:

where α denotes the target risk level. In our implementation, α = 0.05.

Given a test feature z, we first identify its nearest prototype

The final open-set decision is then made by comparing the nearest-prototype distance with the threshold of the predicted class:

This design avoids tuning on real unknown samples and aligns the acceptance region with class-dependent feature geometry.

2.6. Training and inference procedures

The complete training and inference procedures are summarized in Algorithm 1 and Algorithm 2. Training consists of two stages: discriminative representation learning on the proper known-class training split, followed by prototype, variance, and threshold estimation. Inference uses only the trained encoder, class prototypes, class-wise variances, and class-adaptive thresholds.

Algorithm 1.

Algorithm 2.

3.1. Dataset description

The experiments were conducted on real-world multichannel monitoring signals collected from SIL-IoTs systems. As shown in Figure 2, the data used in this study were collected from three field-deployed SIL-IoTs devices in Chuzhou, Anhui Province, China, over a two-month operating period. Following the acquisition protocol of the solar insecticidal lamp fault (SILF) dataset, a raw acquisition record refers to the monitoring segment retained during one fault-diagnosis execution of an SIL-IoTs node. The sensor readings were collected at approximately 0.2 Hz, and each raw acquisition record covered 5 min of device operation. The fault taxonomy is summarized in Table 1, where the fault-free state and eight fault types are denoted by C0-C8.

Figure 2. Real-world SIL-IoTs device used for field data collection in Chuzhou, Anhui Province, China. The SIL-IoT node, electronic box, and Raspberry Pi photographs show the actual deployed device, electrical enclosure, and embedded computing unit, respectively, and were taken by the authors. SIL-IoTs: Solar insecticidal lamp Internet of Things.

Each sample was represented as a temporal window of length 16 with an input dimension of 13. The 13 input features, denoted as Feature 0 to Feature 12, describe the environmental, electrical, and operating states of SIL-IoTs systems, as summarized in Table 2.

The raw monitoring records were first divided into training, validation, and test subsets at a ratio of 7:2:1 before sliding-window segmentation. The sliding-window operation was then performed independently within each subset using a window length of 16 and a stride of 4. After preprocessing, 500 window samples were retained for each fault category, resulting in 4,500 window samples. Therefore, no obvious sample imbalance was present across the nine categories. For each known class in an open-set task, 350, 100, and 50 window samples were used for training, validation, and testing, respectively. Unknown-class samples were excluded from training and were used only in the validation and test stages. Because the experimental subset was generated and balanced after sliding-window segmentation, the class-wise experimental scale is reported at the window-sample level.

This partition protocol was used to reduce the risk of temporal leakage caused by overlapping windows. Specifically, the generated windows were not randomly mixed across the training, validation, and test subsets. Normalization parameters were estimated only from the training data and then applied unchanged to the validation and test data. The training and test data were collected from different SIL-IoTs devices and operating conditions, while the acquisition dates were matched to reduce date-related bias.

To evaluate the proposed method under realistic open-set conditions, only known fault classes were used during training. Unknown fault classes were excluded from model fitting and introduced only during validation and testing. As shown in Figure 3, the inter-class distances were highly non-uniform, and several class pairs were substantially closer than others. The class-center distance map in Figure 3A was used only to guide the construction of the five open-set task partitions and was not used for model training, model selection, prototype estimation, or threshold tuning. Based on this distance distribution, five open-set diagnosis tasks were constructed by varying the partition between known and unknown classes, as summarized in Figure 3B and Table 3. Following the revised task order, Task 1 represents a relatively simple open-set setting. Task 2 and Task 3 introduce more ambiguous unknown-class combinations. Task 4 increases the number of unknown classes, while Task 5 is treated as the most challenging setting because its unknown set contains multiple lamp- and metal-mesh-related current or voltage faults that are close to several known classes in the distance map.

Figure 3. Class-center distance and covariance condition analysis. (A) Normalized class-center Euclidean distance map used to guide open-set task construction. Darker cells indicate smaller inter-class distances, and red boxes highlight closely spaced class pairs; (B) Class-wise full-covariance condition numbers estimated from latent features in the proper training split. Larger values indicate greater potential numerical instability when applying full-covariance Mahalanobis distance.

In the implemented pipeline, the known-class training set was further divided into a proper training split and a calibration split. The proper training split was used for representation learning, prototype estimation, and class-wise variance estimation. The calibration split was used only for class-adaptive threshold estimation. No samples from the unknown classes were used for model fitting, prototype construction, variance estimation, threshold calibration, or hyperparameter selection. The same training, validation, and test protocol was applied to all compared methods in every task to ensure a fair comparison. This setup is consistent with the intended deployment setting, in which unknown samples are unavailable during threshold tuning.

3.2. Experimental setup

4.1. Ablation experiment

As summarized in Table 5, a key question is whether the gain of the full model arises from the combined action of UAPD and CACT, rather than from a single dominant component. Across the five tasks, the complete model achieved the highest OS in Task 1 to Task 4, reaching 0.9689, 0.9333, 0.8849, and 0.8618, respectively, and obtained an OS of 0.8524 in Task 5. These results indicate that enabling both modules generally improves overall open-set performance, although the advantage is not uniform across all task settings. The full model also produced the highest H-score in Task 1 and Task 2, and remained competitive in Task 3 to Task 5, supporting the view that the joint design improves the overall balance between known-class recognition and unknown-class rejection. A more detailed comparison suggests that the two modules play distinct roles. Removing UAPD caused the most pronounced degradation in Task 2, where OS, UNK, and H-score dropped from 0.9333, 0.8960, and 0.9225 to 0.8493, 0.6013, and 0.7435, despite an increase in OS′ to 0.9733. This pattern is consistent with a shift towards known-class bias at the expense of open-set robustness. By contrast, removing CACT led to higher UNK and H-score in Task 3 and Task 5, and also produced a higher OS in Task 5, indicating that CACT mainly adjusts the trade-off between known-class recognition and unknown-class rejection rather than uniformly maximizing every metric in every task. Notably, UNK was saturated in Task 1 and Task 4 across all variants, which limits mechanistic interpretation in these relatively easy settings. Together, these ablations show that the full configuration provides a strong and balanced operating point overall, while also indicating that the effect of CACT on unknown-class discrimination remains task-dependent.

4.2. Hyperparameter sensitivity analysis

To examine the robustness of the selected hyperparameters, we further conducted sensitivity analysis for the main parameters of the proposed framework, including the CACT significance level α, the calibration split, the center-loss weight λ_c, the supervised-contrastive-loss weight λ_sc, and the numerical stabilizer ϵ in UAPD. The dropout rate was kept at 0.15 as an encoder regularization setting, while the sensitivity analysis focuses on the parameters that directly affect prototype compactness, variance normalization, and conformal rejection. The results are summarized in Table 6, where the default settings used in the proposed method are highlighted in bold.

Overall, the sensitivity results show that the selected default settings provide a stable operating point across the five tasks. For α, excessively small values weakened unknown rejection in several tasks, whereas α = 0.05 maintained a better balance between OS and H-score. For the calibration split, the 20% setting generally achieved stronger, more stable performance than the 10% or 15% settings, while the 30% setting reduced the amount of data available for representation and prototype learning. The loss-weight results show that both λ_c and λ_sc are useful: setting either weight to zero degraded performance, whereas the default values preserved compact known-class representations and improved open-set balance. The results for ϵ indicate that a small stabilizer is sufficient for UAPD, whereas an overly large value can weaken variance normalization.

4.3. State-of-the-art open-set diagnosis methods compared

As shown in Figure 4, the proposed method achieved the highest OS in Tasks 1-4 and remained competitive in Task 5 when compared with OpenMax, OpenGAN, EAC, COCL, NNGM, ARPL, ViM, and DCC. This result indicates that the proposed framework provides strong overall open-set diagnosis performance by jointly considering known-class recognition and unknown-class rejection. Although several comparative methods achieved competitive results in individual tasks, their performance fluctuated more noticeably across task settings, whereas the proposed method maintained consistently high OS accuracy.

Figure 4. Comparison of OS overall accuracy across five open-set tasks.

As shown in Figure 5, the proposed method also preserved strong known-class recognition performance. The OS′ results indicate that introducing unknown rejection does not substantially weaken the classification of known SIL-IoTs fault categories. Compared with the baseline methods, the proposed framework maintains more stable known-class accuracy across different open-set partitions, suggesting that the learned prototype representation remains discriminative for known faults.

Figure 5. Comparison of OS′ known-class accuracy across five open-set tasks.

As shown in Figure 6, the proposed method achieved the best or highly competitive H-scores across the five tasks. Since H-score jointly measures the balance between known-class recognition and unknown-class detection, these results further confirm that the proposed method achieves more balanced decision behavior than the compared methods. The advantage is especially evident in challenging tasks where unknown samples are close to known fault categories.

Figure 6. Comparison of H-score across five open-set tasks.

As shown in Figure 7, the proposed method achieved strong UNK performance, demonstrating its ability to reject previously unseen fault types. Compared with OpenMax, OpenGAN, EAC, COCL, NNGM, ARPL, ViM, and DCC, the proposed method more effectively reduced misclassification of unknown samples as known categories. This improvement can be attributed to the UAPD and CACT, which jointly refine prototype matching and unknown rejection.

Figure 7. Comparison of UNK unknown-detection accuracy across five open-set tasks.

4.4. Computational efficiency analysis

To evaluate the proposed framework’s suitability for resource-constrained SIL-IoT devices, we further compared the computational cost and inference efficiency of the proposed method with the baseline methods. All methods were evaluated using the same input size, the same 1D convolutional backbone, and the same batch inference setting. The number of parameters, model size, neural forward floating-point operations (FLOPs), additional open-set mechanism FLOPs, inference latency, and throughput are summarized in Table 7.

As shown in Table 7, the proposed method has 0.0858M parameters and a model size of 0.3274 MB, which are comparable to those of the lightweight baseline methods. Because all compared methods use the same 1D convolutional temporal encoder, their neural forward FLOPs are also similar. The additional computational cost introduced by the proposed UAPD and CACT mechanisms is only 1.542K FLOPs per sample, which is much smaller than the neural forward computation. The proposed method also achieves a low inference latency of 0.0019 ms per sample and a throughput of 534,824 samples/s under the tested batch inference setting. These results indicate that the proposed framework introduces only a small open-set decision overhead while maintaining practical computational efficiency for SIL-IoTs fault diagnosis.

4.5. Statistical significance analysis

To further examine whether the observed improvements are robust to random-seed variability, we performed paired statistical tests between the proposed method and each comparative method across the five repeated runs. As shown in Table 8, the proposed method achieved statistically significant improvements over several baselines, especially in the more difficult tasks. Although some comparisons did not reach statistical significance, the proposed method still showed consistent numerical advantages in most cases. This suggests the improvements are generally robust, and small performance gaps should be interpreted alongside the standard deviations across random seeds. Overall, the statistical results support the effectiveness of the proposed method and clarify the cases where the advantage is relatively modest.

4.6. Confusion matrix and feature visualization

As shown in Figure 8, the challenging setting mainly exposed errors in the separation between unknown samples and the most confusable known classes, while the easier known categories remained largely well classified across methods. The compared methods, including OpenMax, OpenGAN, EAC, COCL, NNGM, ARPL, ViM, and DCC, all showed different degrees of unknown-to-known leakage, especially around the electrically related classes C1 and C2. This indicates that unknown abnormal states can be easily absorbed into nearby known fault regions when their signal patterns are similar. By contrast, the proposed method achieved stronger unknown recognition while maintaining a cleaner diagonal structure for the known classes. These confusion patterns are consistent with the quantitative metrics and further suggest that the proposed method improves class-wise separability under difficult open-set conditions, especially at the decision boundary between unknown samples and the most easily confused known categories.

Figure 8. Confusion matrices under a representative challenging open-set task. The representative task corresponds to Task 5 in Table 3; the known classes C0, C2, C3, C4, and C8 are displayed using task-local labels C0-C4, while the unknown class combines C1, C5, C6, and C7. The proposed method exhibits the clearest diagonal structure and the strongest unknown-class recognition, indicating improved separation between known and unknown samples.

As shown in Figure 9, the challenging setting revealed substantial differences in the feature-space structure learned by the competing open-set methods. Several baseline methods exhibited varying degrees of overlap between known and unknown samples, particularly around the most confusable regions, indicating that their learned representations did not fully separate unknown instances from nearby known classes. In contrast, the proposed method produced a visibly cleaner organization of the embedding space, with known samples forming more compact clusters and unknown samples occupying more distinct regions with reduced intrusion into the major known-class groups. Although limited local proximity between known and unknown samples was still present in a few boundary regions, the overall separation pattern was clearer than that of the competing baselines. This qualitative result is consistent with the confusion matrices and quantitative open-set metrics, and suggests that the proposed method improves feature-level discriminability under difficult open-set conditions.

Figure 9. t-SNE visualization of feature embeddings under a representative challenging open-set task. The proposed method yields more compact known-class clusters and clearer separation between known and unknown samples than competing methods. t-SNE: Distributed stochastic neighbor embedding.

A closer inspection of the failure cases indicates that the remaining errors are mainly concentrated near electrically related known classes, especially C1 and C2 in the representative challenging task. These classes correspond to lure-lamp and metal-mesh open-circuit faults, whose current-voltage signatures can be similar to those of some unknown abnormal states when illumination fluctuation, unstable power supply, or load variation changes the operating signal. Therefore, the residual misclassification of unknown samples is physically plausible: unknown faults with similar electrical responses may fall near the acceptance regions of known electrical faults. Compared with the baseline methods, the proposed framework reduces this leakage by forming more compact known-class clusters and applying class-adaptive rejection thresholds, but a small number of boundary samples remains difficult to separate.