AppendixSensitivity analysis of operational parameters
Table 8 presents the sensitivity analysis of the LLM-augmented classifiers across different decision thresholds \((\tau)\) and temporal persistence windows (k) on the Total dataset. The analysis evaluates how F1 Score, Detection Delay, and Pre-Alarm Rate (PAR) vary as the alarm-formation rule becomes more permissive or more conservative. The default operating setting used in the main experiments is \(\tau =0.50\) and \(k=1.0\text {s}\).
Table 8 Sensitivity analysis of LLM-augmented classifiers on the Total dataset across decision thresholds (\(\tau\)) and temporal persistence windows. Results are presented as mean (± standard deviation) across 5 seeds. Bold values indicate the default operating setting (\(\tau =0.50\) and \(k=10\) frames, equivalent to 1.0 s) used in the main experiments; these results are kept consistent with the main tables.
Overall, the results show a clear trade-off between detection sensitivity and premature-alarm suppression. Lower thresholds generally preserve higher F1 scores but substantially increase PAR, indicating more aggressive alarm behavior. For example, at \(\tau =0.25\) and \(k=1.0\text {s}\), SVM + LLM and GBM + LLM achieve high F1 scores of 87.45% and 90.85%, respectively, but their PAR values rise to 99.25% and 51.32%. In contrast, higher thresholds suppress premature alarms more strongly but often degrade F1. For instance, increasing the threshold to \(\tau =0.75\) at \(k=1.0\text {s}\) reduces PAR to 8.68% for SVM + LLM, 6.04% for RF + LLM, and 12.08% for GBM + LLM, but lowers their F1 scores to 64.78%, 69.21%, and 69.66%, respectively.
The temporal persistence window further modulates this trade-off. Increasing the persistence window from 1.0 to 2.0 s or 3.0 s generally reduces PAR, but excessive smoothing can substantially reduce F1, especially for SVM + LLM and RF + LLM. GBM + LLM is comparatively more robust under moderate smoothing: at \(\tau =0.50\) and \(k=2.0\text {s}\), it maintains an F1 score of 88.33% while reducing PAR from 47.55% under the default setting to 28.68%. However, further increasing the window to \(k=3.0\text {s}\) reduces F1 to 83.46%, showing that overly conservative alarm persistence can also suppress valid fire detections.
Comparative analysis against direct LLM prompting baselines
To justify the architectural design of using the LLM as a semantic feature extractor rather than as an end-to-end decision maker, Table 9 evaluates direct LLM classification under Zero-shot, One-shot, and Few-shot Chain-of-Thought (CoT) prompting strategies52–54. The results show that direct prompting is highly sensitive to both the prompting setting and the LLM backbone when applied to raw multivariate numerical sensor snapshots.
These results indicate that the final alarm behavior is highly sensitive to the selected threshold and persistence window. We retain \(\tau =0.50\) and \(k=1.0\text {s}\) as the default setting for the main experiments because it provides a consistent operating point for comparing classifiers, while the sensitivity analysis clarifies how alternative alarm rules shift the balance between F1, delay, and pre-ignition false alarms.
Table 9 Performance comparison of direct LLM classification using pure prompting strategies (Zero-shot, One-shot, and Few-shot CoT) on the Total dataset. Results are presented as Mean% (± Std) across 5 seeds. Bold values indicate the best performance within each prompting setting.
Under the Zero-shot CoT setting, several evaluated backbones—including Llama-3-8B41, Qwen-2.5-3B58, Qwen-2.5-7B, and Ministral-8B59—produce near-zero Recall and F1, indicating difficulty in directly mapping raw numerical sensor values to stable fire/no-fire decisions. Llama-3.2-3B is a notable exception under Zero-shot prompting (F1 = 78.47%), but its performance drops sharply under the One-shot setting (F1 = 3.90%), illustrating the sensitivity of direct prompting to demonstration context.
Few-shot CoT improves the strongest prompt-only result, with Llama-3-8B reaching an Accuracy of 65.67% and an F1 score of 70.32%. However, this remains below the supervised hybrid approach: GBM + Llama-3-8B achieves an F1 score of 90.77% when the LLM is used as a semantic feature generator rather than as a standalone classifier. These comparisons suggest that, in this setting, LLMs are more reliable when used to produce intermediate semantic representations for supervised downstream classifiers than when used as direct numerical classifiers.
Supplementary implementation details
This subsection provides supplementary implementation details for the proposed framework. The prompt design used for contextual interpretation and ML-based classification is shown in Appendix Fig. 4. The hyperparameter settings for the ML classifiers, LLM semantic extractor, TF–IDF vectorizer, and deep learning baselines are summarized in Appendix Table 10. The scenario-specific thresholds used for the enhanced Rule + ML ablation baseline are reported in Appendix Table 11.
Fig. 4
Prompt design for contextual interpretation and ML-based classification.
Table 10 Hyperparameter settings for the ML classifiers, LLM, TF-IDF vectorizer, and deep learning baselines.Table 11 Scenario-specific multi-level thresholds used for the enhanced Rule + ML ablation baseline. The thresholds generate hand-crafted semantic tags from temperature, smoke, O\(\vphantom{0}_2\), CO, CO\(\vphantom{0}_2\), and inter-sensor difference features, providing a stronger symbolic-feature baseline than the original two-threshold rule setting.