预测模型评估在故障预测与健康管理(PHM)中的应用

120 浏览量更新于2024-08-27 收藏 477KB PDF 举报

"这篇研究论文探讨了在故障预测与健康管理(PHM)中预测模型的评估问题，重点关注如何更准确地评估这些模型在实际应用中的性能。作者包括Chunsheng Yang、Yanni Zou、Jie Liu和Kyle RMulligan，分别来自加拿大国家研究委员会、九江大学、卡尔顿大学和舍布鲁克大学。文章发表于2014年的《国际预测与健康管理期刊》(International Journal of Prognostics and Health Management)。" 在过去的几十年里，机器学习技术，尤其是分类算法，已经被广泛应用于各种实际场景，如PHM（故障预测与健康管理）。在构建高效率的分类器或基于机器学习的预测模型时，如何有效地评估这些预测模型是关键但具有挑战性的任务。通常使用的评价指标，如准确性，可能并不完全适用于预测性模型在预测维护应用中的评估。论文首先回顾了通用的评估方法，并指出它们的局限性。对于PHM系统的预测模型评估，需要考虑更多的因素，比如模型的可靠性、鲁棒性以及预测时间窗口内的性能变化等。作者讨论了预测模型在故障早期检测、故障模式识别以及剩余使用寿命(RUL)估计等方面的评估标准。论文进一步探讨了特定于PHM的评估指标，这些指标能够更好地反映模型在真实世界环境中的表现。例如，时间相关的性能度量，如预测误差随时间的变化趋势，可以帮助分析模型预测性能的稳定性。此外，模型的不确定性量化也是评估的重要组成部分，因为它可以提供关于预测可信度的信息。文章还可能涉及了验证和测试数据集的选择，包括交叉验证策略，以确保模型的泛化能力。作者可能提出了针对PHM应用的特定模型比较方法，以确定最佳预测策略。同时，他们可能讨论了如何处理数据不平衡问题，因为故障事件通常比正常运行状态的数据少，这可能会影响模型的训练和评估。这篇论文对PHM领域的预测模型评估进行了深入研究，提供了针对性的评估策略，旨在改进和优化预测模型在故障预测中的性能，这对于提升工业设备的维护效率和降低运营成本具有重要意义。

INTERNATIONAL JOURNAL OF PROGNOSTICS AND HEALTH MANAGEMENT

where:

 c

is the number of true positives;

 c

is the number of false positives;

 c

is the number of false negatives; and

 c

is the number of true negatives.

Table 1. The scoring metrics

Metrics

Computing method

Accuracy

22211211

2211

cccc





Error Rate

22211211

2112

cccc





TPR

1211



TNR

2221



FPR

1211



FNR

2221



Sensitivity

2111



Specificity

2212



Recall

2111



Precision

1211



F-score

precisionrecall



2

From this confusion matrix, scoring metrics are computed

as shown in Table 1. These scoring metrics are widely used

in comparing or ranking the performance of classifiers

because they are simple, easy-to-compute, and easy-to-

understand. Such an evaluation typically examines unbiased

estimates of predictive accuracy of different classifiers

(Srinivasan 1999). The assumption is that the estimates of

performance are subject to sampling errors only and that the

classifier with the highest accuracy would be the “best”

choice of the classifiers for a given problem. These metrics

overlook two important practical concerns: class distribution

and the cost of misclassification. For example, the accuracy

metric assumes that the class distribution is known in the

training dataset and testing dataset and that the

misclassification costs for false positive and false negative

error are equal (Provost 1998). In real-world application, the

cost of misclassification for different errors may not be the

same. It is sometimes desirable to minimize the

misclassification cost rather than the error-rate in

classification task.

2.3. Graphical methods

To overcome the limitations of scoring metrics and

incorporate the consideration of prior class distributions and

the misclassification costs, several graphical methods had

been developed. These graphic methods are ROC

(Receiver Operating Characteristics) Space, ROCCH (ROC

Convex Hull), Isometrics, AUC (Area under ROC curve),

Cost Curve, DEA (Data Envelopment Analysis), and Lift

curve. They are also useful for visualizing performance of a

classifier. Here is an overview for each method.

2.3.1. ROC Space

The ROC analysis was initially developed to express the

tradeoffs between hit rate and false alert rate in signal

detection theory (Egan 1975). It is now also used for

evaluating classifier performance (Bradley 1997, Provost

and Fawcett 2001, 1997). In particular, ROC is a powerful

way for performance evaluation of binary classifiers, and it

has become a popular method due to its simple graphical

representation of overall performance.

Using TPR and FPR from Table 1 as the Y-axis and X-axis

respectively, a ROC space can be plotted. In this ROC

space, each point (FPR, TPR) represents a classifier

(Fawcett 2003, Flach 2004, 2003). Figure 1 shows an

example of a basic ROC graph for five discrete classifiers.

Based on the position of a classifier in ROC space, we can

evaluate or rank the performance of classifiers. In ROC

space, the point (0, 1) represents a perfect classifier which

has 100% accuracy and zero error rates. The upper left

points indicated that a classifier has a higher TPR and a

lower FPR. For instance, C4.5 has a better TPR and a lower

FPR than nB. The classifier on the upper right-hand side

of an ROC space makes positive classification with relative

weak evidence and has a higher TPR and a higher FPR.

Therefore, the performance of a classifier is determined by a

trade-off between TPR and FPR. However, it is hard to

decide which classifier is best from ROC space. For

剩余10页未读，继续阅读

weixin_38548434

粉丝: 3
资源: 945

预测模型评估在故障预测与健康管理(PHM)中的应用

基于机器学习的故障预测与健康管理（PHM）方法研究

2018-phm-data-challenge:2018 phm数据挑战，离子磨机RUL和故障诊断

PHM2008 挑战赛数据集

phm2012轴承寿命预测

phm 数 据 挑 战 赛

phm2012剩余寿命预测

基于PHM2010数据集进行刀具磨损寿命预测

phm2010数据集处理

使用phm2010数据集进行刀具磨损量预测的matlab代码

使用PHM2010数据集进行刀具寿命预测的代码

最新资源

phm 数据挑战赛