环境科学中的神经网络自定义损失函数指南

版权申诉

85 浏览量更新于2024-07-06 收藏 4.62MB PDF 举报

"CIRA环境科学中神经网络自定义损失函数指南第1版，由CIRA(ECE)的Imme Ebert-Uphoff等多位专家撰写，详细介绍了在环境科学领域如何设计和应用自定义损失函数。该指南旨在帮助科学家更好地适应神经网络在环境科学中的应用，确保模型优化的目标与实际需求相符。" 在环境科学中，神经网络正逐渐成为一种重要的分析工具，用于处理复杂的预测和建模任务。训练神经网络的核心是通过最小化损失函数来优化模型性能。损失函数的选择对于环境科学应用至关重要，因为它直接决定了模型将优化的目标。标准的损失函数，如均方误差(Mean Squared Error, MSE)或交叉熵(Cross-Entropy)，可能无法完全满足环境科学的特定需求。环境科学研究中有许多特有的性能指标，例如气候模型的准确度、生态系统的稳定性评估等，这些都需要定制化的损失函数来更好地反映。本指南详细阐述了如何根据环境科学的特定问题，设计并实现自定义损失函数。这包括但不限于考虑特定领域的目标（如预测的精度、不确定性量化、多元输出的平衡等），以及将经典性能度量（如决定系数(R²)、平均绝对误差(Mean Absolute Error, MAE)等）融入到损失函数设计中。此外，指南可能还涵盖了以下内容： 1. 损失函数的理论基础：介绍损失函数的作用，以及它如何影响模型学习过程。 2. 设计原则：提供设计自定义损失函数的一般准则，包括损失函数的可微性、对异常值的敏感性、以及如何处理多目标优化问题。 3. 实例分析：通过具体的环境科学案例，展示如何选择或构建合适的损失函数。 4. 实现方法：指导读者如何在常用的深度学习框架（如TensorFlow、PyTorch）中实现自定义损失函数。 5. 评估与调优：讨论如何评估自定义损失函数的效果，并提供调整策略以提高模型性能。通过遵循此指南，环境科学家能够更有效地利用神经网络技术，为解决实际环境问题提供更为精准和适用的解决方案。

CIRA GUIDE TO CUSTOM LOSS FUNCTIONS

Items to note:

•

The inputs to the loss function are always two tensors,

y_true

and

y_pred

, in that order.

y_true

represents

the correct output (sometimes called the “label” or “ground truth”), while

y_pred

represents the prediction

generated by the neural network for that sample. You can choose your own variable names for the input

tensors, but always make sure that the label (ground truth) is the ﬁrst input and has a telling variable name, in

order to avoid confusion.

•

Note that

y_true

and

y_pred

represent the tensors for an entire batch! We discuss the implications of this

fact in detail in Section 4.2.

•

In the above examples the math operations

sqrt

square

, and

reduce_mean

are all TensorFlow functions.

We discuss the types of functions one can use in loss functions in Section 6.

Linking the loss function to a model

The loss function is linked to the model using the model.compile call, as shown in the example below.

model.compile(optimizer=keras.optimizers.Adam(), loss=loss_MSE, metrics=['accuracy'])

Note that there are no quotation marks placed around the function name,

loss_MSE

, above. The lack of quotes tells

Keras that this is a custom loss function, rather than a built-in loss function. In contrast, to use the built-in loss function

for MSE, we would call the corresponding function with quotes:

model.compile(optimizer=keras.optimizers.Adam(), loss='mean_squared_error', metrics=['accuracy'])

The metric assigned above, accuracy, also refers to a built-in function, as it is also called with quotes.

Custom loss function to help with class imbalance

The previous example was not very exciting, as MSE is available as a standard loss function anyway. However, we can

now create custom functions that help us with speciﬁc circumstances. For example, let us consider an application, such

as predicting rainfall, where the great majority of output values is small and only very few values are large. Since the

small values are much more common, the NN can achieve very high performance without ever getting the high values

correctly. In other words, because there are only few samples with high values, with a standard loss function the NN

might get away with always predicting low values.

There are many ways to deal with that problem, including creating a more balanced data set. Alternatively, we can

address this by a custom loss function that penalizes the NN more whenever it gets high values wrong. For example,

we can take the standard MSE function and multiply each individual error term by a weight factor that increases

exponentially with the true value. Here is an example that uses e

(5y

true

)

as the weight:

# Loss function with weights based on amplitude of y_true

def my_MSE_weighted(y_true,y_pred):

return K.mean(

tf.multiply(

tf.exp(tf.multiply(5.0, y_true)),

tf.square(tf.subtract(y_pred, y_true))

)

This loss function assigns different weights based on different amplitudes of y_true:

loss_M SE_weighted(y

true

, y

pred

) = mean

i∈I



(5 y

true

)

· (y

pred

− y

true

)



This is a very simple custom loss function but can already be quite useful.

3.4 How to save and load a model that has a custom loss or metric

When saving a NN model, unfortunately, custom metrics and loss functions are not stored in the model ﬁle. Thus, it is

necessary to supply the custom functions when loading the model.

Furthermore, parameters supplied to the custom functions are not automatically stored, either. There are two ways to

deal with this. One is a manual solution: keep track of the parameters (e.g., in conﬁguration ﬁles) and supply them

explicitly after the model is loaded. The more elegant solution is to embed the loss function in a class, which makes the

CIRA GUIDE TO CUSTOM LOSS FUNCTIONS

parameters automatically available after loading. This is not discussed here, but interested readers can ﬁnd it described

starting on page 386 of Geron’s machine learning book [11].

Consider a model that was trained with a custom loss function and saved in the usual manner.

model.save('K12.h5')

This example shows how to load that model, which was trained with the following custom loss function,

import tensorflow as tf

import tensorflow.keras.backend as K

def my_mean_squared_error_weighted_genexp(weight=(1.0,0.0,0.0)):

def loss(y_true,y_pred):

return K.mean(tf.multiply(

tf.exp(tf.multiply(weight,tf.square(y_true))),

tf.square(tf.subtract(y_pred,y_true))))

return loss

and uses the metric,

def my_r_square_metric(y_true,y_pred):

ss_res = K.sum(K.square(y_true-y_pred))

ss_tot = K.sum(K.square(y_true-K.mean(y_true)))

return (1 - ss_res/(ss_tot + K.epsilon()))

We deﬁne both metric and loss in a ﬁle called

custom_model_elements.py

. If the model is stored in the ﬁle

K12.h5

as described above, then you can load the model as follows.

from custom_model_elements import my_r_square_metric

from custom_model_elements import my_mean_squared_error_weighted_genexp

from tensorflow.keras.models import load_model

model = load_model('K12.h5',

custom_objects = {

'my_r_square_metric': my_r_square_metric,

'my_mean_squared_error_weighted_genexp': my_mean_squared_error_weighted_genexp,

'loss': my_mean_squared_error_weighted_genexp()

}

)

Because the loss function weights are not loaded using this approach, a model loaded this way cannot be further trained.

However, it can be used to make predictions.

4 Looking under the hood and common pitfalls to avoid

This section brieﬂy reviews the

compile

command, then, starting with Subsection 4.2, dives into practical implementa-

tion details many of which we have not found discussed anywhere in literature. Bits and pieces were found in web

forums and other parts come from our own experience.

4.1 The compile command in more detail

The key purpose of the

compile

command (

tf.keras.Model.compile

) is to tell the model i) which optimizer, ii)

which loss function, and iii) which metrics to use (if any). It is worthwhile to review for a moment all the options this

command provides. The compile command has the following format

Model.compile(

optimizer='rmsprop',

loss=None,

metrics=None,

剩余36页未读，继续阅读

易小侠

粉丝: 6603
资源: 9万+

环境科学中的神经网络自定义损失函数指南

CIRA大气模型扩展至2000公里高度的Matlab实现

一站式卫星数据下载与处理资源大全

激光雷达光束自动校准系统：设计、实现与精度验证

cira:Cira算法交易变得容易。 来自羊驼市场的羊驼贸易API的一个更简单的库

CIRA大气：大气条件达2000公里-matlab开发

UNB入侵检测数据库 CIRA-CIC-DoHBrw-2020数据集

Jira to Trello-crx插件

DoHlyzer_fixed

什么是近场辐射热传递？ ：在本教程中，我们解释了近场辐射传热 (NFRHT) 的基本原理和局限性。 [测试版]-matlab开发

元器件应用中的CIC系列CIC9102E集成电路实用检测数据

最新资源

cira:Cira算法交易变得容易。来自羊驼市场的羊驼贸易API的一个更简单的库

什么是近场辐射热传递？：在本教程中，我们解释了近场辐射传热 (NFRHT) 的基本原理和局限性。 [测试版]-matlab开发