端到端前列腺癌复发预测：H&E图像的自我注意多实例学习与递归神经网络

版权申诉

38 浏览量更新于2024-07-06 收藏 2.47MB PDF 举报

"这篇论文是关于使用机器学习技术，特别是自我注意多实例学习（Self-Attention Multiple Instance Learning, SAMIL）和递归神经网络（Recurrent Neural Network, RNN），来实现前列腺癌复发的可解释性端到端预测。这项工作主要基于H&E染色图像，这是病理学中常用的一种诊断工具，用于观察组织结构和细胞形态。论文发表在2021年的Machine Learning for Health (ML4H)会议上，展示了如何通过深度学习模型提高临床决策支持的效率和准确性。文章指出，当前临床决策支持系统对于病理图像数据的依赖主要是基于高强度监督的注解，这种方式虽然直观易理解，但受限于专家的表现和可用的标注数据。为了克服这些限制，研究者提出了一种新的方法，结合了自我注意机制的多实例学习和RNN，以处理H&E图像中的复杂信息。自我注意机制允许模型在不同部分之间建立联系，理解图像的整体上下文；而RNN则能够捕捉时间序列或序列数据的动态变化，这在分析肿瘤生长模式时特别有用。在前列腺癌复发预测任务中，由于病变可能分布在多个区域，且并非所有区域都具有相同的病理意义，多实例学习是一种理想的框架。SAMIL在此背景下，能够识别出图像中的关键区域（即“阳性袋”），这些区域可能与癌症复发有关。递归神经网络则用于处理这些关键区域的信息流，以生成连续的预测。论文强调了解释性是机器学习模型在医疗应用中不可或缺的一部分，因此，该模型不仅提供预测结果，还能揭示影响预测的重要特征和图像区域。这有助于医生理解和信任模型的决策过程，从而在实际临床实践中更好地应用。这篇研究为病理学图像分析提供了一种新的、可解释的预测方法，通过集成深度学习的先进组件，提高了对前列腺癌复发预测的准确性和临床相关性。这种方法有望改进现有的诊断工具，并推动病理学向更加智能化和个性化的方向发展。"

eCaReNet

survival and hazard functions are deﬁned as

S(t

) = P (t

∗

> t

), (3)

h(t

) = P (t

∗

= t

∗

> t

j−1

), (4)

S(t

) =

k=0

(1 − h(t

)). (5)

The survival function is a monotonically decreasing

function, as can be seen from Equation 5.

An important characteristic of survival data is cen-

soring. Not all patients in the dataset experience an

event, either because they are lost to follow-up, their

event occurs after the end of documentation or they

never relapse. These patients are right-censored and

here t

∗

is not the time of the event, but the last ob-

served time without any event.

4.2. Model

As a base model for our proposed survival predic-

tion an InceptionV3 network (Szegedy et al., 2015),

pretrained on the ImageNet dataset (Russakovsky

et al., 2015), is chosen, while replacing the last lay-

ers to perform survival prediction as described be-

low. We chose InceptionV3 as it achieved best re-

sults in our experiments. We include two preceding

steps (4.2.1 and 4.2.2), before training our survival

model eCaReNet in a third step. Figure C.1 shows an

overview of the presented models and which datasets

these are trained on.

4.2.1. M

ISUP

In the ﬁrst step, we additionally pretrain the Incep-

tionV3 model to adapt it to our histopathology do-

main. Our model M

ISUP

takes images from the Glea-

son dataset as input (Figure C.1A), downsized with

bilinear interpolation to 1024 × 1024 pixels, and clas-

siﬁes these into one out of six classes (benign or one of

5 malignant ISUP classes). During training, a cross-

entropy loss is used. For training details and results,

see Appendix B.

4.2.2. M

Bin

In the second step, a binary classiﬁcation model M

Bin

is used to predict relapse within 2 years on the sur-

vival dataset (Figure C.1B). 2 years was chosen, as

it lies close to the median (26.8 months) of the re-

lapse times (44% of relapses earlier than 2 years).

For this, we took the model M

ISUP

and modiﬁed the

output to 2 classes. The input image is resized to

1024 × 1024 pixels as in M

ISUP

and a cross-entropy

loss is applied during training. As opposed to the ﬁrst

step, the prediction per image is saved and used in

the third step, which is the survival prediction model

eCaReNet, shown in Figure 1.

4.2.3. eCaReNet

Each image of the survival dataset is cut into square,

non-overlapping patches as input to eCaReNet (64

patches with 256×256 pixels each, see also Section 5).

As this model predicts the hazard over time, one out-

put node per time interval is needed. We chose 28 in-

tervals to cover a time span of 7 years with intervals

of 3-months length, covering the 90% of relapses that

occur prior to 7 years. For eCaReNet, only the ﬁrst

4 inception blocks of M

ISUP

are used to reduce over-

ﬁtting. The following global average pooling layer re-

duces the dimensionality. Then a self-attention block,

as proposed by Rymarczyk et al. (2021), models the

inﬂuence of each patch across all other patches. Next,

the aforementioned binary classiﬁcation is concate-

nated with the output vector of the self-attention

layer. This concatenated vector is repeated 28 times

to model the discrete time intervals. The current time

step is concatenated to each of these vectors. A gated

recurrent unit (GRU) layer (Cho et al., 2014) mod-

els the temporal dependency of the hazard rate in the

output, as proposed by Ren et al. (2019). At the end,

an attention-based MIL-layer weights the predictions

per patch and outputs a prediction per image, as pro-

posed in Ilse et al. (2018).

An individual survival curve per patient is obtained

through Equation 5. Using the normalized area un-

der the survival curve, the patient’s overall risk is es-

timated. Since a large area under the survival curve

indicates a low risk r and vice versa, the normalized

area is subtracted from one:

r = 1 −

i=1

S(t

) · |t

− t

i−1

|, (6)

with the last interval k at time t

(based on the sur-

vival time prediction in Xiao et al. (2020)). Since the

risk score is a single numerical value between 0 and

1, it eases comparison among patients.

As proposed by Kvamme et al. (2019), during

training a maximum likelihood loss is optimized. It

diﬀers for censored (c = 1) and uncensored (c = 0)

patients with the observed event time t

∗

. For un-

censored patients, the loss L

can be deﬁned by the

剩余15页未读，继续阅读

易小侠

粉丝: 6569
资源: 9万+

端到端前列腺癌复发预测：H&E图像的自我注意多实例学习与递归神经网络

基于递归神经网络的网络安全事件预测.pdf

卷积神经网络和递归神经网络（构建神经网络，进行数据处理，包括卷积神经网络和递归神经网络）

使用混合CNN-RNN进行时间序列预测：实现了用于时间序列预测的混合卷积神经网络-递归神经网络（RNN）。-matlab开发

股票短期预测新方法：相空间与递归神经网络结合

递归神经网络驱动的广告点击率精准预测

深度学习与递归神经网络在入侵检测系统中的应用比较

递归神经网络模型：高效处理大图像的新方法

细粒度图像识别的递归注意力卷积神经网络（RA-CNN）

交替卷积与递归池化：准递归神经网络QRNNs

递归神经网络在跌倒检测中的应用

最新资源