RGB-T语义分割新突破：MEFNet融合专家策略提升性能 - CSDN文库

192 浏览量更新于2024-06-22 收藏 5.06MB PDF 举报

身份认证购VIP最低享 7 折!

领优惠券(最高得80元）

资源详情

资源推荐

To fuse the features, RTFNet(Sun et al., 2019b) and FuseSeg(Sun et al.,

2020) simply element-wise add two modal features for further processing.

MFNet(Ha et al., 2017) adopts concatenation to fuse RGB-Thermal fea-

tures. Element-wise summation or concatenation are not good strategies for

features fusion since they fuse the cross-modal features uniformly. There are

two kinds of cross-modal information: complementary information and con-

ﬂicting information. Complementary information has a positive impact on

segmentation, while conﬂict information acts like noises, producing a nega-

tive impact. Both complementary and conﬂict information are spatially dis-

tributed. Here, we use a compound attention mechanism consisting of modal

weight and channel attention to enhance complementary features and discard

noise features. The modal weight is a categorial distribution of modality and

sums to 1. Each modality acts as an expert branch of the multi-expert

network. SSMA(Valada et al., 2020) also uses a multi-expert network to

generate weights of diﬀerent modalities, but it generates a weight map for

each feature channel, which causes unnecessary calculations and increases

the complexity of the network.

As shown in Table 1, unlike the uniform fusion of RTFNet, FuseSeg, and

MFNet, our modal weight and channel attention can explicitly model the

validities of diﬀerent modalities and channel validities. Unlike GMNet(Zhou

et al., 2021) and SSMA(Valada et al., 2020), which redundantly calculate the

weight map for each channel, we decompose the weight matrix into more es-

sential modal weight matrix and channel attention vector, allowing the model

to focus more on the essential factors that cause diﬀerences in information

validities. Reduce the amount of computation required for fusion.

3. Method

3.1. Architecture

An overview of the proposed MEFNet is shown in Figures 1 and 2. The

proposed MEFNet use two independent encoders, which are multi-scale con-

volutional attention networks(MSCAN)(Guo et al., 2022), to extract RGB

and thermal features, considering that RGB and thermal images have diﬀer-

ent features. The information contained in the features obtained at diﬀerent

stages of the encoders is diﬀerent. With the increase of the encoding stage,

the features that can be extracted are more abstract and advanced. Low-level

features generally include low-level information such as the shape and outline

of the object, and high-level features are directly related to the categories of

7

剩余42页未读，继续阅读

byte_hit

粉丝: 18
资源: 6

会员权益专享

图片转文字

全年可省5，000元立即开通

最新资源

资源上传下载、课程学习等过程中有任何疑问或建议，欢迎提出宝贵意见哦~我们会及时处理！点击此处反馈