首页spatial transform network

spatial transform network

时间: 2023-10-21 22:27:50 浏览: 96

1506.02025_Spatial Transformer Networks

Convolutional Neural Networks define an exceptionally powerful class of models, but are still limited by the lack of ability to be spatially invariant to the input data in a computationally and parameter efficient manner. In this work we introduce a new learnable module, the Spatial Transformer, which explicitly allows the spatial manipulation of data within the network. This differentiable module can be inserted into existing convolutional architectures, giving neural networks the ability to actively spatially transform feature maps, conditional on the feature map itself, without any extra training supervision or modification to the optimisation process. We show that the use of spatial transformers results in models which learn invariance to translation, scale, rotation and more generic warping, resulting in state-of-the-art performance on several benchmarks, and for a number of classes of transformations.

空间变换网络（Spatial Transform Network，简称STN）是一种用于深度学习中的空间几何变换的技术。它可以通过学习一个可微的仿射变换，自动调整输入图像的视角、位置和大小，从而提高模型的鲁棒性和泛化能力。 STN 由三个主要组件构成：本地网络（Localisation Network）、网格生成器（Grid Generator）和采样器（Sampler）。本地网络接收输入图像，并输出仿射变换的参数。这些参数被输入到网格生成器中，用于生成规范化网格。最后，采样器根据规范化网格对输入图像进行重采样，生成经过空间变换后的输出图像。通过引入 STN，模型可以在训练过程中自动学习执行空间变换所需的参数，而无需人工标注。这使得模型能够适应不同的视角、尺度和姿态，并具有一定的几何不变性。STN 在许多计算机视觉任务中都得到了广泛应用，例如图像分类、目标检测和图像生成等。

阅读全文