Our paper was accepted for Information Processing Society of Japan

To handle high-resolution images on finite computational resources, many research has been conducted on hierarchical networks to load features in only the most meaningful local regions. However, it is difficult to determine the correct number and location of patch regions at the appropriate scale in these methods. Then, incorrectly selected regions at different scales interfere with feature extraction and information integration. To solve this issue, we propose a hierarchical attention weighted network (HAWK-Net), which consists of a backbone network with differentiable Top-K selection and spatially gated blocks. The Top-K selected patches are identified from multiple image scaled features and extracted from an original high-resolution image. Then, patch features are aggregated via novel gate mechanism under the uncertainty of the predicted information. Not only can multi-scale information uncertainty be modeled, but it also controls the gradient to the feature network coming from patch images with low confidence in the region proposal network in feedback during training. Our model is a simple yet efficient network structure that can learn from multiple scales and patches and is capable of end-to-end training. Based on benchmarks of multiple high-resolution images, our model achieves even higher performance with lower memory usage and reduced computation time.