Multi-Granularity Text and Scale-Aware Visual Encoding for Remote Sensing Image-Text Retrieval and Prompt-Based Scene Recognition

Source abstract

Remote sensing captions often combine domain terms, object relations, and scene context, while the corresponding images contain targets that occupy markedly different spatial scales. A single global text vector and fixed-scale visual aggregation can therefore discard information needed for image-text alignment. We develop a CLIP-style dual encoder with three additions: a text encoder that fuses word-level, phrase-level, and sentence-level representations; a visual encoder that associates four feature levels and uses SAM-guided local views during training and validation; and a semantic membership-aware contrastive loss with detached, bounded weights for semantically close negatives. On cleaned RSICD, the model obtains 87.5% mAP and 78.3% Top-1 accuracy for in-domain prompt-based scene recognition. Under scene-constrained retrieval with transductive dual-softmax reranking, it reaches 34.37% mean recall, compared with 30.58% for RemoteCLIP and 30.39% for fully fine-tuned CLIP. Removing the text encoder, scale-aware visual branch, or membership-aware loss lowers mAP by 6.8, 6.3, and 7.8 percentage points, respectively. The deployable dual encoder has approximately 59.7 million trainable parameters. These results apply to daytime optical imagery, a cleaned closed-category benchmark, in-domain prompts, and a fixed-gallery retrieval protocol.