Spatial Spectral Transformer with Conditional Position Encoding for Hyperspectral Image Classification

Ahmad, Muhammad; Usama, Muhammad; Khan, Adil Mehmood; Distefano, Salvatore; Altuwaijri, Hamad Ahmed; Mazzara, Manuel

Abstract

In Transformer-based Hyperspectral Image Classification (HSIC), predefined positional encodings (PEs) are crucial for capturing the order of each input token. However, their typical representation as fixed-dimension learnable vectors makes it challenging to adapt to variable-length input sequences, thereby limiting the broader application of Transformers for HSIC. To address this issue, this study introduces an implicit conditional PEs (CPEs) scheme in a Transformer for HSIC, conditioned on the input token’s local neighborhood. The proposed SSFormer integrates spatial-spectral information and enhances classification performance by incorporating a CPE mechanism, thereby increasing the Transformer layers’ capacity to preserve contextual relationships within the HSI data. Moreover, SSFormer ensembles the cross-attention between patches and proposed learnable embeddings. This enables the model to capture global and local features simultaneously while addressing the constraint of limited training samples in a computationally efficient manner. Extensive experiments on publicly available HSI benchmarking datasets were conducted to validate the effectiveness of the proposed SSFormer model. The results demonstrated remarkable performance, achieving classification accuracies of 97.7% on the Indian Pines dataset and 96.08% on the University of Houston dataset.

Citation

Ahmad, M., Usama, M., Khan, A. M., Distefano, S., Altuwaijri, H. A., & Mazzara, M. (2024). Spatial Spectral Transformer with Conditional Position Encoding for Hyperspectral Image Classification. IEEE Geoscience and Remote Sensing Letters, 1-1. https://doi.org/10.1109/lgrs.2024.3431188

• Journal Article Type: Article
• Acceptance Date: Jul 16, 2024
• Online Publication Date: Jul 19, 2024
• Publication Date: 2024
• Deposit Date: Jul 20, 2024
• Publicly Available Date: Jul 23, 2024
• Journal: IEEE Geoscience and Remote Sensing Letters
• Print ISSN: 1545-598X
• Electronic ISSN: 1558-0571
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• Pages: 1-1
• DOI: https://doi.org/10.1109/lgrs.2024.3431188
• Keywords: Spatial Spectral Transformer (SSFormer); Hyperspectral Image Classification (HSIC)
• Public URL: https://hull-repository.worktribe.com/output/4740490