DiffFormer: a Differential Spatial-Spectral Transformer for Hyperspectral Image Classification

Ahmad, Muhammad; Mazzara, Manuel; Distefano, Salvatore; Khan, Adil Mehmood; Ullo, Silvia Liberata

Abstract

Hyperspectral image classification (HSIC) presents significant challenges due to spectral redundancy and spatial discontinuity, both of which can negatively impact classification performance. To mitigate these issues, this work proposes the Differential Spatial-Spectral Transformer (DiffFormer), a novel framework designed to enhance feature discrimination and improve classification accuracy. At its core, DiffFormer incorporates a differential multi-head self-attention (DMHSA) mechanism, which accentuates subtle spectral-spatial variations by applying differential attention across neighboring patches. The architecture integrates spectral-spatial tokenization, utilizing 3D convolution-based patch embeddings, positional encoding, and a stack of transformer layers augmented with the SwiGLU activation function—a variant of the gated linear unit (GLU)—to enable efficient and expressive feature extraction. Additionally, a token-based classification head ensures robust representation learning, facilitating precise pixel-wise labeling. Extensive experiments on benchmark hyperspectral datasets demonstrate that DiffFormer consistently outperforms state-of-the-art (SOTA) methods in classification accuracy, computational efficiency, and generalizability. The source code is available at https://github.com/mahmad000/DiffFormer.

Citation

Ahmad, M., Mazzara, M., Distefano, S., Khan, A. M., & Ullo, S. L. (in press). DiffFormer: a Differential Spatial-Spectral Transformer for Hyperspectral Image Classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 18, 10419-10428. https://doi.org/10.1109/JSTARS.2025.3558889

• Journal Article Type: Article
• Acceptance Date: Apr 4, 2025
• Online Publication Date: Apr 8, 2025
• Deposit Date: Apr 25, 2025
• Publicly Available Date: May 7, 2025
• Journal: IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
• Print ISSN: 1939-1404
• Publisher: Institute of Electrical and Electronics Engineers (IEEE)
• Peer Reviewed: Peer Reviewed
• Volume: 18
• Pages: 10419-10428
• DOI: https://doi.org/10.1109/JSTARS.2025.3558889
• Keywords: Hyperspectral image classification; Spatial-spectral transformer; Differential attention
• Public URL: https://hull-repository.worktribe.com/output/5133267