作者：Haijin Zeng Kai Feng Shaoguang Huang Jiezhang Cao Yongyong Chen Hongyan Zhang Hiep Luong Wilfried Philips
使用多光谱滤波器阵列（MSFA）的高光谱成像系统在每个像素中只捕获一个光谱分量。使用高光谱去马赛克来恢复未测量的成分。尽管深度学习方法在这一领域显示出了前景，但它们仍然面临着一些挑战，包括非局部依赖性建模有限，缺乏对可能与周期性伪影有关的周期性MSFA模式的考虑，以及难以恢复高频细节。为了应对这些挑战，本文提出了一种新的去马赛克框架，即MSFA频率awareTransformer网络（FDM-Net）。FDM-Net集成了一种新的MSFA频率感知多头自注意机制（MaFormer）和一种基于滤波器的Fourierzero填充方法，分别重建难度较大的高通分量和相对容易的低通分量。Maformer的优点是它可以利用MSFA信息和
Hyperspectral imaging systems that use multispectral filter arrays (MSFA)capture only one spectral component in each pixel. Hyperspectral demosaicing isused to recover the non-measured components. While deep learning methods haveshown promise in this area, they still suffer from several challenges,including limited modeling of non-local dependencies, lack of consideration ofthe periodic MSFA pattern that could be linked to periodic artifacts, anddifficulty in recovering high-frequency details. To address these challenges,this paper proposes a novel de-mosaicing framework, the MSFA-frequency-awareTransformer network (FDM-Net). FDM-Net integrates a novel MSFA-frequency-awaremulti-head self-attention mechanism (MaFormer) and a filter-based Fourierzero-padding method to reconstruct high pass components with greater difficultyand low pass components with relative ease, separately. The advantage ofMaformer is that it can leverage the MSFA information and non-localdependencies present in the data. Additionally, we introduce a joint spatialand frequency loss to transfer MSFA information and enhance training onfrequency components that are hard to recover. Our experimental resultsdemonstrate that FDM-Net outperforms state-of-the-art methods with 6dB PSNR,and reconstructs high-fidelity details successfully.
论文链接：http://arxiv.org/pdf/2303.13404v1
更多计算机论文：http://cspaper.cn/