Aberration Correcting Vision Transformers for High-Fidelity Metalens Imaging

We fabricated a metalens with the PSF from Neural Nano optics. A metalens with a diameter of 500 µm and a focal length of 1 mm was designed based on the optimization of a polynomial phase equation. The SiN meta-atom library was generated using rigorous coupled-wave analysis simulations for circular pillars with a height of 750 nm. The widths of the selected meta-atoms ranged from 100 to 300 nm, with a lattice period of 350 nm.
A 750 nm thick SiN layer was deposited onto a SiO₂ substrate using plasma-enhanced chemical vapor deposition to fabricate the designed metalens. A 200 nm thick positive photoresist layer was spin-coated at 4000 RPM. The pattern of circular nano-pillar meta-atoms was then transferred onto the positive photoresist using electron beam lithography (Figure right (a)), with a dose of 3.75 C/m². To prevent charging, 100 µL of ESPACER was spin-coated at 2000 RPM for 30 seconds.
The exposed resist was developed in a 1:3 solution of methyl isobutyl ketone/isopropyl alcohol for 11 minutes. Subsequently, a 40 nm thick chromium layer was deposited as a hard mask using an electron beam evaporator (Figure right (b)). The unexposed photoresist was removed through a lift-off process in acetone at room temperature for 1 hour, leaving the Cr hard mask intact. Patterning was finalized using inductively coupled plasma etching with SF₆ and C₄H₈ gases for 10 minutes. Finally, the Cr hard mask was removed using a chromium etchant for 5 minutes. The fabricated metalens is shown on the left-side of the figure.

The above figure illustrates the image capture setup. An optical microscope system was set up to obtain images through the metalens. The images displayed on a 5.5-inch FHD display were captured using a CMOS camera coupled with a magnification system consisting of a 20x objective lens with 0.5 NA and a tube lens. The metalens was positioned such that its focal plane coincided with the focal plane of the objective lens using a linear motorized stage. Camera exposure time was adjusted using a white image prior to recording to prevent saturation. The point spread functions (PSFs) were then acquired using the same setup with 450 nm laser, 532 nm laser, and 635 nm laser for calibration and training of the model. The following images show the restoration results of our model on the real images captured with the fabricated metalens under this setup.

Our model comprises Multiple Adaptive Filters Guidance (MAFG) which produces different representations with various noise-detail balances, and a Spatial and Transposed self-Attention Fusion (STAF) module that aggregates features differently in encoder and decoder.

Multiple Adaptive Filters Guidance (MAFG)

We propose to use multiple Wiener filters to guide aberration correction with several distinct representations. It is not feasible to obtain an optimal Wiener filter with accurate SNR as noise distribution is unknown in the real world. Instead of estimating the noise distribution, we adopt multiple Wiener filters with different \( K \). Such design can approximate the ideal filter by employing \( M \) filters with different \( K \) in a wide range. Various representations are fed to the restoration model, and they can enrich the features complementarily, which in turn improves aberration correction.
We extend multiple Wiener filters to Multiple Adaptive Filters Guidance (MAFG), which determines \( K \) adaptively considering the image intensity. Image with higher intensity tends to have better signal quality, so brighter channels often experience less noise and are less sensitive to noise. Thus, we penalize noise less and capture more information for bright images by adjusting \( K \) with the image intensity. Also, we treat each channel differently to avoid suppressing high-SNR details unnecessarily, since metalens exhibit wavelength-dependent aberrations where each color channel suffers differently due to the chromatic responses.
To dynamically integrate these diverse representations, we employ a cross-attention module that reweights the filtered features according to their relevance to the median representation. This enables spatially adaptive correction, guiding the restoration network with features emphasized based on the severity of local aberration.

Spatial and Transposed self-Attention Fusion (STAF)

We propose a Spatial and Transposed self-Attention Fusion (STAF) module to further ameliorate image restoration. By leveraging both Spatial Attention (SA) and Transposed Attention (TA), the STAF module can capture diverse spatial dependencies. To fully realize the potential of SA and TA in image restoration, it is important to consider the distinct roles of the encoder and decoder in Transformers. The encoder focuses on capturing global context, emphasizing the overall structure and relationships within images—a critical aspect of identifying patterns and features corrupted in degraded images. Meanwhile, the decoder specializes in recovering fine local details and textures necessary for high-fidelity restoration. Therefore, STAF module applies SA and TA separately rather than alternately, as illustrated in Figure (a). Instead, as shown in Figure (b), STAF module fuses their features by assigning different weights in the encoder and decoder stages.

Image Aberration Correction

Video Aberration Correction

3D Reconstruction with Aberrated Images

To evaluate the performance of aberration correction tasks across image, video, and 3D reconstruction domains, we synthesized aberration on existing datasets by applying point spread functions to construct aberrated datasets. These tables present the quantitative results of various methods for each task, demonstrating the superior performance of our model. For video and 3D reconstruction tasks, only Non-uniform Aberration was applied.