A Brief Introduction to Solving Inverse Imaging Problems with AI
From black-box models to physics-informed architectures
High-dimensional inverse problems are central to modern scientific discovery, appearing in fields ranging from medical imaging (MRI, CT) to astrophysics. As we enter the "exascale" era, instruments are producing data at unprecedented volumes and resolutions, pushing traditional computational methods to their limits.
We focus on the ubiquitous setting of linear inverse problems, which can be described by the equation:
where y represents the observed data, x is the underlying signal we wish to reconstruct (e.g., an image), Φ is the measurement operator describing how our instrument observes the world, and n denotes noise.
The challenge is that these problems are often "ill-posed": a unique solution may not exist, or the solution may be unstable with respect to noise. To solve them, we must inject prior information to regularize the problem.
The Challenge of the Square Kilometre Array
A prime example of this challenge is found in radio interferometry. The upcoming Square Kilometre Array (SKA) will effectively act as a giant telescope spanning continents, generating petabytes of visibility data to reconstruct gigapixel images of the radio sky.
Figure 1: Artist's impression of the Square Kilometre Array (SKA), which will be the world's largest radio telescope. Reconstructing images from its massive data volume represents a significant inverse problem. [Image credit: SKAO.]
In this regime, the sheer scale of the data and the computational cost of the measurement operator Φ render traditional Bayesian inference methods infeasible. We require new methods that are computationally efficient, explicitly physics-informed, and expressive enough to capture complex structures.
Reconstruction: Solving the Inverse Problem
Artificial Intelligence (AI) offers powerful new tools for these problems. We can trace the evolution of AI techniques from purely data-driven "black-box" models to hybrid architectures that explicitly integrate the physical measurement operator.
Figure 2: Schematic overview of deep learning-based imaging methods. (a) Fully learned reconstruction maps data directly to the signal. (b) Plug-and-Play (PnP) alternates between data consistency and a learned denoiser. (c) Unrolled architectures unfold optimization iterations into a neural network layers.
Fully Learned Reconstruction
The most straightforward approach is to learn a direct mapping from the observed data y (or a proxy like the "dirty" image Φ†y) to the underlying signal x. Architectures like the U-Net are typically trained to perform this reconstruction as a form of learned denoising or artifact removal (see Figure 2a).
This approach is highly computationally efficient, often requiring only a single pass through the network. This makes it attractive for real-time applications [3]. However, it suffers from significant limitations:
Black-box nature: The measurement operator Φ is not explicitly used during inference, meaning there is no guarantee of data consistency (y ≈ Φx). This can lead to "hallucinations" where the network invents features not present in the data.
Poor generalization: A network trained on one specific telescope configuration may fail catastrophically if the observing conditions change.
Plug-and-Play (PnP)
To address the limitations of black-box models, Plug-and-Play (PnP) approaches bridge the gap between deep learning and traditional model-based optimization (Figure 2b).
In classical iterative algorithms, we alternate between a data-fidelity step (ensuring y ≈ Φx) and a regularization step (imposing prior knowledge). PnP replaces the explicit regularization step with a learned off-the-shelf denoiser, such as a deep neural network [1].
Decouples physics and prior: The denoiser is trained independently, so it remains robust even if the telescope configuration changes.
Computationally slow: Converging to a solution often requires hundreds or thousands of iterations, each involving expensive calls to the measurement operator.
Unrolled Architectures
Unrolled networks offer a hybrid solution (Figure 2c). They "unfold" the iterations of an optimization algorithm into a deep neural network with a fixed number of layers [2]. This allows the network to be trained end-to-end, learning optimal parameters and regularizers from data.
To overcome the high computational cost of standard unrolled methods, we developed the Gradient U-Net (GU-Net) architecture [3, 4]. These integrate the measurement operator at multiple scales within a U-Net architecture, significantly reducing the computational burden while ensuring data consistency is actively enforced at every layer.
Generative AI Extensions
Finally, generative models (like GANs and diffusion models) can be integrated into these frameworks to enhance reconstruction fidelity. By learning expressive distributions, they can recover intricate non-Gaussian structures that simpler methods might smooth over.
For example, our RI-GAN framework [5] builds a conditional GAN on top of a physics-informed unrolled network. This combines the best of both worlds: the generator explicitly incorporates the measurement operator to ensure physical consistency, while adversarial training allows it to capture complex, realistic signal structures.
Summary
No single method is perfect, but the trend is clear: the most effective solutions for high-dimensional scientific imaging are those that combine the expressivity of modern AI with the robustness of physical models. By integrating the measurement operator directly into deep learning architectures—as seen in unrolled networks and their generative extensions—we can achieve reconstructions that are both high-fidelity and physically meaningful.
References
[1] Venkatakrishnan et al., Plug-and-Play priors for model based reconstruction, GlobalSIP (2013).
[2] Gregor & LeCun, Learning Fast Approximations of Sparse Coding, ICML (2010).
[3] Mars, Betcke, McEwen, Learned interferometric imaging for the SPIDER radio telescope, RASTI (2023), arXiv:2311.16876.
[4] Mars, Betcke, McEwen, Learned radio interferometric imaging for varying visibility coverage, RASTI (2025), arXiv:2405.08958.
[5] Mars, Liaudat, Whitney, Betcke, McEwen, Generative imaging for radio interferometry with fast uncertainty quantification (2025), arXiv:2507.21270.