LI Dong's group from Institute of Biophysics of Chinese Academy of Science, in collaboration with DAI Qionghai's group from Department of Automation of Tsinghua University and Dr. Jennifer Lippincott-Schwartz from the Howard Hughes Medical Institute (HHMI) devised the rationalized deep learning super-resolution microscopy (rDL-SRM). The related study was published in Nature Biotechnology on October 6, 2022, entitled "Rationalized deep learning super-resolution microscopy for sustained live imaging of rapid subcellular processes".

In this work, the joint research group reported a method of rationalized deep learning super-resolution microscopy (rDL-SRM), which implants deterministic physical model of specific microscopy into the network training and inference to eliminate/minimize the ill-posedness, model-uncertainty, spectral-bias, and demands for high-quality training dataset in current deep learning super-resolution (DLSR) methods. They demonstrate the rDL-SRM offers the first practical solutions for imaging a variety of dynamic intracellular processes at ultrahigh resolution in space and time for an unprecedented duration, including rapid kinetics inside motile cilium, the multi-phase transition of nucleolar proteins over light-sensitive mitosis, long-time interactions between membranous and membrane-less organelles, etc.

The three-branch neural network architecture of rDL-SRM

Specifically, instead of directly inferring the SR images as current super-resolution neural networks trained in an end-to-end manner, the rDL methods are applied to denoise the raw images, in which the high-frequency information beyond the diffraction limit is down-modulated as low-frequency Moiré fringes, and then reconstruct the SR image using the well-established SIM algorithm. This strategy eliminates the spectral-bias effect of recovering Moiré fringes, but also ensures the high-quality SR image reconstruction without spatial and temporal resolution degradation. After systematic comparison, they demonstrated the physical model rationalized three-branch architecture significantly improves the SR information by >10-fold and alleviates the model-uncertainty by 3~5-fold, providing physically feasible inferences that can be more readily generalized onto the regimes uncovered by training data.

The rDL concept is of wide compatibility and broad functionality. By utilizing the optical transfer function (OTF) of certain imaging modalities and the spatiotemporal continuity of noisy image series, they constructed a learnable Fourier noise suppression module (FNSM) and the temporally/Spatially interleaved self-supervised rDL (TiS/SiS-rDL) denoising networks, which can be implemented without the need to acquire ground-truth (GT) SR training data, but still yield results as good, sometimes even better, as the ones if supervised with GT, therefore, eliminating the need for extra training data.

The overview of multiple biological applications of rDL-SRM

rDL-SRM methods significantly push the boundaries of 2D/3D SR live-imaging significantly, achieving more than 30-fold longer imaging duration at a 10-fold higher frame rate than state-of-the-art SRM. Therefore, rDL-SRM methods enable to investigate the fine spatial details, rapid kinetics, and long-time dynamics of a wide variety of bioprocesses, where SR live-imaging has not yet done so:

· Achieving hour-long time-lapse super-resolution (97 nm) recording of cell spreading after cells were placed on a coverslip, which clearly resolved the coordinated dynamics between F-actin and myosin-IIA, generating the protrusion and retraction oscillation of lamellipodia.

· Characterizing the motile cilia beating pattern and frequency at SR imaging speed up to 684 Hz (limited by sCMOS camera read-out rate) for 60,000 frames without noticeable photobleaching for the first time.

· Revealing multiple novel behaviors of IFT trains colliding, remodeling, and turnaround in the midway of cilia, which was previously known of occurring only at the tip of cilia.

· Expanding the understanding of membranous and membrane-less organelles interaction. cGAS-DNA condensates commonly contact with ER throughout their biogenesis and trafficking, suggesting that cGAS-DNA condensates contain complex active constituents capable of tethering with ER membrane proteins and recruiting molecular motors for directional transportation.

· Analyzing the multi-phase transition process of nucleoli over mitosis at an endogenous level. We consistently observe the large liquid droplets of RPA49 in the inner fibrillar centre phase could partition into several independent foci, indicating that the active fission mechanism might play a role in the assembly of nucleoli.

· In 3D imaging scenario, TiS-rDL LLSM produced high-quality images of the collective behaviors of transport SiT vesicles for 10,000 volumes at a high volumetric imaging speed of 10 whole-cell volumes per second. And SiS-rDL LLMS enables the hours-long three-color SR imaging at seconds interval, which first scrutinizes the diverse strategies cells employ to properly segregate different organelles during mitosis.

Paper link: https://www.nature.com/articles/s41587-022-01471-3

Contact: LI Dong

Institute of Biophysics, Chinese Academy of Sciences

Beijing 100101, China

Email: lidong@ibp.ac.cn

(Reported by Dr. LI Dong's group)