Super-resolution ophthalmoscopy: Virtually structured detection for resolution improvement in retinal imaging

Abstract

Quantitative retinal imaging is essential for advanced study and clinical management of eye diseases. However, spatial resolution of retinal imaging has been limited due to available numerical aperture and optical aberration of the ocular optics. Structured illumination microscopy has been established to break the diffraction-limit resolution in conventional light microscopy. However, practical implementation of structured illumination microscopy for in vivo ophthalmoscopy of the retina is challenging due to inevitable eye movements that can produce phase artifacts. Recently, we have demonstrated the feasibility of using virtually structured detection as one alternative to structured illumination microscopy for super-resolution imaging. By providing the flexibility of digital compensation of eye movements, the virtually structured detection provides a feasible, phase-artifact-free strategy to achieve super-resolution ophthalmoscopy. In this article, we summarize the technical rationale of virtually structured detection, and its implementations for super-resolution imaging of freshly isolated retinas, intact animals, and awake human subjects.

Keywords

Retina photoreceptor super-resolution scanning laser ophthalmoscopy virtually structured detection modulation transfer function optical transfer function

Impact statement

High-resolution retinal imaging is important for eye disease detection and treatment assessment. This article summarizes technical rationale and experimental implementations of virtually structured detection (VSD) for resolution improvement in retinal imaging. In conjunction with rapid line-scan imaging and digital registration to minimize the effect of eye movements, VSD enables in vivo super-resolution ophthalmoscopy of individual rod and cone photoreceptors, promising a solution for better management of eye diseases that can cause photoreceptor dysfunctions.

Introduction

The retina is responsible for capturing photons, converting light energy to bioelectric signal, and preliminary visual information processing. Given its delicate function, the retina can be frequently targeted by eye diseases that can cause vision loss and even blindness. Retinal examination plays one essential role in eye disease detection and treatment assessment. In addition to traditional fundus photography, multiple imaging modalities such as scanning laser ophthalmoscopy (SLO),¹ fluorescein angiography (FA),² optical coherence tomography (OCT),³ and OCT angiography (OCTA)⁴ have been established to enable quantitative assessment of the retina. In principle, a better imaging resolution corresponds to a better opportunity to reveal subtle abnormalities at the early stage of eye diseases. Therefore, spatial resolution of retinal imaging is never more than enough to improve eye disease detection and treatment assessment. Adaptive optics (AO) has demonstrated excellent capability to compensate for optical aberrations of the ocular optics for resolution improvement in fundus camera,⁵ SLO,⁶ and OCT⁷ systems. However, the spatial resolution of conventional optical instruments is diffraction limited. The numerical aperture (NA) of the human eye is limited by the available pupil size, restricting the spatial resolution of in vivo retinal imaging.

By rejecting out-of-focus light, confocal microscopy has been well established to enhance image contrast and sectioning capability.⁸ When the confocal pinhole in the confocal microscopy is small enough (e.g. ≤0.5 Airy Disk Diameter, ADD), the lateral resolution limited by the diffraction can be surpassed. Sub-ADD pinhole detection⁹ and annular pupil illumination¹⁰ have been demonstrated for resolution improvement in confocal ophthalmoscopy. However, the small pinhole in the confocal system also discards in-focus light to reduce the signal-to-noise ratio (SNR) which can be essential for imaging biological tissues, such as the retina in which the useable signal level is frequently low. Pixel reassignment technique has been explored as one alternative to confocal imaging for resolution improvement in SLO without SNR loss.¹¹ Without discarding any useable signal, structured illumination microscopy (SIM) has been established to break the diffraction limit for resolution improvement in light microscopy.¹² However, direct implementation of SIM for in vivo ophthalmoscopy of the human retina is challenging due to unavoidable phase-artifact because of eye movements. Randomly shifted pattern illumination has been theoretically proposed for in vivo SIM of the retina, but the experimental implementation is yet to be validated.¹³

Recently, we have demonstrated the feasibility of using virtually structured detection (VSD) as one alternative to SIM for super-resolution imaging.¹⁴ By providing the flexibility of digital compensation for the effect of eye movements, the VSD provides a feasible, phase-artifact-free strategy to achieve resolution improvement in in vivo retinal imaging.^15,16 In this minireview, we summarize the basic principle of SIM, technical rationale of VSD, and its implementation in scanning laser microscopy (SLM) of freshly isolated retinas, and SLO for super-resolution imaging of intact animals and awake humans.

Basic principle of SIM for resolution improvement

SIM has been developed to surpass the optical resolution limit by a factor of two.¹² As shown in Figure 1(a), the resolution improvement in SIM is based on the recording and processing of a sequence of images with structured pattern illumination at different orientations and phases. As shown in Figure 1(b), the SIM image contrast and resolution can be significantly improved, compared to conventional light microscopy.¹⁷

Figure 1.

(a) Basic principle of SIM. (a1) Moiré fringes, i.e. the dark vertical stripes, are observed in the overlap region of two superposed line patterns. (a2) Spatial resolution of conventional light microscopy is diffraction limited. The observable region of reciprocal space produced by an objective is limited at its edge by the highest spatial frequencies (0.61λ/NA). (a3) A sinusoidally striped illumination pattern has only three Fourier components, i.e. the 0th (red dot) and ±1st (yellow dots) order diffraction components. If the pattern is at the limit of resolution, the 1st order spots fall at the very edge of the “observable field”. (a4) By frequency mixing, the observable region contains, in addition to the normal image of spatial frequencies (center circle), two new offset frequency images, each centered on the edge of the original field. The offset images contain spatial frequencies that are not observed in conventional light microscopy. (a5) From a set of images prepared from three phases at each of three orientations, a super-resolution image can be generated that has twice the spatial resolution in conventional light microscopy. (b) Comparative conventional fluorescence light microscopy (b1) and SIM images (b2) of a living HeLa cell stained with MitoTracker Green. Source: Reprinted from Gustafsson¹² and Murphy et al.¹⁷ (A color version of this figure is available in the online journal.)

SIM has been extensively used for high resolution microscopy of biological cells and tissues.^18,19 However, its practical implementation for imaging non-stationary targets is challenging because the multiple images with different orientation and phase illumination are required (Figure 1(a5)). Any movement between the sequential images can damage the phase relationship for super-resolution reconstruction. In theory, SIM can also be realized in a point scanning system through spatiotemporal modulation, either by modulating light source intensity in the illumination arm with a digital camera (Figure 2(a)) or by placing a moving mask in the light detection arm with a single-channel photodetector (Figure 2(b)).²⁰ The purpose of the spatiotemporal modulation of the point scanning illumination in Figure 2(a) is to produce a set of images with equivalent structured illumination in SIM for super-resolution reconstruction. The spatiotemporal modulation of the detection path illustrated in Figure 2(b) indicates that the structured modulation in illumination path is equivalent to that in the detection path.²⁰ In Figure 2(a) and (b), the objective is shared for both light illumination and detection. If we assume the optical property, i.e. optical transfer function (OTF), is the same for illumination and detection pathways, the spatiotemporal modulations in illumination path in Figure 2(a) and detection path in Figure 2(b) can be equivalent. Given only a single point is illuminated at each time point, crosstalk noise among adjacent volumes can be reduced, compared to wide-field SIM, and thus image contrast and penetration capability can be further improved. However, the spatiotemporal modulation of the illumination/detection arm still requires nine frames to reconstruct a high-resolution image. Sample motion during acquisition of these nine frames leads to reconstruction artifacts, making it difficult for imaging non-stationary targets such as the retina with inevitable eye movements.

Figure 2.

(a) Schematic of super-resolution SLM through spatiotemporally modulated light illumination. The peak intensity of the focused excitation spot is temporally modulated in a controlled manner, while the spot scans across the specimen. The epi-collected ﬂuorescence signal, after spectrally separated from the excitation light by a dichroic and/or a ﬁlter, is imaged onto a nondescanned camera. Each complete frame scanning with a particular modulation sequence builds up one picture cumulatively. Several such pictures are generated with a set of modulation sequences that are phase-shifted in space in order to produce images equivalent to structured illumination required in SIM for super-resolution reconstruction. (b) Schematic of super-resolution SLM through modulated light detection. A mask with modulated transmittance is positioned at the image plane of the epi-ﬂuorescence signal, and a single element large-area nondescanned detector such as a PMT is placed behind the mask. For each scanning position, the detector sums up all transmitted signals through the mask and assigns the integrated intensity to the single pixel corresponding to the current scanning position. After each complete frame is scanned with a particular mask, one PMT-picture is built up. Several such PMT-pictures are generated with a set of masks whose modulated transmittance is phase-shifted in space, in order to produce images equivalent to structured illumination required in SIM for super-resolution reconstruction. Source: Reprinted from Lu et al.²⁰ (A color version of this figure is available in the online journal.)

Technical rationale of VSD-based super-resolution SLM

In principle, a digital camera can be used to record the 2D light profile, corresponding to a point illumination, and thus VSD can be implemented for virtually spatial modulation. In other words, the VSD implementation on 2D light profiles is equivalent to the function of the combined system that includes the photomultiplier tube (PMT), the moving mask, and corresponding laser scanning modulation required in Figure 2(b).²⁰ Differently, the VSD implementation on 2D light profiles can overcome the technical challenge of super-resolution imaging of non-stationary targets. If the camera speed is fast enough, i.e. kHz, the intra-frame movement within each 2D light profile can be ignored. The inter-frame movement can be digitally compensated by precise image registration among raw 2D light profiles, before implementing VSD for super-resolution reconstruction. Therefore, VSD-based super-resolution imaging holds the promise for in vivo retinal imaging to tackle the problem of eye movements.

Figure 3(a) shows an optical diagram of the VSD-based SLM.¹⁴ As conventional SLM, the illumination light is focused to a single spot on the sample. The X- and Y- scanners steer the spot across the sample to produce a raster scanning pattern. The light reflected from the sample is de-scanned by the same scanners and focused to the detection apparatus. Instead of using a single-pixel detector, such as one avalanche photodiode (APD) or PMT, for collecting total reflected light from the sampling volume in conventional SLM, a 2D digital camera is used to map the light profile patterns of each sampling point (Figure 3(b)). For a super-resolution image with frame size $m \times n$ (e.g. 400 × 300) pixels, $m \times n$ (e.g. 120,000) 2D light profile patterns are recorded sequentially from individual scanning points. The sequential 2D light profile patterns can be used for VSD processing to produce the spatially modulated transmittance images (Figure 3(c)), which are equivalent to the images with structured illumination required in SIM for following super-resolution reconstruction.¹⁴

Figure 3.

VSD-based super-resolution SLM. (a) Optical diagram of the SLM system. SLD: superluminescent laser diode; CO: collimator; BS: beam splitter; L1-L3: lens; and OB: objective. (b) Methodology of VSD. A 2D reflectance profile (b1) is recorded, corresponding to each point illumination. Digital sinusoidal modulations can be implemented to the 2D image profile at orientations θ = 0° (b2), 120° (b3), and 240° (b4). (c) The modulated 2D reflectance profile of each sampling point in (b) is used to construct the spatially modulated transmittance images at three orientations (θ₁ = 0°, θ₂ = 120°, and θ₃ = 240°) with three phase shifts (α₁ = 0°, α₂ = 120°, and α₃ = 240°). The spatially modulated transmittance images are equivalent to the images with structured illumination required in SIM for following super-resolution reconstruction. (d) Comparative imaging of one freshly isolated frog retina with conventional SLM (d1) and VSD-based super-resolution SLM (d2). (d3) The white curve and the red curve are normalized intensity profiles along the white line in (d1) and the red line in (d2), respectively. Source: Modified from Lu et al.¹⁴ and Zhi et al.³⁰ (A color version of this figure is available in the online journal.)

In the spatial domain, a point spread function (PSF) can be used to evaluate the resolution of one imaging system. For the imaging system in Figure 3(a), we assume that the PSFs of the illumination ( $h_{i l} (x, y)$ ) and detection ( $h_{d e} (x, y)$ ) paths are identical. With incoherent illumination, the theoretical PSF is¹

h_{i l} (x, y) = h_{d e} (x, y) = \frac{Ω^{2}}{π} {(\frac{J_{1} (Ω ρ)}{Ω ρ})}^{2}

(1)

where J₁ is the first-order Bessel function, Ω = 2πNA/λ and

ρ = \sqrt{x^{2} + y^{2}}

. The resolution of conventional SLM is defined as the radius of the Airy disc R.

In the Fourier domain, the corresponding cutoff frequency of the PSF can be expressed as

f_{c} = 1 / R

(2)

In other words, only frequencies below the cutoff frequency are able to pass through the conventional SLM system

- f_{c} \leq f \leq f_{c}

(3)

In VSD, at the scanning position $(x, y)$ , the image acquired by the camera with digital modulations is

I ({x, y, x}_{o}, y_{0}) = \{[h_{i l} (x_{0} - x, y_{0} - y) s (x_{o}, y_{0})] \otimes h_{d e} (x_{o}, y_{0})\} m (x_{o}, y_{0})

(4)

By integrating equation (4) we have

p (x, y) = \iint \iint h_{i l} (μ - x, ν - y) s (μ, ν) h_{d e} (x_{o} - μ, y_{0} - ν) m (x_{o}, y_{0}) d μ d ν d x_{o} d y_{0}

{= h}_{i l} (x, y) ⨂ \{s (x, y) [h_{d e} (x, y) ⨂ m (x, y)]\}

(5)

where

⨂

denotes convolution and

m (x, y)

is the modulation function

m (x, y) = \cos [2 π f_{m} (x \cos θ + y \sin θ) + α]

(6)

Considering $h_{d e} (x, y) {= h}_{i l} (x, y)$ , exchanging $h_{d e} (x, y)$ and $h_{i l} (x, y)$ , and then rearranging equation (5) yield

p (x, y) = \{[m (x, y) \otimes h_{i l} (x, y)] s (x, y)\} \otimes h_{d e} (x, y)

(7)

Equation (7) exactly represents the acquired image of the conventional wide-field SIM in which the modulation function $m (x, y)$ is implemented spatially in the illumination arm. The Fourier transform of equation (5) after substituting equation (6) into equation (5) yields

\begin{array}{l} \tilde{p} (f_{x}, f_{y}) {= \tilde{h}}_{i l} (f_{x}, f_{y}) [\begin{array}{l} \tilde{s} (f_{x} - f_{m} c o s θ, f_{y} - f_{m} s i n θ) e^{i α} \\ + \tilde{s} (f_{x} + f_{m} c o s θ, f_{y} + f_{m} s i n θ) e^{- i α} \end{array}] \end{array}

(8)

where constant coefficients are ignored, and

\tilde{s} (f_{x}, f_{y})

represents the spectrum of the retina. Thus, the higher frequency

\tilde{s} (f_{x}, f_{y})

is shifted toward the lower passing band of

{\tilde{h}}_{d e} (f_{x}, f_{y})

. If we set

f_{m} = f_{c}

, the retrievable frequency is

- {2 f}_{c} \leq f \leq 2 f_{c}

(9)

In other words, the theoretical resolution is enhanced by a factor of two, which is equivalent to SIM.^12,14,20

We used freshly isolated frog (Rana pipiens) retinas for a functional test of VSD-based SLM. According to equation (2), given that λ = 830 nm, the lateral resolution of the conventional SLM 5X objective with 0.1 NA was 5 µm. Given that the diameter of frog photoreceptors varies from 1 to 8 µm (rods ∼5–8 µm, and cones ∼1–3 µm),^21,22 the conventional SLM with the theoretical lateral resolution of 5 µm resolved photoreceptors partially, while VSD-based super-resolution imaging delineated individual photoreceptors unambiguously (Figure 3(d)). In addition to the super-resolution SLM, we also demonstrated the feasibility of using VSD for resolution improvement in OCT.²³

In vivo super-resolution ophthalmoscopy of animal retinas

VSD-based SLM in Figure 3 has been demonstrated for resolution improvement in imaging of freshly isolated retinas.¹⁴ The VSD method requires neither physical modulation of the light source intensity in the illumination arm nor a physical mask in the light detection arm. Without the system complexity of SIM for precise light phase and pattern controls, the VSD promises an easy, low-cost, and phase-artifact-free strategy to achieve super-resolution imaging. However, practical application of the VSD for in vivo retinal imaging is still challenging due to the limited frame-speed. For the single-point scanning prototype in Figure 3, ∼160 s is required for collecting 2D reflectance profiles to produce a super-resolution image with frame size of 400 × 400 pixels.

In order to minimize the effect of eye movements, line-scanning strategy has been demonstrated for high-speed in vivo retinal imaging.²⁴ Recently, we demonstrated the feasibility of the line-scanning strategy for rapid VSD-based super-resolution imaging of both isolated retinas²⁵ and intact frog eyes.^26,27 Figure 4 shows the experimental setup and representative in vivo super-resolution images captured with 127 frames/s speed. As shown in Figure 4(b), the VSD-based SLO enabled unambiguous observation of individual retinal photoreceptors (Figure 4(b1)) and ganglion fibers (Figure 4(b2)).²⁶ By providing high-spatial (µm) and high-temporal (∼10 ms) resolutions, the VSD-based super-resolution SLO has been used to validate in vivo monitoring of transient retinal phototropism evoked by oblique light stimulation.²⁷

Figure 4.

VSD-based super-resolution SLO. (a) Photographic illustration of the experimental setup. (b) Representative in vivo super-resolution imaging of photoreceptor (b1) and ganglion fiber (b2) layers. (c) In vivo monitoring of photoreceptor movement due to oblique light stimuli. (c1) The time-magnitude courses of photoreceptor movement corresponded to three stimulus intensity levels. Each trace is an average of 12 datasets recorded from six different retinal samples. The colored area that accompanies each trace illustrates the standard deviations. Shaded area represents the 500-ms stimulation period. PR: photoreceptor. The means and standard deviations of peak amplitude (c2) and time-to-peak (c3) of the traces in (c1). Source: Reprinted from Liu et al.²⁶ (b) and Lu et al.²⁷ (c). (A color version of this figure is available in the online journal.)

In vivo super-resolution ophthalmoscopy of human retinas

As shown in Figure 4, VSD has been demonstrated for in vivo super-resolution retinal imaging in anesthetized frogs. However, practical implementation of VSD for in vivo retinal imaging of awake humans is challenged by two factors: (1) involuntary eye movements;^28,29 and (2) uncertain cut-off frequency of the ocular optics of each subject. In principle, increased imaging speed can minimize the intra-frame blurs, and digital image registration can be applied to raw image sequence to compensate for the inter-frame movements before the VSD processing for super-resolution reconstruction. Accurate assessment of the cut-off frequency of ocular optics, i.e. choosing a proper frequency shift in VSD processing is essential. An insufficient frequency shift will result in decreased high-frequency components being involved for image reconstruction and result in reduced resolution improvement. In contrast, an excessive frequency shift will introduce high frequency noises and artifacts to degrade the image quality. For our preliminary study with anesthetized frogs (Figure 4), which are known to have relatively low aberration, empirical frequency shift was adopted for VSD processing.^26,27 However, optical property of human subjects can be significantly variable among different subjects, making the selection of a proper frequency shift difficult. Recently, we developed one objective method to retrieve the modulation transfer function (MTF) of the imaging system to enable an objective identification of the cut-off frequency of ocular optics and validated it for in vivo retinal imaging of awake human subjects.¹⁵

Figure 5 illustrates the experimental setup, i.e. a line-scan SLO system. A near-infrared SLD with 830 nm central wavelength and 60 nm bandwidth was used as the light source. The line illumination was achieved by using a cylindrical lens (CL, Figure 5(b)) to condense the illumination light in one dimension. The galvo scanning mirror was placed at the conjugate plane of the pupil to minimize the vignetting effect. The reflected light from the retina with line-illumination was de-scanned by the scanning mirror and recorded by a high-speed CMOS camera (FastCam Mini Ax50, Photron) as a 2D line-profile pattern (Figure 5(b)). The line profile of the illumination was along the X-direction, and the scanning was performed in the Y-direction. The 2D line-profile patterns were acquired at a speed of 25,000 Hz to minimize the intra-frame blur correlated with eye movements.

Figure 5.

Photograph (a) and schematic diagram (b) of the super-resolution SLO. SLD: superluminescent diode; CO: collimator; CL: cylindrical lens; L1–L4: lens; BS: beam splitter; SM: scanning mirror. The illumination light is projected onto the retina as a focused line. A representative 2D line-profile pattern is shown in the dashed window. Source: Reprinted from Lu et al.¹⁵ (A color version of this figure is available in the online journal.)

Human subjects, without known ocular diseases, were recruited for functional validation of the system in Figure 5. The experiment was conducted in a room with minimized ambient light to maximize the available pupil size, without the requirement of pharmacologically pupil dilation. Digital registration of sequential line-profile patterns was implemented to compensate for inter-frame shifts correlated with eye movements before the VSD processing. Basic principle of VSD processing has been described previously.^14,25,30 Here we focus on the process of retrieving the MTF of the imaging system. As the modulation is achieved by using digital mask, two modulation phases (α = 0 and π/2) can be employed to solve equation (8)

{\tilde{h}}_{i l} (f_{x}, f_{y}) {\tilde{h}}_{d e} (f_{x}, f_{m}) \tilde{s} (f_{x}, f_{y} \pm f_{m}) = {\tilde{p}}_{f_{m}}^{0} (f_{x}, f_{y}) \pm i {\tilde{p}}_{f_{m}}^{π / 2} (f_{x}, f_{y})

(10)

Taking the case of $\tilde{s} (f_{x}, f_{y} - f_{m})$ as an example and omitting $f_{x}$ for simplicity (assuming the scanning is performed in y-direction), equation (10) can be rewritten as

{\tilde{h}}_{i l} (f_{y}) {\tilde{h}}_{d e} (f_{m}) \tilde{s} (f_{y} - f_{m}) = {\tilde{p}}_{f_{m}}^{0} (f_{y}) - i {\tilde{p}}_{f_{m}}^{π / 2} (f_{y})

(11)

As $f_{m}$ is the modulation frequency of the digital mask, it can be assigned with different values, such as $f_{1} \dots f_{i}$ . For each $f_{m}$ , let $f_{y}$ always equal to $f_{m}$ . Taking $f_{m} = f_{i}$ as an example and assuming the OTF of illumination and detection path are same

{\tilde{h}}_{d e}^{2} (f_{i}) \tilde{s} (0) = {\tilde{p}}_{f_{i}}^{0} (f_{i}) - i {\tilde{p}}_{f_{i}}^{π / 2} (f_{i})

(12)

where

\tilde{s} (0)

is an invariant complex. By scanning

f_{i}

across 0 to

1 / (2 \times pixelResolution)

, a series of

{\tilde{p}}_{f_{i}}^{0} (f_{i}) - i {\tilde{p}}_{f_{i}}^{π / 2} (f_{i})

, which reflects the profile of

{\tilde{h}}_{d e}^{2} (f_{i})

, can be obtained.

Figure 6(a) and (b) illustrate representative MTF ( $|{\tilde{h}}_{d e} (f_{y})|$ ) profiles obtained from three subjects and retinal locations. The MTF profiles exhibit significant differences between subjects and retinal locations. To balance the resolution improvement and noise artifacts, the frequency where MTF attenuates to 1/e of its maximum was selected as the cut-off frequency for VSD processing. To verify the effect of cut-off frequency selection, VSD images were reconstructed from the same datasets with different frequency shift values (Figure 6(c)). Figure 6(c3) shows the VSD image reconstructed with the cut-off frequency, while Figure 6(c2) and (c4) corresponds to the insufficient and excessive frequency shift, respectively. Compared with the other two images (Figure 6(c2) and (c4)), individual photoreceptors can be clearly identified in Figure 6(c3), indicating the importance of identifying a proper frequency shift value for the VSD processing.

Figure 6.

(a1) Representative spectra of ${\tilde{h}}_{i l} (f_{y}) {\tilde{h}}_{d e} (f_{m}) \tilde{s} (f_{y} - f_{m})$ with three different $f_{m}$ values. (a2) Profile of the reconstructed MTF ( $|{\tilde{h}}_{d e} (f_{y})|$ ). (b1) MTF profiles retrieved at ∼2° eccentricity from three different subjects. (b2) MTF profiles retrieved at ∼2°, 5°, and 10° eccentricities from Subject 1. (c1) EWF image reconstructed from the same raw dataset acquired at ∼2° eccentricity of Subject 1. Corresponding VSD-based super-resolution images reconstructed with different frequency shifts: 0.02 cycles/μm (c2), 0.06 cycles/μm (c3), and 0.14 cycles/μm (c4). Source: Reprinted from Lu et al.¹⁵ (A color version of this figure is available in the online journal.)

Figure 7(a) and (b) show the representative equivalent wide-field (EWF) image and the VSD image reconstructed from the same dataset acquired at ∼3° eccentricity from fovea. Compared to the EWF image (Figure 7(a)), the VSD image (Figure 7(b)) revealed enhanced resolution and contrast to resolve individual photoreceptors and visualize the projection of micro-capillary structures (yellow arrowheads in Figure 7(b1)). Based on the high speed VSD image sequence, motility processing (standard deviation map of time-lapse VSD images) was applied to further enhance the contrast of individual photoreceptors. The visibility of photoreceptor boundaries was significantly improved in Figure 7(c). Because the signal intensity of rod photoreceptors was typically weaker compared to cones, logarithmic scale display was able to enhance the visibility of rod photoreceptors (yellow arrows in Figure 7(d)). Surprisingly, sub-cellular structure, i.e. the dark center region (red arrows in Figure 7(d)), was observed in some cone photoreceptors. We speculate that the sub-cellular dynamics may reflect motility of cell membrane, disc, or waveguide property of retinal photoreceptors. Further investigation is required to understand the mechanism of the sub-cellular motility properties, which may provide new biomarkers of functional assessment of retinal photoreceptors.

Figure 7.

EWF (a1) and VSD-based super-resolution (b1) images reconstructed from the same dataset with center at ∼3° eccentricity from the fovea. Arrowheads in (b1) identify the projection of retinal vasculatures on the photoreceptor layer. (c1) and (d1) Standard deviation map of a time-lapse recording of 80 super-resolution images in linear and logarithmic scale, respectively. Images are cropped to 400 × 512 pixels for display. (a2), (b2), (c2), and (d2) show magnified views of the areas specified by the yellow dashed rectangles in (a1), (b1), (c1), and (d1), respectively. The yellow and red arrowheads in (d2) specified rod photoreceptors, and the sub-cellular structures at the center of photoreceptors revealed by motility processing. Scale bars represent 50 μm. The scanning direction is along Y-direction. Source: Reprinted from Lu et al.¹⁵ (A color version of this figure is available in the online journal.)

Figure 8 shows representative montaged images to demonstrate the capability of super-resolution imaging at retinal regions with different eccentricities relative to the foveal center (Asterisk, Figure 8).

Figure 8.

Montaged super-resolution (a) and motility processed images (b) of photoreceptor mosaics. Asterisk points to the fovea center. Scale bars represent 100 μm. Reprinted from Lu et al.¹⁵ (A color version of this figure is available in the online journal.)

Discussion

In summary, VSD-based super-resolution SLM and SLO have been developed and validated for resolution improvement in both in vitro and in vivo retinal imaging. Instead of using a single-channel photodetector, such as PMT in a conventional SLM system, a digital camera is employed to capture 2D light profiles corresponding to individual point illumination. VSD can be implemented to the 2D light profiles to achieve equivalent information in SIM to surpass the diffraction limit of optical resolution in the lateral direction.

Wide-field SIM has been extensively used for high resolution study of biological cells and tissues. Practical implementation of wide-field SIM for in vivo retinal imaging is difficult, due to inevitable eye movements. Because multiple images are required to be collected with different illumination orientations and phases (Figure 1(a)), any shift among the sequential images can distort the location/phase relationship for super-resolution reconstruction. The demonstrated VSD method uniquely overcomes the technical challenges, i.e. eye movement and uncertain cut-off frequency of the ocular optics, for in vivo retinal imaging.

In order to minimize the effect of the eye movements, a rapid line-scanning was adopted for the super-resolution SLO in Figure 5. By providing 25,000 Hz speed for recording 2D profiles corresponding to focused line illumination, the intra-frame blur due to eye movements within the 40 µs time interval can be omitted. The inter-frame movement could be digitally compensated by precise image registration among sequential 2D light profiles, before implementing VSD for super-resolution reconstruction. In order to pursue robust in vivo super-resolution imaging of human retina, an objective method has been developed to derive MTF from digital reflectance profiles to enable quantitative estimation of the cut-off frequency required for reliable VSD processing. In conjunction with rapid line-scan imaging and digital registration to minimize the effect of eye movements, VSD enabled resolution improvement for unambiguous observation of individual retinal photoreceptors without the involvement of AO (Figure 7). Dynamic motility processing further enhanced the visualization of individual photoreceptors and allowed differential identification of individual rod and cone photoreceptors (Figures 7 and 8). Interestingly, sub-cellular dark centers (red arrows in Figure 7(d)) were observed in some cone photoreceptors. Because not all cone photoreceptors show such sub-cellular dark centers, we speculate that the sub-cellular dynamics may reflect different functions relative to the motility property of cell membrane, disc, or waveguide property of individual cone photoreceptors. Further investigation is required to understand the mechanism of the sub-cellular motility properties, which may provide new biomarkers of functional assessment of cone photoreceptors.

As a proof-of-concept study, the system in Figure 5 was constructed with all commercially available components. In principle, further optimization of the system, such as customized optical design, can further improve the imaging performance and promise a cost-effective method to foster clinical deployments of quantitative analysis of retinal photoreceptors.

The high-speed and super-resolution imaging with single photoreceptor resolution also promises a feasible solution to advance functional intrinsic optical signal (IOS) imaging,^31,32 also termed as optophysiology³³ or optoretinography (ORG),^34–36 for objective assessment of retinal physiology. It is known that eye diseases, such as age-related macular degeneration (AMD),^37–39 retinitis pigmentosa (RP),^40,41 diabetic retinopathy (DR),^42,43 can cause photoreceptor dysfunctions. Early detection and therapeutic assessment are essential steps to prevent vision loss due to eye diseases. Stimulus-evoked IOS, which is tightly correlated with the activation phase of phototransduction in retinal photoreceptors, has been demonstrated.^44,45 Recent studies suggested that the photoreceptor-IOS attributes to transient outer segment (OS) shrinkage in stimulus activated photoreceptors.^46,47 Therefore, high resolution is essential for robust detection of the localized photoreceptor-IOS due to transient OS change. Using the VSD-based super-resolution imaging system, transient photoreceptor movement has been demonstrated in intact frog eyes²⁷ (Figure 4). We anticipate that further development of the VSD-based super-resolution ophthalmoscopy may provide a feasible strategy to enable functional assessment of human photoreceptors at cellular resolution.

Conclusions

VSD provides a new pathway to achieve super-resolution SLO for high resolution imaging of the retina. In conjunction with rapid line-scan imaging and digital registration to minimize the effect of eye movements, VSD enabled resolution improvement for unambiguous observation of individual retinal photoreceptors without the involvement of AO. We anticipate that further development of the VSD-based imaging system can provide an easy, low-cost solution to foster clinical deployment of quantitative imaging of retinal photoreceptors, which are essential for advanced study and diagnosis of AMD, RP, and other eye diseases that are known to damage photoreceptors, particularly rod cells at the early stages.^37,39,48

Footnotes

AUTHORS’ CONTRIBUTIONS

XY, RL, BW, and YL drafted a first version of the manuscript, and TK contributed to manuscript revision and figure preparation.

DECLARATION OF CONFLICTING INTERESTS

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

FUNDING

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported in part by NIH grants R01 EY023522, R01 EY030101, R01EY030842, R01EY029673, P30 EY001792; by Richard and Loan Hill endowment; by unrestricted grant from Research to Prevent Blindness.

ORCID iD

Tae-Hoon Kim

References

Fischer

Otto

Delori

Pace

Staurenghi

Scanning laser ophthalmoscopy (SLO). In: Bille

(ed) High resolution imaging in microscopy and ophthalmology: new frontiers in biomedical optics. Cham: Springer, 2019 pp. 35–57

Bennett

Quillen

Coronica

Fundamentals of fluorescein angiography. Insight 2016; 41:5–11

Fujimoto

Swanson

The development, commercialization, and impact of optical coherence tomography. Invest Ophthalmol Vis Sci 2016; 57:OCT1–OCT13

Yao

Alam

Toslak

Quantitative optical coherence tomography angiography: a review. Exp Biol Med 2020; 245:301–12

Liang

Williams

Miller

DT.

Supernormal vision and high-resolution retinal imaging through adaptive optics. J Opt Soc Am A Opt Image Sci Vis 1997; 14:2884–92

Chui

TYP

Krawitz

Menon

Choudhury

Gan

Razeen

Shah

Pinhas

Rosen

RB.

Human retinal microvascular imaging using adaptive optics scanning light ophthalmoscopy. Int J Retina Vitreous 2016; 2:11

Miller

Kurokawa

Cellular scale imaging of transparent retinal structures and processes using adaptive optics optical coherence tomography. Annu Rev Vis Sci 2020; 6:115–48

Smith

CL.

Basic confocal microscopy. Curr Protoc Neurosci 2001; Chapter 2:Unit 2 2

Sredar

Fagbemi

Dubra

Sub-airy confocal adaptive optics scanning ophthalmoscopy. Transl Vis Sci Technol 2018; 7:17

10.

Sulai

Dubra

Adaptive optics scanning ophthalmoscopy with annular pupils. Biomed Opt Express 2012; 3:1647–61

11.

DuBose

LaRocca

Farsiu

Izatt

JA.

Super-resolution retinal imaging using optically reassigned scanning laser ophthalmoscopy. Nat Photonics 2019; 13:257–62

12.

Gustafsson

MG.

Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy. J Microsc 2000; 198:82–7

13.

Chetty

Gruppetta

Structured illumination microscopy for in-vivo human retinal imaging: a theoretical assessment. Opt Express 2012; 20:25700–10

14.

Wang

Zhang

Yao

XC.

Super-resolution scanning laser microscopy through virtually structured detection. Biomed Opt Express 2013; 4:1673–82

15.

Son

Kim

T-H

Yao

Virtually structured detection enables super-resolution ophthalmoscopy of rod and cone photoreceptors in human retina. Quant Imaging Med Surg 2020; DOI: 10.21037/qims-20-542

16.

Kim

T-H

Son

Yao

Virtually structured detection enables super-resolution imaging of rod and cone photoreceptors in awake human. Investigat Ophthalmol Visual Sci 2020; 61:PP0016–PP16

17.

Superresolution imaging. In: Murphy

Davidson

(eds) Fundamentals of light microscopy and electronic imaging. 2nd ed. Hoboken, NJ John Wiley & Sons, Inc., 2012. pp. 331–55

18.

Shroff

Faster, sharper, and deeper: structured illumination microscopy for biological imaging. Nat Methods 2018; 15:1011–19

19.

Hirano

Matsuda

Hiraoka

Recent advancements in structured-illumination microscopy toward live-cell imaging. Microscopy 2015; 64:237–49

20.

Min

Conchello

Xie

Lichtman

JW.

Super-resolution laser scanning microscopy through spatiotemporal modulation. Nano Lett 2009; 9:3883–9

21.

Nilsson

SE.

An electron microscopic classification of the retinal receptors of the leopard frog (Rana pipiens). J Ultrastruct Res 1964; 10:390–416

22.

Liebman

Entine

Visual pigments of frog and tadpole (rana pipiens). Vision Res 1968; 8:761–75

23.

Wang

Zhang

Yao

Breaking diffraction limit of lateral resolution in optical coherence tomography. Quant Imaging Med Surg 2013; 3:243–8

24.

Zhang

Curcio

Yao

XC.

In vivo confocal intrinsic optical signal identification of localized retinal dysfunction. Invest Ophthalmol Vis Sci 2012; 53:8139–45

25.

Zhi

Wang

Zhang

Yao

Rapid super-resolution line-scanning microscopy through virtually structured detection. Opt Lett 2015; 40:1683–6

26.

Liu

Zhi

Wang

Thapa

Chen

Alam

Yao

In vivo super-resolution retinal imaging through virtually structured detection. J Biomed Opt 2016; 21:120502

27.

Liu

Yao

In vivo super-resolution imaging of transient retinal phototropism evoked by oblique light stimulation. J Biomed Opt 2018; 23:1–4

28.

Ditchburn

Ginsborg

Involuntary eye movements during fixation. J Physiol 1953; 119:1–17

29.

Robinson

The mechanics of human saccadic eye movement. J Physiol 1964; 174:245–64

30.

Zhi

Wang

Yao

Super-Resolution scanning laser microscopy based on virtually structured detection. Crit Rev Biomed Eng 2015; 43:297–322

31.

Yao

Son

Kim

Functional optical coherence tomography of retinal photoreceptors. Exp Biol Med 2018; 243:1256–64

32.

Yao

Wang

Intrinsic optical signal imaging of retinal physiology: a review. J Biomed Opt 2015; 20:090901

33.

Bizheva

Pflug

Hermann

Povazay

Sattmann

Qiu

Anger

Reitsamer

Popov

Taylor

Unterhuber

Ahnelt

Drexler

Optophysiology: depth-resolved probing of retinal physiology with functional ultrahigh-resolution optical coherence tomography. Proc Natl Acad Sci U S A 2006; 103:5066–71

34.

Kim

T-H

Wang

Son

Yao

Functional optical coherence tomography enables in vivo optoretinography of photoreceptor dysfunction due to retinal degeneration. Biomed Opt Express 2020; 11:5306–20

35.

Pandiyan

Jiang

Maloney-Bertelli

Kuchenbecker

Sharma

Sabesan

High-speed adaptive optics line-scan OCT for cellular-resolution optoretinography. Biomed Opt Express 2020; 11:5274–96

36.

Zhang

Shibata

Peinado

Zawadzki

FitzGerald

Pugh

EN.

Jr.

Measurement of diurnal variation in rod outer segment length in vivo in mice with the OCT optoretinogram. Invest Ophthalmol Vis Sci 2020; 61:9–9

37.

Curcio

Medeiros

Millican

CL.

Photoreceptor loss in age-related macular degeneration. Invest Ophthalmol Vis Sci 1996; 37:1236–49

38.

Yang

Zuo

Xiao

Luo

Liu

Photoreceptor dysfunction in early and intermediate age-related macular degeneration assessed with mfERG and spectral domain OCT. Doc Ophthalmol 2016; 132:17–26

39.

Jackson

Owsley

Curcio

CA.

Photoreceptor degeneration and dysfunction in aging and age-related maculopathy. Ageing Res Rev 2002; 1:381–96

40.

Berson

Gouras

Gunkel

RD.

Rod responses in retinitis pigmentosa, dominantly inherited. Arch Ophthalmol 1968; 80:58–67

41.

Berson

Goldstein

EB.

Early receptor potential in dominantly inherited retinitis pigmentosa. Arch Ophthalmol 1970; 83:412–20

42.

Holopigian

Greenstein

Seiple

Hood

Carr

RE.

Evidence for photoreceptor changes in patients with diabetic retinopathy. Invest Ophthalmol Vis Sci 1997; 38:2355–65

43.

McAnany

Park

JC.

Cone photoreceptor dysfunction in Early-Stage diabetic retinopathy: association between the activation phase of cone phototransduction and the flicker electroretinogram. Invest Ophthalmol Vis Sci 2019; 60:64–72

44.

Yao

Kim

TH.

Fast intrinsic optical signal correlates with activation phase of phototransduction in retinal photoreceptors. Exp Biol Med 2020; 245:1087–95

45.

Wang

Pepperberg

Yao

Stimulus-evoked outer segment changes occur before the hyperpolarization of retinal photoreceptors. Biomed Opt Express 2017; 8:38–47

46.

Benedetti

Yao

Light-Induced length shrinkage of rod photoreceptor outer segments. Transl Vis Sci Technol 2018; 7:29

47.

Zhao

Thapa

Wang

Gai

Yao

Stimulus-evoked outer segment changes in rod photoreceptors. J Biomed Opt 2016; 21:65006

48.

Portera-Cailliau

Sung

Nathans

Adler

Apoptotic photoreceptor cell death in mouse models of retinitis pigmentosa. Proc Natl Acad Sci U S A 1994; 91:974–78