Smartphone based mobile microscopy for diagnostics

Abbreviations

1 Introduction

The history of microscopy begins in the late 16th century and at that time it was primarily a novelty for the aristocracy without any impact for medicine and diagnostics. Only with an improvement of lens qualities and industrial production in the 19th century, the technology became obvious for the detection of pathological indicators [1].

Today, microscopy is often the only possibility to diagnose diseases and therefore cannot be replaced by modern analytical methods. Besides, since the availability of modern devices is higher than ever, about one third of the world’s inhabitants have access to a smartphone [2]. The built-in camera sensor is the most powerful basic for diagnostic applications and the quality of images still increases with an average resolution of about 10 megapixels meanwhile. Another advantage of smartphones is the ability to access the internet; thus creating the possibility for the use of cloud resources for computationally expensive algorithms, and provides a direct connection to a physician or health center for diagnostics. Around 70 percent of people in Africa – where fast and low-priced diagnostics would be highly beneficial - have mobile phones and could principally make use of smartphone supported diagnostic tools [3].

The availability of advanced point-of-care (POC) diagnostic tools is essential for the welfare of a population, particularly in infrastructural under-developed regions where a lack of specialised staff and facilities is prevalent. For that issue, the development of cost-effective smartphone based microscopes is a very promising approach for the diagnosis of several diseases and their timely and correct treatment at the point-of-demand.

2 Smartphone based microscopes

The idea to transform mobile phones into microscopes appeared in the end of 2000, when cameras with higher imaging quality became more available. The main advantage of using smartphones for diagnostics is the possibility to have a mobile tool that can be used almost everywhere and can automatically analyse, digitalize and map the results. Furthermore, results may directly be transferred to a physician or a health center. All smartphone based systems depend on the specification of the camera sensor and its performance. The image quality and the possible practicability for diagnostic applications are mainly determined by two parameters: resolution and field of view (FOV). The microscopic resolution is the shortest distance between two objects that can still be differentiated. The FOV is the image area that can be observed. Therefore, a small minimal distance (misleadingly the term “high” resolution is common) and a great FOV are desirable. For example, a normal laboratory microscope has an average resolution of 0,42 μm [4] and a FOV of 0,6 mm² [5] with a magnification of 400x.

In the field of mobile microscopy for POC diagnostics, the recent developments can be divided into two major categories of mobile phone based microscopes: with lenses and without. Two working groups dominate the research in this field: the Fletcher lab group 1 (FG) and the Ozcan research group 2 (OG). Only a few systems are commercially available. In Table 1 the current state of research in smartphone based microscopes and potential diagnostic applications are summarized.

2.1 Lens based systems

The FG focused on the lens based approach and tried at first to use microscope components (objectives, eyepieces) in front of the camera of the mobile phone. The components are large in comparison to modern day clip-on devices. This first solution is simple and low-priced. It reaches a resolution of 1,2 μm, has a small FOV with a diameter of 180 μm and is dependent on a Nokia N73 [7]. The improvement of this first attempt resulted in a new device - the CellScope. It is a more compact device including optics, illumination and hardware automation. The CellScope is available in two versions: without [8] and with a smartphone [6, 9] (Fig. 1). All solutions have the possibility to use fluorescence and bright field (BF) microscopy. The CellScope - which uses a smartphone – contains a two-axis sliding mechanism for positioning a variety of different mobile phones.

OPEN IN VIEWER

Fig.1

A multi-phone mobile microscope. (A) The lens based approach contains typical microscope components (objective, eyepiece) and the camera sensor of a smartphone. (B) Construction of the mobile microscope (CellScope) developed by the FG. Reproduced from ref. [6] 3 Copyright 2014 by PLoS One is licensed under CC BY 4.0. 4

With the integration of a domed LED array, an advanced version of this CellScope has been developed that is capable of dark field (DF) and phase contrast (PC) microscopy as well as 3D imaging [9]. The control of the LED array is done via Bluetooth and image acquisition and processing is performed through a smartphone application. The image quality of the two phone based devices is dependent on the built-in camera sensor. Their second approach was to simplify the system by combining only two mobile phone camera lenses. For this, a reversed mobile phone camera lens was attached to the existing one. With this combination, it is possible to achieve a resolution of 5 μm in a FOV of 10 mm². This approach makes use of the full image sensor of the smartphone, which was not possible before due to the design of the embedded camera system. Advantageous is the low price of the second reversed lens, dearly bought by a drop of intensity and an inhomogeneously centered illumination, caused by a single LED construction. Additionally, the system is only suitable for the iPhone. Nevertheless, it offers all benefits of every smartphone-based system [10].

The OG pursued both the lens based and the lensless approach. For the former, they created two simple mechanical attachments for mobile phones using only one external lens to reach the desired magnification and resolution. The first attachment is for fluorescent imaging and cytometry in micro-fluidic channels. It has to be used with a Sony Ericsson smartphone and reaches a very large FOV of 81 mm² with a resolution of only 10 μm. The benefits of this add-on are particularly the low-costs, its low weight and size and the possibility to do DF microscopy [11–14]. The second attachment was developed for fluorescence microscopy (FM) and is applicable for standard glass slides. Furthermore, the position of the LED is set to a specific angle to reduce background noise and enables the illumination of small particles [15–17].

Other mechanical attachments with external lenses are similar to the above described and were developed for specific problems:

One clip-on device uses a small adapter for the smartphone where a small lens is integrated [18]. Nor modifications of the phone neither post-processing of the images is needed. The images are taken with the smartphone and therefore can be analysed directly on it or transferred to a professional for evaluation. Because there is no light-source an additional background light is helpful to enhance the contrast of the images.

Another solution uses a relatively large attachment containing a 3D-printed case with polarizer sheets, illumination, diffuser and sample slide for an iPhone to perform polarisation microscopy in order to detect malaria parasites [19]. For the lens based systems, a fancy but simple idea was to reinvent the microscope of Antony van Leeuwenhoek [20], a researcher in the 17th century. With the use of small ball lenses magnification up to 600x can be achieved. These lenses are very small, simple to use and offer lots of possibilities in combination with filters. Since the focus of very small lenses is close behind the lens itself, the sample must also be placed close to the lens producing a small FOV. For all systems, which use ball lenses, the camera-sensor has the greatest influence on image quality [21–27].

2.2 Lensless systems

For their lensless approach, the OG used a holographic microscope: the samples are placed very near to the camera sensor of a Motorola MotoZine ZN5 and are vertically illuminated with an LED (Fig. 2). The reconstruction of the original image is done algorithmically. The lensless system has a resolution of 2,2 μm over a 5 mm² FOV [29].

OPEN IN VIEWER

Fig.2

Lens-free on-chip microscopy. A transmissive sample is placed at a very close distance (less than 1 mm) in front of a mobile phone camera sensor and an external illumination source is applied. The produced holographic images (because of interference between reference and signal light) can be reconstructed algorithmically. Reproduced from ref. [28] 5 with the permission of the American Institute of Physics.

Lee et al. [30] placed the sample directly on the image sensor of a Samsung Galaxy together with a reference target. The user has to move the device to capture multiple images with varying angles of illumination (e. g. using daylight). With a pixel super-resolution algorithm, the original image is calculated. This reconstruction of images applying different algorithms is the biggest disadvantage of the proposed solutions. The limiting factor is that the image reconstruction is carried out on the smartphone and thereby its performance is important for the computational speed. Furthermore, the method involves modification of the phone to remove the camera lens to expose the sensor and obtain the necessary distance to the sample. Even so, lensless systems have a good resolution and a great FOV and they are constituted with simple hardware that is lightweight and compact.

2.3 Commercially available systems

The German company Oculyze developed a mobile microscope (Fig. 3) and an algorithm primarily for the automated measurement of yeast concentration and viability [26]. The microscopic images are transmitted to a cloud where an algorithm performs the automated analysis. Users can see and save their results in an App on the smartphone. Currently, the system is mainly being used by beer brewers but there are a number of other applications in various stages of development [26, 27, 31].

OPEN IN VIEWER

Fig.3

Oculyze smartphone microscope. A combination of an optical module, a corresponding smartphone and automated image analysis in the cloud enables different diagnostic approaches. The image shows a stained blood sample with a magnification of 400x.

The BLIPS^® system consists of a small lens, which is attached directly in front of the smartphone camera by a tape [32]. The challenge in this approach is the need for a high quality camera to take good images. Another project is the μPeek microscope. It is a kick-starter campaign and a very basic version has been available since January 2018. Designed as a flat credit card shaped tool that has a motorized autofocus and a dimmable illumination architecture, μPeek can be attached to almost every smartphone and is connected via Bluetooth [33].

The most basic development was the BF microscopy for the mobile use. For many issues like the detection of parasites [22, 34–36], bacterial spores [23], evaluation of sperm cells [24] and tissue diseases (e. g. macrovesicular steatosis) [37] this method seems to be sufficient. For more challenging analysis tasks, there are many advanced methods: FM, DF, differential PC, and polarized light microscopy (PLM). With FM, researchers are able to diagnose tuberculosis [8, 39], count white blood cells [14], detect viruses [15], parasites [16] and maybe the most impressive: targeted DNA sequencing and mutation analysis [17]. The most special application is an integration of PLM for the detection of hemozoin to detect malaria infections [19]. The lensless systems are comparable with BF microscopy and in the future could also be used to perform FM. Currently such a system was suggested for the analysis of blood smears, microspheres and freshwater green algae [30] but the area of possible applications is even greater.

The FOV and resolution differ based on which optical arrangement is used. Both parameters influence each other. If a wide FOV is needed, the resolution often gets worse and vice versa. Finding a compromise between these parameters is crucial for diagnostic aspects.

All discussed systems – lens based and lensless - as well as their possible diagnostic applications are summarised in Table 1. A brief description of the optical arrangement, the applicable microscopy methods and the most important properties (resolution and FOV) are featured. Furthermore, a timeline and the impact of research regarding the two mentioned working groups (FG and OG) is shown.

4 Conclusion and outlook

Since the development of the first smartphone based microscope, there was an increasing interest in this technology. Many working groups started to develop different mobile microscopic solutions and – as seen in Table 1 – there are solutions for almost every kind of microscopy method. The different approaches can offer many possibilities in diagnostics but also increase the interest for science in the public. In schools such small und low-cost microscopes could motivate children to be future scientists and develop even further techniques. However, the diagnostic benefit is in the focus of this evolving field. Creating easy-to-use smartphone based systems could improve medical support in remote rural areas. Further research is needed since by now there are only a few tools commercially available. Another point is the importance of presenting solutions for the evaluation of taken images to make diagnosis possible without expert knowledge. Currently only the Oculyze system (Fig. 3) makes use of all options of a smartphone: Imaging, calculation and communication. Microscopic images are sent to a cloud server, where computer vision algorithms evaluate the content, and then the results are accessible after a few seconds on the smartphone screen. The focus has to be on developing new devices, which can be used and delivered all over the world for easy use without previous knowledge about microscopy or diagnostics. By such means with the continuous improvement of smartphones and diagnostic methods a substantial improvement of POC should become possible.

Declaration of interest

There is no conflict of interests for the authors Juliane Pfeil, Luise N. Dangelat and Marcus Frohme. The submitted work was performed in cooperation with Katja Schulze from the company Oculyze GmbH, Wildau, Germany. Oculyze works in the field of mobile microscopy and computer vision. Therefore, Oculyze has a vested interest in the success of this field.