The tech is based on the fact that it takes pathologists ages to analyse biological tissue to see if there are signs of chance and Google thinks could benefit from deep learning tools. It said that its microscope could allow groups with limited funds, such as small labs and clinics, or developing countries to benefit from these tools in a simple, easy-to-use manner. Google says the scope could "possibly help accelerate and democratise the adoption of deep learning tools for pathologists around the world".
The microscope is an ordinary light microscope. Google tweaked it with AI technology and augmented reality. First, neural networks are trained to detect cancer cells in images of human tissue. Then, after a slide with human tissue is placed under the modified microscope, the same image a person sees through the scope's eyepieces is fed into a computer. AI algorithms then detect cancer cells in the tissue, which the system then outlines in the image seen through the eyepieces. It's done in real time and works quickly enough that it's still useful when a pathologist moves a slide to look at a new section of tissue.
So far Google has got it to go on breast and prostate cancers, showing the system is pretty accurate. Google says such a set up could be used to detect other cancers as well as infectious diseases like tuberculosis and malaria. It could also be used to display other data or annotations into the view.
This proof of concept study is also described in a paper that's currently still under review. Google says that a more in-depth study is required for more robust assessments of the system's performance and shortcomings. It concluded: "We're excited to continue to explore how the Augmented Reality Microscope can help accelerate the adoption of machine learning for positive impact around the world."