Does big data today keep the doctor away?

There’s a small cadre of highly skilled big data professionals and doctors who are leveraging technology to help you live a longer, healthier life. Armed with mountains of government-funded genomic data sets along with mature and easily accessible analytics tools, these technicians and doctors are building apps, tools, and systems which can help you diagnose and treat illnesses ranging from common to catastrophic.

Leading that charge is Dexter Hadley, unique in that he is both an engineer and a doctor. Dexter runs the Hadley Lab – a big data laboratory at UCSF Health which develops tech to fight disease and promote health. The Hadley Lab has a mandate to derive value from the mountains of clinical data that UCSF continually generates.  With a research background in genomics and clinical training in pathology, Dexter likes to quip that he uses big data to practice medicine.”

We got a chance to ask Dexter about the innovations that are born at the intersection of technology and medicine and tell us about how the democratization of technology is really impacting people’s lives.

So first off, people are probably wondering why and how you became both a doctor AND an engineer?

I have always wanted to be a doctor, but my trajectory changed dramatically when I taught myself to program computers at the age of 10 years old. Since then, I have been obsessed with how to leverage computation to better facilitate medicine. That journey took me from an undergraduate education focused on computer programming to medical school at University of Pennsylvania where I earned a master’s degree in engineering, a Ph.D. in genomics, and an MD for good measure. 

Through stints practicing medicine in an internship in general surgery at Penn, and then later residency in pathology at Stanford, I developed a passion as a physician/scientist to integrate medicine and software engineering in order to improve the delivery of healthcare for doctors and their patients.

So, what does the Hadley Lab do and how do you contribute?

The Hadley Laboratory leverages big data to improve the practice of medicine and the delivery of healthcare.  Our work generates, annotates, and ultimately reasons over large and diverse data stores to better characterize disease. We develop state-of-the-art data-driven models of clinical intelligence that drive clinical applications to more precisely screen, diagnose, and manage disease. We integrate multiple large data stores to identify novel biomarkers and potential therapeutics for disease.

The end point of our work is rapid proofs of concept clinical trials in humans that translate into better patient outcomes and reduced morbidity and mortality across the disease spectrum.  “I’m an equal opportunity scientist.  I care less about the best disease I can study, but more about what disease I can study best– it’s all driven by the data.   

And what would you say is the present, future, and ideal state of R&D in this area?

At present, I think we are experiencing a continued renaissance of medicine that started with the initial sequencing of the human genome well over a decade ago. Now, we are finally in a position to actually quantify human health and disease in “precision medicine,” a fundamentally different approach to healthcare research and its delivery where our focus is on identifying and correcting individual patient differences rather than making broader generalizations. 

While genomics allows us to quantify our molecular self, I think the future is in leveraging all the technology at our fingertips today to better quantify our physical self. As the power of genomics lies in its objective ability to correlate with physical manifestations in the patient, the ideal state of R&D must involve data collection and analysis at both the molecular genotypic level and the more clinical phenotypic level of the patient. 

For instance, in the context of a health system, my research integrates large clinical data stores with state-of-the-art big data algorithms, smartphones, web and mobile applications, etc. to first discover and then deliver precision medicine to patients.

Sounds like a big part of that future is genomics?

Genomics is indeed the future, except it’s clearly more complicated than we initially thought.  Most doctors don’t sit around looking at their patient’s genomic data to develop treatment plans. However, some specialist doctors look at images all day long, such as radiologists and pathologists for instance. We have technology and algorithms today that allows us to build ‘apps’ that can help these specialists.

For instance, we are working on a mobile medical app for doctors and their patients to use smartphones to better screen for skin cancer. However, while digital health apps on smartphones represent a convenient screen for skin cancer, the actual diagnosis and subsequent management of skin cancer remains within the genomics realm.

So, diagnosis is where the need is right now?

The practice of medicine involves screening a general population and diagnosis of suspected cases before intervention on a specific patient. Much of precision medicine research has focused on diagnosis and intervention phases, with less focus on screening. My focus currently is using powerful big data algorithms for population screening of healthy individuals through digital apps. While “anybody” can build an app these days, not everybody has the knowledge, data, and access to the clinical infrastructure to develop clinical-grade algorithms for doctors and their patients.

How big of an impact is the “democratization of technology” having on this space?

About 6 years ago, Mark Andreessen penned a WSJ editorial that lays out the case for “Why Software Is Eating The World.” How does the average person shop today? Or bank? Or trade stocks? Or find a taxi? Mainly through innovative “apps” that we have come to depend on. I think that inevitably this phenomenon will percolate to our medical world where we now have all the ingredients to do magical things with tech, meaning cheap computation, awesome algorithms, and tons of big data that we continue to generate at breakneck speeds in clinical medicine.

For instance, at UCSF Health, we literally have billions of clinical records over almost a million patients that must hold the keys to practice better medicine. If you think about it, the average clinical trial to prove efficacy of an intervention is practically limited to the order of hundreds of patients because of time and monetary constraints. 

Therefore, our modern health systems allow for the largest clinical trials most appropriately powered for rapid discovery of novel medical interventions. I think that building clinical grade apps based on this big data allows us to immediately deliver the innovative discovery power of our health systems to the hands of physicians and their patients.

What would that involve, “building a clinical-grade app”?

Building the app is actually the least rigorous part of the process as the ‘clinical-grade’ performance comes from the algorithms that we develop that underlie the app interface. The magic of what we are doing lies in learning patterns from big data that we generate in healthcare. Deep learning is one such method that is a paradigm shift towards ‘cognitive computing’ where computers are essentially trained to think like humans. 

Deep learning on big data represents state-of-the-art machine learning today and repeatedly outperforms other more traditional methods. Data is the key piece of this process because these deep learning algorithms are incredibly complex. While much of statistics is based on linear models whose parameters can be accurately estimated with only a few data points, some of the most sophisticated deep learning algorithms have more parameters to estimate than there are atoms in the universe. 

Therefore, useful deep learning requires big data to accurately estimate parameters that are most predictive.

Let’s say one of our readers is interested and wants to develop this app for you, what would you share with them to help get them started?

I would definitely encourage them to reach out directly to me through my website. I’m also a member of the Institute for Computational Health Sciences at UCSF, which is dedicated to advancing computational health sciences in research, practice, and education in support of Precision Medicine for all.

If any readers are interested in contributing to the project, you can reach Dexter at

So do you think big data today will keep the doctor away? If you liked this article, read more stories about the data impact on the world at

This article was produced in partnership with Western Digital.

Facebook Comments