When the data doesn't exist, Build it

An exhibit at the Museum of Natural History—the diversity I hope my dataset represents.

What do you do when you have an idea for a research project but not enough resources to carry it out? I had been imagining the perfect execution of my Machine Learning project that would use patient history and travel data to predict likely diseases and outbreaks in high-transmission cities and aid doctors with efficiently treating global pathogens. Then, as explored in my last blog post, I hit a roadblock. This data was very minimal or non-existent, and if it did exist, it was inaccessible due to patient confidentiality agreements. I hadn’t expected to struggle with this part of the project and no easy Plan B was apparent. For weeks, my mentor had been pushing me to search the web and try and gather data, so I was embarrassed to meet him and share my lack of results. However, instead of telling me to try again, he surprised me. 

“Continuing to look for data will be time-consuming with minimal results. I think we should try using synthetic data.” 

Synthetic data? I knew vaguely what this meant but had no idea how it would benefit our project. Then my mentor explained to me—To generate a synthetic dataset for analysing disease transmission and diagnostic accuracy, we would utilize the coding system R. It would create 1 million patient records sampling from empirically-derived distributions. Empirically-derived means that this dataset would be created from past results from surveys or experiments rather than being based on a theoretical formula, therefore increasing the real-world accuracy of the dataset. 

The variables in our dataset would be age, gender, occupation, date of travel, city of stay before travel, city of stay after travel, city of airport at origin, city of airport at destination, disease status before travel, disease status identified after travel, notable medical history, signs and symptoms presented in India, signs and symptoms presented in the U.S, diagnostics used in screening the patient, and sensitivity and specificity of said diagnostics. This would be under the assumption that travelers were screened for disease both at ports of exit from India and at ports of entry into the United States. 

Each variable would also reflect typical traveller demographics. Age would be from 18-80 years with higher weights for the 45–64 age group. Gender would be 49% male, 49% female, and 2% other. Occupation would be from seven different categories. Geographical cities included 13 Indian departure and 12 U.S. arrival cities with logical consistency, meaning that the city of airport at origin matched the origin city and the city of airport at destination matched the destination city. Pre-travel diseases were determined using probabilities calibrated from surveillance data. Symptoms were also assigned conditionally upon disease presence—for example, if you had COVID-19, the data for the symptoms variable would be a cough as opposed to a stomach ache. Post-travel disease was also sampled non-deterministically. This means that instead of saying that if a traveler came from Chennai, they definitely had COVID-19, R would treat infection as a random event based on the probability and distribution of disease occurrence in that particular region. The diagnostic test data would also be complementary to the disease and based on real-world diagnostic usage; for example, a rapid test rather than an X-Ray would be used for COVID-19. 

Not only would our usage of synthetic data be much more efficient and save us the trouble of cleaning up data from a variety of sources, it was also very realistic and calibrated based on past public health observations and surveillance. With my mentor’s detailed explanation about how this synthetic data would be sourced, my fears were assured. I was no longer doubtful about using fake data, nor did I think this would make our model ineffective and inapplicable to real-world scenarios. If anything, I think it would benefit our project even more—instead of using scattered data from different sites, this data was sure to cover a variety of different diseases coming from different cities in India. It would be even more applicable to real-world health conditions. However, regardless of how realistic it was, it is still important to use it responsibly knowing that there are limitations for its realism, which I would cover in my paper. But for right now, it seemed to be the best option. I was excited to begin training my model and achieve results.