My Case for Machine Learning

Over the past year, concerns surrounding AI use have grown exponentially. This is seen through misinformation circulating the internet, adaptive social media algorithms getting better at  capturing your attention, how easy it is to turn to ChatGPT instead of using your brain, and more. Regardless of intent, all demographics have been exposed to AI in some way or another, and fear is understandable. All of these reasons are why I was initially hesitant to use something similar for my project, as neither was I at all familiar with computational bioinformatics. After expressing these concerns to my mentor, he explained to me that utilizing a Machine Learning model would be very different than talking to a chat bot. We would choose a model that was known to be reliable and one with a relatively straightforward procedure. So, obviously, that begs the question: What model? 

A standard Random Forest Model was chosen after thorough research as it aligns well with the intentions of my project and is a reliable method of prediction. No, this is not a model found in a grouping of trees somewhere out in rural New Hampshire (forgive me). In fact, Random Forest models are made by using bootstrapped data sets and making multiple digital decision trees (hence a forest) that each use a different bootstrapped data set. Bootstrapped data sets are data sets that each consider a random selection of variables. Ultimately, this variety is why Random Forest Models are more effective than normal decision trees. Our data was split into 80% training data and 20% testing data. What this means is that the model would first take 240 rows of the 300 row data set and run the data down the trees to identify variables that contribute most to the ultimate disease diagnosis (ex. Knowing the symptoms were a cough contributed most to knowing the disease was COVID19). At each step in the tree, depending on which variable received the most “votes”, it is concluded that this is the more important variable; this is called bagging. After these variables are identified, the testing data is run down all the trees, weighing the previously determined variables as more important and  keeping track of the results. The model then determines if it classified the sample as what was the correct target column in the data set. The proportion of testing samples that were incorrectly vs. correctly classified define my model’s accuracy. 

The results for this model and for Random Forest models generally have four categories: Precision, Recall, F1, and Support. Precision asks: When the model predicts this disease, how many times was it right out of how many times it said it was this disease? The percentage dictates the precision. Recall asks: How often did the model predict another disease when it was not that disease? The percentage is how much it successfully predicted. F1 is the Harmonic mean of precision and recall or the combined score of model accuracy for that disease. Finally, support is the number of rows of data in the testing data set that the disease had, which helps you better visualize the way the model operates. In addition to choosing a model, we had to choose a model-building strategy. 

The model-building strategy chosen for this Random Forest model was Stepwise regression. It was chosen because it is able to include and exclude specific, independent variables in order to identify which variables are the most significant in terms of informing future health policy. It can “streamline model creation” by dealing with large data sets while simultaneously lessening the number of variables that are being considered. This allows for further efficiency. 

After learning more about the model, I was much more confident in its ability to guide this project. I still worried that it would draw patterns between variables that weren’t relevant to disease identification or make assumptions not reflective of real public health situations. He agreed with me—limitations do exist with every model, and caution should be addressed in my paper; I’d discuss how the model might correlate variables that are not necessarily predictive of one another, and similarly could make choices that make sense at a point on a tree, yet do not make sense for the model as a whole, leading to bias and incorrect results due to the model’s lack of knowledge. For now, I’d train the model carefully and to the best of my abilities. I was excited to track its performance with different parameters and analyze the results!