This study does not investigate a clinical hypothesis but performs an empirical evaluation of a deep learning framework for Gram stain image interpretation. The hypothesis to be tested is that the optimization of a deep learning framework will perform better than a scale-up strategy with a single GPU. In order to test this hypothesis, two strategies will be examined, as shown in Figure 1. The scale-up strategy stacks up more computer capabilities (model A, highlighted in blue). On the other hand, the optimization strategy tunes the granular configuration of a deep learning framework (model B, depicted in green).

In order to avoid model bias, 4 pretrained models (Mobilenet [7] with versions 1 and 2, and Inception [3] with versions 3 and 4) will be evaluated. Mobilenet and Inception were originally trained with ImageNet [6], which contains 1.2 million nature images and 1000 classes. Therefore, it is necessary to retrain them on the new data set, as described in the “Data Collection” section, because Gram stain interpretation is barely linked to the original task. The method that retrains a pretrained model without changing network architecture is called fine-tuning. Fine-tuning is a specific transfer learning technique that modifies more layers than the last layer, also known as the fully connected layer, and their corresponding weights and biases. The optimal proportion of the fine-tuning will be empirically determined in this study. Concretely, 10 implementations will be evaluated, with the proportions of frozen layer to unfrozen layer (retraining layer) at 9:1, 8:2, ongoing up to 0:10 (from the shallow strategy to the deep strategy). The computational performance will be measured by time to achieve the target accuracy metric [8] of 95%.

Once the base model is implemented with Tensorflow [9] (Google Corp) as described above, model A will be accelerated with 2 approaches. Model A1 is the scale-up implementation built on top of model A with a single GPU. In contrast, model B is a mutated model of model A because it replaces floating-point arithmetic with an integer arithmetic–only deep learning framework called Tensorflow Lite [10]. Model B1 aims to detect the optimal batch size for Gram stain classification. Model B2 and B3 activate batch normalization and weight normalization. Normalization penalizes large input numbers and weights. Finally, model B4 prunes the model network by adjusting the dropout rate, which speeds up the execution time. These hyperparameters—batch size, batch normalization, weight normalization, and dropout rate—are chosen based on the findings from previous studies [11-14]. The specification of the hyperparameter space is defined in Table 1. The boundary is chosen to be wide to account for many possible combinations of the hyperparameters.

The objective of this study is to understand the relation between computation time and each hyperparameter, not to create a hyperparameter optimization [15] that searches for a global minimum or the local minima of hyperparameters.

This study will use 8728 Gram stain images from between 2015 and 2018 for modeling and images generated in 2019 for testing. Data are archived in a workstation at the Institute for Clinical Chemistry at the Medical Faculty Mannheim of Heidelberg University, Germany. Sample images and labels are shown in Figure 2. The sizes of the cropped images vary from 800×600 pixels to 1920×1080 pixels. Note that the image is not a whole-slide image, but a crop of the interests—the microorganisms and the slide background.

The label data corresponding to the image are stored in a central database for reporting purposes and extracted for this study. Each image is associated with 2 labels: (1) Gram stain class (ie, either gram-positive or gram-negative) and (2) a class for the genus. The genus label includes 5 of the most frequently encountered germs: (1) Staphylococcus, (2) Escherichia, (3) Streptococcus, (4) Enterococcus, and (5) others, for the rest of the germs that are rarely presented. This setting prevents a potential risk of identifying a patient with an extremely rare microorganism. Of the 8728 images, 446 images (5.11%) are associated with multiple classes. These images with multiple class labels are excluded because object recognition, that is, bacterial differentiation, is beyond the scope of this study.

The population for this study is a group of sepsis patients, whose blood samples contain at least one harmful bacterium, such as Staphylococcus, Escherichia, or Streptococcus. This study does not recruit control and treatment groups. However, a set of gram-positive images would be regarded as one group, while gram-negative images would be regarded as a comparison group for this study. This is a retrospective data analysis reusing an archived image data set, as described in the “Data Collection” section.

This section will address and describe the 3 underlying statistical considerations towards a solid study design: (1) the class balancing strategy for the input data set, (2) the proper split ratio for training and evaluation, and (3) the metric for the model evaluation.

This study will use Tensorflow [9] (floating-point arithmetic framework) and Tensorflow Lite [10] for deep learning solutions. Both of these frameworks are open source tools developed by the Google Brain team, and they are able to accelerate deep learning calculation by using multicore central processing units (CPUs) and GPUs. With regard to a model-debugging tool, TensorBoard will be used to graphically track all execution history.

All solutions will be developed and deployed in the data center at the Heinrich-Lanz-Center for Digital Health. The hardware configuration for this study is one Intel Xeon Silver 4110 CPU (Intel Corp), one Tesla V100 PCIe 32 GB GPU (Nvidia Corp), and 189 GB memory. The server is virtualized by Docker technology (Docker Inc) [19] for reproducible research.

Distributed computing across multiple machines will not be covered in this study. Although it is the usual method to process big data, it is not always the most efficient choice to process the data. According to Boden et al [20], distributed systems are surprisingly inefficient in training machine learning models compared with a single workstation. The size of the input data for this study is 25 GB and it fits into the capacity of a standalone workstation. In future work, we would like to study Apache Spark [20], which enables distributed machine learning model training when the data do not fit into the memory of a single computer.

This study does not aim to propose novel neural network architecture, which requires many days of GPU processing time with state-of-the-art computational infrastructure that is not available within the scope of this project. Also, designing an outperforming architecture for image classification is a saturated topic, as many researchers have devoted their endeavors to this problem in the last decade. Nevertheless, for those who are interested in this topic, Elsken et al published a state-of-the-art review paper [21] that provides an overview of existing works and categorizes them into 3 dimensions.

An insufficient amount of image data could lead to an underpowered deep learning solution. The proper input data size is still an open question in the computer vision community. The answer is, “it depends.” It depends on the number of classes, image size, image quality, and complexity of the problem. For instance, classifying a black image versus a white image demands fewer input data compared with classifying a gram-positive image versus a gram-negative image.

In medical data analysis, power analysis is widely applied for determining the minimum sample size required. Unfortunately, power analysis is not applicable to unstructured data such as images. A rule of thumb for a good input size is 1000 images per class [21], which was the basis of an object recognition competition that was part of the Pascal Visual Object Classes challenge [22]. This study is anticipated to have low failure, since about 5800 gram-positive labeled images and 2700 gram-negative images are available for modeling.

Although this study does not use any personal information for data analysis, the name of the input data consists of a unique identifier for the experiment. This experiment identifier harbors a remote risk of linking back to personal information in the database. In the interest of data protection, this identifier is anonymized and securely stored at the Heinrich-Lanz-Center for Digital Health , which is protected by the hospital network firewalls. Unlike data pseudonymization, which transforms the identifier, data anonymization is an irreversible technique that removes the identifier permanently. The anonymized data will be archived for reproducibility.

The study will comply with the latest version of the Declaration of Helsinki [23] and Professional Code for Physicians in Germany. Patient names and other personal data are subject to the legal requirements concerning confidential medical communication. They comply with European Directive 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of individuals with regard to the processing of personal data and on the free movement of such data, the EU General Data Protection Regulation and the German Federal Data Protection Act, and the State Data Protection Act Baden-Württemberg.