JMIR Publications

JMIR Research Protocols


Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 07.06.12 in Vol 1, No 1 (2012): Jan-Jun

This paper is in the following e-collection/theme issue:


    HIV Drug-Resistant Patient Information Management, Analysis, and Interpretation

    Department of TeleHealth, Nelson R Mandela school of Medicine, University of KwaZulu-Natal, Durban, South Africa

    Corresponding Author:

    Yashik Singh, BSC, BSC(Hons), MMedSc

    Department of TeleHealth

    Nelson R Mandela school of Medicine

    University of KwaZulu-Natal

    719 Umbilo Road, Dept of Telehealth, 5th floor, Main Building, Umbilo

    Durban, 4001

    South Africa

    Phone: 27 312604117

    Fax:27 312604737



    Introduction: The science of information systems, management, and interpretation plays an important part in the continuity of care of patients. This is becoming more evident in the treatment of human immunodeficiency virus (HIV) and acquired immune deficiency syndrome (AIDS), the leading cause of death in sub-Saharan Africa. The high replication rates, selective pressure, and initial infection by resistant strains of HIV infer that drug resistance will inevitably become an important health care concern. This paper describes proposed research with the aim of developing a physician-administered, artificial intelligence-based decision support system tool to facilitate the management of patients on antiretroviral therapy.

    Methods: This tool will consist of (1) an artificial intelligence computer program that will determine HIV drug resistance information from genomic analysis; (2) a machine-learning algorithm that can predict future CD4 count information given a genomic sequence; and (3) the integration of these tools into an electronic medical record for storage and management.

    Conclusion: The aim of the project is to create an electronic tool that assists clinicians in managing and interpreting patient information in order to determine the optimal therapy for drug-resistant HIV patients.

    JMIR Res Protoc 2012;1(1):e3



    Crowdfunding campaign to support this specific research

    We help JMIR researchers to raise funds to pursue their research and development aimed at tackling important health and technology challenges. If you would like to show your support for this author, please donate using the button below. The funds raised will directly benefit the corresponding author of this article (minus 8% admin fees). Your donations will help this author to continue publishing open access papers in JMIR journals. Donations of over $100 may also be acknowledged in future publications.

    keyboard with crowdfunding key instead of enter key

    Suggested contribution levels: $20/$50/$100


    The current trend in patient health care is personalized medicine where treatment is individualized, rather than a response to set physical presentations. Thus, access to and interpretation of personal patient information is vital in order to provide a sustainable and useful medical service. The science of information systems, management, and interpretation plays an important role in the continuity of care of patients. This is becoming more evident in the treatment of human immunodeficiency virus (HIV) and acquired immune deficiency syndrome (AIDS). This paper describes proposed research where the aim is to develop a physician-administered artificial intelligence-based decision support system tool that will facilitate the management of patients on antiretroviral therapy.

    The enveloped human immunodeficiency virus infects and destroys the human immune system over a long period of time [1]. The two known strains of HIV are HIV-1 and HIV-2. The rate of replication and infection of the HIV-2 is substantially slower than that of the HIV-1 and accounts for 95% of all HIV infections [2]. HIV-1 is subdivided into four groups representing four separate introductions of simian immunodeficiency virus into humans:

    1. Group M is the major HIV-1 group with respect to prevalence (the number of people infected) and incidence (the number of new infections) of the virus;

    2. Group O is the outlier group and is mostly restricted to west-central Africa;

    3. Group N was discovered in 1998 in Cameroon and is extremely rare; and

    4. Group P is a strain closely resembling the gorilla simian immunodeficiency virus.

    Currently, Group M is subdivided into nine subtypes or clades—A, B, C, D, F, G, H, J, and K—based on variations in genetic sequence characteristics [3]. However, it is possible for viruses from different subtypes to form mosaic genomes called circulation recombinant forms (CRF). In sub-Saharan Africa, HIV/AIDS is the leading cause of death [4] and it is one of the fastest growing epidemics in South Africa [5-8], where currently there are 5.7million confirmed cases of HIV/AIDS [9]. Demographic information on confirmed HIV-infected patients in South Africa is presented in Table 1.

    Table 1. Estimated HIV prevalence rates in South Africa [9].
    View this table

    HIV infection can be effectively managed with antiretroviral (ARV) drugs, usually in the form of highly active antiretroviral therapy (HAART), which is comprised of a regimen of three drugs from at least two of the following five drug classes [10-13]: reverse transcriptase inhibitors (RTI), nucleoside reverse transcriptase inhibitors (NRTI), protease inhibitors (PI), integrase inhibitors (II), and fusion inhibitors (FI).

    Factors that influence treatment of HIV/AIDS with ARVs include poor treatment regimen prescribed by the physician; the World Health Organization (WHO) stage of the disease, which is related to the progression of the disease; levels of plasma drug concentration achieved; how strictly the patient adheres to the regimen; drug resistance [14]; and toxic effects of the drug. Drug resistance is the most critical aspect of treatment. Three common reasons leading to the development of HIV antiretroviral drug resistance are high replication rates, selective pressure, and initial infection by resistant strains of HIV. Thus, it is inevitable that drug resistance will become a reality in most patients’ treatment.

    Preventative measures must be taken in order to develop infrastructure that will aid in the management of drug-resistant HIV patients. It is essential to develop techniques that will extract valuable information from little patient data. There must be a means developed to manage, analyze, and interpret patient data.

    The aim of this study is to develop a physician-administrated information system that facilitates the clinical management of HIV-positive patients on antiretroviral therapy. This system should be Web-based, patient centric, ascribe to the principles of personal medicine, promote complete health management, and incorporate continuity of care. Creation of this tool will involve:

    • Development of an artificial intelligence computer algorithm that analyzes HIV drug resistance data and provides singular interpretable information for a physician indicating which ARVs a patient will be resistant to;
    • Investigation of the application of a computer algorithm to predict current and future CD4 lymphocyte cell count information given a genomic sequence and other data;
    • Integration of the above tools with an electronic medical record system such that it facilitates the storage, acquisition, and management of patient information; and
    • Development of a Web-based electronic tool that assists clinicians in determining the optimal therapy for drug-resistant HIV patients.


    Medical Informatics

    The appropriate application of computer science and associated technology has extended medical care beyond traditional diagnosis and patient management, resulting in extensive cost efficiencies and improved public health outcomes [15]. Areas of medical informatics application include patient records, practice management, clinical measurements, patient education, prescription writing, Web/database resources, clinical records, data collection, clinical decision support, and clinical measurements. Recently, there has been an intentional move towards investigating the synergy between medical informatics and bioinformatics [9]. Bioinformatics is the application of computer science techniques to study how information is represented and transmitted in biological systems starting at the molecular level [16].

    The application of genomes in medicine has altered many aspects of medicine. Genome analysis that enhances clinical practice has been successfully applied to asthma [17], cancer [18-20], diabetes [21], and cardiovascular disease [22].

    HIV Drug Resistance Prediction Algorithms

    Testing for HIV drug resistance may consist of wet or dry chemistry laboratory tests, or by employing electronic computerized algorithms [23]. The use of computer algorithms falls under the field of medical informatics. Computer based interpretation algorithms using genomes can also be used to predict HIV drug resistance.

    These interpretation algorithms can be generally divided into one of two groups:

    • Those based on known domain knowledge (ie, they are based on the fact that certain combinations of known genome mutations cause unequivocal resistance), and
    • Those not based on predefined domain knowledge. These algorithms include machine learning and statistical methods.
    Interpretation Algorithms Based on Domain Knowledge

    Domain knowledge interpretation algorithms are based on scientific and published interactions between certain mutations and/or combination of mutations with resistance. This means that all computational decisions concerning resistance are based on known mutation-resistance rules found in published scientific literature. REGA, Agence Nationale de Recherches sur le SIDA (ANRS), and Stanford’s HIVdb algorithm [24] are three examples of well-known domain knowledge interpretation algorithms. These algorithms are widely used in the field and are regarded as gold standards.

    REGA and ANRS classify ARV resistance according to three levels: susceptible, intermediate, and resistant. “Susceptible” indicates that a particular ARV drug will be effective against HIV; “intermediate” indicates that the ARV drug is partially effective; and if the ARV is not effective at all, it is classified as “resistant.” HIVdb classifies HIV drug resistance according to five levels: susceptible, potential low-level resistance, low-level resistance, intermediate resistance, and high-level resistance. These algorithms employ Boolean-based rules, some with penalties, and predict resistance by determining which mutations are present and/or absent.

    Two other domain-based algorithms are the Drug Resistance SEQuence ANalyzer (DR_SEQAN) and RetroGram. DR_SEQAN was coded by the Universidad Autónoma de Madrid for a Windows environment using Visual Basic. DR_SEQAN classifies three levels of resistance: high-level resistance, increased susceptibility, and no resistance. RetroGram was developed by InferMed Ltd (London, UK) and is built using Arezzo and PROforma. RetroGram generates a suitability ranking for ARV drugs using expert rules. Table 2 describes the accuracy of predicting drug resistance of some algorithms based on domain knowledge.

    Table 2. Accuracy of predicting ARV drug resistance by the domain-based algorithms, Drug Resistance SEQuence ANalyzer (DR_SEQAN), RetroGram, REGA, and HIVdb [28].
    View this table
    Interpretation Algorithms Not Based on Known Domain Knowledge

    Many different pattern recognition and machine-learning algorithms have been applied to find a predictable correlation between genotypic and phenotypic data (called “virtual phenotyping”) [25]. Machine learning may be used to develop a model that predicts virological response. Machine learning is an artificial intelligence computer science technique that tries to find a mathematical model that maps between inputs and outputs of a domain problem.

    Virtual phenotyping is growing in popularity. Kuritzkes supports virtual phenotyping as a tool for interpreting viral genotypes [26]. The following are some of the algorithms that have been used:

    • Least absolute shrinkage and selection operator (LASSO)
    • Ridge regression
    • Neural networks, such as multilayer perceptron (MLP) and radial basis neural networks (RBNN)
    • Principle component analysis
    • Support vector machines (SVM)
    • Linear regression models
    • Hidden Markov models
    • Decision trees
    • Multiple correspondence analysis
    • Associative classifiers
    • k-nearest neighbor algorithm (kNN)

    Results produced by these interpretation algorithms are shown in Tables 35. These interpretation algorithms have achieved various levels of success, but there are shortcomings in some of the current versions [27]:

    • Resistance is interpreted separately for each drug even though therapy consists of combination therapy;
    • There is a general lack of data, especially for non-B HIV-1 subtypes;
    • Rule-based interpretation is based on the algorithm creator’s knowledge;
    • Interpretation algorithms are not always updated even though HIV drug resistance is a rapidly evolving field; and
    • Other factors that contribute to treatment failure are not taken into account, such as treatment history, resistance history, viral load history, CD4 count history, or plasma drug concentrations.
    Table 3. Accuracy of predicting ARV drug resistance using the interpretation algorithms, support vector machines (SVM), multilayer perceptrons (MLP), and radial basis neural networks (RBNN) [29].
    View this table
    Table 4. Accuracy of predicting ARV drug resistance using k-nearest neighbor (kNN), decision tree [30], and associative classifier [31] algorithms.
    View this table
    Table 5. Accuracy of predicting ARV drug resistance (%) or correlation coefficient (r) reported for various other algorithms and machine-learning techniques.
    View this table

    These shortcomings have led to the creation of many different interpretation algorithms, which produce different resistance measures even if applied to the same resistance profile. These differences are because the studies each used different datasets, subtypes, analysis on drug-naive and drug-experienced patients, and so forth. Conclusions of some studies that reported on the discrepancy of the interpretation algorithms are shown in Table 6.

    Table 6. Summary of discrepancies reported using various interpretation algorithms.
    View this table

    Collation and interpretation of the contradictory outputs of these algorithms is difficult for physicians treating complex drug-resistant HIV cases, as information is only valuable when it is presented in a clearly interpretable way.

    Predicting CD4 Count

    HIV can be successfully managed with ARV drugs, but information relating to the progression of HIV is vital. HIV infection may be monitored using laboratory [40,41] and clinical marker information [42,43]. Information about a patient’s CD4 lymphocyte cell counts are the most widely used data for HIV progression and is recognized as a standard measure of immunodeficiency in HIV-positive patients [44,45]. Thus, the proper use and analysis of information regarding CD4 cell counts is vital in CD4-guided treatment of HIV [46].

    Although the use of CD4 count is part of the standard of care in developing countries, the measurement of CD4 count requires many complex and expensive flow cytometric procedures, which burden the minimal resources available [45]. The ability to predict current CD4 cell count will aid in easing the burden on these resources. A physician may use an electronic tool to economically determine an approximate CD4 cell count. If the predicted count is low or indicates that a change in treatment is required, then the physician might order the more expensive laboratory procedure to determine the exact CD4 cell count. The ability to obtain information about future CD4 count changes will have many benefits to physicians. For example, it will facilitate definite treatment actions, such as changing the regimen in order to prevent opportunistic infection (eg, pneumocystis pneumonia) and delay the onset of AIDS.

    Neural network machine-learning algorithms have been used to predict viral load [7,47]. Altmann et al [48] created a machine-learning algorithm that predicts success or failure of therapy, based on viral load, with 80% success. This was later changed by predicting the probability of treatment success based on a degree of predicted HIV drug resistance [49]. However, there is not a chemical test or computer model developed yet to forecast changes to the CD4 count.

    Decision Support System Tool for Managing Therapy

    Although models have been created to choose treatment regimens, very few are available in the public domain and/or are easily accessed through a graphical human (user) interface. Currently, there are Web portals that allow one to determine some aspects of HIV drug resistance treatment. These information portals allow one to determine the current HIV resistance profile, graph trends in viral and CD4 counts with basic alerts, or store basic patient information. BioAfrica ( is an African-based bioinformatics resource [50]. BioAfrica contains bioinformatics resources that can perform sequence alignments, epitope analysis, tools for proteomics, subtyping and virus genotyping, an RNA virus database, and an HIV drug resistance database and tools. The HIV drug resistance database and tools section is based on the REGA collaborative mode and the Calibrated Population Resistance Tool (CPT). REGA is a drug resistance database developed by the REGA Institute, MyBioData Biomedical IT Solutions, and the Katholieke Universiteit Leuven. It contains interpretation algorithms and stores some clinical data related to HIV treatment. CPT was developed at Stanford University and determines the prevalence of HIV drug resistance in a population.

    Some of the other international Web portals for managing HIV treatment information are listed in Table 7.

    Table 7. Descriptions of Web portals for managing HIV treatment information.
    View this table

    These individual information portals are limited by the following:

    • The tools they employ in determining HIV drug resistance information. Each information portal uses its own interpretation algorithm and, if collaboration does exist, it consists of simply reporting the outputs of the various algorithms. This causes confusion as some of these interpretation algorithms are disparate, even when the same mutations are analyzed.
    • They do not have any means of a real-time expert consultation.
    • They are not integrated into a full electronic medical record, which will add the advantage of continuity of care and facilitate tele-HIV-management.
    • No individual portal has a variety of tools that can be used to manage HIV therapy.


    Part 1: Developing a Single Interpretation Algorithm

    The goal of Part 1 is to develop an HIV drug resistance interpretation algorithm capable of providing a single interpretation to genomic analysis.

    This part of the study is divided into three main objectives: (1) determining the extent of the disparate information provided by some gold standard interpretation algorithms using the latest version of the interpretation algorithms; (2) developing a novel algorithm to collate the HIV drug resistance interpretation information of these gold standard algorithms into a single easily understandable output; and (3) analyzing the collated algorithm in terms of specificity, sensitivity, and accuracy.

    1. Determining the extent of the disparate nature of some gold standard interpretation algorithms using the latest version of these algorithms.

    Over time with each new version, interpretation algorithms have improved in predicting ARV drug resistance. Previous comparisons between interpretation algorithms have had some shortcomings:

    • Each interpretation algorithm has different measures or levels of resistance;
    • Non-contemporary versions of interpretation algorithms were used in the interpretation;
    • The interpretation algorithms were applied to different data sets; and
    • Few interpretations make use of complex statistical analysis to determine if the differences are in fact significant or not.

    The latest versions of different interpretation algorithms will be applied to a single data set extracted from a publicly available anonymized database, the Stanford HIV drug resistance database [51]. The measures of resistance for each interpretation algorithm will be determined, grouped, and analyzed.

    2. Developing a novel algorithm to collate the HIV drug resistance interpretation of these gold standard interpretation algorithms into a single output.

    The gold standard algorithms may be collated by:

    • Weighted output. Different levels of complexity may be applied to determine a single interpretation from multiple interpretations. A simple majority-voting scheme may be applied, where a count of the interpretations of each algorithm is kept. The single interpretation is obtained by determining the resistance outcome with the highest weighting.
    • Machine learning on gold standard outputs. Different machine-learning techniques may be applied to the data in order to obtain a single interpretation. Machine-learning techniques work by determining a mapping between a given set of input and desired outcomes and then, using this learnt mapping function, it predicts the output, given a set of inputs. Each interpretation produced for a single resistance profile by the different interpretation algorithms can be the input to a machine-learning algorithm. The output will be the actual HIV-ARV resistance measure determined by fold resistance values. The algorithm will then learn a mapping between the interpretation results obtained using various interpretation algorithms and the actual HIV-ARV resistance measure. One such algorithm that may be employed is a support vector machine.
    • Creating a simulated boosted dataset both by modeling the strengths and weaknesses of the gold standards.

    3. Analyzing the collated algorithm in terms of specificity, sensitivity, and accuracy.

    The specificity, sensitivity, and accuracy associated with predicting ARV drug resistance will be calculated for each algorithm and then compared using statistical analysis.


    The literature does not indicate the current state of disparity between gold standard interpretation algorithms. Combining the interpretation algorithms to form one single interpretation is novel.

    Part 2: Predicting CD4 Count From Genome Data

    This part of the study may be divided into three parts:

    1. Investigating the possibility of creating a machine-learning algorithm that predicts the current CD4 count of a patient using genome sequences, viral loads, and time;
    2. Investigating the possibility of creating a machine-learning algorithm that forecasts the medium term change in CD4 count of a patient using current genome sequence;
    3. It is acknowledged that genome sequencing is more expensive and resource intensive than CD4 cell count measurement. However, the cost of genome sequencing is offset by the numerous bioinformatics applications that may be applied to the genome sequence to predict and analyze other physiological measurements and diseases. This study, however, will also investigate the possibility of creating a machine-learning algorithm that forecasts the medium term change in CD4 count of a patient using standard of care data.

    Datasets will be obtained from the Stanford HIV drug resistance database (, which is publically available and contains data from clinical trials. Subtype B consensus protease (PR) genome sequences, CD4 count, viral load, and the number of weeks from the baseline measure of CD4 count for each patient sample will be determined by joining individual datasets using the sample identifier (the unique number that identifies a sample) and date. Data of patient’s genome sequences and associated viral load and CD4 count data at different time points will be extracted.

    The changes in CD4 count will be grouped into categories and a classification model will be built based on the changes. Different groups of inputs will be created and each will feed into the machine-learning algorithm separately, forming three models. Some of these input groups will be:

    • Input1: consisting only of genome sequence;
    • Input2: consisting of genome sequence and current viral load; and
    • Input3: consisting of genome sequence, current viral load, and number of weeks from the current CD4 count to baseline CD4 count.

    Currently, there is not a chemical test or computer model developed to forecast future changes to the CD4 count.

    Part 3: Developing Web-based Tool for Determining Optimal Therapy

    The goal of Part 3 is to develop a Web-based electronic tool that assists clinicians in determining the optimal therapy for patients indicative of HIV drug resistance.

    There is evidence that suggests that resistance testing is beneficial:

    • A two-factorial (genotyping and expert advice), randomized, open label, multicenter trial [54] was undertaken to determine if there is any benefit in using genotyping rather than the expert’s direct knowledge when prescribing ARVs. The conclusion was that genotyping benefits the overall optimal care of HIV patients.
    • The VIRalliance SAS [55] group clearly demonstrated in their study that “resistance testing prior to initiating or switching antiretroviral therapy” is essential.
    • Mascolini et al [56] questioned 600 clinicians about the effect of resistance testing on their diagnosis and regimen they prescribe. They confirmed that “if the assay detected partly or multidrug-resistant virus, then the large proportions of respondents (indicated that they would) change their treatment choice.”
    • Hirsch et al [57] found that “resistance testing can improve virological outcome among HIV-infected individuals.”
    • The Can Resistance Enhance Selection of Treatment (CREST) [25] study (a 48-week follow-up randomized trial) found that genotypic drug resistance testing may be beneficial in the management of HIV infection.

    The goal is to combine the tools mentioned previously, and possibly other bioinformatics tools, into one seamless application.


    Java, HyperText Markup Language (HTML), PHP: Hypertext Preprocessor (PHP), and other paradigms will be used to create a Web-based portal that will integrate the different tools. An important aspect to take into account when building the model for the HIV management system is security. Dwivedi et al [58] argued that electronic medical records will only become a reality if security takes a prominent role in design considerations and during implementation. Two of the most promising techniques for incorporating security into any information system are public key infrastructure and biometrics. Public key encryption is a nondeterministic polynomial time complex technique that ensures high-level security. Biometrics use physical of behavioral traits to identify an individual. The exact means of integration and security model to be used will only be determined after the individual tools are built.


    The creation of an electronic medical record-based virtual HIV clinical support system that aids in the determination of the best HAART combination, using a combined ARV resistance interpretation, CD4 count prediction, and the other methods described is novel.


    The outcome of this study is to facilitate the acquisition, storage, management, analysis, and interpretation of information by physicians. In personalized medicine, it is essential that information be interpreted and presented clearly and concisely. We expect that the proposed tool will aid in this aspect.


    This work was supported by the National Institutes of Health Fogarty International Centre (grant number 5D43TW007004-11).

    Conflicts of Interest

    None declared.


    1. Albert D, Altfeld M, Aries SP, Behrens G, Bredeek UF, Buhk T, et al. HIV medicine 2007. Paris: Flying Publisher; 2007. Pathogenesis of HIV-1 Infection   URL: [accessed 2012-05-15] [WebCite Cache]
    2. Quinn TC. Molecular variants of HIV-1 and their impact on vaccine development. Int J STD AIDS 1998;9 Suppl 1:2. [Medline]
    3. Kanki PJ, Hamel DJ, Sankalé JL, Hsieh C, Thior I, Barin F, et al. Human immunodeficiency virus type 1 subtypes differ in disease progression. J Infect Dis 1999 Jan;179(1):68-73 [FREE Full text] [CrossRef] [Medline]
    4. Campbell C, Nair Y, Maimane S, Sibiya Z. Supporting people with AIDS and their carers in rural South Africa: possibilities and challenges. Health Place 2008 Sep;14(3):507-518. [CrossRef] [Medline]
    5. Walker L, Walker L. They (ARV's) are my life, without them i'm nothing- experiences of patients attending a HIV/AIDS clinic in Johannesburg, South Africa. Health and Place 2009. [CrossRef]
    6. Simon-Meyer J, Odallo D. Greater involvement of people living with HIV/AIDS in South Africa. Evaluation and Program Planning 2002;25:3850-3855.
    7. Larder B, Wang D, Revell A, Montaner J, Harrigan R, De Wolf F, et al. The development of artificial neural networks to predict virological response to combination HIV therapy. Antivir Ther 2007;12(1):15-24. [Medline]
    8. United Nations country proile: Implementation of agenda, Commission of Sustainable development. 1997.   URL: [accessed 2012-05-15] [WebCite Cache]
    9. Rispel LC, Metcalf CA. Breaking the silence: South African HIV policies and the needs of men who have sex with men. Reprod Health Matters 2009 May;17(33):133-142. [CrossRef] [Medline]
    10. Mitton J. The sociological spread of HIV/AIDS in South Africa. J Assoc Nurses AIDS Care 2000;11(4):17-26. [Medline]
    11. Bartlett J, Gallant JE, Pham PA. Jhons Hopkins University, School of Medicine.: Knowledge Source Solutions; 2004. Medical management of hiv infection   URL: [accessed 2012-05-15] [WebCite Cache]
    12. Pierret J. An analysis over time (1990-2000) of the experiences of living with HIV. Soc Sci Med 2007 Oct;65(8):1595-1605. [CrossRef] [Medline]
    13. Beerenwinkel N, Schmidt B, Walter H, Kaiser R, Lengauer T, Hoffmann D, et al. Diversity and complexity of HIV-1 drug resistance: A bioinformatics approach to predicting phenotype from genotype. PNAS 2002;99(12):8271-8276.
    14. Richman D. A practical guide to HIV drug resistance and its implications for antiretroviral treatment strategies. International Medical Press 2000.
    15. Lovell NH, Celler BG. Information technology in primary health care. Int J Med Inform 1999 Jul;55(1):9-22. [Medline]
    16. Shortliffe EH, Perreault LE, Wederhold G, Fagan LM. Medical Informatics Computer applications in health care and biomedicine. Bioniformtics, pp. 638-662, 2000.
    17. Allen M, Heinzmann A, Noguchi E, Abecasis G, Broxholme J, Ponting CP, et al. Positional cloning of a novel gene influencing asthma from chromosome 2q14. Nat Genet 2003 Nov;35(3):258-263. [CrossRef] [Medline]
    18. van de Vijver MJ, He YD, van't Veer LJ, Dai H, Hart AA, Voskuil DW, et al. A gene-expression signature as a predictor of survival in breast cancer. N Engl J Med 2002 Dec 19;347(25):1999-2009 [FREE Full text] [CrossRef] [Medline]
    19. Weigelt B, Glas AM, Wessels LF, Witteveen AT, Peterse JL, van't Veer LJ. Gene expression profiles of primary breast tumors maintained in distant metastases. 2003 Dec 23 Presented at: Proccedings National Academy Sciences; 2003; USA p. 15901-15905   URL: [CrossRef]
    20. Wong YF, Selvanayagam ZE, Wei N, Porter J, Vittal R, Hu R, et al. Expression genomics of cervical cancer: molecular classification and prediction of radiotherapy response by DNA microarray. Clin Cancer Res 2003 Nov 15;9(15):5486-5492 [FREE Full text] [Medline]
    21. Bell GI, Polonsky KS. Diabetes mellitus and genetically programmed defects in beta-cell function. Nature 2002;9:788-791.
    22. Brindle JT, Antti H, Holmes E, Tranter G, Nicholson JK, Bethell HW, et al. Rapid and noninvasive diagnosis of the presence and severity of coronary heart disease using 1H-NMR-based metabonomics. Nat Med 2002 Dec;8(12):1439-1444. [CrossRef] [Medline]
    23. Ravela J, Betts BJ, Brun-Vézinet F, Vandamme AM, Descamps D, van Laethem K, et al. HIV-1 protease and reverse transcriptase mutation patterns responsible for discordances between genotypic drug resistance interpretation algorithms. J Acquir Immune Defic Syndr 2003 May 1;33(1):8-14. [Medline]
    24. de Oliveira T, Deforche K, Cassol S, Salminen M, Paraskevis D, Seebregts C, et al. An automated genotyping system for analysis of HIV-1 and other microbial sequences. Bioinformatics 2005 Oct 1;21(19):3797-3800 [FREE Full text] [CrossRef] [Medline]
    25. Hales G, Birch C, Crowe S, Workman C, Hoy JF, Law MG, CREST investigators. A randomised trial comparing genotypic and virtual phenotypic interpretation of HIV drug resistance: the CREST study. PLoS Clin Trials 2006;1(3):e18 [FREE Full text] [CrossRef] [Medline]
    26. Kuritzkes DA. Reliability of virtual phenotype resistance assay. Medscape HIV/AIDS 2002 [FREE Full text]
    27. Vercauteren J, Vandamme A. Algorithms for the interpretation of HIV-1 genotypic drug resistance information. Antiviral Research 2006;71:335-344.
    28. Garriga C, Menéndez-Arias L. DR_SEQAN: a PC/Windows-based software to evaluate drug resistance using human immunodeficiency virus type 1 genotypes. BMC Infect Dis 2006;6:44 [FREE Full text] [CrossRef] [Medline]
    29. Bonet I, Garcia MM, Salazar S, Sanchex R, Saeys Y, Grau R. Predicting human immunodefciency virus (hiv) drug resistance using recurrent neural networks.. 2006 Nov 1 Presented at: 10th International Electronic Conference on Synthetic Organic Chemistry; 2006; Switzerland.
    30. Murry J. Predicting HIV type 1 drug resistance from genoptype using machine learning. Master's thesis, University of Edinburgh 2004.
    31. Srisawat A, Kijsirikul B. Using associative classification for predicting HIV-1 drug resistance. 2004 Presented at: HIS 04: Proceedings of the Fourth International Conference on Hybrid Intelligent Systems; 2004; Japan p. 280-284.
    32. Zazzi M, Romano L, Venturi G, Shafer RW, Reid C, Dal Bello F, et al. Comparative evaluation of three computerized algorithms for prediction of antiretroviral susceptibility from HIV type 1 genotype. J Antimicrob Chemother 2004 Feb;53(2):356-360 [FREE Full text] [CrossRef] [Medline]
    33. Drăghici S, Potter RB. Predicting HIV drug resistance with neural networks. Bioinformatics 2003 Jan;19(1):98-107 [FREE Full text] [Medline]
    34. Beerenwinkel N, Däumer M, Oette M, Korn K, Hoffmann D, Kaiser R, et al. Geno2pheno: Estimating phenotypic drug resistance from HIV-1 genotypes. Nucleic Acids Res 2003 Jul 1;31(13):3850-3855 [FREE Full text] [Medline]
    35. Snoeck J, Kantor R, Shafer RW, Van Laethem K, Deforche K, Carvalho AP, et al. Discordances between interpretation algorithms for genotypic resistance to protease and reverse transcriptase inhibitors of human immunodeficiency virus are subtype dependent. Antimicrob Agents Chemother 2006 Feb;50(2):694-701 [FREE Full text] [CrossRef] [Medline]
    36. Vergne L, Snoeck J, Aghokeng A, Maes B, Valea D, Delaporte E, et al. Genotypic drug resistance interpretation algorithms display high levels of discordance when applied to non-B strains from HIV-1 naive and treated patients. FEMS Immunology and Medical Microbiology 2006;46(1):53-62.
    37. De Luca A, Cingolani A, Di Giambenedetto S, Trotta MP, Baldini F, Rizzo MG, et al. Variable prediction of antiretroviral treatment outcome by different systems for interpreting genotypic human immunodeficiency virus type 1 drug resistance. J Infect Dis 2003 Jun 15;187(12):1934-1943 [FREE Full text] [CrossRef] [Medline]
    38. De Luca A, Cozzi-Lepri A, Perno AF, Balotta C, Di Giambenedetto S, Poggio A, ICoNA drug resistance study group, ICoNA study group. Variability in the interpretation of transmitted genotypic HIV-1 drug resistance and prediction of virological outcomes of the initial HAART by distinct systems. Antivir Ther 2004 Oct;9(5):743-752. [Medline]
    39. Poonpiriya V, Sungkanuparph S, Leechanachai P, Pasomsub E, Watitpun C, Chunhakan S, et al. A study of seven rule-based algorithms for the interpretation of HIV-1 genotypic resistance data in Thailand. J Virol Methods 2008 Jul;151(1):79-86. [CrossRef] [Medline]
    40. Fahey JL, Taylor JM, Detels R, Hofmann B, Melmed R, Nishanian P, et al. The prognostic value of cellular and serologic markers in infection with human immunodeficiency virus type 1. N Engl J Med 1990 Jan 18;322(3):166-172 [FREE Full text] [CrossRef] [Medline]
    41. Moss AR, Bacchetti P, Osmond D, Krampf W, Chaisson RE, Stites D, et al. Seropositivity for HIV and the development of AIDS or AIDS related condition: three year follow up of the San Francisco General Hospital cohort. Br Med J (Clin Res Ed) 1988 Mar 12;296(6624):745-750. [Medline]
    42. Cahn P, Perez H, Casiro A, Crinberg N, Muchinik G. Progression of HIV-disease: the Buenos Aires cohort study. 1991 Presented at: International conference on AIDS; 1991; Florence.
    43. Montaner JS, Le TN, Le N, Craib KJ, Schechter MT. Application of the World Health Organization system for HIV infection in a cohort of homosexual men in developing a prognostically meaningful staging system. AIDS 1992 Jul;6(7):719-724. [Medline]
    44. Post FA, Wood R, Maartens G. CD4 and total lymphocyte counts as predictors of HIV disease progression. QJM 1996 Jul;89(7):505-508 [FREE Full text] [Medline]
    45. Schechter M, Zajdenverg R, Machado LL, Pinto ME, Lima LA, Perez MA. Predicting CD4 counts in HIV-infected Brazilian individuals: a model based on the World Health Organization staging system. J Acquir Immune Defic Syndr 1994 Feb;7(2):163-168. [Medline]
    46. Osmond D, Charlebois E, Lang W, Shiboski S, Moss A. Factors affecting changes in time from CD4 = 200 to death in two San Francisco cohorts-1992. 1983 Presented at: International Conf on AIDS; 1993; Germany.
    47. Wang D, DeGruttola V, Hammer S, Harrigan R, Larder B, Wegner S, et al. A CollaborativeHIV Resistance Response Database Initiative: Predicting Virological Response Using Neural Network Models. 2002 Jul 2 Presented at: The XI International HIV Drug Resistance Workshop; 2-5 July 2002; Seville.
    48. Altmann A, Rosen-Zvi M, Prosperi M, Aharoni E, Neuvirth H, Schülter E, et al. Comparison of classifier fusion methods for predicting response to anti HIV-1 therapy. PLoS One 2008;3(10):e3470 [FREE Full text] [CrossRef] [Medline]
    49. Altmann A, Däumer M, Beerenwinkel N, Peres Y, Schülter E, Büch J, et al. Predicting the response to combination antiretroviral therapy: retrospective validation of geno2pheno-THEO on a large clinical database. J Infect Dis 2009 Apr 1;199(7):999-1006 [FREE Full text] [CrossRef] [Medline]
    50. de Oliveira T, Cassol S. Scourge of a Continent: The BioAfrica website. Science 2005;310:1877.
    51. Shafter R, Stevenson D, Chan B. Human immunodeficiency virus reverse transcriptase and protease sequence database. Nucleic Acids Research 1999;27(1):348-352.
    52. Vertibrae Inc and HIVresistanceWeb. HIVresistanceWeb. 2012.   URL: [accessed 2012-01-16] [WebCite Cache]
    53. Los Alamos National Laboratory. 2012. HIV databases   URL: [accessed 2012-01-16] [WebCite Cache]
    54. Tural C, Ruiz L, Holtzer C, Schapiro J, Viciana P, González J, Havana Study Group. Clinical utility of HIV-1 genotyping and expert advice: the Havana trial. AIDS 2002 Jan 25;16(2):209-218. [Medline]
    55. VIRalliance SAS. HIV phenotyping - A technique to evaluate drug resistance availible worldwide for patients, clinicians and pharmaceutical industries. European Pharmacotherapy 2003.
    56. Mascolini M. A suite of new therapies. 2003 Feb Presented at: 10th Conference on Retroviruses and Opportunistic Infections; 2003; Boston p. 10-14.
    57. Hirsch MS, Brun-Vézinet F, Clotet B, Conway B, Kuritzkes DR, D'Aquila RT, et al. Antiretroviral drug resistance testing in adults infected with human immunodeficiency virus type 1: 2003 recommendations of an International AIDS Society-USA Panel. Clin Infect Dis 2003 Jul 1;37(1):113-128 [FREE Full text] [CrossRef] [Medline]
    58. Dwivedi A, Bali RK, Belsis MA, Naguib RNG, Every P, Nassar NS. Towards a practical healthcare information security model for healthcare institutions. In: Information Technology Applications in Biomedicine. 2003 Presented at: 4th International IEEE EMBS Special Topic Conference; 2003 April; Birmingham p. 114-117. [CrossRef]


    AIDS: acquired immune deficiency syndrome
    ANRS: Agence Nationale de Recherches sur le SIDA
    ARV: antiretroviral
    ASI: algorithm specification interface
    CPT: Calibrated Population Resistance Tool
    CREST: Can Resistance Enhance Selection of Treatment
    CRF: circulation recombinant forms
    DR_SEQAN: Drug Resistance SEQuence ANalyzer
    FI: fusion inhibitors
    HAART: highly active antiretroviral therapy
    HIV: human immunodeficiency virus
    kNN: k-nearest neighbor algorithm
    LASSO: least absolute shrinkage and selection operator
    MLP: multilayer perceptron
    NIAID: National Institute of Allergy and Infectious Diseases
    NRTI: nucleoside reverse transcriptase inhibitors
    PHP: PHP: Hypertext Preprocessor.
    PI: protease inhibitors
    RBNN: radial basis neural networks
    RTI: reverse transcriptase inhibitors
    SVM: support vector machines
    WHO: World Health Organization

    Edited by G Eysenbach; submitted 20.09.11; peer-reviewed by R Foster; comments to author 09.01.12; revised version received 27.01.12; accepted 22.04.12; published 07.06.12

    ©Yashik Singh, Maurice Mars. Originally published in JMIR Research Protocols (, 07.06.2012.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.