The impact of the 0–0 cell on measures of amino acid covariation

The covariation between the amino acids expressed at different residues on the HIV genome has been examined in numerous studies ^[1–6]. In particular, the phi statistic (which is equivalent to the Pearson correlation coefficient for 0/1 data) has been widely utilized as a measure of covariation. Studies of samples from antiretroviral therapy (ART)-naive patients may provide insights into fitness epistasis, although this cannot be easily disentangled from the effect of evolutionary relationships between sequences ^[2,6]. Studies of samples from ART-experienced patients may identify mutations that lie on a common mutational pathway under specific selective drug pressure (positive association) or mutations that are biologically antagonistic (negative association), such as K65R and T215F/Y ^[7].

We recently examined the covariation between the N348I connection domain mutation and key reverse transcriptase (RT) mutations, based on a case–control analysis of ART-experienced patients in a large clinical database ^[8]. Cases were defined as the first sample per patient in which N348I was detected (n = 198). The control series consisted of 10 N348N samples that were closest in calendar time to the sampling date of each of the cases (n = 1980). Associations were quantified by the odds ratio (OR), the only estimable measure of association in case–control studies.

The initial analysis indicated a strong positive association between N348I and virtually all key RT mutations. For example, the OR between N348I and T215F/Y was 2.72 [95% confidence interval (CI) 2.00–3.69, P < 0.0001] (Table 1). However, an increasing proportion of samples from ART-experienced patients lack any major resistance mutations (25% in 2001, 56% in 2007) (http://www.hivrdb.org/public/surveillance.asp); this is presumed to largely reflect changes in the clinical indication for conducting an HIV resistance test ^[10]. As the detection of wild-type virus suggests an absence of antiretroviral selection pressure, we, therefore, repeated our analysis excluding such samples to better approximate the target population of ‘ART-experienced’ patients.

Table 1: Association between N348I and T215F/Y.

By definition, the re-analysis results in a reduction in the number of samples in the bottom-left cell of the table (from 1462 to 654) while leaving the other cells unchanged. The effect of this is to nullify the previously observed strong association between N348I and T215F/Y (OR = 1.22, 95% CI 0.89–1.67, P = 0.21). A large attenuation in the strength of the associations for other key mutations was similarly observed, although some remained statistically significant. Thus, the conclusions reached on covariation between amino acids are seen to depend critically on analytical assumptions.

The Jaccard coefficient, which ignores samples that lack both the mutations being assessed, has recently been proposed as a measure of covariation ^[3,6]. It has been argued that this coefficient ‘…does not inflate the correlation between two mutations that may appear correlated by other measures when both mutations are nearly always absent’ ^[3]. Unfortunately, however, the statistic to test whether the observed Jaccard coefficient differs significantly from the expected value under the null hypothesis of zero covariation fundamentally depends on the number of samples harboring neither mutation.

Many subtle biases underlie analyses of amino acid covariation. It is important to note that although the procedure we adopted is an improvement on the initial analysis, it does not provide a definitive solution. Another generally overlooked point is that observed mutational associations are a function of the patients' treatment histories, which are often highly variable, especially in observational studies. Analyses that do not stratify by drug exposure provide only limited biological insights.