Background

Outbreaks such as Severe Acute Respiratory Syndrome (SARS) and H1N1 influenza have triggered acute policy debates around community-based infection control measures. Quarantine is the segregation of healthy persons exposed to an infectious disease from unexposed persons, until the exposed person becomes ill or the incubation period has passed. Quarantine is also one of the most controversial measures available in outbreak control, with important impacts on economic activity and civil liberties. Public health agencies need to have weighed evidence of the potential costs and benefits of these manoeuvres before the next outbreak of a severe, novel infection and to have publicly and widely communicated the importance of planned control measures. This makes outcome evaluation of modern quarantine activity an important area for research.

Health policy and practice guidelines rely heavily on experimental and quasi-experimental research comparing outcomes under intervention and non-intervention conditions. Statistical estimates used in health decision-making tend to derive from differences in outcome rates in treated versus non-treated individuals (e.g., risk difference, RD; attributable risk, AR and population attributable risk, PAR), as well as variations on the number needed to treat (NNT) statistic. Research to evaluate the impact of modern quarantine is less well-developed, and no clear tradition has emerged in the measures of association used to communicate net harm and benefit. The quantitative impact of control measures for infectious diseases is commonly assessed using variants of the effective reproductive number, R (the average observed number of secondary infections per index cases in an observed population). Rt the daily effective reproductive number, is the observed number of secondary cases to index case at a specific time. When Rt, has a value below1.0, this is evidence that an outbreak is dying out.

Here, we discuss the quantitative evidence available to inform policy-makers on the potential benefit of community-based quarantine in outbreak control, based upon the SARS experience. To bridge the gap between evaluative research on quarantine and other areas, we also use data from the 2003 Ontario, Canada (hereafter, Ontario), SARS outbreak to demonstrate statistical approaches used elsewhere to the evaluation of modern quarantine measures.

We propose a series of measures of reduction of adverse outcomes (analogous to those used in other contexts), to evaluation of quarantine. Specifically, we estimated the difference in secondary transmissions that is attributable to community quarantine as the Secondary Case Count Difference (SCCD), which is comparable to risk difference statistics and interpretable as reduction in average transmissions per existing case, attributable to intervention. We also report the inverse of this statistic, interpretable as Number Need to Quarantine (NNQ).

Subjects and data extraction

The contact tracing and quarantine procedures used in the 2003 Ontario SARS outbreak have been described previously. From source public health records, we extracted data for all 332 index cases with a final disposition of suspect or probable SARS of whom 204 had at least one community contact uniquely associated with them in Public Health records. Contacts included had non-overlapping periods of exposure to SARS index cases within 10 days, as previously reported. The total number of community contacts associated with these index cases, for this analysis, was 8,498. This number excluded health care providers who were contacts of those SARS index cases only as a result of providing care to SARS cases, but included health care workers exposed through social or family contacts. Community contacts were classified by closest level of exposure to the index case (e.g., level 1 being closest at ≥30 minutes within a distance of one metre). For all community contacts, outcome status as a secondary SARS case was defined. We also consider data for an additional 140 individuals who were quarantined as contacts of a known SARS case, became ill and were considered to be potential cases themselves, but who subsequently had SARS ruled out. These additional 140 observations were used only in sensitivity analyses.

Analysis

In the first step, we carried out an analysis where the 8,498 individual contacts were treated as the unit of analysis. The outcome was whether or not each contact became a SARS case (binary outcome). The predictor variable of interest was whether the associated index case had been in community quarantine at symptom onset. This, seemingly intuitive, analysis treats the individuals in whom the outcome is measured as though they were assigned to treatment conditions. Therefore, information regarding quarantine status is used only from the 204 SARS index cases with one or more community contacts. In place of typical logistic regression which would yield an odds ratio (OR), we obtained risk difference (naïve secondary attack rate difference - ignoring index cases with no contacts) as the measure of association, using a generalized linear regression model with binomial error term and identity link function. Clustering of multiple contacts to index cases was accounted for using robust variance estimates.

In the next step, we treated all 332 SARS index cases as the unit of analysis and applied Poisson regression analysis to model the number of secondary cases per index case. As before, the exposure of interest is whether or not the index case was quarantined. We calculated several measures of association to describe the difference in transmission observed in the quarantine and non-quarantine group. These included the secondary case count ratio or SSCR, the ratio of secondary cases (per index case) in the quarantine condition relative to the non-quarantine condition. The SSCR is a function of the ratio of secondary cases per quarantined index case (SCq) and of the ratio of secondary cases per non-quarantined index case (SCnq) as discussed below. We also calculated the difference in average secondary cases per index case between the two groups (secondary case count difference, SCCD), and the inverse of the SCCD which we label as "NNQ".

Secondary Case Count Ratio (SCCR) was estimated using the most familiar form of Poisson regression (Poisson error term and the canonical log link function). Poisson regression is often used to estimate incidence rate ratios (IRRs), where the outcome variable is the numerator for a rate in a stable population, or where a variable person-time denominator is accounted for by specifying an offset variable for the model. IRR is equal to the exponent of the beta coefficient for the regression term of interest. For Poisson models used here, no offset was specified for the number of community contacts at risk for each index case. Said differently, the outcome variable here is not the rate of secondary transmission among a variable number of potentially exposed community contacts per index case, but the number of secondary SARS cases arising from an index case. The SCCR is the exponentiated beta term for this model (where quarantine status was entered as a binary variable). SCCR is the ratio of (secondary cases per quarantined index case, SCq) to (secondary cases per non-quarantined index case, SCnq); therefore, SCCR = SCq/SCnq.

Estimation of SCCD (attributable difference in number of secondary cases, per index case) was achieved using a Poisson model estimating additive effects (Poisson error term and identity link function). SCCD is the difference in the average absolute number of secondary cases in the quarantine versus non-quarantine conditions (SCCD = SCnq-SCq). This model yields a (non-exponentiated) beta coefficient interpreted as the difference in outcome. Here, beta = SCCD. The inverse of this SCCD we define as NNQ, a statistic analogous to NNT. NNQ is the number of cases that would need to be in quarantine at symptom onset to prevent one case among contacts of an index case. This is defined as:

Additional analysis quantified the degree to which reduction in total contacts, and close contacts, mediated the impact of quarantine. An estimate of the reduction in size of the SCCR was tested against a null (no change) using a bootstrap method. This mediation analysis was carried out using the canonical (log link) Poisson regression model, above.

For the sensitivity analysis, we re-estimated SCCD and NNQ including data for an additional 140 case observations. This group consisted of quarantined contacts who were, prospectively, under suspicion of being SARS cases themselves (i.e., during the outbreak), but in whom SARS was later excluded. These people also had at least one community contact themselves, in order to be included in the database. This sensitivity analysis was to show the effect of false positive index cases on apparent outcomes of quarantine.

For all regression models, diagnostics were performed. This included post-fit examination for non-linearity for ordinal independent variables (i.e., numbers of contact by level of exposure). Where found, non-linearity was corrected by taking a linear transformation of the predictor variable to obtain normally distributed residuals. Nonnormality of residuals in models including total contacts was corrected by transforming the total number of the contacts (replaced with square root of value) to reduce the influence of the positive skew in this covariate. For all models using the Poisson error term, tests for over-dispersion were carried out. Identical models using the negative binomial error term were also run, correcting for over-dispersion. Both are presented.

Because of the limited sample size, confidence limits based on large sample assumptions are questionable. Therefore, we also present confidence intervals estimated using bootstrap variance estimation. For bootstrapping, 5000 replications were used in all cases. All analyses were carried out with Stata, Version 10.

This study received ethical review and approval from the Health Sciences I Research Ethics Committee of the University of Toronto (Protocol #10764).

Results

Table 1 summarizes quarantine status for 332 probable and suspect SARS cases in the 2003 Ontario SARS outbreak, with numbers of contacts by level of contact and transmission status.

Table 2 presents several measures of association describing the impact of quarantine using regression models described above. In the first, naïve, regression analysis (Approach A in Table 2) the estimated effect was positive (indicating a harmful effect for quarantine) but not statistically significant.

The first model treating index cases as the unit of analysis (Approach B in Table 2) yields an estimate of SCCR of 0.316 indicating less than one third the number of secondary cases for quarantined versus non-quarantined index cases. Approach C provides the estimated SCCD obtained using an additive Poisson regression model (Poisson error term and identity link function). The average difference in secondary SARS cases in moving from non-quarantine to quarantine status is estimated at 0.133 secondary cases per index case. The final estimate in Table 2 is NNQ suggesting 7.51 SARS index cases be placed under quarantine to reduce the number of secondary cases by one. When variance estimates and confidence intervals are estimated using a large-sample assumption (asymptotic variance) all Poisson regression models presented were statistically significant. When negative binomial regression was used to estimate SCCR, this became non-significant (p = 0.057). When using bootstrap variance methods, confidence intervals for all estimates became very wide and none of these estimates were significantly different from the null value.

Table 3 presents supplemental analyses of the degree to which the effect of quarantine is explained by reduction in the number of contacts. Adjustment of the effect of quarantine only for the total number of contacts had little impact on the point estimate for SCCR (0.35245 versus 0.31598; data not shown), nor its significance (when compared to Table 2). However, when quarantine status was also adjusted (see Table 3) for the number of close contacts (level 1; see Table 1), the SCCR attributable to quarantine was reduced by 3.6% and it's asymptotic level of significance went from p = 0.026 to p = 0.046. This change falls just short of satisfying criteria for significant mediation using the Baron and Kenny method. A bootstrapped test for reduction of effect was not significant (p > 0.999).

The adjusted models in Table 3 also show that the number of close contacts was a significant predictor of the number of secondary cases, even when adjusting for total number of contacts. This remained statistically significant even in the bootstrapped model.

Finally, the SCCD and NNQ estimates (Table 2) were rerun including 140 additional quarantined false-positive SARS cases with no secondary cases. In this analysis, the NNQ estimate drops from 7.51 to 5.74, giving the appearance of a still greater benefit (data not shown; again, statistically significant under the large sample assumption, and not when using bootstrap methods).

For all Poisson-based analyses, model diagnostics did not indicate significant over-dispersion (p-value > 0.99 in all cases). Using negative binomial models to account for over-dispersion did not affect point estimates, but resulted in slightly wider asymptotic confidence limits (Table 2).

How has quarantine been evaluated?

Previous studies evaluating quarantine for control of SARS used two general methods: simulation studies; and, case-study reports describing specific settings (e.g.).

Simulation studies serve several purposes in outbreak research, one of which is to estimate the impact of control measures in outbreak scenarios. A review of simulation studies on quarantine effectiveness for SARS has been presented by Bauch and colleagues. This review, and other individual reports suggest that quarantine measures are most effective when mobilized at the very start of the outbreak when numbers are small. While decision-makers must rely heavily on simulation studies, a persistent concern raised is that results are driven by prior assumptions; which may be unrealistic. Simulations rely on some degree of simplification such as considering only point-source outbreaks in non-overlapping populations. More sophisticated models incorporate heterogeneity in parameters including variations in size and behaviour of human contact networks, differences in transmissibility due to the host, and levels of adherence with control measures, such that scenarios simulated reflect what happens in real-world experiences. Simulation studies, typically, do not present impact statistics familiar to health policy makers, and may not provide evidence as compelling to decision-makers as real world outbreak experiences. To give an example, a recent World Health Organization review on control measures for influenza cited no simulation studies but only real-world observational data.

Quarantine-related studies from real SARS outbreaks typically present the epidemic curve with case counts plotted against the timeline of events including arrival of index cases, and changes in public health response. In many reports, the impact of control measures is implied from qualitative observation, such as visible deceleration of new cases in plots, or the eventual end of the outbreak. Some of these reports appeared well after the outbreak, with efforts made to complete data on onset dates and transmission in hindsight (e.g.,). Drawing inference from epidemic curves suits only point-source outbreaks, and amount to one-group, pre-test post-test designs, providing weak evidence for causation. The true value of case reports, arguably, is the rich contextual information on challenges faced and unexpected events, and showing socio-political feasibility of aggressive control across settings.

Some case reports have taken a more quantitative approach. A few studies reported on the effect of quarantine in shortening the time from onset of symptoms to isolation, or the proportion of SARS cases who developed symptoms while already under quarantine. Others report quarantine yield (the proportion of individuals quarantined who eventually develop SARS), which reflects specificity of contact tracing versus burden from unnecessary quarantine. These are intermediate outcomes, however, evaluating processes as opposed to final outcomes.

In others, control interventions are examined pre- post-quarantine in relation to subsequent changes in R. Wallinga and Teunis, 2004, present the average daily effective reproductive number, R, both prior to alert and after, for each of four SARS outbreak locations. Table 1 of their analysis reveals fairly consistent transmission numbers pre-alert, but more variable R values post-alert, highlighting less effective control in Ontario relative to elsewhere. This is also an uncommon example of presentation of R estimates along with confidence limits from observed (as opposed to simulated) data. Their analysis would permit reporting of differences in transmissions before and after alert, but no reduction in R attributable to control was presented (with or without confidence limits). The approach also permitted no consideration of individual-level covariates.

Quantitative estimates of quarantine impact

We estimated that use of community quarantine in the 2003 Ontario SARS outbreak reduced transmission to one third, with an absolute difference of 0.13 secondary cases per index case under quarantine, relative to not quarantined by symptom onset. For discussion purposes, we present several effect measures, including Secondary Case Count Difference (SCCD) and "number needed to quarantine" (NNQ), a novel adaptation of NNT. Our point estimate of NNQ for the Ontario outbreak was 7.5 persons in quarantine to one SARS case averted using data from probable or suspected SARS cases. As a point estimate, this NNQ compares very favourably with NNTs reported for public health interventions such as chemoprophylaxis for leprosy and meningococcal disease, or vaccination against pertussis and influenza and pneumococcal disease, particularly with a condition like SARS with significant morbidity and a high case-fatality rate. All estimates we present for the impact of quarantine, however, are imprecise. Bootstrapped confidence intervals include values for no impact. Statistical power is a limitation to this and many analyses of real outbreak data.

We also show, not surprisingly for an infection now known to be transmitted by droplet spread, a statistically significant association between the number of close contacts and number of secondary cases, per index case. Number of close contacts (level 1 in Table 1) had some overlap with the observed (non-significant) effect of quarantine, whereas the number of more distant contacts was unrelated to any apparent benefit of quarantine. Our analysis also suggests (without statistical significance) that reduction in the number of close contacts contributed to reduction in spread, and this may have implications for targeting of quarantine toward closer contacts.

Statistical challenges

Research in quarantine effectiveness presents many challenges. One complication is the unit of analysis for the outcome relative to the intervention. Clinical decision-making looks at outcomes in the same individuals assigned to treatment. Interventions such as vaccination are more complex in that outcomes may be assessed at the individual or population level, with different implications, although the individual vaccinated is part of the same population. Number Needed to Vaccinate (NNV) has been estimated incorporating herd immunity. The case of quarantine is distinct even from vaccination, in that all potential benefit is to other persons. It is theoretically possible to study sets of index cases and their contacts as independent units of analysis, although it is often difficult to identify precisely which persons exposed which others. Here, we have worked with contacts matched to an exclusive index case. The creation of sets of cases and associated controls goes only part-way toward a complete network-based analysis, although future studies could address non-independence of networks.

A second challenge to evaluators might be non-familiarity with regression models for count data. The generalized linear model used here to obtain an NNQ estimate, with a Poisson error term and identity link function, is less commonly used, but long-described in biostatistics texts. Regression models used here are available in all major statistical packages (Stata, SAS, SPSS and others). The distribution of secondary cases (per index case) may be positively skewed (with a few cases generating large numbers of transmissions). Over-dispersion may need to be addressed through means such as use of negative binomial models in place of Poisson models, as above. Negative binomial models have interpretation very similar to Poisson models.

Statistical power was a limitation of our analysis and most studies of real outbreaks. As the goal of outbreak management is to minimize events, small samples must be considered. Procedures assuming large samples tend to overstate precision relative to bootstrapping. Other authors in this field have used bootstrap variance methods as well.

Methodological challenges

Random assignment of individuals to quarantine is not ethical; and in some jurisdictions, comprehensive quarantine procedures may eliminate any control arm. In North America and Europe, voluntary quarantine practices are favoured and some degree of non-adherence is inevitable, so both non-quarantined and quarantined groups will be observed. However, selection bias related to health status, employment, family structure and other factors may confound the association between quarantine status and observed transmissions. The best possible observational design would permit evaluation of the decision made by public health officials to place individuals under quarantine and apply analyses based on both the intention to treat approach and taking compliance into account.

Retrospectively, we explored the possibility of identifying all individuals screened by public health staff for potential quarantine and contact tracing, regardless of final disposition. This was not feasible. Practices varied with respect to when a record was initiated (i.e., in one health unit a file might have been opened even with an unfounded inquiry, elsewhere a record was generated only with a confirmed contact link and symptoms). Within Public Health records, we were able to confirm 140 quarantined false positive "cases". These cases were quarantined contacts who became ill with possible SARS symptoms but were subsequently excluded as SARS cases, and they had at least one identified community contact. Future cost-benefit studies should include information on such groups (e.g.,). Legitimate costs are incurred for and by these false positive cases and their contacts which should be taken into account. Because people without the disease can't spread it, uneven distribution of such individuals across infection control conditions being compared could bias estimates of impact. As we found, the rate of false positive cases (resulting in no transmission) may have a large influence on the apparent benefit. Several case-reports discussed problems with prospective record-keeping, in terms of detailed contact tracing and the implications of time-lags in serological testing (including those never tested), as challenges to both outbreak management and research.

Measurement error is also likely with other important information, such as documenting level of contact and therefore numbers of individuals at risk by contact level. With delayed contact tracing, assignment of contact level may even be done after secondary infection (unblinded) and so could be biased toward closer contact where transmission already happen, and toward less close contact where the contact remained well.

Papers on the SARS experience have spoken about the importance of data management resources and described core data to be tracked during an outbreak (e.g.,). None made explicit recommendations for statistical evaluation of control measures. Planning to report statistics familiar to other areas of health care evaluation may improve the quality and comparability of data collected.

Our approach demonstrates that existing outbreak data may yield more information to evaluate outbreak control measures than has been reported. Further research, presenting quantitative differences in outcomes attributable to measures such as quarantine, would be useful in many ways. First, this would add to evidence on cost-effectiveness. Second, it would facilitate further methodological development in this field. Pooled re-analysis of existing outbreak data across several settings, would ameliorate statistical power problems, and increase the scientific contribution from these important databases.

Challenges in interpretation and communication

Policy-makers rely on estimates of the impact of population-based preventive measures, which should derive from actual experience, as well as theoretical forecasting. Evidence also needs to be understood. It has been debated whether the NNT statistic achieves its goal to facilitate decision-making in other health care settings. Further thought and discussion are needed as to how meaningful a NNQ statistic might be for decision-making in outbreak planning, relative to other expressions of attributable case reductions, such as SCCD also proposed here, or other metrics.

No variant on attributable risk difference or NNT can be interpreted without consideration of the absolute costs of not acting, and the harm side of the decision. This discussion must include the severity of the illness, as well as harms of intervention to the individual and society which are difficult to quantify and value-laden. Quarantine includes potential non-health related harms including civil liberties and may include economic and other costs to the individual.

Finally, studies to evaluate control measures for one agent may not be generalizable to other agents. Measures to restrict close contact probably made an important contribution to the control of SARS outbreaks. Evidence of this is accumulating slowly and should be taken into consideration for future outbreaks of SARS or similar droplet spread agents without significant transmission in the asymptomatic phase. However, the applicability of this evidence to the current experience with pandemic influenza is less certain. Ferguson et al present simulation data suggesting community quarantine and isolation may play a role in influenza control but comment that this would presume such measures were feasible. Transmission patterns for influenza however, make it less likely that contact tracing and quarantine would be fast enough to avoid transmission which is greatest in the earliest stage of infection. Under such circumstances, use of a 'severe' measure such as quarantine is likely not justified where such efforts are likely to yield little benefit.

Conclusions

Relative to other health policy areas, literature on quarantine tends to lack in quantitative expressions of effectiveness, or agreement on how best to report differences in outcomes attributable to control measure. The study of quarantine effectiveness presents several methodological and statistical challenges. Further research and discussion are needed to understand the costs and benefits of enacting quarantine, and this includes a discussion of how quantitative benefit should be communicated to decision-makers and the public, and evaluated.

Competing interests

There are no financial or non-financial competing interests. The funding source had no role in study design; collection, analysis or interpretation of data; writing of the report; nor decision to submit the paper for publication.

Authors' contributions

All listed authors made substantial contributions to conception and design of the analysis presented in this manuscript. SB, ER and JL were involved in initial collection of data for the underlying outbreak database. All authors were involved in defining the objectives for the present analysis; in determining case selection and definitions for the present analysis; and in interpretation of results. SB and JL took lead responsibility for data analysis. All authors were involved drafting the manuscript and revising it critically for intellectual content. All authors have given final approval of the version to be published.

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-2458/9/488/prepub