The statistical package
The statistical analysis employed has been reviewed and officially approved by DEFRA, who have also provided the criteria they require for the package to be operated under different circumstances (e.g. the herd’s testing parish). It is important to understand that because this tool uses a number of estimated inputs (e.g., test sensitivity, test specificity, etc.) and analyses clinical situations that are by definition uncertain, its predictions cannot be 100% accurate. The use of this tool is expected, however, to guard significantly against the risk of inaccurate diagnosis of TB being made in the herd.
The need for such a statistical approach stems inherent uncertainty in testing for disease in individual animals. Although at the four-antigen level the high specificity of the Enferplex test (which is close to 100%) implies that any positive result can safely be interpreted as confirmation of disease, situations where a proportion of the herd tests positive only at the two-antigen level (with no positives at the four antigen level) need to be addressed. At the two antigen level, the specificity of the test is currently estimated as 96.9%. This implies that there is significant risk of the occurrence of false positives, which would be highly problematic under a strict herd screening approach. Ignoring positive tests, on the other hand, would risk overlooking true TB herd infections.
The statistical package developed by Surefarm Ltd. addresses these issues. For a given herd population size, the package determines the upper bound for the number of animals that could test positive at the two antigen level while the herd would still be considered likely to be disease free (because a diseased herd would almost certainly have generated more positives).
This upper bound will be set such that the threshold will only be breached when the confidence level that the herd is truly diseased is reached (herd level specificity set at 99.5%): i.e., when we can be very confident that disease truly exists in the herd.
Therefore, where this threshold is breached the herd will be deemed to be positive and SureFarm to pass details to APHA and the client’s veterinary surgeon who will be informed to notify their local APHA office. This APHA office will decide on appropriate actions on the test positive animals, based on the herd’s location, biosecurity and TB testing history.
Please see the different testing scenarios to understand how these further steps will be applied by DEFRA.
Outcome of potential false positives
As can be seen from above, in cases where the number of two-antigen positive animals is below the given threshold for notification of DEFRA, the issue arises of how to proceed regarding these two-antigen positive animals.
In this situation, the positive animals will be considered as inconclusive reactors. As such, they should not be considered definitively negative and at a minimum should be isolated and subjected to a re-test within 30 days. Failure to re-test such animals would obligate the reporting of the herd to DEFRA.
If these animals subsequently test negative, then no further actions will be required other than to return to annual surveillance. However, a repeat positive result from any animal(s) will require DEFRA to be notified of suspicion of disease: as repeat positive testing is highly unlikely to be due to false positives.
Testing in small herds or small numbers
In cases where small numbers of animals are tested (e.g., small herds), a high degree of confidence that all the animals in the herd are disease-free may not be attainable. As a result the following protocols will apply:-
1. In cases when the threshold number of positives has not been exceeded but the number of animals tested was insufficient to be fully confident of freedom of the herd from disease, any two-antigen positive test(s) be need to be defined as an inconclusive reactor(s) as above.
2. In cases where the threshold number of positives has been exceeded but the numbers are still insufficient to be fully confident that the herd is truly diseased then likewise any two-antigen positive tests) will also be defined as inconclusive reactor(s) as above.
3. In cases where the threshold number of positives has been exceeded to the point where the confidence level agreed with Defra on the statistical package has been reached, then suspicion of disease will have been deemed to occur and SureFarm to pass details to APHA and the clients veterinary surgeon who will be informed to notify their local APHA office. This APHA office will decide on appropriate actions on the test positive animals, based on the herd’s location, biosecurity and TB testing history.
Methodology of the Statistics used in the Analysis
Most users will not wish to enter into the details of the statistical methodology. Salient points include the following:
- The importance of expected disease prevalence in known infected herds. Seeking a high degree of confidence that a herd is disease-free entails interpreting negative tests as evidence against the maintained (null) hypothesis that the herd contains diseased animals. The degree of confidence that a herd is disease-free therefore depends on prior beliefs or expectations about (a) the prevalence of the disease and, related but distinct, (b) the likely spread of the disease within an infected herd. The more infected animals are likely to be in an infected herd, the more likely we are under the maintained hypothesis to have tested them, and therefore the stronger the interpretation of a lack of positive tests in favour of the alternative that the herd is in fact disease-free. Conversely, if there are few diseased animals even in infected herds, we must test many animals before a lack of positive tests is strong evidence in favour of a disease-free herd.
- Sensitivity and specificity of individual tests. The degree of confidence that a herd is disease-free also depends on the sensitivity and specificity of the tests administered. The Enferplex test has relatively low sensitivity and high specificity. The high specificity is an attribute in that positives are strong evidence in favour of disease. But high specificity combined with low sensitivity tends to make both true and false negatives more common, increasing the number required for testing to be able to reach a given confidence level that a herd is disease-free.
- Small herds. Testing in small herds requires tailored analysis and approach. In herds of less than about 30-40 animals, our analysis suggests that the entire herd would have to be tested before any statistical conclusions could be drawn. In even smaller herds, repeat testing of the same animals would be needed before any conclusion could be drawn about the disease-free status of the herd at conventional statistical significance levels.
Inputs used to generate statistics
As can be understood from the above, one of the key inputs that needs to be determined before the analysis can proceed is the likely prevalence of the disease. Following discussion with DEFRA, the prevalence parameter is to be set at 10% for annual testing parishes and 5% for other areas.