The Doctor’s Dilemma/Statistical Illusions

Public ignorance of the laws of evidence and of statistics can hardly be exaggerated. There may be a doctor here and there who in dealing with the statistics of disease has taken at least the first step towards sanity by grasping the fact that as an attack of even the commonest disease is an exceptional event, apparently over-whelming statistical evidence in favor of any prophylactic can be produced by persuading the public that everybody caught the disease formerly. Thus if a disease is one which normally attacks fifteen per cent of the population, and if the effect of a prophylactic is actually to increase the proportion to twenty per cent, the publication of this figure of twenty per cent will convince the public that the prophylactic has reduced the percentage by eighty per cent instead of increasing it by five, because the public, left to itself and to the old gentlemen who are always ready to remember, on every possible subject, that things used to be much worse than they are now (such old gentlemen greatly outnumber the laudatores tempori acti), will assume that the former percentage was about 100. The vogue of the Pasteur treatment of hydrophobia, for instance, was due to the assumption by the public that every person bitten by a rabid dog necessarily got hydrophobia. I myself heard hydrophobia discussed in my youth by doctors in Dublin before a Pasteur Institute existed, the subject having been brought forward there by the scepticism of an eminent surgeon as to whether hydrophobia is really a specific disease or only ordinary tetanus induced (as tetanus was then supposed to be induced) by a lacerated wound. There were no statistics available as to the proportion of dog bites that ended in hydrophobia; but nobody ever guessed that the cases could be more than two or three per cent of the bites. On me, therefore, the results published by the Pasteur Institute produced no such effect as they did on the ordinary man who thinks that the bite of a mad dog means certain hydrophobia. It seemed to me that the proportion of deaths among the cases treated at the Institute was rather higher, if anything, than might have been expected had there been no Institute in existence. But to the public every Pasteur patient who did not die was miraculously saved from an agonizing death by the beneficent white magic of that most trusty of all wizards, the man of science.

Even trained statisticians often fail to appreciate the extent to which statistics are vitiated by the unrecorded assumptions of their interpreters. Their attention is too much occupied with the cruder tricks of those who make a corrupt use of statistics for advertizing purposes. There is, for example, the percentage dodge. In some hamlet, barely large enough to have a name, two people are attacked during a smallpox epidemic. One dies: the other recovers. One has vaccination marks: the other has none. Immediately either the vaccinists or the antivaccinists publish the triumphant news that at such and such a place not a single vaccinated person died of smallpox whilst 100 per cent of the unvaccinated perished miserably; or, as the case may be, that 100 per cent of the unvaccinated recovered whilst the vaccinated succumbed to the last man. Or, to take another common instance, comparisons which are really comparisons between two social classes with different standards of nutrition and education are palmed off as comparisons between the results of a certain medical treatment and its neglect. Thus it is easy to prove that the wearing of tall hats and the carrying of umbrellas enlarges the chest, prolongs life, and confers comparative immunity from disease; for the statistics show that the classes which use these articles are bigger, healthier, and live longer than the class which never dreams of possessing such things. It does not take much perspicacity to see that what really makes this difference is not the tall hat and the umbrella, but the wealth and nourishment of which they are evidence, and that a gold watch or membership of a club in Pall Mall might be proved in the same way to have the like sovereign virtues. A university degree, a daily bath, the owning of thirty pairs of trousers, a knowledge of Wagner's music, a pew in church, anything, in short, that implies more means and better nurture than the mass of laborers enjoy, can be statistically palmed off as a magic-spell conferring all sorts of privileges.

In the case of a prophylactic enforced by law, this illusion is intensified grotesquely, because only vagrants can evade it. Now vagrants have little power of resisting any disease: their death rate and their case-mortality rate is always high relatively to that of respectable folk. Nothing is easier, therefore, than to prove that compliance with any public regulation produces the most gratifying results. It would be equally easy even if the regulation actually raised the death-rate, provided it did not raise it sufficiently to make the average householder, who cannot evade regulations, die as early as the average vagrant who can.