The present article is aimed to reply and clarify some issues from the article “Much ado about nothing” published by Apgaylard in www.wordpress.com that uses many arguments to criticize and reduce the impact of the report of the homeopathic intervention against Leptospirosis in Cuba. I consider some of the arguments used as inappropriate and sometimes based on unsupported assumptions, and thus could be confusing to the readers. I have kept the focus away from the subjective and ironic arguments but instead look at the scientific principles. I have pasted the complete text from the Apgaylard´s article and added some comments and analysis in blue bold letters within the text giving the discussion of main ideas from another point of view.
Please follow the COMMENTS (blue paragraphs)
So, the much-trailed* paper on the homeopathic intervention in an outbreak of Leptospirosis in Cuba during 2007 has finally been published (Bracho et al, 2010) along with a very useful companion editorial (Roniger and Jacobs, 2010).
The editorial claims that, “the size of the population treated and the dramatic decrease in disease incidence compared to previous years make it difficult to dismiss these results as spurious or occurring by chance.”
Before finding it quite easy to dismiss these results, some background information on Leptospirosis is in order.
What is Leptospirosis?
Also called pea picker’s disease or swineherd’s disease is an, “acute systemic illness of animals, occasionally communicable to humans, that is characterized by extensive inflammation of the blood vessels. It is caused by a spirochete, or spiral-shaped bacterium.” (Encyclopædia Britannica, 2010)
Humans catch the disease by coming into contact with animal urine, usually as a contaminant of water. The spirochetes gain access through breaks in the skin, the eyes or mucous memranes.
The Encyclopædia Britannica (2010) goes on to point out that after an incubation period of around a week**, “the first symptoms to arise in humans are the abrupt onset of fever, chills, muscle aches, headache, abdominal pain, and vomiting. Another characteristic symptom is congestion of the conjunctival blood vessels around the corneas of the eyes.”
Unpleasantly, “After a latent period of five to seven days, during which the infected person may improve, fever returns and the infection may involve the brain. In a less-common, but more-severe, form of the illness known as Weil disease or icteric (associated with jaundice) leptospirosis, the infected person turns yellow as a result of the destruction of red blood cells and liver disease. The death rate is approximately 30 percent of the severely ill and jaundiced patients.”
The Encyclopædia Britannica (2010) concludes by pointing out that the disease is diagnosed, “by identification of the causative organism in urine or blood and by blood cultures on special media.”
The disease can be treated with antibiotics; this should be done within the first four or five days, as “Antibiotic efficacy drops rapidly after day [five] of symptoms and can be of no benefit whatsoever after day [eight]. Ideally it should be initiated no later than 10 days after exposure”.
Prevention requires limiting exposure to freshwater or mud, covering cuts and grazes, washing after any exposure. If this is not possible, then using waterproof gloves and boots is recommended. Finally, in affected areas, it is important to only drink sealed bottled water or fresh water that has been boiled.
For individuals who are at risk the antibiotic Doxycycline is used as a phrophylactic.
Clearly, this is a disease that poses relatively little risk in developed countries with good medical infrastructure. Conversely, in the developing world it is a serious, and sometimes fatal, problem.
Why was homeopathy used?
Although the Cuban Government has, quite remarkably, developed an effective vaccine for humans (vax-SPIRAL), according to Bracho et al (2010), they have been unable to produce it in sufficient quantities to meet the demand. In stepped the homeopaths with the suggestion that they could knock something up cheaply and quickly to help with a crisis.
The Cuban Scientific Centers have developed and produced 12 different vaccines, some of them only available from Cuban manufactures (i.e. meningococcal B vaccine, synthetic Hib vaccine and three valent Leptospirosis vaccine). The production facilities guarantee to provide for the demand for the national immunization program but also export a considerable amount of vaccines to other countries. As far as can be seen, there were many reasons to begin the choice of using homeopathy to confront the leptospirosis epidemic.
- The stockpile of vaxSPIRAL (Cuban leptospirosis vaccine) at the time when the epidemic peaked and the risks increased (impact of natural disasters) was limited.
- Vaccination is of reduced effectiveness in an epidemic setting in the short term because of the time needed to induce protection.
- The large period of time needed (months) to get a vaccine ready to use even on the most modern production plant.
- Lack of time because of the danger of the situation.
- Previous reports on the use of homeoprophylaxis to control epidemics and infectious diseases as referred in the article.
- The advantages of using homeoprophylaxis in these settings and the potential of possible positive effects as reported previously.
- Finally, none of the authors are homeopaths, but orthodox scientists who surely apply skepticism and scientific principles to the analysis.
So the above assumptions have been taken without any supporting information.
What did the homeopaths do?
They developed a ‘nosode’ treatment, nosoLEP, which is described as ‘containing’:
“four highly-diluted strains of inactivated leptospiras: L. interogans Serovar Canicola, L. interogans Serovar Copenhageni, L. kirschneri Serovar Mozdok and L. borgpetrsenii [sic] Serovar Ballum. ***
Serovar Copenhageni, L. kirschneri Serovar Mozdok and L. borgpetrsenii [sic] Serovar Ballum. ***
References for Leptospirosis classification need to be updated. The taxonomy group from the International Society for Leptospirosis during the meetings of 2005 and 2007, decided to classify Leptospirosis species according to genotypes following by serovars. So grouping by serotyping is not the currently used classification. Please check:
As we shall see later, “highly diluted” is not so much an understatement as a misstatement.
They aimed to give this homeopathic prophylactic to:
“The entire population over 1 year of age from the provinces of Las Tunas (LT), Holguı´n (HG) and Granma (GR) in eastern region of Cuba”
This gave a target population of 2.4 million people, of whom they ultimately ‘treated’ 2.3 million (96%).
The intervention started in week 45 of 2007, “with two oral doses of nosoLEP 200C with an interval between doses of 7 – 9 days. Ten to twelve months later, the schedule was completed by the administration of another two oral doses (7 – 9 days apart) of nosoLEP 10MC.”
How was each individual treated?:
“Each dose consisted of five drops (250 – 300 μL) administred [sic] sublingually 20 m away from eating, smoking or drinking. It was administered by about 5000 personnel of public health system of Cuba which included family doctors, nurses, social workers and paramedics that were trained in the administration procedure.”
This represents a huge effort from healthcare workers. Although the cost of the preparation must have been small, I am left wondering about the opportunity cost of deploying so many skilled individuals on such a task.
There is no additional cost of deploying such a distribution system because they were already deployed previously. It has been there for more than 20 years since this is the basis of the Cuban Health System, to have a network of medical doctors spread throughout every region of Cuba and in charge of medical care of small areas. In that way, the intervention could be conducted by using this already established system.
Why am I not convinced?
Roniger and Jacobs (2010) claim that the sheer size of the intervention plus correlation with a dramatic reduction in reported cases makes this study unassailable is nowhere near as strong an argument as it seems. Let’s see why.
Homeopathy is not a proper Journal.
Before looking at the technical merit of the work, the first thing that always comes to mind is what a terrible pseudojournal Homeopathy is. Their disgraceful standards of quality control and peer review were exposed for all to see in the infamous ‘Memory of Water’ special edition†.
They allowed the publication of specious quantum flapdoodle (Weingartner, 2007; Milgrom, 2007), just plain wrong assertions, childishly transparent ‘mathematical’ reasoning (Weingartner, 2007), obviously sloppy (Rao et al, 2007; Rey, 2007) and incomplete experiments (Vybiral and Voracek, 2007), clutching at straws (Elia, Napoli and Germano, 2007), and missing the blindingly obvious (Beauvais, 2008) all topped off with authors drawing wild conclusions (e.g. Vybiral and Voracek, 2007).
To their credit, the editors did publish a number of critical letters; though this has had no discernable effect on standards at Homeopathy. For instance in 2008 they published a ‘special’ homeopath-friendly version (Rutten and Stolper, 2008) of a paper (Lüdtke and Rutten, 2008) re-analyzing the allegedly controversial meta-analysis of Shang et al (2005, 2006). This was shown to be partisan, error-strewn and rash (Wilson, 2009). Comparing the two versions shows nicely that this journal is a mouthpiece for homeopathic propaganda† †, not a respectable scientific journal.
Given the lack of scholastic standards at Homeopathy, then this comment from Roniger and Jacobs (2010) really says it all:
“The non-randomized cohort design may explain why this innovative and well-designed study with its remarkable results was rejected by conventional journals.”
A telling admission; it’s clearly been hawked around proper journals and has ended up in Homeopathy. If it really was an “innovative and well designed study” then it would have been published elsewhere. Clearly the authors would have preferred this. Instead it’s ended up in the Homeopath’s trade journal; a remarkable story, given that even good journals can be quite homoeophile.
“Give more credence to trials published in well known medical journal and give no credence at all to those published in CAM journals.”
However, the reasons for rejections remains unknown.
Good advice, I’m sure. But in the interests of being overly fair, I’ll plough on anyway.
There’s nothing in it
The biggest problem with this sort of homeopathy is that there is no medicine in the medicine. This intervention used mind-bending levels of dilution. Staring with “1 x 106 bacteria/ml” and using Korvsakovian [sic] dilutions (plus succession) the manufacturers produced two potencies: “200°C [sic] (200 X 1:100 dilutions) and 10 MC (104 X 1:100 dilutions)”
So the most concentrated form of the medicine diluted the initial preparation to one part in 10400! The highest potency represents a dilution of one part in 1020000!!
Given that there are something like 1050 atoms in the world, one thing is crystal clear: the manufacturing process washed the bugs clean out of the medicine. This fact alone rules out the homeopathic preparation having any specific effect against the disease.
Homeopaths may declaim about ‘the memory of water’, but there’s no evidence to support this effect (if you exclude hand waving theories and badly run experiments). It also contradicts the knowledge embodied in modern Chemistry and Physics. Finally, homeopaths and their apologists tend to gloss over that often water is not the only dilutent. In this case it’s 30% ethanol in water. So, what the homeopaths need is ethanol/water memory. It goes without saying that the evidence for this is even more dilute.
This incredible level of dilution leads to quite odd ‘quality control’ methods:
“The quality of final products was controlled by measuring the alcohol content, water quality, pH and microbiologic load.”
These quality control measurements are just recommended for any oral product produced on a large scale.
Sensible measures, no doubt, but making a product in which you cannot detect the alleged ‘active’ ingredients should be some sort of red flag.
Korsakovian dilutions, named for Semen Korsakov, essentially make each dilution by filling a container, emptying it, assuming that the volume of liquid adhering to the container wall is sufficient for the dilution ratio claimed, re-filling with dilutent, shaking and repeating the procedure for the desired number of dilutions. Clearly this is more suited to industrial scale preparation, but seems to be based on a rash assumption.
In essence, the two potencies used were made by washing a flask that been drained of the ‘mother tincture’ either 199 or 9,999 times and accepting the next refill as medicine. It’s just not plausible.
It also makes the title of the paper, “Large-scale application of highly-diluted bacteria for Leptospirosis epidemic control” misleading. The bacteria are not “highly-diluted” they have been washed away. The homeopaths know that to be true. In the editorial Roniger and Jacobs (2010), bemoan the fact that they could not have this nosode preparation made in Europe because of the need to sterilise animal and disease products, resulting in, “mother tincture will either be free of infectious material or contain infectious organisms changed by heat to be beyond recognition by the immune system.” They think that this is unfair because:
“… one could easily argue this to be unnecessary because after only a few by conventional definition potentisation steps homeopathic remedies would be sterile (usually 6C)”
Yes, they do understand the Avogadro limit, when it suits them.
- How can we be absolutely sure that there is nothing in it?
- Is there any plausible or even possible theory or any evidence that could support at least part of what is happening with agitated ultra dilutions?
- Do we have all the knowledge needed to make final irrevocable certain conclusions about what is happening beyond Avogadro’s number?
To find answers in a fair, impartial and scientific manner will be the only way to discuss and to define reality.
Now, given that there is no trace of the inactivated bacteria in the preparations and no credible explanation as to how the washed away bacteria might have influenced the diluent any claims of a real effect would need to be based on an unimpeachable trial design, comprehensively and accurately reported. However, this is not a well designed trial, as its rejection from credible journals evidences.
Moving on to the design of the trial; it has real problems.
No real controls, No Randomization
First, it contains no appropriate controls. It’s clear that the Intervention Region (IR) and the external comparator, the rest of Cuba (RC), were already quite different before the homeopathic intervention. Just looking at Figure 4 (please note: all figures are based on measuring data points off the page so may be subject to small errors, below) in the paper shows that annual Leptospirosis incidence from 2000 to 2008 was around 4 per hundred thousand. The Intervention Region (IR) was indistinguishable from the rest of Cuba in 2004, and then started to increase: 6 (2005), 11(2006), 16.6 (2007). Clearly, something more has happened in the RC than the crisis described in the paper. This problem had been building for some years. This makes the RC a very poor control for this study.
The trial could have been designed with internal control regions. It would then have been possible to introduce randomization. A point alluded to in Roniger and Jacobs (2010) editorial:
It is unethical to introduce randomization and internal control groups (placebo) during emergency situations and a strong epidemic, particularly with such a large number of persons at risk. There were no initial research objectives in this intervention. The authors are just reporting the results from surveillance after the intervention as an observation that could point out a remarkable finding, without irrefutable conclusions, but which could have a huge impact on the approaches to confront epidemics.
Precisely, in the region of greatest risk and epidemic (IR), with less probability to reduce the disease incidence, after the intervention a significant reduction of cases was observed, whilst in the region with the lower risk (RC) the incidence in the same period increased. Although the differences between IR and RC could be discussed, they were more favored a failure than a success in IR, so in this case the impact of the intervention on the results was remarkable.
“In retrospect, it has been su
ggested that randomization could have been done by province to maximize the rigor of the study design.”
It’s unclear why this wasn’t thought of by the homeopaths. The slightly sniffy “in retrospect” dismisses a very important aspect of trail design. Without randomization to an internal control group there is the very real chance that some other change could have caused the sudden, dramatic decline in reported cases.
Perhaps this is why no respectable journal would publish it?
So with no appropriate controls and no randomization, how do the homeopaths know that their intervention made the difference? Well, actually they are not in a position to claim causation. All they have is a correlation in an uncontrolled experiment. The authors attempt to address this by the use of predictive models.
Reliance on undefined models
One of their main approaches is to compare what happened in the IR and RC, with what might have been. To do this they took five “available forecast models” and tested them for, “best fit to temporal series of Leptospirosis cases (dependent) and rainfall (independent variable)”
This seems a very simple approach, so I’m looking for a detailed description of what they did, along with some data to back up their claims. The authors do provide a little more detail:
“To select the best fitting model, all were tested to determine how well they predicted the real temporal series of 2000 – 2004. The differences between forecast and real values (residual error) were analysed for statistical significance. All models gave similar forecast curves, but simple exponential smoothing was selected as no significant differences were observed when the residual errors were analysed in five out of five different tests with a 95% confidence level while the other five models all failed in one or more tests.”
This seems a very short period over which to calibrate the models, particularly as something happened to make the IR results very different to the rest of Cuba, from 2005 to 2007.
Given the central importance of this technique to the claims, I am surprised that the paper contains no real details of the model used and no results for this selection process. We are just told that a, ” simple exponential smoothing” model worked best on five years of data.
The authors also say that, “Adjusted forecast curves, lower and upper confidence limits were validated with real data sets from different years”, with no description of how the forecast curves were adjusted. There is an oblique comment that the models’ predictions were based on disease history and rainfall; indicating some sort of adjustment made on previous years’ data. Overall the description of the modeling is wholly inadequate given the reliance placed upon it.
There is also something odd about the description of the model selection process. Elsewhere we are told, “Unfortunately, no data on rainfall are available before 2004.” If that’s the case how were these models run and assessed? It could just be poor reporting, but it’s an obvious anomaly in the paper.
So when I’m told that, “A simple exponential smoothing model based on the 2004 – 2007 data was used to estimate the probable trend” I can’t really have much confidence in the predictions of case numbers during the trial period. Particularly if it’s being taken to indicate that a few drops of 30% ethanol in water is preventing infection.
All models could be questionable, just because it is a mathematical approach trying to model real and mostly complex phenomena that can only incorporate a range of probabilities. So a comparison with a mathematic model should not be used as a solid evidence to take conclusions. As pointed out by the authors, the incidence of Leptospirosis is affected by a several factors. The authors acknowledge this and neither use the prediction of the model to support any conclusion. The main objective for the use of the model appears to be to measure the risk of occurring epidemics. The authors did not use the comparison between predicted values and real incidence as the main evidence and it was only mentioned in a couple of sentences in the article; the strongest evidence is based on changes in the trend, comparisons with historic media and the behavior of the disease on the rest of the country.
Despite the reliability of the model needs to be tested over the years and is not done yet; interestingly the forecast curves meet the real data on the predicted period of 2007 in the RC (no intervened region) suggesting that it could be useful. However, even when accuracy of the model could be questionable, the lack of details on the model is not a reason for rejecting it. In addition, the mathematical development of a forecast model and its validation could be a matter for another full article and it is difficult to summarize in a paragraph.
Time to treat?
Another concern I have with this study is the sheer scale of the logistical challenge. Getting 2.3 million people to take drops of the remedy, twice, is something of an organizational triumph. But did they do it quickly enough?
The campaign started during week 45 of 2007 and the bulk of the effect, for which credit is being claimed, is seen by the end of week 47. Given a typical one week incubation period, this would mean these people would have been infected in week 46, were it not for the homeopathy.
What level of coverage, I wonder, had the homeopaths achieved by week 46? The paper does not say, but includes some limited data on the progress of this campaign. It reports 70% coverage after three weeks (week 48), 92% by week 50 and ultimately 96% coverage, presumably by the end of the year. This looks to me like I would expect coverage in week 46 to be less than 40% (See below). I could be wrong, but given the extraordinary nature of the claims made in this paper, I don’t think it unreasonable to query this.
The percent showed here (week 46-40%) is calculated on the assumption that the coverage curve will follow a similar trend before (weeks 45-48) and after (weeks 45-52) the 70%, thus it can be modeling using a polynomial equation. However, if you check other coverage curves the shape could be completely different, particularly in campaign interventions. Usually it follows two phases, an exponential curve with a fast increase between 0 and 70-80% and a logarithm curve on higher percentages. Thus, the 40% of coverage calculated from the hypothetical and probably unrealistic curve in week 46 and assumed to criticize the article, could be higher and thus inappropriate.
Short data sequence
Another argument that the authors make to support their claim that the fall in cases towards the end of 2007 would be entirely unexpected. This is based on a frequency analysis by week, using data from 1990 to 2006 (Their Figure 1).
They use this to argue that a peak in the number of reported cases is to be expected during the last few weeks of the year. However, the range of these data is huge. For instance, the last week of the year has an historic median of close to 100 cases, but the range runs from around 15 to close to 550. Accounting for the fact that the IR holds about 21% of the population of Cuba, the bottom of the range could be around three cases for the last week of the year.
Now, this is taking the extreme end of the range of the data. However, given the relatively short historical dataset (16 years) for a highly variable pattern of disease, the authors’ argument that the year end will always show a spike of Leptospirosis is less than convincing.
The analysis of patterns and trends should not be based only on ranges since it only tells you the maximum and minimum values of the data, thus it only represents the variability of data. It is impossible to know from the graphics the series to which the minimum or maximum values (ranges) belongs. Just having a single uncommon point far away from the population of dots grouping the large number of data (central tendency), you will have a maximum or a minimum (range) that’s tells you nothing about the real trend of the population. Thus, one must better base the analysis on the central tendency of the data (medians and quartiles in this case). Just have a look into the following graphic (hypothetical data). Two series from two years with different number of cases in the last week, thus the values from week 52 of the two years represent the maximum and the minimum for this week (range) but individually the two series follow the same trend i.e. higher number of cases at the end of the year. This pattern can be only checked by analyzing the full data, otherwise it will be inappropriate or speculative. If during the previous 16 years, the large number of cases are truly being observed in the last weeks, then the changes observed after intervention could be considered as an effect of an intervention.
Given that they just handed out drops of a water/alcohol mix, I think that arguing that the year-end trends may not have been as the authors assume is not such a stretch.
Why jump straight into a huge trial?
Reading this paper also makes me wonder why they jumped straight to this scale of epidemic intervention with no basic or clinical research data behind them. Why no basic laboratory studies? Surely, if they had confidence in this treatment, why didn’t they at least do an RCT?
The authors have an excuse:
“Double blind controlled and randomised clinical trials have been seen as the gold standard to demonstrate efficacy of any health intervention. One limitation of such trials is often the size of the population and the levels of exposure to risk factors. Thus, even when successful results are obtained from controlled trials demonstrating efficacy, the real effectiveness needs to be tested in large populations with high exposure to the target disease, preferably in endemic areas.”
It’s not very convincing. Here they have access to a large population with substantial exposure to the risk factors. It would seem that a large randomized control trial would have been well within their means. But the authors argue that because RCTs produce false positives their uncontrolled, un-randomised methodology is preferable. This, of course, is even more likely to provide a spurious positive outcome.
Surely it is unethical to subject a large population to an experiment in controlling a serious disease, without evidence from basic scientific investigation or smaller clinical trials to suggest that it might have some worth?
Is the result really all that good?
Although the reduction in cases in the IR is dramatic, this does no more than return the level of Leptospirosis to a historic level comparable to the rest of Cuba. Why, if the initial homeoprophylaxis was so good, with 96% of the population covered and a follow-up with a reinforcing dose, wasn’t Leptospirosis completely conquered in the IR? Why is it actually no better than the rest of Cuba? Why is it no better than it was in 2004?
In fact at the end of 2007 and 2008 the IR decreased below the historic media of both the IR and Cuba. Notice that this occurred because of a reduction of 337 cases from 2007 (401) to 2008 (64). No health intervention are able to 100% control any disease. Moreover, if the driving forces of an epidemic are not fully eliminated (risk factors or pathogens), then any preventive alternative will not have a 100% impact.
Rather than compare with 2004 it will be better to know what happened in 2009-2011 in this region as the authors propose. Do we even know or have the possibility to determine the baseline for this disease to consider as the controlled or desirable situation? The answer is no based just in the graphics from the paper, so 2004 could not be considered as controlled conditions (mean) but still the reduction observed in 2008, particularly at the end of the year is highly significant.
What could have caused the reported effect?
For all the weaknesses in the trial design, it’s clear that something happened to reduce the incidence of Leptospirosis in the Intervention Region (IR). If it wasn’t the homeopathic prophylaxis (HP) that caused the reported reduction in Leptospirosis, what might have done?
Non Specific Effects
My favourite candidate for a plausible explanation are non-specific effects; changes caused by the intervention, but not the magic water. Back in 2004 the IR had the same rate of Leptospirosis infection as the rest of Cuba. Then something changed. There seems to have been increasing rainfall – a risk factor for the transmission of the infection to humans. However, in 2005 the rest of Cuba combined higher rainfall than the IR with a lower (and flat) rate of infection.
In fact there was an increment of about 0,8 in the incidence per 100 000 inhabitant at RC in 2005 coincident with the increase of rainfalls. Given the population of RC this increase represents around 60-70 more cases.
Although rainfall and the number of cases seem to correlate reasonably well for the IR, they don’t for the rest of Cuba (See above). In fact, in 2008 the data from the IR show the poorest correlation with rainfall. Rather than making the post-homeopathic treatment IR something special, it has just made it more like the rest of Cuba – where rainfall and Leptospirosis cases don’t seem to correlate very well. This can be seen in the simple plot presented below. The analysis, such as it is simplistic (just fitting a trendline in Excel), but it’s more analysis than is contained in the paper.
The relationship of Leptospirosis incidence with rainfall is a phenomena described since last century. Truly this relationship could be affected by measurements to decrease risks and to protect susceptible or exposed individuals. However the relationship is more evident and strong when the endemic or epidemic levels increase. So it is fair to expect a better correlation in an epidemic setting where the high levels of vectors, circulating pathogens and larger size of group at risk maximize the probability of infection.
Anyway, to compare the degree of correlation in IR and RC was not a target of the study. Furthermore, the analysis of the correlation of rains with incidence as stated in the original paper was done with a larger amount of data of monthly reports over the 4 years.
It is evident that a single point after intervention (2008) will not provide a significant modification of the relationship of rains and incidence when analyzing the pool of data, but individual analysis of 2008 suggests that in this year the relationship was different. In that way when the above graphic is split into 4 (by regions including or not the 2008 year) is possible to get a better analysis.
In IR there was a stronger relationship of incidence and rainfalls from 2004 to 2007 when compared with RC. However the decrease in the incidence observed in 2008 (right graphic) significantly modified the relationship from 0.97 to 0.77. This evidence suggests that in 2008 the relationship between rains and incidence was affected and that this year was completely different than previous ones.
In RC the relationship between rains and incidence was even much more modified from 0.84 to 0.38 when the 2008 year is included in the analysis. However, this modification was because of an increment of the incidence in RC rather than a decrease as observed in IR.
Truly when you compare the two regions in the pool of data it is evident that the relationship is stronger in IR but when comparing before and after the intervention major differences could be detected. In both regions the relationship was modified in different extent and by different causes, an increase in RC but a decrease on IR. But what’s clear is that any correlation with rain falls was lost in 2008 in IR.
In addition, to drastically modify any trend it will be necessary to have several points following a different tendency than the previous one, to get a population of data large enough to drive a trend modification. However, we will need to wait for the observation of incoming years to check if the overall relationship of rains and Leptospirosis incidence is modified in IR.
It is also important to note that correlation between variables is a different concept apart from tendency of a given series. Confusing these two analyses could bring uncertain conclusions.
The authors briefly discuss other changes in the IR:
“From 2004 and up to 2007 an increase in the annual incidence of the disease was observed in Cuba, particularly in the IR although there had been no modifications in the strategies for Leptospirosis control. The main cause of this observation is likely to be the implementation of policies promoting agriculture and animal breeding that caused rapid and continuous changes in the size and composition of risk groups, making identification difficult.”
It seems like changes in agricultural practice may have affected the IR, more than the rest of the country. This seems to have changed who is at risk. It’s not unreasonable to argue that organizing this study may have facilitated a better identification of those in risk groups. This would have made the targeted conventional interventions more effective. So maybe just organizing this trial has helped the interventions that work in the rest of Cuba work in the IR?
A second level of non-specific effect could relate to personal precautions. As I noted earlier:
“Prevention requires limiting exposure to freshwater or mud, covering cuts and grazes, washing after any exposure. If this is not possible, then using waterproof gloves and boots is recommended. Finally, in affected areas, it is important to only drink sealed bottled water or fresh water that has been boiled.”
An army of 5,000 health workers visiting (nearly) everybody twice may well have increased compliance with these precautions. It should have been unconscionable for health workers to have put a few drops of the elixir under the tongue of each person without reminding them of what they could do to protect themselves. This level of focus might also have made officials responsible for disease prevention look with more care at these regions.
The authors note, in their discussion of the ethics of this trial:
“Information about the product and the intervention was provided by local TV, radio programs, newspapers and was also free available through information desks spread over the IR.”
So there was a comprehensive publicity campaign as well. Again, I would argue that putting this disease at the forefront of people’s minds is likely to increase compliance with personal protective measures.
In another way, it could also sound implausible that given the epidemic since the beginning of 2007 in IR and the health system working over years, that the disease was not at the forefront of the people and that the conventional measurements were not also applied with stronger emphasis. However no effect was observed in controlling the disease before week 46 2007.
Moreover, the impacts of natural disasters make even worse the effectiveness of conventional measures, especially in large population in endemic areas. The extent of the damage caused by the natural disasters was extremely large. One can get an idea from the official numbers published by the Cuban government after hurricanes Gustav (category 5), Ike (category 4) and Paloma (category 3): 500,000 homes destroyed, 4 billion dollars lost, interruption of national power supply systems, over 30,000 km of roads affected and lack of communication. You have to be in a place affected by such strong hurricanes to see the incredible damage they cause and to feel the reaction of the population within the disaster area. They need to recover what is possible and rebuilt everything with the lack of electricity and water for several days, large flooded areas and mountains of garbage. This makes it almost impossible to follow the recommendations for personal protection against Leptospirosis, even when having promotion through the media and doctors taking care of this. Particularly if because of the extent of damages and floods it is almost impossible to avoid contact with water potentially contaminated and the materials needed for personal protection (boots, gloves, etc) are not available or have been lost. In addition, after hurricanes, Leptospirosis is only one of the diseases that need special attention, though in IR it was the most dangerous one. Note that the risk groups at the time of the emergency comprise the full population of these provinces.
This would explain why the IR has ended up recovering to its 2004 position and is no better than Cuba as a whole.
Proper medicines muddy the waters
Next on my list is the use of conventional prevention measures. It’s made absolutely clear in the paper that vaccination and preventative use of Doxicycline, an antibiotic (“chemoprophylaxis” ) were routinely deployed in the Intervention Region.
Vaccination and chemoprophylaxis did not have any significant effect before the intervention.
Several factors could influence the incidence of Leptospirosis. They include the density and behavior of animal vectors, risk perception, virulence and infectivity of circulating strains, level of personal protection of the population, contamination of water, among others. The most effective measure to confront the disease is to improve or induce protective status on the exposed individuals.
The effect of vaccination, as stated in the paper, will be significant only over the long term (years) because of the time needed to induce immune response. In that way there is another detail missed, the vaccine vaxSPIRAL is only applicable to individuals over 15 years of age. So there is a significant cluster of the population unprotected even if the vaccine has 100% effectiveness and is applied with 100% coverage. In contrast, as reported in the paper, only 15,000 individuals were vaccinated in IR during the emergency, so it is impossible that vaccination could have caused the sharp decrease of incidence.
Chemoprophylaxis has a short temporal effect and thus not useful for long term prevention. Thus doxicycline treatment dose not explain why in IR no outbreaks were reported in 2008 in contrast to RC. However, even if that doxicycline treatment was the cause of the drop of the incidence in 2007, then how could it last for more than a year? In contrast, when the outbreak appears in a well controlled and limited population, the treatment could be highly effective as described, but this is not the case most of time.
“The individuals treated with either vaccination or chemoprophylaxis in the IR amounted to about 3% of the population. Individuals within risk groups were vaccinated when identified with two intramuscular doses (6 – 8 weeks apart) of vaxSpiral following manufacture’s instructions […] Chemoprophylaxis was applied mainly for focal treatment and outbreak control to high-risk groups when identified and consisted of a weekly oral dose of Doxicycline 100 mg.”
Although the reported percentage is small, it nevertheless means that 72,143 high-risk individuals in the IR were treated conventionally. It is quite possible that this could make a significant impact on the number of cases reported for the IR as a whole. Particularly as Doxicycline was used for, “focal treatment and outbreak control”. And the paper does demonstrate the impact Doxicycline can have on an outbreak, and the effect that an outbreak can have on the overall results.
Going further with this calculation. Let’s consider an incidence of 20 x 100 000 inhabitants in weeks 46-52 (hypothetical number extremely higher than the real situation in IR in 2007) and an effectiveness of 100% for both vaccination and chemoprophylaxis. From this 72 143 high risk individuals that received either vaccination or chemoprophylaxis 14 cases could be prevented. However the decrease observed on weeks 46-52 in IR was almost the double of this hypothetical number.
“In week 42  an outbreak of the disease was reported in a closed population of RC but fortunately was quickly controlled by chemoprophylaxis.”
During this week 116 cases were reported in the the rest of Cuba; by the following week there were around ten. The authors do not state how large this “closed population” was, but the term tends to imply that it’s relatively small. They should really clarify this.
As an aside, this directly contradicts one of the authors’ stated reasons why the homeopathic intervention was the cause. Discussing the fall in reported cases in the IR during the closing weeks of 2007 they state:
“This sharp decrease of incidence does not suggest an expected effect of vaccination or chemoprophylaxis considering the time needed to induce a protective immune response by vaccines and the short temporal protection of antibiotics.”
And yet this outbreak was knocked on the head in a week, using conventional measures.
The grand claim that this paper makes for the homeopathic intervention is that it caused the fall in weekly cases of Leptospirosis, from 38 to around four, over a three week period. However, there are no data or analysis in this paper that could rule out the effect being due to targeted intervention with conventional treatments in high-risk groups.
The authors do make a feeble attempt in their discussion section, as the third reason why the homeopathy must have caused the reductions, “coverage of conventional measurement [sic] of control including vaccination and chemoprophylaxis was similar in both regions since their application followed the current guidelines from MPHC”. As we have seen, the extent of infections in the IR was much greater, so applying the same protocols should have resulted in more conventional intervention in the IR.
How plausible is this as a possible explanation? If I have to choose between magic water and actual medicines as an explanation, the magic water loses. If someone expects me to give the magic water credit, they will need something better than the contents of this paper.
Regression to the mean?
Could there be other potential candidates for the dramatic reduction in cases seen in the IR? Observing that the regions making up the IR had a problem that appeared to start in 2005 and peaked in 2007 (see Figure 4) could it just simply be that this is a disease with variable incidence and that a strong peak will likely be followed by a fall? In other words, could the homeopaths have struck lucky, intervening at a peak and benefitting from regression to the mean?
Maybe not the strongest candidate for an explanation, but more plausible than magic water, certainly.
It is certainly the worse candidate.
More research required?
The paper ends, as most do, with a call for more research. Is this reasonable? No – effective prophylaxis measures are available. Cuba has an effective vaccine. There is absolutely nothing in the homeopathic ‘vaccine’. It would be unethical to divert resources to magic alcoholic water.
Five unassailable reasons?
According to the editorial by Roniger and Jacobs (2010), “Other possible explanations for these findings were explored by the authors but cannot explain the results.” Actually, the authors did the inverse: providing reasons to support their conclusion that the homeopathic intervention caused the reduction in Leptospirosis. I think that this says more about the authors pre-existing commitment to homeopathy than the lack of other, more plausible, explanations.
The five reasons the authors declare to demonstrate that they must be right are:
“… the risks for Leptospirosis infection are present all across the country.”
Yet, from 2005 – 2007 , the rest of Cuba did much better than the regions included in the IR without resorting to homeopathy. Even after the improvements in the IR, the level of Leptospirosis is no better than in the rest of Cuba.
Risks in IR and RC are not equal in the epidemic. In endemic areas risks are spread within any place with the condition for the bacteria to live. Several other factors should be present to have epidemics in high risk endemic areas as described.
“… more risk in IR than RC …”
And there was also more scope for improvement. In 2004 the IR had the same level of Leptospirosis as the rest of Cuba. The levels in the IR returned to their 2004 level, pretty similar to the rest of Cuba. Could there have been an improvement in the implementation of the conventional controls?
Normally is easier to control a disease during no epidemics periods.
“… coverage of conventional measurement [sic] of control including vaccination and chemoprophylaxis was similar in both regions since their application followed the current guidelines from MPHC.”
Yet more Leptospirosis in the IR should have driven more conventional intervention, if the same rules were used.
The higher amount of conventional interventions does not mean a commensurate higher effect, it has to consider the coverage, the risks for infection, the localization and composition of exposed individuals, and the effectiveness of applied measures.
“… the extent of vaccination and HP was very different in the IR. The HP intervention covered over 96% of the target population while the coverage of vaccination was limited to 0.6% because of the reduced stockpile of the vaccine vaxSpiral at that time.”
The use of the vaccine, if the authors are to be believed, was targeted at high risk groups. This is a bit of a red herring though. As the 2008 data from the rest of Cuba shows, an outbreak can be dealt with by targeted conventional intervention. Also an outbreak in a “closed community” can dominate the overall number of infections, even in the much larger RC region. Finally, the bulk of the initial decrease in cases in the IR seems to have occured, adjusting for the typical incubation period, when less than 40% of the population had been covered. Anyway, what’s the relevance of giving 2.3 million people ten drops of 30% alcohol in water?
“…. the reduction in the number of confirmed cases in IR occurred within 2 weeks but was sustained for the next 57 weeks. This sharp decrease of incidence does not suggest an expected effect of vaccination or chemoprophylaxis considering the time needed to induce a protective immune response by vaccines and the short temporal protection of antibiotics. In fact, because of the vaccination schedule of vaxSpiral, the immunization of newly exposed individuals was finished in a time frame several weeks after the effects observed at IR. The reduction of confirmed cases on IR was coincident with the achievement of 70% of coverage of HP treatment.”
Yet in 2008 chemoprophylaxis (antibiotics) controlled an outbreak in the rest of Cuba in about a week. This same strategy should have been followed in the IR if the MPHC guidelines were being followed. Also, as noted above, claiming coincidence between the reduction in cases and 70% homeoprophylaxis (HP) coverage appears to be ignoring both the incubation period and the fact that the bulk of the reduction occured two weeks after the start (week 47) of the campaign.
Conventional measures controlled an outbreak in a closed population in RC but were unable to control the increase observed at the end of the year, so is more likely to be a phenomena particularly to the closed population affected by the outbreak. In addition, conventional measures were not able to prevent this outbreak in RC.
Anyway in IR, even considering the incubation period of Leptospirosis, it is implausible to think that infections stop at week 46 in 2007 or the susceptible population was exhausted at week 45. This dramatic effect (in two weeks) observed in IR is reported for the first time in the history of Leptospirosis epidemics.
So, why couldn’t it have been the homeopathy? Because there were no inactivated bacteria in the homeopathic preparation. It would have been a miracle if someone has ingested a single dead bug. They had all been washed away.
If not the homeopathy, then what caused the change? No one can know for sure, but here are some massively more plausible explanations:
- Improved implementation of conventional controls (including change in behaviour of individuals).
- Conventional antibiotic intervention.
- Regression to the mean
Finally, the accompanying editorial seeks to support a delusional and dangerous agenda with this flawed paper.
A dangerous agenda
It’s perhaps not that surprising as the second author, Jennifer Jacobs has prior form on homeopathic interventions for dangerous diseases. She was involved in an ethically dubious, unsuccessful trial treating chronic diarrhea in children in Honduras (Jacobs et al, 2006). This was a follow-up to a similar trial conducted in Nicaragua, that achieved marginal statistical significance but was clinically irrelevant (Jacobs et al, 1994)‡ ‡.
Now they want to move on to other dangerous things …
“However, research could be hindered by perceived competition with conventional vaccines. Therefore other common infections with significant morbidity and/or mortality lacking effective and/or feasible conventional treatment, such as dengue fever, malaria, and antibiotic-resistant bacterial infections should be targeted first. In spite of our lack of theoretical understanding of homeoprophylaxis, ethical approval of future studies will be facilitated by this impressive study by our Cuban colleagues.”
This is a most dangerous proposal. Dengue fever may not have a specific treatment, but a keen focus on vector control is vital and should not be diluted by homeopathic distractions. Homeopathic delusions already exist about preventing and treating this disease, it is dangerous to indulge them further.
However, still vector control is far from being an effective measure to control Dengue particularly in developing countries and the tropics, otherwise the disease would not be considered by WHO as the next highly dangerous epidemic of the planet. In addition, if from this paper on Leptospirosis the homeopathic intervention is supposed to have a positive impact on the application and effectiveness of conventional measures that resulted in a better control of the disease, then a homeopathic intervention in Dengue would have similar effect.
Effective treatments and control methods do exist for malaria (WHO, 2010). The editorial is not telling the truth. Again, vector control and personal protection are vital. There is no possible role, or ethical justification, for homeopathy in protecting people from this disease. Delusional homeopaths are already dabbling in Africa, putting lives at risk. Treating antibiotic resistant infections with magic water are unconscionable. There is no reason to suppose that homeopathy can be of any benefit. Using it for a life-threatening condition is a very dangerous proposal.
The idea that this trial could provide an ethical justification for this agenda is unfathomable. It is an uncontrolled, un-randomized, poorly-reported, piece of homeopathic propaganda published in a pseudojournal.
As far as can be seen in the conclusions of the article there is no danger or propaganda in it:
“The homeoprophylactic intervention was strongly associated with a dramatic reduction of the disease incidence resulting in complete control of the epidemic. The results support the use of homeopathic prophylactic formulations as a feasible strategy to help control epidemic situations. Integrated approaches should be designed according to regional conditions and epidemic characteristics. Scientific rigour and responsibility should direct further research and application of HP”.
It ignores plausible explanations in favour of the fantastical. It’s a cracking example of Cargo Cult science.
Some Useful Sources of Information
I try to make sure that what I write is both accurate and fair. If you think that I have got anything wrong please let me know. If you are right I will happily change what I have written.
This is not medical advice. If you need that see a properly qualified and registered doctor.
*Some scant descriptions have been passed around the homeopathic community. The Quackometer ” Hasta el Absurdo Siempre!” (Friday 9th January 2009) responded to some of these reports.
**NHS Choices summarises the incubation period as, “seven to 14 days after the initial contact with infection, although they can develop in as little as three days, or as long as 30 days.”
***See here for Serological classification and grouping.
†For a reasonably complete summary of the criticisms that were levelled at most of the papers in this ‘special’ issue see my round up, which includes the authors’ responses to the criticisms made, or visit the Journal Club which ran at Bad Science – where all the papers are available in full. For a good example of the homeopathic apologists failing to engage with any sunstntial criticisms, see Kerr et al (2008) and the reply made by Rao (2008).
‡ ‡ See also these articles by Wallace Sampson on the Science-Based Medicine blog: Homeocracy, Homeocracy II, Homeocracy 3, Homeocracy IV , which give a comprehensive analysis of Jacobs’ work in this area.
My thanks to gimpy for sending the papers my way and darkside, at the Bad Science Forum for a couple of keen observations.
Guidelines for the treatment of malaria. World Health Organization, 20, avenue Appia, 1211 Geneva 27, Switzerland: WHO Press; 2010. Available from: http://www.who.int/malaria/publications/atoz/9789241547925/en/index.html.
leptospirosis. (2010). In Encyclopædia Britannica. Retrieved August 05, 2010, from Encyclopædia Britannica Online: http://www.britannica.com/EBchecked/topic/336973/leptospirosis
Beauvais F. Memory of water and blinding. Homeopathy. 2008 January;97(1):41–42. Available from: http://dx.doi.org/10.1016/j.homp.2007.10.001.
Bracho G, Varela E, Fernández R, Ordaz B, Marzoa N, Menéndez J, et al. Large-scale application of highly-diluted bacteria for Leptospirosis epidemic control. Homeopathy. 2010 July;99(3). Available from: http://dx.doi.org/10.1016/j.homp.2010.05.009
Elia V, Napoli E, Germano R. The ’Memory of Water’: an almost deciphered enigma. Dissipative structures in extremely dilute aqueous solutions. Homeopathy. 2007 July;96(3):163–169. Available from: http://dx.doi.org/10.1016/j.homp.2007.05.007
Jacobs J, Gloyd SS, Gale JL, Jimenez LM, Crothers D. Treatment of Acute Childhood Diarrhea With Homeopathic Medicine: A Randomized Clinical Trial in Nicaragua. Pediatrics. 1994 May;93(5):719–725. Available from: http://pediatrics.aappublications.org/cgi/content/abstract/93/5/719.
Jacobs J, Guthrie BL, Montes GA, Jacobs LE, Mickey-Colman N, Wilson AR, et al. Homeopathic Combination Remedy in the Treatment of Acute Childhood Diarrhea in Honduras. The Journal of Alternative and Complementary Medicine. 2006 October;12(8):723–732. Available from: http://dx.doi.org/10.1089/acm.2006.12.723.
Kerr M, Magrath J, Wilson P, Hebbern C. Comment on “The defining role of structure (including epitaxy) in the plausibility of homeopathy”. Homeopathy : the journal of the Faculty of Homeopathy. 2008 January;97(1). Available from: http://dx.doi.org/10.1016/j.homp.2007.10.004.
Lüdtke R, Rutten AL. The conclusions on the effectiveness of homeopathy highly depend on the set of analyzed trials. Journal of Clinical Epidemiology. 2008 December;61(12):1197–1204. Available from: http://dx.doi.org/10.1016/j.jclinepi.2008.06.015.
Milgrom L. Conspicuous by its absence: the Memory of Water, macro-entanglement, and the possibility of homeopathy. Homeopathy. 2007 July;96(3):209–219. Available from: http://dx.doi.org/10.1016/j.homp.2007.05.002.
Rao M, Roy R, Bell I, Hoover R. The defining role of structure (including epitaxy) in the plausibility of homeopathy. Homeopathy. 2007 July;96(3):175–182. Available from: http://dx.doi.org/10.1016/j.homp.2007.03.009.
Rao M. Authors’ reply to Kerr et al. Homeopathy. 2008 January;97(1):45–46. Available from: http://dx.doi.org/10.1016/j.homp.2007.11.011.
Rey L. Can low-temperature thermoluminescence cast light on the nature of ultra-high dilutions? Homeopathy. 2007 July;96(3):170–174. Available from: http://dx.doi.org/10.1016/j.homp.2007.05.004.
Roniger H, Jacobs J. Prophylaxis against Leptospirosis using a nosode: Can this large cohort study serve as a model for future replications? Homeopathy. 2010 July;99(3):153–155. Available from: http://dx.doi.org/10.1016/j.homp.2010.06.004
Rutten AL, Stolper CF. The 2005 meta-analysis of homeopathy: the importance of post-publication data. Homeopathy. 2008 October;97(4):169–177. Available from: http://dx.doi.org/10.1016/j.homp.2008.09.008.
Rutten ALB, Stolper CF. Reply to Wilson. Homeopathy. 2009 April;98(2):129. Available from: http://dx.doi.org/10.1016/j.homp.2009.01.002.
Shang A, Huwiler-Müntener K, Nartey L, Jüni P, Dörig S, Sterne JA, et al. Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy. Lancet. 2005 September;366(9487):726–732. Available from: http://dx.doi.org/10.1016/S0140-6736(05)67177-2/S0140-6736(05)67177-2
Shang A, Jüni P, Sterne JAC, Huwiler-Müntener K, Egger M. Are the clinical effects of homoeopathy placebo effects? – Authors’ reply. The Lancet. 2006 January;366(9503):2083–2085. Available from: http://dx.doi.org/10.1016/S0140-6736(05)67881-6.
Vybiral B, Voracek P. Long term structural effects in water: autothixotropy of water and its hysteresis. Homeopathy. 2007 July;96(3):183–188. Available from: http://dx.doi.org/10.1016/j.homp.2007.03.007.
Weingartner O. The nature of the active ingredient in ultramolecular dilutions. Homeopathy. 2007 July;96(3):220–226. Available from: http://dx.doi.org/10.1016/j.homp.2007.05.005.
Wilson P. Analysis of a re-analysis of a meta-analysis: in defence of Shang et al. Homeopathy. 2009 April;98(2):127–128. Available from: http://dx.doi.org/10.1016/j.homp.2008.12.003.