|
Visit the MDMA Research Page
MDMA and Memory Impairment: Proven or Not?
Critique of the article:
K. Thomas Nelson, Ph.D.
This recent study claimed to have discovered evidence suggesting that MDMA use above certain amounts is linked to memory problems. Much has been said critically about this study on the Internet (see erowid.org). On the one hand, the authors do seem to be implementing the study with state of the art neuropsychological measures, as well as with physiological measures which appear appropriate (CSF 5-HIAA levels to measure 5-HT depletion and Urine Drug Screens to establish current abstinence). Nevertheless, there are serious statistical and methodological weaknesses which undermine the credibility of the conclusions, which are themselves overstated. In addition, the procedures are not explained clearly enough for the study to be replicated, and the statistics used to report the main effects and interactions are confusing and probably inappropriate. [Editors's Note: For more information about this statistical analysis, please contact the author directly.] Furthermore, the magnitude of the purported MDMA-related deficits is not clearly stated in the paper as published, making any interpretation of the clinical significance of those deficits impossible. An appraisal of the clinical implications of the study became possible once I had been supplied with Table 6: Means and Standard Deviations for the Memory Tests, which was not included in the article as published in the Journal of Neurology. I was provided with Table 6 by Rick Doblin who, in turn, had been supplied with that table by principal investigator, Dr. Karen Bolla. I was motivated to write this critique because I think that there is a lot of research being published in mainstream journals which makes statistically unwarranted conclusions about the dangerousness of MDMA and other entheogens. Because the FDA relies in part on such studies, obtaining permission for research into the possible benefits and applications of entheogens has been and continues to be more difficult than the existing data seem to warrant. Worse yet, the popular media take such unsound research and publicize it widely to an audience which has very little knowledge on how to evaluate the soundness of the studies. This leads to the use of "scare tactics," which tends eventually to undermine the credibility of drug abuse prevention efforts and increases the extent to which valid warnings are ignored by the public.
Presentation and Interpretation of Results To quote the authors: The authors are flatly admitting that there was no significant difference between the MDMA-User and Control subjects on any of the memory measures. In addition: "no significant associations were found between the Memory factors and other estimates of exposure, including duration of MDMA use or the cumulative lifetime use of MDMA." (p. 1534)
The results section would ordinarily have stopped right there, since the
main hypotheses have been tested and the results found to be non-significant. While
the interactions are of secondary interest, the statistical testing of memory factors
in Table 4 of the article and the statistical testing of variables (actual test
scores) comprising the memory factors is nothing more than what we used to call
"data-sifting" or "data-milking" when I was
in graduate school. Selected Percentile Equivalents from Scores Prepared in Table 6:
This is how well the subjects scored who were tested compared to the American adult population matched for age. Fiftieth percent is the midpoint, scores above 50% are better than average, scores below are below average. The "Normal" range is 16th-84th percentile. These percentile differences between the High Dose and Low Dose/Control groups really amount to practically nothing. When I am evaluating patients for disability or for forensic purposes, I ordinarily consider a score in the 10th percentile to represent "borderline" impairment of memory and scores at or below the 5th percentile to represent clinically significant impairment. I have encountered a majority of people with memory scores in the 15th to 40th percentile who are fully employed, sometimes affluent, and often owning their own homes. Control and MDMA User Groups The authors report no significant differences between MDMA and control subjects in overall memory scores and no significant associations between memory and total MDMA consumed or duration of use when comparing all 24 MDMA users with all 24 controls. However, as (unpublished) Table 6 reveals, in order to demonstrate memory impairment, the subjects had to be divided into "Control Group" which included all 24 subjects and the 13 low dose MDMA subjects (N=37) and "High Dose" N=11. The low dose MDMA users were defined in the study as persons who consumed less than an average of 400 milligrams per month for the entire period of time they had been using MDMA, with a cumulative lifetime minimum of 25 doses. Neither the number of MDMA subjects in the high and low dose groups, nor the inclusion of the 13 low dose MDMA subjects into a larger control group, were mentioned in the paper. This is significant information that should not have been omitted. Lumping those 13 Low Dose MDMA subjects into the Control group clearly implies that the authors considered low dose MDMA users and people who had never used MDMA to be equivalent neuropsychologically. Such a position would support the notion that low dose MDMA use is relatively harmless and that research involving the administration of MDMA to human subjects is of relatively low risk from the perspective of memory impairment. The authors took great pains to assure that the MDMA users and Control subjects were matched for age, gender, and verbal intelligence. However they had vastly different histories of prior drug use in general. For example, the MDMA User group had five times as many cocaine-experienced subjects (20 as opposed to four in the Control group) and five times as many people with prior amphetamine experiences (10 as opposed to two). There are also twice as many solvent abusers in the MDMA User group (four as opposed to two) and four times as many past users of PCP (four vs. one). The groups are clearly not equivalent in terms of non-MDMA "street" drug use as reported by Bolla et al. I also suspect that the two groups may not be equivalent in their frequency of prior drug and alcohol use, but the authors provide no data on such frequencies. The effects reported later (especially since they are small) could easily have resulted from the other drugs previously used by the MDMA group. Further, while the prior drug use patterns reported in Table 2 (p. 1534) compare the 24 Subject "Control" and "MDMA User" groups, the effect sizes reported in (unpublished) Table 6 were based upon a comparison between the 37 subject low MDMA dose group and the 11 subject high dose group. One wonders how many of the four past inhalant users and how many of the four past PCP users were among the small High Dose MDMA group. Since inhalants and PCP are among the most indisputably memory impairing substances, any disproportionate representation of these subjects in the 11-subject High Dose group could in and of itself have accounted for the (relatively modest) effects reported. The 11- subject High Dose group may also have had significantly higher rates of other prior drug use than the 24 subject MDMA User group. I am of the opinion that the multiple contamination of the Control and MDMA User groups is the most disturbing flaw in this study. Reference Tom is a consulting and forensic psychologist who is gradually phasing out an unsatisfying private practice and leaving Arizona for California. His main interests are schizophrenia and cognitive flexibility. He is presently looking for a clinical/research position. He had the tremendous good fortune to have been taught research design by Professor Glenn E. Tagatz, Ph.D. of Marquette University. See also MDMA Memory Studies Overview at Erowid
|
 |