Volume 1 Number 4 - June 2004

Correlation Research Home  |  Consulting Services  |  About the Company  |  Contact Form

Taking a Sample

Introduction

News in Brief

Feature Article

How to Contact Us

Statistical samples can be remarkably useful in a variety of situations in which it is impractical to obtain complete data on a population of interest. In this issue, we focus on two situations in which both the power and the potential limitations of statistical samples are important. The first article was written jointly with David Schwartz, Ph.D., one of the founding partners of Innovative Science Solutions, LLC. The second article considers the use of statistical sampling in a very different and timely context: political polling.

In his extensive work defending pharmaceutical products in liability cases involving complex scientific issues, Dr. Schwartz often advises attorneys who confront adverse drug reaction reporting data. We discuss how to think about situations in which such data are used to allege that a particular drug causes a specific adverse event, either in a particular plaintiff or as a general proposition. The second article explains the statistical "magic" that can sometimes allow a relatively small sample of potential voters to tell us who the next U.S. President is likely to be.

Interpreting ADRs Correctly

Case reports relating to possible adverse drug reactions (ADRs) play a valuable role in the process of scientific discovery. In the context of medical research, hypotheses about causal relationships are often generated as a result of case reports. A physician may notice that a patient exhibited an adverse health effect after taking a particular pharmaceutical product. Perhaps that physician (or other physicians)have witnessed a series of such apparent reactions over several years. Such a pattern of observations may then lead to the design of a more rigorous epidemiological investigation to evaluate whether there is a true association between the drug and the adverse event.

A problem can occur, however, when a series of ADRs is adduced as evidence of a causal relationship between the drug and the adverse event. Anecdotal, subjective, and often unreliable, ADRs alone cannot prove cause and effect between a drug and an adverse event. They are simply a signal that a drug may present a safety hazard. In product liability litigation, ADRs can effectively sting the defense when presented out of context. In recent years, for instance, plaintiff experts have trumpeted a study of 140 ADRs reported by users of ephedra-containing products in support of the allegation that ephedra causes specific adverse cardiovascular events.

There are many reasons why reports of ADRs do not constitute scientific proof. One important limitation of ADRs results from the fact that apparent associations can arise by chance. An article entitled "Cluster or Coincidence" in a previous issue of this newsletter (December, 2003) described how highly "suspicious" patterns can occur when observing many essentially random events. Being able to reproduce the patterns in a study designed to test for them is the hallmark of the scientific method and is a fundamental tenet underlying the need for randomized controlled drug trials.

Another potential problem is the "denominator problem." It is not the number of ADRs that counts, but the frequency in a relevant population. Does the ADR occur once in a thousand doses, or once in a billion? Equally important is the issue of comparison. Is the frequency greater than expected in general? There are many other potential hazards in relying on ADRs as evidence. Putting ADRs in a broader biomedical and biostatistical context is critical in order to properly interpret their significance.

Statistical Wizardry and Presidential Politics

As fans of the Harry Potter books know, the Sorting Hat is an infallible tool for assigning students to the four houses of the Hogwarts School of Witchraft & Wizardry. If only we "muggles" possessed a similarly accurate methodology for selecting our political leaders. While our methods of electing the President and other officials may be more prosaic, there is an almost magical element in one aspect of the process. I refer of course to that triumph of the statistical art known as political polling.

Is it not remarkable that political polls, if properly done, are so incredibly reliable? Consider that a sample of only 400 eligible voters, taken from the entire United States, can estimate the percentage of Bush or Kerry voters with only a 5% margin of error? With a mere 1600 respondents, this margin is cut in half. While not quite up to Sorting Hat standards, this is not too shabby for mere science.

Now for the fine print. Just like the magical incantations performed by Harry and his friends, the conduct of political polls must also follow certain rules in order to avoid unpleasant surprises. Remember the debacle that attended the widely predicted defeat of another famous Harry on election eve in 1948. Fortunately, current day polling has come a long way since those early days.

So, what are the main principles that must be followed by modern statistical wizards? In the first place, the sample must be properly drawn as a random sample of eligible voters. Then, interviewers must screen the respondents to include only individuals likely to vote. The questions about voting preferences must also be framed in such a way that bias is minimized.

Finally, the complexities introduced by the U.S. Electoral College must be taken into account. While not affecting our estimate of the nationwide popular vote, the Electoral College system makes prediction of a winner considerably more complicated. In effect, we must conduct 50 separate polls in order to arrive at the answer.

Of course we will still be left with one little problem. What you measure will of course be only a snapshot in time. So, what is true today may not be true on Election Day. Now, if only we could find one of those time-travel devices to take a little peek into the future..........

Contact Correlation Research at:

Herbert I. Weisberg, Ph.D.
61 Pheasant landing Road
Needham, MA 02192-1000

Phone (781) 455-6850

Fax (781) 444-9563

email: hweisberg@correlation.com
Please let us know what you think about our newsletter. Thanks.

Back to top