© 2000

At one time, published scientific studies on training and nutrition were for researchers and health care practitioners to interpret. Now, however, the consumer is constantly hit with a barrage of scientific studies. The evening news, radio and television commercials, and magazines are full of references to scientific studies. Some supplement manufacturers have even cited studies right on the jars of their products! Today’s consumer is well-served by gaining at least a cursory understanding of scientific studies and the jargon that accompanies them.

Strictly speaking in scientific lingo, "research" is not what students do when they spend time pouring over books in the library in preparation for writing a paper for a school assignment. The process of combing through books is called reading. In contrast, research is the tedious process of attempting to solve problems and answer unanswered questions. Research is carried out through scientific studies.

The scientific method is a way of gathering knowledge about our world. The scientific method consists of the following steps: asking a question, hypothesizing (developing a tentative explanation), designing a study to test the hypothesis, manipulating only one variable (influential characteristic) at a time, controlling as many confounding (interfering, nuisance) variables as possible, statistically analyzing the results, and submitting the study to a peer-reviewed science journal for publication. Some of these concepts and terms are further explained below.

Another very important aspect of the scientific method is reproducibility. All too frequently, a new product or technique is aggressively marketed with only one scientific study to support its use. Because there is always a slight probability that a study’s results were due to nothing more than chance, a single study does not constitute conclusive evidence that a supplement, training method, or whatever is or is not effective for its intended purpose. There is also the possibility that the original researcher was biased and could have unconsciously or consciously influenced the outcome of the study. Therefore, most reputable scientists prefer to evaluate the sum results of several well-controlled studies. In addition, it is best if the studies are conducted by different researchers. For example, it would be curious if a particular inventor demonstrated twice that his miraculous product, Lard-B-Gone (not the name of a real supplement--I hope), could decrease body fat, yet many other competent researchers subsequently failed time and time again to reproduce the findings.

Have you ever seen a scientific journal? Perhaps you were exposed to scientific journals at a college library. Or, if you think about your physician’s, dentist’s, physical therapist’s, or other health-care professional’s office, you may have seen some drab magazines floating around the reception area. If you’ve ever picked up a magazine in such an environment only to find its pages filled with primarily black and white text, graphs, and tables all expressed in a seemingly foreign language, chances are that you laid eyes on a scientific journal. Scientific journals, also called academic journals, are usually painfully boring and/or incomprehensible to people outside the journal’s topic area of publication. Honestly, the intended audience of many such journals may be unable to readily grasp the meaning of all of the articles.

In addition to publishing the findings of individual studies, some journals also publish summaries or critical reviews of several prepublished studies to determine what the majority of studies regarding a particular topic indicate. Journals may also publish substantiated opinions, or perspectives, as well as letters to the editor and textbook reviews. Abstracts, which are brief summaries (often one paragraph) of research findings that have been presented at a conference, are also published in peer-reviewed journals.

Scientific journals often have cumbersome or peculiar titles. Well-known journals that you’ve probably heard of include the Journal of the American Medical Association and the New England Journal of Medicine. Titles more specific to exercise and nutrition include Journal of Applied Physiology, Medicine and Science in Sports and Exercise, Journal of Exercise Physiology Online, and International Journal of Sport Nutrition and Exercise Metabolism.

When properly cited, a journal article reference should provide you with at least the following information: author name(s), article title, journal name, year of publication, volume number, and inclusive page numbers. Be suspicious if insufficient information is provided in the reference to readily look up the study; it may indicate that the claimant may not really want you to find the study, granted that it does indeed exist. Oftentimes, the titles of the journals are abbreviated, as in one of the following widely accepted formats:

Poortmans JR, Francaux M. Long-term oral creatine supplementation does not impair renal function in healthy athletes. Med Sci Sports Exerc. 1999;31:1108-1110.

Antonio, J. & Sanders, M. (1999). Effects of self-administered androstenedione on a young male body builder: A single-subject study. Current Therapeutic Research, 60, 486-491.

The term "peer-reviewed," sometimes called "refereed," indicates that a manuscript submitted for publication in a journal will first be scrutinized by other experts in the field prior to its acceptance. Training and nutrition studies are published in peer-reviewed scientific journals. On the other hand, popular books and magazines accept and publish material based on its novelty and potential appeal as opposed to scientific accuracy. The process of peer-review is certainly an imperfect system, but it is the only system that has been agreed upon in an effort to enhance the quality of published studies.

The overall purpose of blinded peer-review, in which reviewers don’t know the identity of the submitting authors, is not to determine whether the results themselves are correct, but rather whether the authors of the study followed generally accepted protocols and had considered alternative explanations for their findings. Reviewers also double-check the statistical methods to see if the appropriate model was applied to the problem. Peer-review is an ugly process and sometimes fails miserably as evidenced by some journals that are obviously biased in the studies published and the reckless freedom granted to certain authors to present their opinions as facts. So, it's far from perfect, but we can't let every nutty professor publish something conducted under questionable conditions!

Recall that abstracts are brief presentation summaries that are published in scientific journals. However, abstracts are not as rigorously reviewed as full journal articles. (By unfortunate coincidence, the brief summary that appears at the beginning of each journal article is also referred to as an "abstract.") In general, an abstract alone is not regarded as strong evidence of anything, but it is better than nothing and may be viewed as preliminary evidence until the study’s details are published in their entirety.

Let’s look at how an abstract may be misused. If an advertisement for Lard-B-Gone claimed the product was shown in a study to miraculously reduce body fat but that sole support was reported only in abstract form, then we could say that the product’s promoters may be guilty of being overly optimistic. Here’s why: a single study doesn’t prove anything. Furthermore, a single study published only as an abstract proves even less. Provided that marketers or popular authors have properly cited their references (which they often do not!), you can spot an abstract because the reference will say "abstract." Look at the following examples, presented in two different but equally acceptable formats:

Dominick, G., Quindry, J., Brittingham, K., Panton, J., Breuel, J., Earnest, C., Olson, M., & Broeder, C. (2000). The andro project: Androstenediol or androstenedione use on body composition in men [Abstract]. Medicine and Science in Sports and Exercise, 32 (Suppl.), S177.

Sugiura K, Takasaki M, Ohta A. Creatine supplementation increases the requirement of nutrients (ntrogen [sic] and minerals) in rats with spontaneous exercise [abstract]. Med Sci Sports Exerc. 2000;32(suppl):S136. Abstract 563.

Statistics is a branch of mathematics that describes measurements (data). You can think of it as a tool to evaluate the findings of a study. Commonly, when the findings of studies are reported, the results are referred to as "significant" or "non-significant." These terms, used extensively throughout scientific literature, refer to whether or not chance may have played a large role in the results of a study, as determined by certain statistical calculations. In the early 1900s, a bunch of mathematical bigwigs got together to determine what level of chance was acceptable when reporting the outcome of a study. As a result, the generally accepted level of significance is p equals less than .05, where "p" stands for "probability." A p value of less than .05 means that the probability of the outcome occurring by chance is less than 5%. Expressed another way, at least 95% of the time, the findings are truly the result of the variable that was manipulated.

Sometimes a piece of data will almost reach statistical significance, but not quite. For example, the percent body fat lost by using Lard-B-Gone versus a placebo (inactive substance) over the course of several weeks may show that Lard-B-Gone resulted in a 2% greater loss of fat, with a calculated p value of .052, or 5.2%. Although this is technically not statistically significant because it’s more than a 5% probability that the results were due to chance, some reporters of the findings may state that there was a "trend" showing those who used Lard-B-Gone lost a greater percent body fat than those who did not. The problem is that there is no widely accepted definition for trend in this context, and therefore, a trend is subject to the eye of the beholder. It goes without saying that some unscrupulous or uninformed marketers of a product may suggest a trend when the researchers of the product themselves would not.

Just because something was found to have a statistically significant effect does not necessarily mean that the effect is clinically significant, or meaningful. For example, if new and improved Lard-B-Gone was shown to significantly decrease the average person’s percent body fat by .25%, many users of the product would likely not find the hassle, cost, or any side effects to be justifiable for such a measly effect. You can think of clinical significance as assessing the cost-to-benefit ratio of an intervention. Do the product’s effects translate into real-world application? Is it practical to use this thing?

Exercise and nutrition research are important for the continued development of the fields of physique and performance enhancement. Research should be viewed critically, applied specifically, and incorporated slowly into daily life according to the growing preponderance of evidence. My hope is that this article has provided a bit of insight into how ideas are developed, tested, analyzed, and disseminated among scientists and a broader general audience. Perhaps this article will enhance your appreciation for and understanding of the language of science, which may help you better comprehend mention of scientific studies in a variety of settings, even in articles at this very web site.

Dr. Greg Bradley-Popovich holds dual master's degrees in Exercise Physiology and Human Nutrition from West Virginia University as well as a doctorate in Physical Therapy from Creighton University. He is the Director of Clinical Research at Northwest Spine Management, Rehabilitation, and Sports Conditioning in Portland, Oregon.

Back to Library Main Page