Science Current Events | Science News |

The importance of (experimental) design

February 25, 2014

One of the hottest debates in evolutionary biology concerns the origin of behaviour: is it genetically encoded or do animals and birds copy their parents or other individuals? A classic experiment published in 2000 seemed to provide overwhelming evidence that a particular behavioural choice (whether individuals of a species of swallow breed in a small colony or a large one) is largely genetically determined. Together with colleagues in France, Richard Wagner of the University of Veterinary Medicine, Vienna has re-examined the data and shown that the findings could be explained by random choice. The design of the original experiment - which represents a blueprint for a vast range of studies of heritability of behaviour - contains two pitfalls that combine to undermine the conclusions. The results are published in the journal Scientific Reports.

There are clear advantages to living in cities: safety, ready availability of infrastructure, plenty of company etc. Nevertheless, a large number of people eschew them for the benefits of country life, such as clean air and lots of space. Many species of animals, and particularly birds, face the same choice between living in large groups or remaining in smaller ones, thereby avoiding disadvantages of larger colonies such as the increased risk of disease and increased aggression from neighbours. What causes different individuals of a particular species to take the decisions they do?

It's all in the genes

One possible explanation is that animals and birds might be genetically influenced to breed in smaller or larger colonies. This idea gained widespread acceptance in 2000, when Charles and Mary Brown reported the results of a field experiment of unprecedented scope. The Browns worked on Cliff Swallows, a species of American swallow that breeds in colonies of variable size. Some individuals appear to be more resistant to nest parasites so can breed in larger, more parasite-infested colonies where they can achieve higher foraging efficiencies by following their many neighbours to the insect swarms on which they feed. The scientists cross-fostered an astounding 2,000 nestlings from nests in small colonies to nests in large colonies and vice versa and recaptured over 700 of the birds when they returned to breed in the following years. Analysis of the data showed convincingly that swallows hatched in larger colonies (but reared in smaller ones) chose to breed in large colonies, whereas the reverse was true for swallows hatched in small colonies. In other words, the birds' choice is determined by genetic factors.

Or is it?

Appealing though the conclusion might be, something does not sound quite right. How would a genetic component of the choice of group size be selected and maintained throughout evolution? Étienne Danchin of the University of Toulouse and Richard Wagner of the Vetmeduni's Konrad Lorenz Institute of Ethology wondered whether the results might not be influenced too much by the experimental design. Together with Éric Wajnberg, a modelling specialist at the INRA in Sophia Antipolis, they simulated the data in the original paper and found that the results could be generated by chance.

The problem stems largely from the fallacy known as "regression to the mean" (RTM), which was first identified by Sir Francis Galton, a cousin of Charles Darwin, in the 19th century. It results from the fact that uncommonly large or small measurements are generally followed by more normal measurements simply because normal values are by definition far more common than extreme ones. In the Browns' experiment, when individuals are fostered from small to large colonies they will on average recruit to colonies that are statistically smaller than their foster colony because these are closer to the mean colony size and vice versa.

Danchin and Wagner actually found a second problem with the original data analysis. The so-called "spatial fallacy" was discovered by Arie van Noorwijk in 1984 and occurs when the set of potential dispersal sites differs among individuals according to where they were born. Together the two fallacies can account for the experiment's results, even in the absence of an inherited component of the selection of colony size.

A question of design

The latest calculations are important for two reasons. First, they call the original conclusions into question and should thus encourage researchers to reconsider the role of genetic factors in certain behavioural decisions. Secondly, they sound a clear warning to all those working in behavioural science. As Wagner says, "The biggest shock is that the very experimental design that is widely used to control for all extraneous effects actually creates the RTM fallacy. Even the most carefully designed studies can suffer from the problem, making their results invalid." Fortunately the scientists have proposed a way to avoid both pitfalls. Their simulations showed that restricting comparisons to individuals with the same set of options (in the present case by fostering all offspring to a single colony) can give rise to data that are immune to the pitfalls.

The Nature Publishing Group has just published the paper "Avoiding pitfalls in estimating heritability with the common options approach" by Étienne Danchin, Éric Wajnberg and Richard H. Wagner in its online journal Scientific Reports, where it is freely available. (Scientific Reports Volume 4, doi:10.1038/srep03974)

University of Veterinary Medicine Vienna

Related Experimental Design Current Events and Experimental Design News Articles

Paper highlights best practices for omega-3 clinical trials with cardiovascular outcomes
A paper published this week in the journal Prostaglandins, Leukotrienes and Essential Fatty Acids sheds new light on recent neutral studies questioning the benefits of omega-3s for heart health.

Seeing where energy goes may bring scientists closer to realizing nuclear fusion
An international team of researchers has taken a step toward achieving controlled nuclear fusion--a process that powers the Sun and other stars, and has the potential to supply the world with limitless, clean energy.

Brief cognitive-behavioral therapy helps those with problematic caffeine use
Engaging in brief, cognitive-behavioral therapy is an effective treatment for helping people with problematic caffeine use lower their caffeine consumption, according to a new study coauthored by Laura M. Juliano, Ph.D., a professor of psychology at American University.

Side stream emissions from 'heated tobacco' products similar to secondhand cigarette smoke
A new study published in the Journal of Environmental Analytical Chemistry today found that, despite claims made by some manufacturers, next generation 'heated' tobacco devices produce side-stream emissions similar to secondhand cigarette smoke.

Novel competitors affect species' responses to climate change
Climate change is proceeding unchecked and average temperatures are rising, forcing many wild animals and plants to move into new habitats. As species come together in new combinations, climate change reshuffles the deck of ecological interactions, with unexplored consequences.

Sandia's Z machine helps solve Saturn's 2-billion-year age problem
Planets tend to cool as they get older, but Saturn is hotter than astrophysicists say it should be without some additional energy source.

Unravelling the mysteries of carbonic acid
Blink your eyes and it's long gone. Carbonic acid exists for only a tiny fraction of a second when carbon dioxide gas dissolves in water before changing into a mix of protons and bicarbonate anions.

New release of Glioblastoma Atlas sheds light on deadly disease
Robust new data added to the Ivy Glioblastoma Atlas Project (Ivy GAP) changes the scope and impact of this publicly available resource for researchers and clinicians searching for treatments for this most deadly and aggressive of brain cancers, glioblastoma multiforme.

Consumption rises with automated bill payment
The adage "out of sight, out of mind" applies to electricity use, according to new research from Duke University's Sanford School of Public Policy.

Scientists unknowingly tweak experiments: ANU media release
A new study has found some scientists are unknowingly tweaking experiments and analysis methods to increase their chances of getting results that are easily published.
More Experimental Design Current Events and Experimental Design News Articles

Experimental Design: Procedures for the Behavioral Sciences

Experimental Design: Procedures for the Behavioral Sciences
by Roger E. Kirk (Author)

Experimental Design: Procedures for Behavioral Sciences, Fourth Edition is a classic text with a reputuation for accessibility and readability. It has been revised and updated to make learning design concepts even easier. Roger E. Kirk shows how three simple experimental designs can be combined to form a variety of complex designs. He provides diagrams illustrating how subjects are assigned to treatments and treatment combinations. New terms are emphasized in boldface type, there are summaries of the advantages and disadvantages of each design, and real-life examples show how the designs are used.

Design and Analysis of Experiments

Design and Analysis of Experiments
by Douglas C. Montgomery (Author)

The eighth edition of this best selling text continues to help senior and graduate students in engineering, business, and statistics-as well as working practitioners-to design and analyze experiments for improving the quality, efficiency and performance of working systems. The eighth edition of Design and Analysis of Experiments maintains its comprehensive coverage by including: new examples, exercises, and problems (including in the areas of biochemistry and biotechnology); new topics and problems in the area of response surface; new topics in nested and split-plot design; and the residual maximum likelihood method is now emphasized throughout the book. Continuing to place a strong focus on the use of the computer, this edition includes software examples taken from the four most dominant...

Experimental Design for Biologists, Second Edition

Experimental Design for Biologists, Second Edition
by David J. Glass (Author)

The effective design and analysis of experiments in biology are critical to success, yet graduate students in biological and medical sciences typically receive very little formal training in these steps. With feedback from readers of the first edition, colleagues, and students taking the very popular experimental design courses taught by the author, this second edition of Experimental Design for Biologists retains the engaging writing style while organizing the book around the four elements of experimental design: the framework, the system, the experiment, and the model. The approach has been tested in the classroom, where the author has taught numerous graduate students, MD/PhD students, and postdoctoral fellows. The goal of every scientist is to discover something new and with the aid...

Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 4th Edition

Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 4th Edition
by John W. Creswell (Author)

The eagerly anticipated Fourth Edition of the title that pioneered the comparison of qualitative, quantitative, and mixed methods research design is here! For all three approaches, Creswell includes a preliminary consideration of philosophical assumptions, a review of the literature, an assessment of the use of theory in research approaches, and refl ections about the importance of writing and ethics in scholarly inquiry. He also presents the key elements of the research process, giving specifi c attention to each approach. The Fourth Edition includes extensively revised mixed methods coverage, increased coverage of ethical issues in research, and an expanded emphasis on worldview perspectives.

Experimental and Quasi-Experimental Designs for Generalized Causal Inference

Experimental and Quasi-Experimental Designs for Generalized Causal Inference
by William R. Shadish (Author), Thomas D. Cook (Author), Donald T. Campbell (Author)

This long awaited successor of the original Cook/Campbell Quasi-Experimentation: Design and Analysis Issues for Field Settings represents updates in the field over the last two decades. The book covers four major topics in field experimentation:

Testing 1 - 2 - 3: Experimental Design with Applications in Marketing and Service Operations (Stanford Business Books)

Testing 1 - 2 - 3: Experimental Design with Applications in Marketing and Service Operations (Stanford Business Books)
by Johannes Ledolter (Author), Arthur Swersey (Author)

This book is about the power of statistical experiments. In the past, books on experimental design focused almost entirely on manufacturing problems. In contrast, although this book is relevant to manufacturing and includes useful examples in that area, its emphasis is on applications to marketing and service operations. The authors provide a fresh and practical treatment of the key topics in designing and analyzing experiments. Testing in the business world is commonplace, and the usual approach is to change one factor at a time while holding other factors constant. This approach may seem logical and appealing, but, as the authors explain, it is highly inefficient, and may lead to wrong conclusions. The better method, the authors propose, is to test all factors simultaneously. Doing so...

Experimental Design for the Life Sciences

Experimental Design for the Life Sciences
by Graeme Ruxton (Author), Nick Colegrave (Author)

At the core of good research lies the careful design of experiments. However, all too often a successful design comes only after a painful trial-and-error process that wastes valuable time and resources.

Now in a new edition, Experimental Design for the Life Sciences teaches today's students how to effectively design experiments, equipping them with the skills they need to become tomorrow's top researchers. Employing a refreshingly approachable and articulate style, the text explains the essential elements of experimental design in clear, practical terms, so that students can grasp and apply even the most challenging concepts, including power analysis and pseudoreplication. In addition, it emphasizes the interrelatedness of experimental design, statistics, and ethical...

Experimental Designs, 2nd Edition

Experimental Designs, 2nd Edition
by William G. Cochran (Author), Gertrude M. Cox (Author)

The past six years have seen a substantial increase in the attention paid by research workers to the principles of experimental design. The Second Edition of brings this handbook up to date, while retaining the basic framework that made it so popular. Describes the most useful of the designs that have been developed with accompanying plans and an account of the experimental situations for which each design is most suitable. Examples come from diverse fields of research, with an emphasis on biology and agriculture, two of the authors' specialties. New chapters have been added: one discusses the fractional replication of experiments. A second is concerned with experiments of the factorial type that present new methods and designs in which the factors represent quantitative variables...

Experimental Design and Analysis (Quantitative Applications in the Social Sciences)

Experimental Design and Analysis (Quantitative Applications in the Social Sciences)
by Steven R. Brown (Author), Lawrence E. Melamed (Author)

This volume introduces the reader to one of the most fundamental topics in social science statistics: experimental design. The authors clearly show how to select an experimental design based on the number of independent variables and the number of subjects. Other topics addressed include variability, hypothesis testing, how A.N.O.V.A. can be extended to the multi-group situation, the logic of the t test and completely randomized designs.

Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building

Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building
by George E.P. Box (Author), William G. Hunter (Author), J. Stuart Hunter (Author)

Introduces the philosophy of experimentation and the part that statistics play in experimentation. Emphasizes the need to develop a capability for ``statistical thinking'' by using examples drawn from actual case studies.

© 2017