Fixed Effect Versus Random Effects Models in Meta-analysis: As Simple as It Gets

Fixed Effect Model

Imagine that the studies that we plan to pool in meta-analysis were conducted in largely the same manner in patients who were largely similar in sociodemographic and clinical characteristics. So, we can reasonably assume that the samples in the different studies represent the same clinical population.

In this situation, for the outcome of interest, there is a single effect in the population. This is called the true effect, and the values of the outcome obtained in the different studies are variations of this single true effect. Because different studies examined different samples, the variations are due to differences in the samples; that is, due to sampling error within studies.

Returning to the subject, the single true effect in the population is the fixed effect; note that the word “effect” is singular because there is only one population and hence only one effect. In this situation, we select a fixed-effect model to pool the studies in meta-analysis.

In summary, we use a fixed-effect model in meta-analysis when the studies to be pooled are largely similar in design, methods, and sample characteristics, and when we can reasonably assume that values of the outcome (effect) in these studies represent sample-related variations of a single true effect in the population.

Expressed otherwise, we use a fixed effect model when we believe that there is only a single true effect and that different findings in different studies represent variations of this single true effect.

Random Effects Model

Imagine that the studies that we plan to pool in meta-analysis were conducted in different ways and in patients who were dissimilar in important sociodemographic and clinical characteristics. That is, there is meaningful diversity across studies. There will therefore be sampling error between studies because different studies represent samples from different populations. Now, we can no longer assume that the outcomes (effects) identified in individual studies represent a single true effect.

As examples, consider studies conducted with high versus low doses of medication, in ordinary versus treatment-resistant samples, in pediatric versus adult versus geriatric subjects, and in patients with versus without medical, neuropsychiatric, or substance use disorder comorbidities. It is reasonable to expect that study outcomes (magnitude of effects) would vary with such study-related differences. So, the samples in these studies will not represent a single population, and the study outcomes will not represent a single true effect. Rather, different studies will represent different populations, each with its own true effect. In this situation, we select a random effects model to pool the studies in meta-analysis. Note that the word “effects” is plural because there is more than one population, each with its own true effect.

In summary, we use a random effects model in meta-analysis when the studies to be pooled are meaningfully dissimilar in their design, methods and/or samples, and when we can reasonably assume that values of the outcomes (effects) in these studies represent sample-related variations of true values in different populations. That is, there are many true values.

When Should the Model Be Chosen?

The choice between fixed effect and random effects models is best made a priori, and at the time that the systematic review and meta-analysis protocol is registered with PROSPERO. This is because it is often possible to predict in advance that, for example, existing studies are very likely to represent different populations and that, in consequence, there will be many true effects, necessitating the use of a random effects model. However, the choice can also be made after a superficial literature search is completed; that is, after identifying relevant studies and examining their design, methods, and sample characteristics. This allows the investigators to make a more reasoned choice.

The choice of model is based on conceptual assumptions, and not on eyeballing the results. So, for example, the choice of model should not be made after running the analyses and examining the magnitude of heterogeneity. This is because low or high heterogeneity can occur by chance, regardless of the conceptual assumptions. Heterogeneity can be falsely low or high for other reasons, too, based on the number of studies, their precision, and other factors.

Importantly, the choice of model should not be made after running the analyses and deciding that the results from one model align better with the study hypotheses than the results from the other model; this is research malpractice.

When to Prefer a Fixed Effect Model

A fixed effect model, to assign study weights, uses sampling error (variance) from a single source: within studies in a single population. A random effects model, to assign study weights, uses sampling error from two sources: within studies plus between studies. When the number of studies is small (e.g., fewer than five to seven studies), the error between studies is hard to estimate accurately, and so the results of a random effects model will be imprecise. It may then be better to select a fixed effect model. If a random effects model is nevertheless chosen, the results should be interpreted with caution. Results should anyway be interpreted with caution if the number of studies in a forest plot is small.

Sometimes, the investigators may want to generate results that are applicable only to the set of studies that were identified in the search. That is, the investigators do not want to generalize their findings to a wider population. In such a situation, they can opt for a fixed effect model, regardless of the number of studies being pooled and regardless of the conceptual assumptions about the distribution of the true effect(s). This approach is unusual because investigators typically want to generalize their results for clinical relevance.

Choice of Model and Impact on Forest Plot Study Weights

Meta-analysis does not mathematically average results across studies. Rather, it averages results after applying weights to the results of individual studies. In general, studies that have more precise results (i.e., studies with larger samples and narrower confidence intervals) receive larger weightage than studies with less precise results (i.e., studies with smaller samples and wider confidence intervals). This happens in fixed-effect meta-analysis, where the results of one or more large studies may dominate the pooled effect size.

In random effects meta-analysis, however, the assumption is that there are many true effects, and so variations in precision could be a genuine result of different population characteristics. So, studies are less penalized for apparent imprecision. The result is that small and large studies, or studies with imprecise and precise results, tend to be treated more equally. For this reason, it is bad in principle to use a random effects model when a fixed effect model is more appropriate. The converse is also true; it is bad in principle to use a fixed effect model when a random effects model is more appropriate.

Choice of Model and Effect on Results

From the previous section, it is clear that differences in study weightage will result in differences in the value of the pooled estimate in the forest plot. Of note, when heterogeneity is low, fixed-effect and random-effects models will yield similar results. When heterogeneity is zero, fixed-effect and random-effects models will yield the same results.

As explained earlier, there is only one source of sampling error in fixed effect models and two sources of sampling error in random effects models. That is, in fixed effect analysis, the variance is defined as the variance within studies; in random effects analysis, the variance is defined as the variance within studies plus the variance between studies. So, there is greater imprecision in random effects model results, as shown by wider confidence intervals. Consequently, it may be harder to obtain statistically significant results in random effects models.

Other Notes

The conceptual assumptions underlying the choice between fixed effect and random effects models are subjective and can be a matter of opinion. Therefore, even though investigators may prefer one model and report results using that model, it is customary, such as in supplementary materials, to present results using the other model, as well, so that readers who have contrary opinions about the choice of model can examine results in the model that they believe to be appropriate.

Convergence of results between random effects and fixed effect models may suggest robustness of the results and of the conclusions of the study.

A Realistic Comment

Although this article has focused on true effect(s) within and between populations to differentiate between the need for fixed effect versus random effects meta-analysis, it should be understood that differences in study design and methods can also be responsible for differences in true effects, regardless of the population from which the sample was drawn.

As an example, if some studies underdose a drug and other studies dose the drug appropriately, there could be two true effects even if the samples had been drawn from the same population. A fixed effect meta-analysis would therefore be inappropriate. It would be better, of course, to exclude, a priori, studies that underdose the drug; however, it may not always be possible to know what variations in study methods will create variations in true effects. So, this concern applies to other methodological variations, as well, and not to dosing alone. As a realistic precaution, therefore, if studies vary significantly in design and methods, a random effects approach may be preferable. This precaution is frequently necessary in medical research.

Simple Summary

A fixed effect model assumes that all studies are measuring the same numerical value and that differences in values across studies are due to chance. A random effects model assumes that all studies are measuring the same outcome, and that differences in values across studies are due to meaningful differences in contexts (e.g., study design, study methods, and/or sample characteristics).

Parting Notes

This article emphasizes the distinction between fixed effect (singular) and random effects (plural). However, some authors shun convention and use “effects” as a plural noun for both models.

This article did not hyphenate the terms “fixed effect” and “random effects.” Some articles do, presenting the terms as “fixed-effect” and “random-effects.” The hyphenation is a less common usage.