Abstract
As universities cut library funding and forego expensive journal subscriptions, many academic organizations and researchers, including the American Educational Research Association (AERA), are moving toward open-access publications that are freely downloadable by anyone with a working internet connection. However, the impact of paywalls on the consumption of academic articles is unclear. We provide novel evidence on this question by exploiting a natural experiment in which six high-impact, usually gated AERA journals became open access for a 2-month period in 2017. Using monthly download data and an always-open-access journal as a comparison group, we show that making journals open access likely increased article downloads in those journals by 55% to 95% per month. Given a per-article download price of $36, this suggests a relatively elastic response: The average price elasticity of demand for downloads is 1.2, with individual journal elasticities ranging from 0.6 to 2.
Academics, university libraries, and the general public are increasingly frustrated by the rising costs of access to academic journal articles, particularly those published by for-profit firms (Bergstrom, 2001; McCabe et al., 2013; McKenna, 2012). Concerns include the growing market power of a handful of private publishers that enables them to set high prices and drain library budgets, socioeconomic gaps in access to research between universities, and an inability to get research into the hands of practitioners, teachers, and policy makers who might use that research to improve lives. This is particularly troubling when you consider that much academic research is either directly funded or indirectly subsidized by public funds and that authors of said research are not directly compensated for their contributions. Indeed, many funding agencies now require that results of funded projects be published as open access (OA; “Brought to Book,” 2012).
Outside funding agencies, a broader movement is afoot to make OA the norm in academic publishing. OA means that no paywall, registration, fee, or subscription is required to download and read material. OA journals typically generate revenue by charging authors a flat fee, although in rare instances the publication costs are covered by a third party, such as a professional organization or foundation associated with the journal. This is in direct contrast to traditional pricing models that generate revenue by charging readers an access or subscription fee. The sustainability of the OA business model requires that authors (or some other third party) voluntarily pay to publish this work, which in turn requires the existence of both authors (or third parties) who benefit from a wide readership and can afford to pay publication fees and readers who have relatively low benefits of individual access (McCabe et al., 2013).
For many authors, academic units, and universities, the quality of a publication is viewed in terms of page views, downloads, or citations. However, measuring the impact of OA, which is essentially a price change from a positive price to zero, on a publication’s readership is a thorny methodological problem (McCabe & Snyder, 2014). The reason is that there might be various types of selection bias associated with the decision to publish an OA article. For example, paywalled journals might make their best papers OA to increase the journal’s standing, which depends on citation data. Similarly, authors might be more willing to pay OA fees for articles they perceive to be of higher quality. Alternatively, authors might relegate their lower-quality work to OA journals if they perceive their chances of publication in those journals to be greater.
Because of these potential selection problems, credible evidence on readers’ responsiveness to the cost of academic articles is sparse and, indeed, nonexistent in the case of education research. The extant literature is focused on the hard sciences, which were early entrants into OA publishing. McCabe and Snyder (2014) used a journal-volume fixed-effects strategy to show that, in a sample of top ecology, botany, and multidisciplinary science and biology journals, OA increased citations by about 8% between 1996 and 2005. We know of no such analyses in the social sciences or education. We address this gap by providing arguably causal evidence on readers’ responsiveness to price, or price elasticity of demand, by exploiting a natural experiment in which several leading education journals unexpectedly became free.
Specifically, in the first 2 months of 2017, the paywall of the six leading journals of the American Educational Research Association (AERA) went down, essentially granting free access to anyone with a working internet connection. Meanwhile, access to articles in the always-OA AERA Open journal was unaffected by this change, as the articles in this journal are always ungated (never behind a paywall). Thus, AERA Open serves as a comparison group in the natural experiment created by the paywall failure. A difference-in-differences-style identification strategy can therefore provide suggestive evidence of OA’s effect on readership by comparing the various journals’ monthly downloads before, during, and after the paywall failure. 1
Using monthly data on downloads from 2015 through early 2019, we estimate an event-study time series regression for each of the seven journals. Monthly downloads in the six paywalled journals increased by 55% to 95% during the 2 months the paywall was down. This was unlikely to be due to any confounding aggregate shock to demand for education research, as downloads of AERA Open articles actually fell by about 42% during this time, suggesting that newfound free access to the other six journals may have crowded out demand for AERA Open articles. However, total journal downloads increased during this time, suggesting it was not a 1:1 crowding out. Regardless of the size of any crowd-out effect, the lack of a spike in downloads in AERA Open during the time the paywall was down suggests that the increase in downloads at the other journals was not due to a universal increase in demand for education research.
Institutional Details and Data
AERA Journals
The AERA is the largest organization of education researchers in the United States, with more than 20,000 members. The association operates seven journals, which are described in Table 1. Six of the journals were founded between 1931 and 1979 and are available in hard copy and online. Educational Researcher (ER) and American Educational Research Journal (AERJ) publish primarily original empirical research on any aspect of education, with the main difference being the format (ER articles are generally 4,000 to 5,000 words, and AERJ and all other AERA journals except Review of Educational Research [RER] are double that) and frequency (ER is published nine times a year with 46 total articles in 2017; AERJ, six times with 38 total articles). RER and Review of Research in Education (RRE) are review journals; the distinction is that RRE publishes one themed issue per year (with 12 articles in 2017), whereas RER publishes longer review articles (up to 15,000 words) on any topic spread over six issues per year (containing 23 articles in 2017). Educational Evaluation and Policy Analysis (EEPA) and Journal of Educational and Behavioral Statistics (JEBS) are more specialized journals; the former publishes analyses of the implementation and effects of educational policies (in four issues per year, with 24 articles in 2017), and the latter publishes research that advances quantitative methods in educational measurement and statistics (in six issues per year, with 29 articles in 2017). All AERA members are entitled to receive paper copies of ER and are allowed to select one of the other five to receive for free; additional journals are available to AERA members for a small fee and to the general public for an annual subscription fee. These six journals are all published by Sage and are available on Sage’s website behind a paywall.
Seven American Educational Research Association (AERA) Journals
Note. N/A = not applicable.
One-year impact factors and rankings are from 2017 InCites Journal Citation Reports published by Web of Science.
AERA Open is an open-access (OA) journal. It is the only AERA journal that is OA. It is still too new to have an impact factor. It is also fully online.
The newest AERA journal, published since mid-2015, is AERA Open. Like AERJ and ER, AERA Open is a general-interest journal publishing mainly original empirical research on any topic in education. Unlike the other six AERA journals, AERA Open is OA. Anyone can go to the Sage website and download AERA Open articles, and there are no paper issues created or disseminated. Articles are generally published on a rolling basis as they are accepted, although they are sometimes rolled out in groups as part of “special topics” (48 articles were published in 2017).
The Natural Experiment
In early 2017, the publisher of the AERA journals, Sage, switched to an online platform provided by Atypon. To ensure that all subscribers had uninterrupted access during the platform change, the paywall was removed. Ultimately, the platform change was successful and the paywall was down for most of January and February 2017. The timing and duration of the paywall outage were neither announced nor expected, so there were unlikely any strategic or anticipatory responses on the part of readers. This also means that had the outage been advertised, the effects would likely have been bigger.
The Data
Sage and AERA provided monthly download data for each of the seven journals spanning January 2015 to February 2019. Figure 1 displays the raw download data for each journal. Panel A of Figure 1 shows the monthly downloads of AERA Open and the average monthly downloads of the other six AERA journals that were “treated” by the paywall going down. The average journal has about 20,000 full-text downloads per month. Panel B of Figure 1 plots the monthly downloads separately for each journal. Two patterns are noteworthy. First, individual journals vary in download activity, as expected given the differences in prestige and topics identified in Table 1. AERJ, ER, and RER tend to have more downloads. Second, a seasonal pattern in downloads is apparent for all journals: Downloads spike early and late in the calendar year. The spikes in some journals, like ER and RER, are larger than the others.

Monthly downloads, in 1,000s: (A) average monthly downloads; (B) monthly downloads for each American Educational Research Association journal.
Two aspects of Figure 1 suggest a causal effect of OA on downloads. First, the spike in download activity in the six usually gated journals during the first 2 months of 2017 is bigger than the January-February spikes in other years. Second, there is no such spike in early-2017 AERA Open downloads. A similar pattern is observed when averaging the six gated journals together. Figure 2 shows additional evidence consistent with a causal effect of OA by differencing out the same-month, prior-year’s downloads. We do this because there is a seasonality to the demand for education research driven by the academic year of university professors and students, who are major consumers of peer-reviewed research, as well as the 9-month school year experienced by many teachers and principals. 2 Again, we see a departure from trend for the six usually gated journals in January and February of 2017, when the paywall was down. The next section formalizes this analysis using time-series event-study regression models.

Adjusted monthly downloads, in 1,000s: (A) adjusted average monthly downloads; (B) adjusted monthly downloads for each American Educational Research Association journal.
Method
We estimated simple event-study time-series models for each journal (Wooldridge, 2018). The unexpected paywall removal is a shock that potentially disrupts general trends and seasonal patterns in monthly article downloads. Specifically, we estimated models of the form
where t indexes the 50 months that span our data, Downloads is the count of monthly downloads, Month is a set of month indicators, t is a linear time trend, Ungated is a binary indicator equal to 1 in January and February of 2017 (when the paywall was down) and 0 otherwise, and u is a possibly heteroskedastic and serially correlated error term. The parameter of interest is β, which represents the impact of the paywall removal on monthly article downloads.
We conditioned on linear time trends, although results are robust to using exponential, quadratic, or cubic trends and to ignoring the trend altogether. This is unsurprising, as Figure 1 shows that the trends are relatively flat. More importantly, we conditioned on a full set of month indicators (fixed effects) to explicitly control for seasonality in downloads. Seasonality is apparent in Figure 1, where there are spikes in download activity in at least some of the journals during March through May and October through November, perhaps in tune with university academic calendars. Alternatively, it is possible that article quality is correlated with month, if journal editors put strong papers in the first issue of the year, for example. Whatever the reason for this seasonality, though, the month fixed effects effectively adjust for it. The identifying assumption is that seasonality effects are constant over time, such that the only shock to download demand in early 2017 was the paywall being down.
We report HAC (heteroskedasticity and autocorrelation) robust standard errors (Newey & West, 1987) that allow for arbitrary forms of heteroskedasticity and autocorrelation (up to three lags). We chose three lags because this is the integer suggested by the “quartic-root of T” rule of thumb described in Wooldridge (2018), although the main results are robust to using either one, two, or three lags and to ignoring possible serial correlation altogether.
We considered two modifications of the model presented in Equation (1). First, we replaced the Ungated indicator with two mutually exclusive “January 2017” and “February 2017” indicators, to allow the “no-paywall effect” to vary between the 2 months the paywall was down. Second, we replaced the dependent variable Downloads with its natural log, which given the non-negative nature of downloads, implements an exponential trend model and provides an intuitive semielasticity interpretation of β (Wooldridge 2018). Specifically, in the log-level specification of Equation (1), 100 × β represents the percentage change in downloads during the ungated period (e.g., a coefficient estimate of 0.5 implies a 50% increase in downloads).
We also used estimates of β to compute own- and cross-price elasticities of demand for article downloads. Elasticity is a fundamental concept in economics used to measure the responsiveness of quantity demanded to a price change (Frank, 2014). Elasticities rely on percentage changes in price and quantity to facilitate comparisons across goods with different baseline levels of demand. This is an appealing way to compare the effects across journals given the baseline differences across journals in average monthly downloads.
Of primary interest is the own-price elasticity of demand, which is simply the ratio of the percentage change in quantity demanded (downloads) to the accompanying percentage change in price. When this ratio is less (greater) than one in absolute value, elasticity is said to be inelastic (elastic). Own-price elasticities are commonly computed as
Finally, we computed the cross-price elasticity of demand for AERA Open with respect to the price changes observed in the other six AERA journals. Cross-price elasticities measure the percentage change in demand for good j attributable to a 1% change in the price of good k (Frank, 2014). The sign of the cross-price elasticity determines whether goods j and k are complements (goods that are consumed together, like peanut butter and jelly) or substitutes (either one or the other is consumed but not both, like shoes and sandals). The preceding discussion of computing and interpreting own-price elasticities applies analogously to the case of cross-price elasticities.
Results
Estimates of various specifications of Equation (1) are reported in Table 2. Panel A reports the baseline estimates where the outcome is total monthly downloads (in 1,000s). Column 1 reports estimates for the AERA Open time series, which was OA (freely downloadable) for the entire duration of the time period covered by the analytic sample. We use only data from 2016 forward for the AERA Open regressions because there was only a handful of articles posted in 2015; all other regressions use the full 2015-to-2019 data set. The negative point estimate of –4.16 means that when the other six AERA journals’ paywalls went down (and became freely downloadable), AERA Open article downloads decreased by about 4,160 per month. Relative to the same months (January and February) of the previous year, when the other AERA journals were paywalled, this amounts to a monthly reduction of about one third.
Time-Series Event-Study Estimates of Effect of Taking Down Paywall on Monthly Downloads
Note. For AERA Open (column 1), T = 38 monthly data points, from January 2016 (t = 1) through February 2019 (t = 38). For all other journals (columns 2–7), T = 50 monthly data points, from January 2015 (t = 1) through February 2019 (t = 50). We exclude 2015 data for AERA Open because AERA Open was not regularly publishing articles in 2015. The “no-paywall” period lasted 2 months: January (1) and February (2) of 2017. All models condition on month fixed effects and a linear time trend. The results are robust to using a quadratic time trend or eliminating the trend altogether. HAC (heteroskedasticity and autocorrelation) robust standard errors are reported in parentheses. Specifically, these are Newey-West standard errors that allow for three lags, which was determined following the “quartic-root of T” rule of thumb. However, the results are robust to allowing for one, two, or three lags. For each model, we report the mean of the outcome for the analogous pretreatment month(s). For example, Panel A reports mean downloads in January and February of 2016 to provide a reference point for interpreting the effect of the January-February 2017 treatment effect. Elasticities (ε) are computed based on the stated per-article download price of $36 becoming $0 during the treatment period of no paywall. Column 1 reports cross-price elasticities; the negative sign implies that AERA Open articles are substitutes for, not complements to, the other journals’ articles.
p < .01.
The negative effect on AERA Open downloads suggests that there is limited demand for education research articles and that the availability of other free downloads crowded out some potential AERA Open downloads. Indeed, the implied cross-price elasticity of –0.35 indicates that AERA Open articles are substitutes for articles in the other six AERA journals (that readers read in place of other articles) and not complements to articles in the other six AERA journals (that readers read in addition to other articles). Still, they are not perfect substitutes and the cross-price demand is relatively inelastic. Moreover, as we see in the subsequent columns of Table 2, the increase in downloads for the other six journals far outweighs the decrease in AERA Open downloads, indicating that total downloads of AERA journals went up during the 2-month period in which the paywall was down. We do not know how non-AERA journal article downloads were affected, but the drop in AERA Open downloads suggests that other education-focused journals may have experienced a small dip in downloads, as well.
Columns 2 to 7 of Table 2 report the same estimates of the time series specified in Equation (1) for each of the six gated AERA journals that experienced a 2-month paywall elimination. This effectively reduced the per-download price from $36 to $0. We now explain and interpret the results for AERJ in column 2, which is arguably AERA’s flagship research journal; estimates for the other journals can be interpreted similarly. The point estimate of 22.22 is positive and strongly statistically significant, suggesting that the elimination of the paywall increased monthly downloads by more than 22,000. This is a relatively large effect, representing a 127% increase (more than doubling) over the total downloads over the same 2-month stretch in the previous year. This was driven by a price reduction from $36 to $0, implying an elastic elasticity of demand for AERJ articles of 1.27.
The results for AERJ are similar to those for the other five usually gated AERA journals, as reported in columns 3 to 7 of Table 2. Although the point estimates vary in magnitude from effects of about 3,000 downloads for JEBS to more than 41,000 for RER, all are positive and strongly statistically significant. The variation in absolute effect sizes across journals is largely driven by the underlying average monthly download rates of those journals. Expressed as percentage increases based on results from the exponential trends model reported in Panel B, effects range from 55% to 95%. Corresponding elasticities range from 0.6 to 2.0. It is interesting and intuitive that the smallest effects and elasticities are observed for EEPA and JEBS and the largest effects and elasticities are observed for the review journals and AERJ, as JEBS (and to a lesser extent, EEPA) are the most technical and specialized of the six journals, whereas RER, RRE, and AERJ have larger impact factors and cater to a broader audience.
Panel B estimates a version of Equation (1) that takes the natural log of monthly downloads as the outcome variable. These are our preferred estimates, as they account for the strictly non-negative nature of monthly downloads, allow for an exponential trend, and provide a useful semielasticity interpretation of the paywall effect (Wooldridge 2018). These estimates suggest that the removal of the paywall causes a 42% reduction in AERA Open monthly downloads and, for example, a 79% increase in AERJ monthly downloads. Once again, the point estimates are positive and strongly statistically significant for each of the six previously gated AERA journals and range from effects of 55% (EEPA) to 95% (RER). These results are broadly consistent with those reported in Panel A and provide robust evidence of a sizable, arguably causal effect of OA on journal downloads.
Panel C provides yet another scaling that shows just how large and meaningful the estimated effects are. Here, we standardize journals’ monthly downloads to have a mean of 0 and standard deviation of 1. Panel C shows that the removal of the paywall was associated with statistically significant increases of 2 to 3 download standard deviations for all six gated journals.
Finally, Panel D explores the dynamics of the temporary paywall removal, which lasted 2 months, by differentiating the effect in Month 1 (January 2017) from that in Month 2 (February 2017). In all cases, the two month-specific effects are strongly jointly significant. Column 1 shows that the crowd-out effect of ungated access to the other AERA journals on the OA AERA Open seen in Panel A occurs primarily in January, as the effect on downloads in February remains negative but is indistinguishable from zero and significantly smaller than the January effect. This is perhaps partly explained by the significant (nearly 50%) drop in AERA Open downloads observed between January and February in 2016.
Columns 2 to 7 of Panel D show that the effect of the paywall removal on downloads of the six previously gated AERA journals was larger in February than in January. These differences are at least marginally significant for all but one of the journals. Perhaps some of these monthly differences were due to interested parties learning about the fallen paywall, but more likely it has to do with seasonal patterns in downloads: There are more downloads in February than in January, as seen in the 2016 figures. Indeed, in percentage terms, the monthly effects tend to be within 20 to 30 percentage points of each other and constitute similar percentage increases over the same month’s downloads in 2016 as the baseline results in Panel A (80% to 150%).
Conclusion
We estimate the effect of OA to academic journal articles on article downloads by exploiting a natural experiment in which the paywall to several prestigious education research journals was unexpectedly taken down for 2 months. Using the always-OA AERA Open as a comparison group, we find arguably causal evidence that removing the paywall increased monthly downloads by about 55% to 95%. This is a fairly large effect on downloads and suggests that thousands of potential readers interested in education research are stymied each month by paywalls.
Our focus on downloads is necessitated by data availability, as this is the only proxy for article consumption available to us. This is a limitation of the study, in the sense that downloaded articles are not necessarily read, which makes downloads a noisy measure of consumption. A related limitation of the current study is that we know neither the intent nor the background of the individuals who are downloading these articles. For example, whether the increase in downloads was driven by public school teachers and principals or by students and researchers at colleges that lack subscriptions to the AERA journals is a question with important policy implications that we necessarily leave to future research.
Nonetheless, downloads are arguably a useful first-order approximation of an article’s impact. We are of course unable to address the important question of how many downloads are actually read and, of those read, how many alter the reader’s behavior or knowledge. Previous work has focused on citations rather than downloads (e.g., McCabe & Snyder 2014, 2015). We cannot say how many of the downloads induced by the temporary period of OA led to citations, although descriptive research on the determinants of citations find that every 50 to 140 downloads lead to one citation (Gorraiz et al., 2014). Assuming that about 1/100 (1%) of downloads lead to a citation suggests that the temporary removal of the paywall generated anywhere from 30 (JEBS) to 250 (ER) citations per journal per month. That said, in the case of education research, citations are not necessarily the best measure of impact, as teachers, schools, and districts may use journal articles to inform their policy and practice without ever formally citing the research. Here, there is scope for rigorous qualitative research to increase our understanding of how school leaders and teachers consume academic research.
This work has important implications for policy and practice. Our findings suggest that there is demand for greater access to education research; this implies that there is a need for new structures or systems to make education research more available, particularly to individuals at institutions that do not have paid subscriptions to large numbers of academic journals. One approach is to more widely and aggressively disseminate journal article preprints or working-paper versions of published papers (which are typically allowed under copyright restrictions). On a personal level, researchers can and should make close-to-final versions of their research easily searchable and downloadable on their personal websites. Similarly, the academic community could do a better job of communicating to practitioners and journalists (a) that ungated working-paper versions of research articles are often available online and (b) that “cold emailing” a scholar to request an electronic copy of an article not only is reasonable but will often make the scholar’s day. At the professional level, there are already several groups disseminating working papers, including the recently launched, relatively inclusive EdWorkingPaper series from Brown University (https://edworkingpapers.com/). More efforts along these lines are warranted, and we strongly encourage researchers at all levels to consistently make their research available through such working-paper series. Finally, our results indirectly suggest a need for new organizations or structures that focus on making research findings more accessible in other ways, such as through distilling or simplifying education research findings and presenting them in shorter formats. We encourage funders and researchers to experiment with ways to increase the availability and use of education research given the apparent untapped demand. One example of this is the research briefs that are now a submission option at journals such as EEPA and Education Finance & Policy.
Future work should assess the economics of publishing education research more generally, including who submits articles to and publishes in AERA Open, who pays the submission fees, perceptions and citation rates of the quality of AERA Open and other OA journals relative to subscription-based journals, and the price sensitivity of university libraries to journal subscription prices.
