Reply to critique of “A randomized trial of the effects of the no-carrageenan diet on ulcerative colitis disease activity”

Abstract

This article is an invited response to a critique by industry of our published study about the impact of carrageenan supplement on the interval to relapse in ulcerative colitis patients on a no-carrageenan diet.

Keywords

Carrageenan ulcerative colitis log-rank test inflammation

1 General comments

The food industry has increasingly utilized carrageenan in food products in recent decades and, as shown by this critique of our published report, vigorously defends the use of carrageenan in processed foods. Prior to detailed responses to their critique of our study, we present several general comments about carrageenan:

First, carrageenan is well-known to induce inflammation. Carrageenan has been shown to cause inflammation in thousands of published scientific experiments over several decades. There are over 11,000 references about carrageenan in PubMed. Carrageenan-induced inflammation is used in tests of the effectiveness of pharmaceuticals or botanicals on reducing inflammation and the associated pain. The stimulation of inflammation by carrageenan occurs with either high or low molecular weight (degraded) carrageenan, since carrageenan-associated inflammation is attributable to its unique chemical structure, including alpha-1,3-galactosidic bonds alternating with beta-1,4-galactosidic bonds and high degree of sulfation [1]. Alpha-1,3-galactosidic bonds have been associated with human rejection of organ transplants from mammals due to innate immune responses, since humans do not make these bonds [2]. Experiments have shown extra- intestinal inflammation following oral ingestion of carrageenan [3], indicating systemic effects which may be influenced by circulating immune cells and responses of the intestinal microbiome. Carrageenan producers have tried for several decades to distinguish low (degraded; poligeenan) from high molecular weight carrageenan. However, the disaccharide structure of each carrageenan is constant, regardless of the molecular weight. As with sucrose, lactose, heparan sulfate, heparin, other glycosaminoglycans and sugars, the number of disaccharide units may vary, but the structural unit does not change. Tests have repeatedly shown that higher molecular weight carrageenan used in food applications contains a significant percentage of low molecular weight carrageenan [4]. In addition, food preparation and digestion, including exposure to heat, acid, mechanical processing, and intestinal bacteria may further lower the molecular weight of carrageenan ingested in food [2]. Human consumption of carrageenan may be as much as several grams daily, and some of the carrageenan is low molecular weight.

Regulatory bodies have been concerned about inclusion of carrageenan in food products for decades [5, 6], and carrageenan is currently under intensive review by the European Food Safety Authority (EFSA) [7]. Ongoing work by EFSA is intended to clarify the safety issues with regard to carrageenan exposure in food and considers the current acceptable daily intake to be temporary, pending the results of their inquiry.

When cDNA microarray was performed to test the effect of exposure to a low concentration of high molecular weight lambda carrageenan on human colonic epithelial cells in cell culture, hundreds of genes were significantly modified by carrageenan exposure [8]. This suggests profound effects of carrageenan exposure and contrasts with the effects of dextran sodium sulfate on expression of only a few genes. These transcriptional effects of carrageenan may arise due to inflammation, activation of immune responses, or interference with normal cellular metabolism related to effects on proteoglycans, glycosaminoglycans, and signaling pathways.

The non-food uses and exposures to carrageenan have increased in recent decades, including uses in fertilizer, toothpaste, infant formula, nutritional supplements, pharmaceuticals, cosmetics, and room air deodorizers. Hence, human exposure to carrageenan is an ever-increasing threat to human health. Carrageenan has no known nutritional value, so removal from the diet is not expected to have any negative effect on human health. There are many anecdotal reports about benefit by removing carrageenan from the diet. The effects of chronic exposure may be insidious.

With this background, we developed a study to assess the impact of the no-carrageenan diet, with or without capsules containing food-grade carrageenan, on the interval to relapse in patients with ulcerative colitis in remission. This was a carefully considered study design; participants were randomly assigned to either carrageenan supplementation or control, and investigators in contact with patients and patients were blinded about whether the capsules contained carrageenan or placebo. Capsules were prepared using food-grade carrageenan obtained from FMC Biopolymer by scientists at Purdue University. Participants were recruited through Digestive Disease clinics at the University of Chicago and the University of Illinois at Chicago.

2 Specific comments

In response to the critique of our published study by food industry scientists, who have an undeniable conflict of interest, we present the following responses to their statements (shown in italics):

“These regulatory bodies have all agreed on the safety status of CGN used as a food additive.”

In fact, there has been significant skepticism and controversy on the part of regulatory bodies, in spite of intense industry pressure. In the 1970’s, the FDA considered restriction of carrageenan to permit no more than 10% of carrageenan of molecular weight <100,000 [5]. More recently the European Union recommended no more than 5% of carrageenan of less than 50,000 molecular weight [6]. Industry can not achieve these standards in food-grade carrageenan, as shown by their own report [4]. Recently, the European Food Safety Authority called for additional technical and toxicological data about carrageenan in order to consider further the use of carrageenan in food products [7]. “According to one of the interested parties (Documentation provided to EFSA n. 19), carrageenan has a molecular weight distribution from 30 kDa to as high as 5,000 kDa and is defined as having a weight-average molecular weight between 200 and 800 kDa. Furthermore, it has been confirmed by industry (Documentation provided to EFSA n. 19) that commercial carrageenan (E 407) may have a weight-average molecular weight as low as 200 kDa. In view of the Panel, the molecular weight distribution of such a carrageenan product may have a considerable fraction of molecules encompassing weight-average molecular weight of degraded carrageenan.”

“A few groups have continually published reports using misinformation and misinterpretation of results to suggest that CGN is harmful.”

Thousands of reports listed on PubMed show that carrageenan predictably causes inflammation. Experiments have been performed worldwide by thousands of investigators. The harmful effects of carrageenan in animal studies are well documented, and carrageenan induced inflammation has been used to study mediators of inflammation and effectiveness of pharmaceutical and biological treatments.

“Being a food additive, carrageenan is mostly ingested as part of the diet, and typically bound to food protein. It is never ingested as a bolus in capsule form.”

The uses of carrageenan extend to its use as an excipient in pharmaceuticals and as a biopolymer used to make capsules. In food uses, carrageenan is anticipated to be in combination with protein. However, the process of digestion, including the action of digestive enzymes and intestinal bacteria, involves hydrolysis of chemical bonds, so that nutrients are extracted and can be absorbed. Degradation of ingested carrageenan has been reported by investigators who showed decline in the molecular weight of orally administered carrageenan in rat experiments [4, 9].

“cited studies that used degraded CGN, not food grade CGN”

The carrageenan industry has recognized that degraded carrageenan is present in food grade carrageenan and that harmful effects are associated with low molecular weight carrageenan. Mechanical processes, acid, and heat can further increase the exposure to lower molecular weight carrageenan.

“how this study could have been approached to eliminate obvious bias.”

The study was designed to be free of bias, and it is ironic that industry representatives are admonishing about bias. The authors of the study have no financial or personal interest with regard to the study outcome, and it is inappropriate to suggest that such bias might be present.

3 Statistical tests

“There are also issues with the use of statistical tests and data interpretation.”

The critique addresses sample size, randomization, SCCAI vs. SIBDQ, the definition of relapse, and the choice of statistical tests. No prior studies were available to inform the initial sample size calculation. We anticipated enrollment of more subjects, but found it difficult to enroll participants in a randomized study, in which one arm might be at risk of earlier relapse, and in which the diet was restricted for up to one year. Due to occurrence of relapses in three subjects, a statistically significant difference was evident between the two groups with a smaller sample than initially anticipated. The randomization did not consider medications taken for ulcerative colitis, and both study arms had participants on multiple drugs and treated with immunosuppressives, as well as participants who did not require intensive drug regimens while in remission.

The definition of relapse was determined prior to enrollment of any study subjects and included increase in the SCCAI, a measure of disease severity. SIBDQ was not considered as an outcome measure, since it is a quality-of-life scale, not determined by manifestations of disease. Relapse was defined to include intensification of treatment for ulcerative colitis, as well as increase of at least 2 points in the SCCAI.

We provide the following responses to the critiques about statistical tests used in the study:

3.1 Power of the study

The critic raises questions about the power about the sample size for testing SCAAI score difference. We must point out a misuse of the term “predictive power” in the critique.

“The fact that the authors ignored their own stated participant numbers necessary to achieve 80% predictive power is the first sign of a potentially improperly designed study and immediately calls into question the robustness of the results.” We’d like to point out that we are conducting a hypothesis testing rather than a predictive study. The use of “predictive power” appears to imply that our study has both higher Type I and Type II error rates than a similar study with large sample sizes. An under-powered study affects the Type II error rate (i.e., falsely retain the null when the null is incorrect) of the test result. However, the Type I error rate for the test is still set at 0.05, meaning that the probability of falsely rejecting a null when the null is true is still controlled at the stated rate of 0.05. Thus, the small sample size can render the study overly conservative for testing the difference in SCCAI score, but does not affect the validity of the test result. The post-hoc power analysis based on the observed effect size and standard deviation (4.20±3.70 and 0.86±1.46) with a Type I error rate of 0.05 using a two-sided two-sample unequal-variance t-test, shows that the study of 12 subjects achieved 33% power to reject the null hypothesis of equal means in the SCCAI score. This means that the sample size of 12 subjects may be unable to detect the observed effect size in the SCCAI score with high probability (67% = 1–33%). However, although underpowered, the test for SCCAI is still valid as the Type I error rate is controlled at 0.05 level of significance, meaning that we have a low rate of 5% to false positive result for the SCCAI score difference.

3.2 The use of t-tests in inflammatory markers and fecal calprotectin analysis

The critic notes “This was done using itp-values greater than 0.05 as statistically significant, despite the fact that itt-tests are not very stringent due to the expected high level or type 1 error (5% at minimum), and despite the fact that the authors admit this rate of error, and the small sample size of the study.” To clarify, the primary t-test is on the SCCAI score difference. The t-tests performed on the inflammatory markers and fecal calprotectin are conducted as secondary analysis and should be viewed as such.

3.3 The use of Log-rank test for survival data

The critic questions the use of log-rank test with two reasons. First, it says “the data used as the endpoint was the SCCAI scores, which provides a numerical value between 0 and 19. These are not censored data as obviously the values of the measurements are known.” As shown in Fig. 3. x-axis and the text on page 187, “the primary outcome measure was (time to) occurrence of relapse, which was defined as an increase of two or more points on the SCCAI, in association with an increase in treatment.” Thus, unlike what the critic noted above, the endpoint is the time to event data with censoring, for which the standard log-rank test is the appropriate test. Second, the critic says “It seems that the most appropriate statistical test to be used in this study is the F* test, which, under the conditions of this study, may have been a far more accurate test for statistical significance.” We disagree with this statement. The F* test, as described in Berty, Shi and Lyons-Weiler [10] is designed for the setting of survivorship prediction modeling for the purpose of assessing statistical significance of survivorship differences among model- predicted groups and the F* test can incorporate prediction errors in model-predicted group memberships that are unobserved by current methods of evaluation, including the log-rank test. Our setting here is different from that described in [10]; our aim is the typical hypothesis testing for comparing survival distributions on two different groups of subjects, for which setting the group membership is fully observed and requires no prediction (i.e., no prediction error). Thus F* test is not needed here and the log-rank test remains the standard method in our setting.

Fig.1

SCCAI Before and After Paired Samples.

3.4 Raw data for SCCAI scores

In response to the critique, we present the raw data for the SCCAI scores (see accompanying Fig. 1). Unpaired t-test, two-tailed with equal variance yields p-value of 0.05 and t-test with unequal variance yields p-value of 0.11. The t-tests are conducted in Excel. As we explained above, this study is underpowered for testing the difference in SCCAI scores, and thus does not have sufficient power to detect the clinically meaningful differences in SCCAI scores. However, the more robust and informative comparison using the log-rank test reveals statistical significance in the difference of the time to relapse between the two arms, which is reported in the abstract and as the primary finding of the study.

4 Summary

Additional research about the impact of carrageenan is of considerable interest, and we hope that the findings from this study will help other researchers to design informative clinical studies.

Funding

The authors report no funding.

Conflict of interest

The authors have no conflict of interest to report.

Footnotes

Acknowledgments

The authors have no acknowledgments.

References

Tobacman

JK.

Review of harmful gastrointestinal effects of carrageenan in animal experiments. Environ Health Perspect. 2001;109(10):983–94.

Tobacman

JK.

The common food additive carrageenan and the alpha-gal epitope. J Allergy Clin Immunol. 2015;136(6):1708–9.

Bhattacharyya

, O-Sullivan

, Katyal

, Unterman

, Tobacman

JK.

Exposure to the common food additive carrageenan leads to glucose intolerance, insulin resistance and inhibition of insulin signalling in HepG2 cells and C57BL/6J mice. Diabetologia. 2012;55(1):194–203.

Marinalg Working Group on Molecular Weight Determination. Technical position on measurements related to meeting the EC molecular weight distribution specification for carrageenan and PES. Report Jan 2006. https://www.cornucopia.org/wp-content/uploads/2016/04/CarageenanReport-2016.pdf

Proposed Revision of Food Additive Regulations and Deletion of Chondrus Extract (Carrageenin) from Generally Regarded as Safe (GRAS) List. 37 Fed Reg 15434. 1, 1972.

Scientific Committee on Food. Opinion of the Scientific Committee on Food on Carrageenan. SCF/CS/ADD/EMU/199, 2003. https://www.cybercolloids.net/sites/default/files/EU-carrageenan-opinion.pdf

EFSA Panel on Food Additives and Nutrient Sources added to Food (ANS). Re-evaluation of carrageenan (E 407) and processed Eucheuma seaweed (E 407a) as food additives. EFSA Journal 2018;16(4):5238–5350.

Borthakur

, Bhattacharyya

, Dudeja

, Tobacman

JK.

Carrageenan induces interleukin-8 production through distinct Bcl10 pathway in normal human colonic epithelial cells. Am J Physiol Gastrointest Liver Physiol. 2007;292(3):G829–38.

Taché

, Peiffer

, Millet

A-S

, Corpet

DE.

Carrageenan gel and aberrant crypt foci in the colon of conventional and human flora-associated rats. Nutr Cancer. 2000;37:193–8.

10.

Berty

, Shi

, Lyons-Weiler

Determining the statistical significance of survivorship prediction models. J Eval Clin Pract. 2010;16(1):155–65.