A review of: "Myo-inositol rather than D-chiro-inositol is able to improve oocyte quality in intracytoplasmic sperm injection cycles. A prospective, controlled, randomized trial"
By: Vittorio Unfer, Giafranco Carlomagno, Paola Rizzo, Emanuela Raffone, and Scott Roseff
European Review for Medical and Pharmacological Sciences, 2011
The evidence presented in this paper is inconclusive at best. The title and conclusions of the paper are questionable.
This paper serves as a reminder that not all science is good science.
Poor study design
Ill-defined study objectives
No description of selection process, blinding, or randomization
Multiple undisclosed conflicts of interest
Numerous statistical, typographic, spelling, and grammatical errors throughout
Questionable peer review
We will try to be brief here, as we have commissioned a detailed review by a biostatistician. This report, which was produced using the CONSORT criteria and co-authored with our own biochemist, is reproduced in full below.
The authors had hoped to show that myo-inositol supplementation would improve IVF outcomes in women with PCOS. They based this on the hypothesis that elevated insulin levels reduce the myo to DCI ratio, which results in lower oocyte quality. But women with elevated insulin were excluded from the study:
We excluded from the study patients that showed insulin resistance and/or hyperglycaemia.
Moreover, there are disturbing inconsistencies in the two treatment groups. Analysis by our third-party researcher reveals that there were clinically significant differences in the age of the women in the two groups, as well as statistically significant differences in prolactin levels—both of which are important, independent predictors of oocyte quality. Any differences in the outcomes of the two groups could easily be a result of these differences.
Conspicuously absent from the discussion section is a discussion of the implications of undergoing IVF therapy. High levels of hormones are administered; invasive surgical procedures are performed. Extrapolating the results or outcomes from women undergoing this dangerous, powerful procedure to other women with PCOS is questionable. Some discussion of this limitation of the study was warranted, considering the focus on the PCOS community.
The authors wisely do not draw a straight line between these IVF patients and the PCOS community at-large. Because this paper was produced by a supplement company (see last paragraph, below), they left that task to the marketing department. Instead, the authors make this outrageous statement:
Ooocyte [sic] retrieved from PCOS patients are indeed characterized by poor oocyte quality and, therefore, any treatment able to improve oocyte quality could be considered the "holy grail" for IVF procedures.
PCOS is complicated: the presentations and the reasons behind it are myriad. The very idea that there would be a universal, magical solution to any medical condition (much less PCOS) is ridiculous. One thing that should be clear to anyone with more than a passing interest in PCOS—there is no "holy grail". I realize that it is just a metaphor, but it is incredibly inappropriate.
In fact, both treatment groups were so much better than IVF therapy alone, that the differences (even if we granted they were in fact driven by differences in treatment modality rather than differences in age and prolactin) were of questionable clinical significance.
Lastly, there is no conflict of interest or funding statement. The lead author, Vittorio Unfer, is president of LoLi Pharma, an Italian company that sells myo inositol to women with PCOS. The second author listed, Giafranco Carlomagno, is also employed by LoLi Pharma. This undisclosed corporate affiliation is unconscionable. When considered along with the omission of any blinding or randomization description, this issue raises serious concerns about the peer review and the validity of the results.
There are far too many serious problems with this paper to take any of it seriously. It serves as a reminder of how important it is to read clinical literature with a critical eye. Nevertheless, in the years since this paper first appeared, it has been parroted and misappropriated to make sales pitches for myo-inositol (and to denigrate DCI therapy) all over the internet.
To be clear, we at Chiral Balance are not down on myo-inositol. We know from the literature and our own experiences that myo-inositol is an appropriate solution for many women with PCOS. But, we are down on inappropriate claims backed by junk science.
By: Phillip Watkins, MS Statistics; Travis Johnson, VP Chiral Balance, MS Medical Sciences, MA Biochemistry
Prepared using the CONSORT criteria
Scientific Background And Explanation Of Rationale
The rationale is poorly reasoned. The authors note the prevalence of both impaired glucose tolerance and insulin resistance in PCOS. They conjecture that this leads to a depletion of myo-inositol (MI) in the ovary. This rests on a number of poor assumptions and factual inaccuracies. First, the authors state that "[conversion of MI to DCI] is insulin dependent." This is not supported by the provided citation, although it is supported by other literature. A number of studies, including the provided citation, suggest that this conversion is impaired in women with PCOS, i.e. the conversion does not occur to the same extent in PCOS women as it does in the general population.
Specific Objectives Or Hypotheses
The authors list their study objective as: "to compare the effects of myo-inositol (MI) and D-chiro-inositol (DCI) oocyte quality in euglycemic PCOS patients." The skeptical reader cannot be certain what aspect of "oocyte quality" the investigators planned to compare based on this somewhat generic study objective. Given the absence of a consensus definition of "oocyte quality"among researchers, defining endpoints at study outset is especially important. The casual observer is left to guess whether the outcomes detailed in this paper were pre-specified or cherry-picked from a larger data set with largely insignificantly different oocytes outcomes.
Even if we grant that MI is depleted in the PCOS ovary by the action of elevated insulin, then an appropriate hypothesis would be that MI supplementation would improve outcomes for women with PCOS and depleted MI, i.e. those with elevated insulin. However, the authors test their hypothesis "in euglycemic PCOS patients" and specifically "excluded from the study patients that showed insulin resistance and/or hyperglycaemia." This is the exact opposite of the patient type discussed in their hypothesis.
Study Design & Randomization
The lack of a testable hypothesis and missing power analysis may indicate that this study was inadequately planned. With no pre-specified detail regarding the primary outcome of interest or how the sample size was determined, we are uncertain how many factors the investigators measured. One in twenty factors will appear statistically significantly different (p<0.05) by mere chance, so it's important to think twice and test once.
Enrollment duration, participant flow, losses, and exclusions are not detailed at all. A flow chart detailing the patients screened, excluded, enrolled, and losses and reasons would have helped clear up any ambiguity here. Without information on number of exclusions, we have no idea how many women were screened for the study. With no losses detailed, the reader is left to assume a 100% completion rate based on the reported 84 subjects recruited. While a 100% completion rate is extremely rare for a prospective study, given the significant cost of IVF, it's not surprising that every study subject came back for follow-up.
The randomization process is completely trivialized by saying only "patients were randomly assigned." The type, method, concealment/blinding mechanism (if any), and implementation of randomization are conspicuously absent from the paper methods. The unequal allocation ratio (43:41) leaves the reader wondering if a balanced design was planned and mistakes occurred or if some random mechanism (coin flip or investigator whim) was the method of determination. Furthermore, it doesn't appear that this simple random sample achieved the desired effect of producing comparable groups: table I suggests the groups are comparable with respect to age, duration of infertility, and BMI, which is confirmed in the body of the paper. Prolactin (PRL) and thyroid (TSH) are also compared, but it is unclear if the comparison is a baseline (pre-treatment) or outcome (post-treatment). The censored non-significant (NS) p-values can be estimated using the reported sample sizes, means, and standard deviations: age (p=0.116) and duration of infertility (p=0.069) were closer to statistically significantly different than BMI (p=0.694) and TSH (p=0.651), which appeared more or less comparable. However, a statistically significant difference in PRL (p=0.043) was discovered that appears to have been erroneously reported as NS. As such, it is possible that a clinically significant difference in age or the statistically significant difference in baseline PRL is driving the difference in fertility outcomes, rather than differences in the treatments.
Given that age is known to affect fertility outcomes, age matching might have been employed in the design stage to ensure groups were comparable with respect to this critical variable, yet it appears that simple random sampling was employed, and we ended up with two groups that were not quite comparable with respect to age or PRL levels. In light of the potential differences between our groups, an adjusted analysis was warranted to attempt to answer the question, "Do the two treatment groups differ after we account for differences in age and/or PRL levels". The absence of any attempt to adjust for baseline differences, along with some statistical oversights in their tabular summaries (listed below), may reflect that limited scientific review was conducted prior to study publication. Furthermore, there are numerous misspellings, typographical and grammatical errors, which raises the question of whether there was any serious copyediting prior to publication.
The statistical testing procedures for the three tables (t-test, Wilcoxon, Fisher's exact for tables I, II, and III respectively) were generally appropriate. As detailed above, Table I did appear to have an error on the summary of the PRL baseline level. This may have been a result of a clerical error in reporting the means/SD, an error in computing, or incorrectly interpreting the p-value of 0.043 as NS. In table II, reporting means and standard deviations suggests a normal distribution, or one that depends on the two parameters of the mean/SD. However, a normal distribution is not assumed by the non-parametric Wilcoxon test. Reporting medians and (inner quartile) ranges is generally held as standard. Furthermore, it would have been sufficient to test just one of the number of MII or immature oocytes, given that that they must add up to the total number of oocytes.
In Table III, the breakdown of types of pregnancies does not add up to 100% for the DCI group. The correct percentages for the 10 pregnancies in that group would be 2/10 = 20% biochemical, 5/10 = 50% clinical, and 3/10 = 30% spontaneous abortion. One should compare these percentages to the respective 14%, 68%, and 18% of the MI group in a single 3 x 2 chi-square test, which has a p-value of 0.568, rather than 3 individual chi-square tests for each outcome that are all equally NS.
Lack of blinding is a serious limitation to this study. Concealing group status from researchers serves to minimize any conscious or unconscious bias in measuring or interpreting their data. However, no blinding methods are mentioned or detailed. As such, we cannot rule out the possibility that the bias inherent to a financial interest in the study findings enlarged the observed differences between the fertility outcomes of interest. Furthermore, it is possible that researcher bias crept in when "patients were randomized to two groups", which might have caused the age or PRL differences we observed if a subconscious investigator bias due a conflict of interest (listed below) could affect which patients were allocated to which groups.
Conflicts Of Interest
The paper lacks any conflict of interest statement. The institutional affiliation of the lead author, Vittorio Unfer, is conspicuously absent. An internet search reveals that he is the president of LoLi Pharma, an Italian company that sells myo-inositol to women with PCOS. At least one other of the authors is an employee of LoLi Pharma as well: Giafranco Carlomagno. Clearly they have a vested interest in the findings of the study, which should have been disclosed in the paper.
No funding statement is provided. If LoLi Pharma provided some or all of the MI supplements, we have another financial conflict of interest for the investigators. However, no details were provided on the source of the MI and DCI supplements. As such, we are uncertain of the type or quality of the supplements used in this study.
The authors failed to conduct an age-adjusted analysis and appear to have erroneously reported baseline prolactin difference between the treatment arms as NS. Given that age and baseline prolactin are known independent predictors of IVF outcome, the internal validity of the study is questionable. In view of the facts that the DCI group was both older and had higher prolactin levels, the conclusion that myo-inositol was a superior treatment is unsupported by this study. Given that the authors' broad hypothesis presupposes insulin-dependent depletion of ovarian myo-inositol in hyperinsulinemic PCOS women, their decision to exclude these women from treatment raises concerns about the external validity as well.