The International Network of Cholesterol Skeptics

Home

 

Is The Peer-Review System A Guarantee For Quality?

Most medical journals use the so-called peer-review system. This implies that for being published all manuscripts are scrutinized by one or two external specialists (referees) in the issue. Here follow (in bold) the comments from five referees, who were asked to review a manuscript by Uffe Ravnskov, sent to and rejected by three different medical journals before it was accepted for publication in Quarterly Journal of Medicine. Dr. Ravnskov's comments were also sent to the editors of the journals that rejected the article. A few explanatory comments (in italics) have been added later for non-medical readers.


You can read the article here  

 

Reviewer A

 

Even if atherosclerosis growth is causally related to LDL-cholesterol concentrations medication based reductions of LDL-cholesterol concentrations need not necessarily lead to reductions of atherosclerosis growth

Most trial directors claimed that they had reduced atherosclerosis growth. If the concentration of LDL-cholesterol was not associated with this reduction, it cannot be causal, this is the very meaning of causality. 

 

In a paper addressing the issue of a relationship of LDL-cholesterol concentration with atherosclerosis progression the choice of coronary angiographic outcomes seems not optimal, since the % stenosis or luminal diameter is not straightforwardly related to the amount of atherosclerosis. Direct measurement of intima-media thickness (eg by US) would seem more appropriate.

I have already met that objection on page 7, third paragraph: “This objection is not relevant here because associations were sought to the changes, not to the degree of atherosclerosis”

 

It is unclear whether the (even not appropriately documented) lack of a dose response relationship at the level of the coronary artery can be extrapolated to atherosclerosis of the entire vascular system.

It seems highly unlikely that the cause of atherosclerosis is another one in another part of the vascular system. Even if it were, the main target for cholesterol screening and treatment is the coronary system.

 

Introduction. P-values cannot be used to indicate the strength or existence of a relationship, they merely point to likelihood of chance.

If a p-value for the association between outcome and degree of cholesterol lowering is close to one, as it was in the statin trial mentioned, a relationship must be highly unlikely.

(A p-value of 1.0 means that there is no relationship at all. P-values should generally be below 0.05 to indicate a statistically significant relationship)

 

Lack of relationship can be explained by more factors that only absence of it: small numbers, incorrect or indirect measurements of variables of interest, imprecision in measurement, confounding factors, etc.

If it is impossible to find exposure-response between changes of blood cholesterol and atherosclerosis growth in 22 studies including almost 2500 individuals a relationship between the two, if any, must be trivial.

     Angiographic recordings of  atherosclerosis changes and the serum concentration of LDL-cholesterol are certainly direct measurements. Which measurements could be more direct? Imprecision in the measurement of atherosclerosis growth is discussed in detail on page 7.

     As atherosclerosis growth was not associated with any of the major risk factors (except physical exercise), none of them could have confounded the result.

 

Dose-response relationships tend to be complicated, having steep and flat parts in their slopes.

I agree, but as the baseline LDL-cholesterol in the reviewed studies varied between 3.11 and 8.89, at least some of the studies should have found exposure-response.

 

Methods.  Eligibility criteria for studies to be included are not clear.

They are as clear as crystal: “studies of sequential coronary angiographs that had included recordings of spontaneous or therapeutically induced blood lipid changes during the study”

 

Quality assessment of included studies was not performed.

As there was a uniform lack of exposure-response it makes no sense to grade the quality of the studies. A grading would have been appropriate only if some of the studies found exposure-response, others did not.

 

No attempt to data synthesis (e.g. meta-regression of LDL change versus atherosclerosis change) was not attempted. This is of consequence since most included studies were small and the absence of a significant association could be due to low numbers in the individual studies, while clearly present if individual study results were to be combined (one of the potential strengths of systematic reviews.)

A meta-analysis would have been appropriate if some of the studies had shown exposure-response, others not, but apart from one of the studies, there was a general absence of exposure-response; some of the studies even found an inverse association between atherosclerosis growth and LDL-C changes. It is therefore highly unlikely that a meta-analysis could have changed the result. Besides, individual data, necessary to perform a meta-analysis were not given in the studies.

     I do not agree that the studies were small; some of them included several hundred individuals. The objection would have been correct if I had reviewed studies that use morbidity or mortality as outcome, because even large studies may have few outcomes. But the studies used change of atherosclerosis growth as outcome. Each individual thus contributes to the calculation  and therefore much fewer individuals are necessary.

 

Results. No quantitative information is provided in the tables. This limits their information value. In their current form tables are too complicated to read and get the (a) message.

As mentioned above, individual patient data or odds ratios were absent in the studies. Most papers stated that “no association was found between degree of cholesterol lowering and atherosclerosis growth”, or similarly. I also doubt that the tables would have become more lucid had I added more data.

 

 

Reviewer B

 

I strongly suggest that a more balanced view concerning the relation of LDL-C changes and end-points should be presented. For example, the data of the primary and secondary prevention trials are not adequately presented. These studies have clearly established the benefits of the lipid lowering, which correlate with both the LDL cholesterol reduction, and with the extra lipid effects of these drugs (the so-called pleiotropic effects). For example in the 4S study the reduction in major coronary events within the simvastatin subgroup was highly correlated with on treatment levels and changes from baseline in total and LDL cholesterol levels, ApoB and less so with HDL cholesterol levels. It was estimated that each additional 1% reduction in LDL cholesterol reduced major coronary events risk by 1.7% (p<0.00001).(Circulation, 1998).

My review does not concern the effect of cholesterol lowering on coronary or other events, but its effect on atherosclerosis growth! See also the introduction (page 3): “statin treatment prevents cardiovascular disease but cardiovascular disease is not synonymous with atherosclerosis”. The effect of the statins on the clinical outcome most probably depends on other mechanisms, as I have explained on the same page; and that was the reason why I chose the direct way of studying the problem, namely by using atherosclerosis growth as end-point, not clinical events.

 

In the CARE study, even though the absolute or percentage reduction in LDL had little relation to coronary events, the LDL cholesterol levels achieved during treatment with pravastatin or placebo were associated with reductions in coronary events down to an LDL cholesterol of 125mg/dl. Furthermore, within the LDL cholesterol concentration observed (115 to 174mg/dl), baseline values influenced both the risk of events in the placebo group as well as the clinical effectiveness of pravastatin therapy (Circulation, 1998).

Again, my review concerned atherosclerosis growth, not coronary events!

.

Additionally, a meta-analysis of many trials has shown that the extent of cholesterol reduction following statin therapy was the most powerful factor in explaining the difference in mortality reduction across trials, a finding which provides further support to the suggestion that the greater benefit of statins is related to their ability to lower serum cholesterol (Arterioscler Thromb Vasc Biol, 199;19:187).

Same answer as above; I did not use mortality as end-point!

 

Another previously published meta-analysis of many studies, has also suggested that there is a relationship between cholesterol lowering and reduction in CHD and mortality risk (Gould et al. Circulation 1998;97:946)

Same answer; I did not use reduction in CHD or mortality as end-points!

 

Finally, even though, as pointed out by Ravnskov, isolated angiographic trials do not support any relation between LDL cholesterol reduction and CHD progression (possibly because of the small number of subjects participated in these studies as well as the problems arisen from the absence of relation between extension of atherosclerosis and coronary events), meta-analysis of angiographic trials showed that there was a very good correlation between changes in minimal lumen diameter and either LDL cholesterol on treatment or percentage change in LDL cholesterol. From this analysis LDL cholesterol on treatment appears to be as good a predictor of angiographic benefit as percentage reduction in LDL cholesterol. These findings are similar to another meta-analysis, which also indicated that LDL cholesterol on treatment and the absolute change of LDL cholesterol were equally predictive of change in percentage diameter stenosis (Curr Opin Lipidol 1997;8:354 & 1996;7:341).

What these reviews have shown is that in cholesterol-lowering trials atherosclerosis progresses more in the control groups, where LDL-C is high, than in the treatment groups, where LDL-C had been lowered. Again, this effect may be due to other mechanisms than cholesterol-lowering, but by plotting the mean values from the treatment and control groups from a number of trials together in a diagram, the authors create a false impression of exposure-response. A calculation of exposure-response demands individual data, but such data were not given in any of the included trials in the mentioned reviews, and in only one of the trials, the MARS trial (included in my review), the authors had calculated exposure-response. The diagrams from these reviews are typical examples of how to cheat with statistics. One can also speculate why the authors had included only one of the many trials reviewed in my paper. The concept of meta-analysis demands that all published trials are included.

 

 

Reviewer C

 

This review is based on the statement in non-immunologic diseases lack of exposure-response indicates lack of causality. This may be true in some cases, but not for the majority (especially for multifactorial diseases).

I disagree; if the LDL-C concentration is causal, some kind of exposure-response should be present, even if other factors may interfere. Besides, none of the major risk factors could have interfered because as mentioned above, none of them were associated with atherosclerosis growth either.

 

There are a lot of other mechanisms (mainly immunologic), besides lipids, which are involved in the formation and growth of atheroma. In addition, atherosclerosis is a procedure, which begin from the first years of our lives (there are evidence that it may begin even before our birth). So, it looks very optimistic to believe that an atherosclerotic plaque, which was enlarged for 4 or 5 decades, would be disappeared with a few months of lipid-lowering treatment. Our effort, by lipid-lowering treatment, is to slow down its progression and many studies have shown that this is enough to achieve a significant decrease of the risk of CVD.

I agree that many factors may be involved in the formation of atheroma, this is the conclusion of my study. What I question is that the LDL-C concentration is one of them.

     The formation of atherosclerotic plaques does not begin in childhood; in most people they appear after age 40-50. What is seen in childhood and in foetuses are fatty streaks, but this is not equivalent with atherosclerosis (See also: Ravnskov U. Prevention of atherosclerosis in children.Lancet 2000;355:69).

Most of the reviewed studies followed the patients for several years, not for a few months only, and during that time all studies noted atherosclerosis growth, a few noted regress in some of the patients. The crucial finding was that growth or regress were seen whether LDL-C or t-C went up or down.

 

The lumen diameter of the vessels, as it is measured by angiography or by ultrasound, is not the best way to measure the atherosclerosis, because the plaque growth not only towards the lumen of the vessel, but also towards its epithelium. So, we cannot estimate by them the correct size of the plaque.

Again, the end-point was degree of plaque changes, not the very size of the plaques.

 

The author referred only in studies, which failed to show a significant decrease in the lumen diameter with the decrease of serum concentration of LDL and tC. However, in the majority of these studies were used mild lipid-lowering methods (diet and regins). There are a lot of other studies using statins (ACAPS, PLAC-I, PLAC-II, KAPS, 4S(?), etc) which have showed a significant decrease in the atheroma growth.

I do not question that statins may reduce atherosclerosis growth, what I have suggested is that their effect is due to other mechanisms than cholesterol lowering.

     The mentioned studies could not answer that question because they did not calculate exposure-response or gave data that allowed this calculation and thus did not satisfy the criteria for inclusion.

 

It is true that statins may prevent cardiovascular disease with additional mechanisms (platelets, endothelium, plaque stability), but LDL and tC decrease is the main way of their action.

Where is the evidence? This is the very allegation I have falsified in my review. The absence of exposure-response demonstrates that a decrease of blood cholesterol is not the main way.

 

Homozygote FH patients die in their 20s from MI. This is one of the most powerful proofs of the link between tC, LDL and atherosclerosis. There are also animal studies, which showed that animals that have no atherosclerosis in their nature lives became atherosclerotic, by eating foods very rich in cholesterol.

(homozygote FH = a severe form of familial hypercholesterolemia, where the patient has inherited the disease from both parents)
     The pathologic changes in homozygote FH patients are not identical with atherosclerosis. For instance, these patients have deposits of cholesterol in most of their tissues, not only in their vessels. I do not question that the extremely high levels of  cholesterol in the blood of these patients may cause these deposits, but both macroscopically and microscopically the vascular deposits have very little in common with atherosclerotic plaques. This has been pointed out by several authors,  most meticulously by William Stehbens, see for instance Pathology 1991;23:54-61.  

     How and why rabbits and other herbivorous animals react to excessive amounts of dietary cholesterol lies far beyond the scope of my review. However, I would like to comment it shortly. First, the vascular changes in the rabbit after cholesterol feeding are fatty streaks. No one has succeeded in producing atherosclerotic plaques by cholesterol feeding. Furthermore, certain animal species do not react, for instance rats, baboons, pigs and carnivores. How do we know whether man reacts as a rabbit or as a rat?

 

Reviewer D

 

The question should be investigated in prospective studies

All studies included in my review are prospective!

 

An exemption should be made for children with homozygous familial hypercholesterolemia who do not have other risk factors, where angiographic studies do show accelerated atherosclerosis growth

The vascular changes in homozygous familial hypercholesterolemia are certainly not atherosclerosis (see for instance Stehbens W. Pathology 1991;23:54-61). Even if they were, there are no studies of homozygous FH that have addressed the question of exposure-response and I doubt there will ever be any because this condition is extremely rare.

     The reviewer refers to four studies that I should have included in my study. These studies did not satisfy the inclusion criteria because, like many other studies, none of them had calculated exposure-response, neither gave they any data to allow this calculation. I have spent at least a year reviewing hundreds of  trials and observational studies to single out those that satisfied my selection criteria, but found only the few that have been included in my review. As I mentioned in the paper I may have overlooked studies, but the mentioned four did not belong to that category. (One can only speculate why most of the angiographic trial directors did not calculate exposure-response, the most important parameter in a study of the effect of a treatment on atherosclerosis growth).

 

The list of studies is extremely heterogeneous

This is not a problem because my review is not a meta-analysis of cholesterol-lowering trials. If the aim of my review had been to study the effect of a certain treatment on atherosclerosis growth, the objection would have been relevant. But what I studied was the association between change of serum cholesterol and change of atherosclerosis; therefore, any measure that could have changed the serum cholesterol concentration during the study could be used, as could studies of spontaneous changes.

 

The mean changes (of coronary anatomy) were usually very small. These studies were not designed to demonstrate a relation in degree of LDL lowering and changes in coronary anatomy; they were designed to demonstrate an effect.

What is the difference between “changes in coronary anatomy” and “effect”? The studied effect in these trials and observational studies was changes in coronary anatomy! This is the very idea with angiographic studies: to use the change of atherosclerosis growth as outcome instead of clinical events. Besides, the mean changes of serum cholesterol concentrations or coronary anatomy are irrelevant measures – it is the individual changes, and the correlation between these changes and the individual change of the cholesterol concentration, that matter. A study of exposure-response demands individual measures and they may vary considerably even if the mean measurements change only a little, or not at all.

 

The Post CABG trial, not mentioned by the author, was designed to investigate the degree of LDL-cholesterol lowering

Again, exposure-response was not addressed in that study. Like most other trials they studied the mean changes only, not the individual changes.

 

 

Reviewer E

 

The so-called “Global estimates” from one study cannot really be compared to the “cross-sectional area determination by quantitative coronary angiography” utilized in other studies, or “percent stenosis” or “minimum lumen diameter” used in others.

The aim of my study was not to compare the effect between different studies, but to record exposure-response. The limitations of the different ways of expressing change of atherosclerosis degree are discussed in detail in the paper.

 

To be significant, the change in total cholesterol or LDL-cholesterol has to be determined from the onset and end point of each study. It is important for the author to show that this was the case in the studies reviewed.

All studies gave the change in total and/or LDL cholesterol from the onset and end point of the study, this was one of the inclusion criteria.

 

The title of the paper is somewhat misleading. It does not address, in any way, the issue of LDL-receptors, that are responsible for the binding and cellular penetration of LDL lipoprotein particles.

According to Brown and Goldstein´s LDL receptor hypothesis (ref. 1) the cholesterol of atherosclerotic plaques is derived from particles called low-density lipoprotein that circulate in the bloodstream. To cite the authors: “The more LDL there is in the blood, the more rapidly atherosclerosis develops” They developed it further by stating that “the number of LDL-receptors displayed on the surface of cells varies with the cells´ demand for cholesterol. When the need is low, excess cholesterol accumulates; cells make fewer receptors and take up LDL at a reduced rate. This protects cells against excess cholesterol, but at a high price: the reduction in the number of receptors decreases the rate at which LDL is removed from the circulation, the blood level of LDL rises and atherosclerosis is accelerated.” I think that this idea is what most researchers understand with the LDL-receptor hypothesis. I admit that what I question is only the first part of their hypothesis, the crucial importance of the LDL-C concentration in the blood.  I have no problems with a change of the title to, for instance, “Evidence against the importance of  the serum LDL-cholesterol concentration for atherosclerosis growth”

(The original title of the paper was “LDL-receptor hypothesis questioned”)

 

 

    Home