Do’s and don’ts in clinical studies

Can you tell the difference between a validation and a clinical study? Do you know what to consider when starting a randomised controlled trial? Being author or co-author on more than 60 peer reviewed papers I have a keen interest in current regulation regarding clinical studies and will share some of my experience on conducting clinical studies in this blog post. I hope I will be able to help you sort out some of the many variables related to clinical studies and what you need to consider.

Evidence based medicine - the established way to discover similarities or differences

Decide on the correct level of scientific evidence

Although an RCT is considered the best way to collect scientific evidence, it can also be collected in other ways. Different levels of scientific evidence exist. The lowest level of evidence is the description of a single case. In any other type of investigation involving multiple cases, the single case is the unit. Higher levels of evidence are obtained from observational studies. This can be done by summarising a series of cases, comparisons involving case-controlled or cohort studies. Such studies can be either retrospective or prospective.

Compared to an RCT, the types of studies mentioned are more prone to bias. Scientific journals in the field of reproductive medicine stress the importance of RCT’s and take measures to increase the use of RCT’s as the basis of scientific evidence for publication². An RCT is however not necessarily the only method of investigation. Certain types of information can be difficult to obtain in a prospective way or studies may take so much time that retrospective summaries are the best possible evidence available.

A study can be described in many ways

There are many ways to describe studies. Wording used, in terms of basic terminology, can be a cause of confusion. In EU, regulators describe a clinical study as a clinical investigation, while FDA in the USA refers to clinical trial. The meaning of RCT is quite clear and a good summary is available on Wikipedia. There can be important differences between investigations despite using rather similar wording in the description of the type of study.

Considerations when setting up and performing a randomised controlled trial

Although the definition of RCT is quite clear, performing an RCT in a correct way is quite complex. It involves a number of aspects and requires a lot of preparation. Setting up an RCT starts with defining the study question. Today one can still read published material stating that the aim of the study is to investigate the effect of A on B and C as well as the impact of X on Z. This is not possible in a properly designed RCT.

The basis of an RCT is a single study question with a single primary endpoint. One can of course include other parameters but these are defined as secondary endpoints. Obviously, when discussing the study question in a team, this can be subject of extensive discussions as different approaches will result in different study questions and possibly a different primary endpoint. Once the study question is defined, it is important to know how many cases are required for the RCT. A power calculation is used to determine this number. The information required for the power calculation depends on the study question and may require a calculation for either;

• A superiority trial (is A better than B)

• An equivalence trial (is A as good as B) 

• A non-inferiority trial (is A not worse than B)

For any power calculation, the current and expected success rate for the primary endpoint is required as input, as well as the statistical power and the level of significance. The outcome of the power calculation will tell how many samples (patients) that is required per study group. 

Avoid bias through randomisation

When the required number of patients is known, a randomisation list can be prepared. Proper randomisation is very important to avoid bias. Also here different possibilities exist and it can be worth discussing the optimal method of randomisation for the study question with a statistician.

With the information collected so far, a detailed study plan including the study question, power calculation, randomisation, statistical plan as well as the study protocol can be developed. This documentation, together with additional information such as patient consent form, is part of the preparation for ethical approval.

Authorities approving clinical studies

Approval by a competent authority is required for any clinical study. Depending on regulation, ethical approval may be obtained from a local and or national authority. Besides ethical approval, registration of clinical studies to authorities regulating medicine and medical devices is required in some countries.

When approval by the competent authorities is obtained, the study should be registered in a recognised database before the start of the study, prior to the first patient enrollment to the study. Different databases exist, but the major scientific journals refer to ICMJE for databases accepted for trial registration.

All this work required before actually starting with a study takes a considerable amount of time and effort.

Do not mix up validations with clinical studies

Almost 40 years after the birth of the first IVF baby, IVF now is a routine method and around 1.5 million treatment cycles are performed yearly. Gradually, quality management systems are being introduced in IVF laboratories, which require validation of new methods or products before introducing them in clinical routine.

The quality management system describes how a validation is performed and what the requirements are in terms of numbers of oocytes or patients as well as results. The size of validations largely depends on the size of the clinic. Obviously, a clinic performing around 200 cycles yearly cannot do validations in the same way as a clinic doing 2,000 cycles or more over the same time period. With increasing numbers of cycles performed, it is easier to include larger numbers of cases and obtain a more powerful result.

Also, validations can be performed in different ways avoiding too much impact on clinical routine. For a clinical study the randomisation list has to be followed. This can have substantial impact on daily routine. To minimise this, validations are frequently performed by changing the product or procedure involved on a daily, weekly or monthly basis. Such a way of working should also be considered with the results from the validation analysed and interpreted.

Although validations are important, it is not the same as a clinical study. Even if these findings do not have the same scientific value as an RCT, publication of the results can be very valuable for others.

In conclusion

Today, an RCT is considered the basis of solid scientific evidence. Although an RCT is the best possible scientific evidence, this evidence can also be collected in other ways. For certain aspects, retrospective studies or cohort studies can provide valuable and useful information.

Performing an RCT requires a lot of effort and energy from the whole team. Considering the size of many IVF clinics around the world, running proper RCT’s is not always practical or possible. Collaborations between clinics performing studies in a multicenter way can however be very useful to reach the highest possible level of scientific evidence.

In summary the following needs to be in place to perform an RCT:

• Defined study question and primary endpoint
• Power calculation to obtain the number of cases required
• Randomisation list
• Study plan
• Approval from relevant and competent authorities

Many clinics run validations for introduction of new products or methods. Validations are not the same as clinical studies. Nevertheless, information on performance of products and methods obtained in this way can be of importance for colleagues and should also be incorporated in programs of scientific meetings organised globally.

Learn more

To get more insight into the topic of clinical studies you can watch Hubert´s presentation on the topic. This presentation was filmed at a Vitrolife seminar in 2015.

References