Original ArticleA simple formula for the calculation of sample size in pilot studies
Introduction
A pilot study can be defined as a small-scale study that helps to examine the practicality and feasibility of the methods to be used in a subsequent larger and more comprehensive investigation [1]. Because conducting an adequately powered study often requires the inclusion of a large number of participants and therefore may be very costly in terms of time and money, piloting a study on a smaller scale can help to identify unforeseen problems that could compromise the quality or flow of the study [2].
For example, one may encounter nonanticipated reasons why potential participants have to be excluded, questionnaire items that are interpreted in unintended ways by the participants or whose answer options are not sufficiently comprehensive, or unclear information about the delivery of the intervention (eg, dosing or visiting schedules).
If such problems are discovered during the course of a pilot study, the necessary steps can be taken before the actual large-scale study is started to minimize or entirely avoid their negative impact. For example, the study protocol or materials (eg, questionnaire items) could be adapted accordingly, or contingency plans could be set up ahead of time to handle any problems adequately and in a timely manner. However, this is only possible if such problems are actually discovered during the conduct of the pilot study. Therefore, a pilot study aimed at discovering such problems should have sufficient power to do so, or in other words, the sample size of a pilot study must be sufficiently large, such that the probability of detecting such problems is high.
Existing methods for sample size calculations typically focus on how to select an appropriate sample size for a pilot study such that various parameters of interest can be estimated with sufficient precision (eg, the effect size, the standard deviation of the outcome measure, its reliability, or adherence or attrition rates) [1], [3], [4], [5]. Such calculations may also play an important role in deciding whether to proceed with the primary trial in the first place [6], [7]. These considerations have led to various guidelines for choosing an appropriate sample size for a pilot study, such as 12 participants per group [3], values in the range of 10 to 40 participants per group depending on the parameter of interest [4], [5], at least 9% of the main trial's sample size [6], or at least 50 participants [8].
However, none of these approaches is directly applicable when the goal of a pilot study was the detection of unforeseen problems. Therefore, in this article, we take a different approach and describe a simple method for determining the sample size necessary to identify problems with a chosen level of confidence in pilot studies.
Not surprisingly, the sample size determined in this manner depends not only on the confidence level with which we would like to detect a particular problem but also the actual probability that the problem manifests itself in a potential study participant. Because the true problem probability is unknown in practice, what we really need to consider is a lower bound for the problem probability: if the true probability is in fact this low (or higher), then we achieve (or exceed) the desired confidence level, but if it is really lower (so that we are more likely than desired to miss the problem), then it would be infrequent enough to not be considered a problem worthy of detection. Choice of this value therefore depends on the context and on how detrimental a problem would be to a trial. We will return to this issue further in the following.
Section snippets
Required sample size to detect a problem in a pilot study
For now, assume that a particular problem has a given probability of occurring in a potential study participant. For example, if there is a 0 probability of encountering unanticipated reasons for exclusion in a given participant, then there is 0 probability that this problem does not manifest itself. In a group of participants, there is then a 0 probability that the problem will not occur at all. Therefore, the probability of observing at least one occurrence of this problem in
Sample size calculations in practice
To use Equation (1) for deciding on a sample size for a pilot study, we must first choose values for and . By convention, we could adopt a confidence level of 95% (analogous to the level commonly chosen for confidence intervals) or choose the confidence level in accordance with the severity of the issue that we want to detect. (ie, a potentially disastrous problem that could ruin the entire study should be detected with higher confidence than a problem that would merely be a nuisance to deal
Discussion
Despite the extensive levels of planning involved in the design of a study, experience shows that unforeseen problems can and typically will arise during the conduct of a study that must be handled with care. A pilot study provides an excellent opportunity to uncover such problems ahead of time, minimizing the need to adapt procedures or to develop contingency plans on short notice when the larger study is being conducted. In this article, we described a simple method for choosing a sample size
References (14)
- et al.
Sample size calculations for pilot randomized trials: a confidence interval approach
J Clin Epidemiol
(2013) - et al.
The size of a pilot study for a clinical trial should be calculated in relation to considerations of precision and efficiency
J Clin Epidemiol
(2012) - et al.
A tutorial on pilot studies: the what, why and how
BMC Med Res Methodol
(2010) - et al.
Caution regarding the use of pilot studies to guide power calculations for study proposals
Arch Gen Psychiatry
(2006) Sample size of 12 per group rule of thumb for a pilot study
Pharm Stat
(2005)Considerations in determining sample size for pilot studies
Res Nurs Health
(2008)- et al.
Initial scale development: sample size for pilot studies
Educ Psychol Meas
(2010)
Cited by (312)
A long-term case study indicates improvements in floodplain biodiversity after river restoration
2024, Ecological EngineeringFeasibility of a physical activity intervention for children and adolescents with anxiety and depression
2024, Pilot and Feasibility StudiesFeasibility and preliminary efficacy of iConquerFear: a self-guided digital intervention for fear of cancer recurrence
2024, Journal of Cancer SurvivorshipNew Technique for Wedge Selection in Direct Class II Restorations: A Pilot Study
2024, Journal of Clinical Medicine
W.V. derived the equation. W.V. wrote the first draft and produced the figure. All authors contributed to revising the article and approved the final version. W.V. is the guarantor.
Conflict of interest/Financial disclosure: Neither conflict of interest nor financial support regarding the present study.