What is Sample Size?
In this article
- What is sample size?
- Why your research type matters
- Study Design Considerations
- Statistical Considerations:
- Sample Size Calculators
Sample size refers to the total number of participants that have taken part in your research study. We refer to this as a ‘sample’ because, for logistics reasons, in research we often focus on a selection of a smaller group of individuals from your
population of interest.
The size of your sample has important implications for the accuracy of your results and therefore the conclusions you can make. If you don’t have enough participants, you are not likely to get an accurate representation of
the population you’re interested in. If you have too many participants, while you are more likely to obtain more accurate results this raises concerns about how practical it is to collect data from a very large group.
It’s therefore important for research that we get the sample size as close to optimal as possible. To do this, several important factors need to be considered.
Research and Sample Size Considerations
A key determining factor for how many participants you need for your study is the type of research you are conducting. Quantitative research involves the collection and analysis of data that can be quantified (or is numerical in nature), whereas qualitative
research involves the collection and analysis of data that is thematic(or is non-numerical in nature).Whether your research is quantitative or qualitative, and therefore the study design you use, depends on your research question.
Qualitative studies will often have smaller sample sizes compared to quantitative ones because of saturation. Saturation occurs when, after a certain number of responses to the same question, no new data are identified. This is common in
interviews or focus groups; typically, after around 15-20 responses, a lower degree of variability between responses becomes apparent (i.e., they become consistent with one another) and no new patterns or trends are observed. Saturation is
also possible in quantitative research, however because there is a higher degree of variability in numbers the threshold for saturation is considerably higher. You will therefore often see qualitative studies report on a smaller, and quantitative
studies reporting on a larger, number of participants.
Study Designs and Associated Practicalities:
We also need to weigh up the practicalities of our study designs and data collection methods.Some study designs, such as the ones outlined in the table below, and data collection methods are easier to coordinate and administer to large groups of participants compared to others.For example, it is much easier to collect data from 500 participants using an online questionnaire or survey compared to conducting 500 one on one interviews, or conducting an experiment with 500 participants in a laboratory environment.
| Type | Definition |
|---|---|
| Longitudinal | Examination of a group of individuals over the course of a specified time frame. |
| Cross-Sectional | Examination of a group of individuals at a specific point in time. |
| Case Study | In-depth investigations of a small number of individuals. |
| Observational | A non-experimental study where the behaviour of individuals is observed and recorded, and not changed/manipulated. |
| Experimental | The systematic, and tailored, manipulation of two or more variables to investigate how the change in one variable influences the other(s). |
| Correlational | Examination of a potential relationship between two variables. |
| Quantitative Data Methods | Involves measurement of numerical data and is analysed using statistical techniques. Common methods include questionnaires, surveys, or experiments. |
| Qualitative Data Methods | Involves measurement of non-numerical data and is analysed by reporting broad themes in the language used. Common methods include interviews or focus groups (text, video, audio). |
The logistics of how you are going to collect the data are important, and can be broken down into three main, and highly interrelated, points:
- Cost: Financially, a larger group of participants means more money needing to be spent to conduct the research.However, funding is finite, and it may not be fiscally feasible to have a large group take part in your study. Relatedly, if taking part in your research comes at too great a cost to your participants, such as giving up a significant portion of their time without reimbursement or the study requirements are too demanding, then they aren’t likely to register their interest and you won’t have a large enough sample.
- Time: The amount of time it takes to participate in your study will be a deciding factor for whether a participant takes part. The greater the time demand on participants, the less likely they are to participate. Shorter studies are therefore easier to recruit for. The same could be said for the researchers – the study may be operating on a specific timeline, and so if there is not enough time to collect a large sample a smaller one will need to be used, and vice versa.
- Adherence: this refers to the commitment needed by participants to take part in research, and is a greater concern for longer, more burdensome studies compared to ones that are shorter and easier in duration. Depending on the length of the study in combination with what they need to do over that timeframe, participants may discontinue their participation and a high attrition (drop out) rate is likely, resulting in a smaller sample.
Statistical Considerations:
Where the aforementioned points relate to how you are going to collect your data, how you intend on analysing your data is equally important. Each statistical test is associated with a minimum sample size required to see a significant effect, and these are based on:
- Significance: in statistics, the p value is the probability or likelihood that the null hypothesis is true (i.e. there is no effect to be observed). If your p value is less than 0.05, this suggests that there is a less than 5% chance that the null hypothesis is correct (i.e. there is over 95% chance that there is an effect to be observed). P values are typically set to 0.05, but sometimes can be 0.01 if we wish to be more conservative.
- Power: Statistical power refers to the probability that a statistical test will correctly reject a false null hypothesis. Generally, power is kept at around 80% or higher. If your study is underpowered because you don’t have enough participants, this increases the likelihood of making an error, particularly:
a. Type I error: where the null hypothesis is incorrectly rejected, i.e. a false positive. This means that results are identified as significant, when they have occurred by chance.
b. Type II error: where the null hypothesis is incorrectly retained, i.e. a false negative. This means that results are identified as not significant when they actually are. Type II errors are reduced by having enough power. - Effect size:Effect size is a numerical index of how much your dependent variable is affected by the independent variable, and determines whether the observed effect (IV on DV) is important enough to translate to the real world. Cohen’s d is normally used for effect size in the context of sample size calculations, where 0.2 is a small effect, 0.5 is a medium effect and 0.8 is a large effect.
Remember, the combination of significance, power and effect sizes will result in different minimum sample sizes that you will need for your study, and will also change depending on each type of statistical test that you use. You will therefore need to check this every time. Calculating your minimum sample size when designing your study and before you start data collection will give you a benchmark of the number of participants you will need. Determining sample size is a relatively complicated, but incredibly important, part of research. It is a delicate balance between the type of research you are conducting, how you are conducting it, and what kind of results you would like to see. While bigger is generally better, remember – each study is unique and it will vary with each study you run!
Helpful References - Sample Size Calculators:
It is possible to calculate your sample size by hand. However, in the age of technology, there is an abundance of online websites or computer programs that you can use, provided you know your margin of error, how much power you need and the effect size you’re after. Some of the more popular programs that are available (some are free, others are subscription based) include:
- G*Power https://stats.idre.ucla.edu/other/gpower/
- PASS: https://www.ncss.com/software/pass/
- nQuery: https://www.statsols.com/nquery/sample-size-software-options