How to Plan your Quantitative Investigation
Aims
- Make a quantitative version of the research question.
- Visualise your quantitative results.
- Outline an analysis plan.
- Estimate the expected effect size, and the sample size required.
- Consider sampling/experimental methods.
Making a Quantitative Research Question
Please view this 20-minute video:
Video not working? Watch on Re:View.
Download the slides for part one.
- Play the Research Data Management Adventure if you're going to conduct an experiment.
- Selecting, Designing and Developing your questionnaire, a good short article.
- Looking for validated questions? Try these places:
- Good resources for experimental design:
- NC3R has good advice and guidelines when animals are involved,
- Experimentation for Improvement - a lively, free online course from McMaster University,
- Design of Experiments (DOE) is a useful tutorial,
- Design of Experiments and Observational Studies - a comprehensive online book.
Plan your Analysis, Data Collection and Sample Size
Please view this 20-minute video:
Video not working? Watch on Re:View.
Download the slides for part two.
- Wikipedia gives a useful summary on effect size.
- A useful online effect-size calculator.
- An introduction to different sampling methods from qualtrics/
- Looking for sampling methods? Try these:
Test yourself
Here are some quick-check questions and answers:
Research Questions
- True or False: "Data from surveys cannot be continuous data but must be ordinal because subjects answer 1,2,3,4 or 5"
- True or False: "A scatter plot illustrates the relationship between two categorical variables"
- True or False: "ANOVA and Regression statistical tests always require normally distributed data."
- False. Several Likert-questions are averaged together as a "scale" to measure attitudes etc. These averages can be considered continuous.
- False. Scatter plots are for two continuous variables measured along the x-axis and y-axis. A stacked bar-chart (or even better a mosaic plot) may be best for two categories.
- False. ANOVA is robust given large group sizes with similar distributions, and regression does not require normal data at all.
Analysis, Collection, and Sizing
- True or False: "Your literature search should provide you with: valid measurement methods, estimates of effect size, and examples of the analysis and presentation of data".
- True or False: "Smaller effects need larger samples to improve the probability of detecting those effects".
- True or False: "Statistical Power and sample sizing calculations are based on the assumption that data will be collected from the population in a random sample"
- True - and also a description of an acceptable data collection/sampling method.
- True.
- True - so sample size calculations are often based on shaky assumptions concerning effect size, and sampling method.
Statistical Advice
- What is the first question you ask your statistical advisor?
- What must you bring, and what is the second question you ask your statistical advisor?
- True or False: " Statistical advisors are experts in survey design, experimental design and sampling methods?"
- Given your research question and measurement methods, what statistical tests might be suitable?
- You must bring an estimated effect size, and ask for help in estimating a suitable sample size.
- False - statistical advisors are applied mathematicians who have acquired knowledge of good and bad methodology through seeing their effects in analytical results.
Survey and experimental design and data collection methods are an area of study distinct from statistical analysis - it is the core business of international companies such as the Rand Corporation and A.C.Nielsen.
Getting Help
We recommend www.statstutor.ac.uk and MASH Moodle pages for statistical tutorials and introductory videos.
Test yourself
1. How often can you visit the MASH drop-in for Maths and Stats help?
1. Every week-day Find out more on our webpages.