Populations, samples and sampling methods

Sampling determines what a statistical investigation can legitimately conclude. A large sample collected badly may be less useful than a smaller sample selected carefully. Calculations cannot remove bias already built into the data collection.

Prerequisites

You should be able to:

• calculate fractions and percentages;
• round to the nearest integer;
• interpret probability as a measure from $0$ to $1$;
• distinguish categorical and numerical variables.

Review GCSE statistics if these ideas are unfamiliar. After this lesson, continue to presenting and interpreting data.

Population, sample and census

The population is the complete collection of people or items about which information is wanted. A sample is a subset of that population from which data are collected. Each individual person or item that can be selected is a sampling unit.

A census collects data from every member of the population.

These terms depend on the investigation. If a school studies the journey times of its current Year 12 pupils, the population is all current Year 12 pupils at that school. It is not all pupils, all teenagers or all journeys. One pupil is a sampling unit, and the recorded journey time is an observation.

State the population precisely

Include the relevant place, group and time. Saying only “students” is usually too vague to justify an inference.

Census or sample?

Census	Sample
can give information about every population member	usually quicker and cheaper
avoids sampling variation	may permit more detailed measurements
may be expensive or take too long	essential when testing destroys the item
may be impossible for a changing, infinite or inaccessible population	results vary from sample to sample
can still suffer from non response, measurement error and poor questions	can be biased if selection is unrepresentative

A census is not automatically accurate. If many people do not reply, or a faulty instrument is used, observing the whole intended population has not removed the error.

Worked example 1: identify the statistical objects

A manufacturer wants to estimate the mean lifetime of light bulbs made by one production line during July. It tests $80$ bulbs until they fail.

• Population: all bulbs made by that production line during July.
• Sample: the $80$ tested bulbs.
• Sampling unit: one bulb.
• Variable: lifetime, a continuous quantitative variable.
• Why not a census? Lifetime testing destroys the useful life of every tested bulb. Testing all July production would leave nothing to sell and would take too long.

The conclusion should concern July bulbs from that line. It should not automatically be extended to other production lines or months.

Self check 1

A council surveys $300$ households to estimate the proportion of households in the borough that recycle food waste. State the population, sample, sampling unit and parameter of interest.

Answer

The population is all households in the borough. The sample is the $300$ surveyed households. One household is a sampling unit. The parameter is the true population proportion $p$ of borough households that recycle food waste.

Parameters, statistics and sampling variation

A parameter is a numerical feature of a population. It is fixed, but usually unknown. A statistic is calculated from a sample and is used to estimate a parameter.

Population parameter	Sample statistic
population mean, $\mu$	sample mean, $\bar{x}$
population proportion, $p$	sample proportion, $\hat p=\dfrac{x}{n}$
population variance, $\sigma^2$	variance calculated from the sample data

Different random samples usually contain different units, so they produce different values of $\bar{x}$ or $\hat p$. This natural difference is sampling variation. It is not evidence that somebody made a mistake.

Increasing the sample size generally reduces sampling variation, but does not cure systematic bias. Asking $10\,000$ visitors to a fitness website about weekly exercise could still overestimate exercise in the general population.

Worked example 2: statistic and parameter

In a random sample of $250$ voters, $145$ support a proposal. The sample proportion is

$$\hat p=\frac{145}{250}=0.58.$$

The value $0.58$ is a statistic. It estimates the unknown population proportion $p$ of all relevant voters who support the proposal. It does not prove that exactly $58\%$ of the population supports it.

Sampling frames

A sampling frame is a numbered or otherwise identifiable list of all sampling units in the population from which a sample can be selected. Examples include a school register, an employee database or a list of serial numbers.

A useful frame should be:

• complete, with every population unit represented;
• free from duplicates;
• current;
• relevant to the defined population.

If the frame omits part of the population, those units have no chance of selection. This is undercoverage. A telephone directory, for example, excludes households not listed in it and may include obsolete entries.

Not every population has a practical frame. There is no complete list of all fish in a lake or all customers who may enter a shop next month.

Frame and sample are different

The frame is the list from which selection is made. The sample is the set of units actually selected.

Simple random sampling

In a simple random sample of size $n$, every possible group of $n$ population units has an equal chance of selection. Consequently, each unit has selection probability

$$P(\text{unit selected})=\frac{n}{N},$$

where $N$ is the population size.

How to select one

1. Obtain a complete sampling frame of the $N$ units.
2. Assign each unit a distinct number from $1$ to $N$.
3. Use a random number generator or random number table to generate numbers.
4. Ignore numbers outside the range and repeated numbers.
5. Continue until $n$ distinct units have been selected.

Ignoring repeats is important because a sample of people normally selects each person at most once.

Worked example 3: random numbers

A college numbers $80$ students from $01$ to $80$. Reading two digit random numbers gives

$$17,\ 84,\ 03,\ 17,\ 60,\ 00,\ 41,\ 92,\ 28.$$

For a sample of five:

• accept $17$;
• reject $84$ because it is outside $01$ to $80$;
• accept $03$;
• reject the second $17$ because it is a repeat;
• accept $60$;
• reject $00$;
• accept $41$;
• reject $92$;
• accept $28$.

The sample is $17,03,60,41,28$. Selecting the first five numbers shown would not produce five valid distinct students.

Simple random sampling reduces conscious selection bias and supports probability based inference. Its main difficulty is the need for an accurate frame, which may be costly to construct.

Systematic sampling

A systematic sample selects units at a regular interval from an ordered frame. For population size $N$ and desired sample size $n$, calculate the sampling interval

$$k=\frac{N}{n}.$$

When $k$ is an integer:

1. choose the first position $r$ randomly from $1,2,\ldots,k$;
2. select positions

$$r,\ r+k,\ r+2k,\ldots$$

until $n$ units have been chosen.

Worked example 4: systematic positions

A factory produces $1200$ items during a shift and wants a systematic sample of $50$.

$$k=\frac{1200}{50}=24.$$

Suppose a random integer from $1$ to $24$ gives $r=9$. Select items in positions

$$9,33,57,81,\ldots,1185.$$

The last position is

$$9+49(24)=1185,$$

so exactly $50$ items are selected.

Systematic sampling is fast and spreads observations across the frame. However, a hidden periodic pattern can bias it. If every $24$th item is produced just after a machine adjustment, the sample may repeatedly observe an unusual part of the production cycle.

The starting point must be random

Always starting with the first unit makes the procedure systematic but not randomly started. The first $k$ units must have equal chances of providing the start.

Stratified random sampling

A population often contains identifiable subgroups called strata, such as year groups, regions or age bands. In a stratified random sample, the population is divided into non overlapping strata and a random sample is taken from each.

For proportional allocation, the sample size from a stratum is

$$\boxed{n_h=\frac{N_h}{N}n},$$

where $N_h$ is the stratum size, $N$ is the population size and $n$ is the total desired sample size.

The allocations should be whole numbers and add to $n$. Round thoughtfully, then adjust if necessary. Selection within each stratum must still be random.

Worked example 5: calculate stratified allocations

A sixth form has $420$ students:

Group	Population size
Year 12	$240$
Year 13	$180$

For a proportional stratified sample of $70$:

$$n_{12}=\frac{240}{420}(70)=40,$$ $$n_{13}=\frac{180}{420}(70)=30.$$

Randomly select $40$ students from the Year 12 register and $30$ from the Year 13 register. Taking the first names on each register would meet the allocation but would not be random.

Worked example 6: rounding allocations

A workforce has departments of sizes $35$, $50$ and $65$, giving

$$N=35+50+65=150.$$

For a sample of $20$, the exact proportional allocations are

$$\frac{35}{150}(20)=4.666\ldots,$$ $$\frac{50}{150}(20)=6.666\ldots,$$ $$\frac{65}{150}(20)=8.666\ldots.$$

Rounding all three upwards gives $5+7+9=21$, which is too many. Allocate $5,7,8$ instead, giving $20$ in total. The final allocation is as close as possible to the required proportions.

Stratification guarantees representation from every chosen stratum and can reduce sampling variation when strata differ meaningfully. It requires reliable information about stratum membership, and irrelevant strata add complexity without necessarily improving the sample.

Self check 2

A school has $320$ pupils in Key Stage 3, $280$ in Key Stage 4 and $150$ in the sixth form. Find proportional allocations for a stratified sample of $60$.

Answer

The population size is

$$N=320+280+150=750.$$

The allocations are

$$\frac{320}{750}(60)=25.6, \qquad \frac{280}{750}(60)=22.4, \qquad \frac{150}{750}(60)=12.$$

Use $26$, $22$ and $12$, which add to $60$.

Non random sampling methods

Some investigations cannot use a complete frame. Non random methods can be practical, but their selection probabilities are unknown, so bias is harder to assess.

Quota sampling

In quota sampling, the population is divided into groups and an interviewer fills a specified quota from each group. The proportions may resemble stratified sampling, but selection within a quota is not random.

For example, an interviewer may be instructed to question $24$ people aged under $30$ and $16$ aged $30$ or over. The interviewer chooses convenient people until each quota is full.

Advantages include:

• no complete sampling frame is required;
• it is quick and relatively inexpensive;
• specified groups are represented.

Disadvantages include:

• interviewer choice can introduce bias;
• easy to approach people may be overrepresented;
• selection probabilities are unknown.

Stratified is not quota

Both methods divide the population into groups. Stratified sampling selects randomly within each stratum. Quota sampling allows non random choice within each quota.

Opportunity sampling

An opportunity sample, also called a convenience sample, uses units that are readily available. Asking the first $30$ people leaving a library is quick, but library users at that time may differ systematically from the target population.

Opportunity sampling is useful for pilot studies and situations with little time or no frame. Its weakness is selection bias, so findings should not be generalised confidently beyond the observed group.

Voluntary response

In a voluntary response sample, people choose whether to participate, perhaps by answering an online poll. People with strong opinions are often more likely to respond, producing self selection bias.

Voluntary response should not be confused with non response. In voluntary response, participation begins by opting in. In non response, a unit was selected or contacted but did not provide usable data.

Bias and sources of error

A sampling method is biased if it systematically favours some outcomes or parts of the population. Sampling variation is random fluctuation; bias consistently pushes results in a direction.

Problem	What goes wrong	Possible improvement
undercoverage	the frame omits some groups	update or combine frames
non response	selected units do not provide data	follow up at different times and record response rates
voluntary response	people with strong views opt in	select units using a random method
leading question	wording encourages one answer	use neutral, precise wording
interviewer bias	interviewer choice or behaviour affects responses	randomise selection and standardise questions
timing bias	collection time excludes or overrepresents groups	sample across relevant days and times
measurement error	the recorded value differs from the truth	calibrate instruments and use clear definitions

Non response matters when respondents differ from non respondents in the variable being studied. Replacing a non respondent with the nearest convenient person may create further bias because the replacement did not have the intended random selection probability.

Worked example 7: diagnose and improve a design

A gym emails a questionnaire to its current members asking, “Do you agree that our excellent new equipment has improved your fitness?” It uses replies received during the first day to estimate the proportion of all local adults whose fitness has improved because of the gym.

There are several distinct problems:

1. Wrong frame: current gym members do not represent all local adults. Non members cannot be selected.
2. Voluntary response: recipients decide whether to reply, and early respondents may have unusually strong views.
3. Leading wording: “excellent” and “improved” encourage a positive response.
4. Unclear measurement: respondents may interpret “fitness” differently.

A stronger design would define a measurable outcome and target population, obtain an appropriate local resident frame if available, select randomly, use neutral wording and follow up selected non respondents. A larger number of the same biased replies would not solve these problems.

Choosing a method

Match the method to the available information and the investigation.

Situation	Usually suitable	Reason
complete current frame, modest population	simple random	known and equal selection mechanism
ordered production or customer flow	systematic with random start	efficient and well spread
important subgroups must be represented	stratified random	controls subgroup allocation
no frame, but group proportions are known	quota	practical representation of named groups
early pilot with limited resources	opportunity	quick, with limitations stated clearly

No label makes a design automatically good. A systematic sample can fail because of periodicity, and a stratified sample can fail because selection within strata was not random.

Worked example 8: compare two proposals

A college wants opinions from its $900$ students, who study on three campuses of sizes $450$, $300$ and $150$.

Proposal A: ask $90$ students in the cafeteria at the largest campus.

This is an opportunity sample. Students at two campuses have no chance of selection, and cafeteria users at the chosen time may be unrepresentative.

Proposal B: take a proportional stratified random sample by campus.

The allocations are

$$\frac{450}{900}(90)=45, \qquad \frac{300}{900}(90)=30, \qquad \frac{150}{900}(90)=15.$$

Randomly choose $45$, $30$ and $15$ students from the three campus registers. Proposal B represents each campus in its population proportion and has a defensible random selection mechanism.

Self check 3

Name the method and one important limitation in each case.

1. From an alphabetical list of $600$ patients, choose a random start from $1$ to $20$, then every $20$th patient.
2. An interviewer questions any available shoppers until age group targets are filled.
3. Every employee is numbered and $40$ distinct numbers are generated randomly.

Answer

1. Systematic sampling. A periodic pattern in the ordered frame could align with the interval, although alphabetical order is unlikely to have period $20$ for many health variables.
2. Quota sampling. Choice within each age quota is non random and subject to interviewer or availability bias.
3. Simple random sampling. It requires a complete, accurate employee frame and non response may still bias the collected data.

Exam checklist

Before accepting a sampling design, ask:

1. What exactly is the population?
2. What is one sampling unit?
3. Is there a complete, current sampling frame?
4. Does every relevant group have a chance of selection?
5. Is selection random, and where does the randomness enter?
6. Could ordering or timing create a pattern?
7. Could non response, wording or measurement introduce bias?
8. Is the conclusion restricted to the population actually represented?

For method questions, describe a usable procedure rather than merely naming it. For evaluation questions, connect each limitation to the context and explain the likely effect.

Next steps

Once data have been collected:

• choose suitable displays in presenting and interpreting data;
• calculate summaries in averages and measures of spread;
• investigate suspicious observations in outliers and cleaning data;
• distinguish data collection from probability assumptions in probability modelling.