Research methods common mistakes

Common mistakes

• Confusing Field and Natural Experiments

  Students often describe field experiments as having no manipulation or natural experiments as having controlled settings.

  Fix itClarify that field experiments involve deliberate manipulation in a real‑world context, whereas natural experiments rely on naturally occurring events with no researcher control over the independent variable.

• Confusing Covert with Overt Observation

  Assuming that covert observation always involves hidden cameras or technology, and overlooking the fact that it can simply be a researcher remaining unseen.

  Fix itCovert observation is defined by the lack of participant awareness, not by the use of technology. The researcher can remain physically present but hidden, or use discreet recording devices.

• Assuming all self‑report data are accurate

  Students often claim that self‑report techniques provide perfectly reliable data, ignoring issues such as social desirability bias and lack of insight.

  Fix itHighlight that self‑report data can be influenced by participants’ motives, memory, and interpretation, and that researchers should use techniques (e.g., anonymity, validated scales) to mitigate these problems.

• Assuming causation from correlation

  Believing that a significant correlation automatically means one variable causes the other.

  Fix itExplain that correlation does not imply causation; additional evidence such as experimental manipulation or longitudinal data is needed to establish causal links.

• Confusing content analysis with thematic analysis

  Assuming that content analysis is purely qualitative and thematic analysis is purely quantitative.

  Fix itContent analysis is quantitative (counts and frequencies) but can incorporate qualitative coding; thematic analysis is primarily qualitative, focusing on identifying themes without systematic counting.

• Confusing aims with hypotheses

  Students often write a hypothesis as the aim or vice versa, leading to vague or untestable statements.

  Fix itEnsure the aim is a general research goal, while the hypothesis is a specific, testable prediction derived from the aim.

• Confusing random and systematic sampling

  Assuming systematic sampling is the same as random sampling because it involves selection from a list.

  Fix itExplain that random sampling gives each individual an equal chance, whereas systematic sampling selects at regular intervals, which can still introduce bias if the list is ordered.

• Confusing Reliability with Validity

  Assuming that a questionnaire is reliable simply because it shows high internal consistency.

  Fix itHigh internal consistency indicates reliability but does not guarantee that the questionnaire measures the intended construct. Validity must be assessed separately, for example through content, criterion, or construct validity evidence.

• Confusing Peer Review with Replication

  Assuming that peer review guarantees that a study has been replicated.

  Fix itPeer review evaluates the quality of a single study; replication involves repeating the study to confirm its findings.

• Confusing data type with measurement scale

  Assuming that any numerical data is automatically quantitative.

  Fix itNumerical coding can be used for categorical variables (e.g., gender coded 1, 2). Such data are qualitative in nature because they represent categories rather than continuous measures.