Research design and data sources

Learning objectives

• Distinguish primary from secondary data and quantitative from qualitative methods, and pick the right one for a planning question.
• Explain what the decennial census and the American Community Survey (ACS) each do well, and when to use each.
• Describe common sampling methods and the biases that undermine survey results.
• Explain the difference between validity and reliability, and use triangulation to strengthen findings.
• Choose a research design (descriptive, evaluative, cross-sectional, longitudinal, quasi-experimental) that fits the question.
• Use community-based methods, such as asset mapping and key informant interviews, to understand a place.
• Match planning questions to data sources beyond the census, and to the right census geography.
• Design a survey that avoids common wording problems, and explain how sample size affects the margin of error.
• Recognize the ethical duties that come with collecting data about people.

Key concepts

Start with the question, not the data

Good planning research starts with a clearly stated question: How many housing units will the city need by 2040? Why do residents avoid the new park? Which intersections have the most serious crashes? The question decides what data you need, how precise it must be, and which method fits. A common mistake, and a common exam trap, is to reach for a familiar dataset or method before deciding what you're actually trying to learn.

A research design spells out:

• the question and the decision it supports;
• the unit of analysis (people, households, parcels, census tracts, trips);
• the data sources and how they'll be collected;
• the method of analysis; and
• the limits of what the results can tell you.

12 more sections follow in the full lesson.

Key terms

• Primary data: Data you collect yourself for a specific question.
• Secondary data: Existing data collected by someone else for another purpose.
• Decennial census: The complete count of the U.S. population taken every ten years.
• American Community Survey (ACS): The Census Bureau's continuous sample survey of detailed population and housing characteristics.
• Margin of error: The range around a sample estimate within which the true value likely falls, at a stated confidence level.
• Probability sample: A sample in which every member of the population has a known, nonzero chance of selection.
• Stratified random sample: A probability sample drawn separately from defined subgroups to ensure each is represented.
• Self-selection bias: Distortion caused when respondents choose whether to participate.
• Leading question: A survey question worded to suggest a preferred answer.
• Validity: The degree to which a measure captures what it is meant to capture.
• Reliability: The consistency of a measure across repeated use.
• Triangulation: Confirming a finding with multiple methods or sources.
• Metadata: Documentation describing a dataset's origin, definitions, and limits.
• Independent variable: The factor expected to explain or cause a change in an outcome.
• Dependent variable: The outcome a study tries to explain.
• Longitudinal study: Research that follows the same cases over time.
• Quasi-experimental design: An evaluation design that compares cases with and without a program when random assignment isn't possible.
• Asset mapping: Inventorying a community's existing strengths, organizations, and resources.
• Key informant interview: An interview with someone who has deep knowledge of a community or issue.
• Census tract: A small, relatively permanent statistical area within a county, typically home to a few thousand people.
• Double-barreled question: A survey question that asks about two things at once.
• Response rate: The share of people contacted for a survey who respond.