Designing Experiments: Principles and Methods

Introduction to Experimental Design¶

In the previous chapter, we explored the various ways researchers compare models with real-world systems. The diversity we encountered suggests a crucial insight: there is no universal approach to planning experiments. The techniques and methodologies researchers employ necessarily depend on specific circumstances and objectives.

Despite this diversity, certain fundamental principles remain valid across virtually all experimental situations. Perhaps most important among these is keeping your experimental purpose clearly in mind: the fundamental requirement in experimentation, regardless of what else is happening, is to compare the properties of a physical system with the properties of one or more theoretical models.

Testing an Existing Model¶

Remember that determining whether a model is appropriate for a given system must be based on experimental evidence. We aren’t attempting to decide whether models are “true” or “false” in some absolute sense—all models are imperfect approximations. Rather, we need to determine if a particular model is adequate for our specific purposes at our desired level of precision.

Since conventional graphs are two-dimensional, we initially need to limit ourselves to examining relationships between two variables at a time. When dealing with multiple input variables, we can simplify by holding all but one constant while studying how the output variable depends on the remaining input variable. After completing this analysis, we can adjust one of the previously fixed variables and repeat the process. Through successive measurements of this kind, we can construct a comprehensive picture of the system’s behavior.

For now, assuming we’re working with a single input variable (either because only one exists or because we’ve isolated one by controlling the others), our procedure is straightforward: measure how the output variable changes with the input variable, then plot these measurements for comparison with the model’s predictions. As noted earlier, the advantages of linear representation are so significant that we’ll focus primarily on transforming our data into straight-line form.

Converting Equations to Straight-Line Form¶

Basic Transformations¶

Consider a function describing the time of fall for an object:

To represent this in linear form:

We might choose:

Multiple valid transformations often exist for a given equation. The function above could be equivalently expressed as:

While convention often suggests plotting input variables horizontally and output variables vertically, there’s no strict requirement to do so. Choose the representation that best serves your analytical purposes.

Practical Considerations¶

For example, when analyzing the period of a physical pendulum, the equation is given by:

Where:

• $T$ = period of oscillation
• $I$ = moment of inertia about the pivot point
• $m$ = mass of the pendulum
• $g$ = acceleration due to gravity
• $d$ = distance from pivot to center of mass

For a compound pendulum with multiple masses at different positions, the moment of inertia becomes more complex:

Where:

• $m_i$ = mass of each component
• $r_i$ = distance from pivot to center of each component
• $k_i$ = length of each component (for extended objects)

A better approach: square both sides of the original equation to get:

Then plot $T^2$ versus $\frac{1}{d}$ for a fixed configuration. This gives a straight line with slope $4\pi^2\frac{I}{mg}$. After measuring the slope, you can calculate the moment of inertia using:

This principle—plot variables in their simplest form and leave arithmetic for the final calculation—will serve you well in experimental design.

Working with Compound Variables¶

Converting this to linear form using single-variable functions of h and T proves impossible. However, using compound variables makes it possible. Starting by squaring both sides:

Multiplying both sides by h:

Rearranging to isolate h²:

This gives us a linear equation where:

• Vertical variable = h²
• Horizontal variable = T²h
• Slope = g/4π²
• Intercept = -k²

Compound variables also prove valuable with multiple input variables. When measuring specific heat using flow calorimetry, the heat balance equation is:

Where $Q$ is heat generation rate, $m$ is mass flow rate, $C$ is specific heat, and $\Delta T$ is temperature difference.

If plotting with compound variables reveals unexpected patterns (scattered data or nonlinearity), you can always revert to plotting individual variable pairs to investigate further.

Logarithmic Transformations¶

Logarithmic plotting applies to simple power relationships too. For:

Taking logarithms:

Plotting log y versus log x (a “log-log plot”) yields a straight line with slope n.

For instance, if measurements follow y = x¹·⁸ rather than y = x², plotting y versus x² would show systematic deviation from linearity without revealing the true relationship. A log-log plot would still produce a straight line, immediately indicating a power relationship, with the slope revealing the actual exponent (1.8).

We’ll explore log-log plotting further when discussing empirical model construction in the next chapter.

Step-by-Step Experimental Planning¶

The planning process includes:

1. Identify system and model: This seemingly obvious step can be surprisingly challenging. The phenomenon under study is often surrounded by measurement apparatus, obscuring the fundamental system. If you struggle to identify your system, ask: “What entity’s properties does my model describe?”

   Similarly, clearly define your model’s limitations. When studying falling objects, will you account for air resistance? Neglecting air resistance isn’t irresponsible—it’s defining one aspect of your model. The experiment itself will reveal whether this simplification is justified at your desired precision level.

2. Select variables: Typically, one quantity presents itself as the natural output variable. If there’s only one input variable, selection is straightforward. With multiple input variables, identify your primary independent variable and vary others in discrete steps.

3. Transform the equation: Put your model equation into straight-line form as described earlier. Remember, multiple valid transformations usually exist. Choose one that serves your purposes effectively. When the equation contains unknown parameters to be determined experimentally, structure your transformation to place these unknowns in the slope rather than the intercept whenever possible. Intercepts are more susceptible to systematic errors from instrument defects.

4. Determine variable ranges: Plan for an input variable range spanning at least a factor of 10. Wider ranges provide better basis for comparing system and model behaviors. While you can’t directly control output variable ranges, carefully consider instrument limitations. Circuit components have maximum current ratings, materials have elastic limits, and sensors have operating ranges. Perform trial measurements to determine input variable ranges that avoid damaging equipment or exceeding measurement capabilities.

5. Consider experimental precision: Begin with a target precision level for your final result. This guides your measurement method selection. A request to “measure g using a pendulum” is meaningless without specifying whether you need 10% precision (achievable with simple equipment in minutes) or 0.01% precision (requiring sophisticated apparatus and days of work).

   With a clear precision goal—say, measuring $g$ within 2%—you can work backward to determine requirements for each component measurement. For a pendulum experiment, if you need $g$ within 2%, you might aim for uncertainties in length ($\ell$) and period-squared ($T^2$) below 1% each.

   If you can measure length with ±1mm uncertainty, the minimum acceptable length measurement would be:

   $$\frac{0.001 \text{m}}{\ell} = 0.01$$

   (23)
   $$\ell = 0.1 \text{ m}$$

   (24)

   Similarly, if timing uncertainty is ±0.2s, and period measurement requires 0.5% precision (for 1% in $T^2$), the minimum timing interval would be:

   $$\frac{0.2 \text{s} }{t} = 0.005$$

   (25)
   $$t = 40 \text{ seconds}$$

   (26)

   This analysis helps ensure all measurements contribute meaningfully to your desired final precision. If any measurement appears limited to uncertainties exceeding your target, you’ll need either more precise measurement methods or a revised precision goal.

The complete experiment design process is illustrated in Appendix A4 with a sample experiment.

Designing Experiments Without Existing Models¶

Even without detailed theoretical understanding, empirical models prove extremely valuable. They help organize thinking about complex systems and enable mathematical operations like interpolation, extrapolation, and forecasting.

Even without established theories, consider any available hints about potentially applicable functions, testing them against your observations. One powerful technique for obtaining such hints is dimensional analysis.

Dimensional Analysis¶

This approach can’t determine dimensionless constants (like π), but it reveals functional relationships between variables.

For example, analyzing the velocity ($v$) of waves on a string under tension ($T$) with mass per unit length ($m$):

Dimensionally:

• $v$: $LT^{-1}$
• $T$ (tension): $MLT^{-2}$
• $m$: $ML^{-1}$

Therefore:

Matching powers:

• For M: 0 = a+b
• For L: 1 = a-b
• For T: -1 = -2a

Solving gives $a= \frac{1}{2}$, $b=-\frac{1}{2}$, yielding:

Difference-Type Measurements¶

Null-Effect Measurements in Physical Sciences¶

For steel wires, you might cut a single wire in half, load one piece while leaving the other unloaded, and measure the difference in length. Both experience identical temperature variations, but only one responds to loading. This approach reveals small effects that would otherwise be lost in environmental noise.

Control Groups in Biological Sciences¶

This approach often requires refinements like placebo treatments and double-blinding (keeping both experimenters and subjects unaware of group assignments) to prevent psychological effects from contaminating results.

Observational Studies with Uncontrollable Variables¶

In such cases, careful observational techniques become crucial. With well-defined systems and models (like celestial mechanics), precise measurements may still allow meaningful conclusions—determining that general relativity better explains Mercury’s orbit than Newtonian mechanics, for instance.

Your best approach: meticulous sampling procedures. Create artificial null-effect measurements by constructing treatment groups under the influence you’re studying and control groups exempt from it but otherwise matched as closely as possible.

When facing such complexity, conventional concepts like “proof” require modification. Mathematical theorems can be proven from axioms, and some physical measurements are certain enough to be considered “proven” (the moon is closer than the sun). But in complex systems with probabilistic effects, “proof” gives way to correlation—statistical relationships between variables that differ fundamentally from direct cause-effect relationships but remain valid for identifying influencing factors.

We’ll examine correlation analysis further in Chapter 6 when discussing experimental evaluation.

Glossary¶

Problems¶

For problems 6-23, state which variables or combinations of variables should be plotted to verify the proposed relationship, and explain how to determine the unknown parameter(s) from your graph (slope, intercept, etc.).