Introduction
Numerical methods provide approximate, computational solutions to problems (root-finding, integration, differential equations) that often cannot be solved exactly in closed form, or that must be evaluated automatically by a computer. Carrying this chapter’s highest syllabus weight, it rewards a systematic approach: for each method, know its purpose, its formula, the assumptions required for it to work, and its typical error behaviour.
1. Errors and approximation
Round-off error arises from a computer’s finite precision in representing numbers; truncation error arises from approximating an infinite or continuous process (like a derivative or an infinite series) with a finite calculation. Understanding which type of error dominates in a given method, and how it behaves as step size or iteration count changes, is essential to judging a numerical result’s reliability.
2. Roots of equations
Bisection method: repeatedly halves an interval known to contain a root (identified by a sign change in the function), guaranteeing convergence but relatively slowly. Newton–Raphson method: uses the function’s derivative to make a much faster (quadratically convergent) correction, xₙ₊₁ = xₙ − f(xₙ)/f′(xₙ), but is not guaranteed to converge from every starting point and fails outright if f′(xₙ) = 0. Choosing between methods trades reliability (bisection) against speed (Newton–Raphson) — often bisection is used to find a rough estimate before switching to Newton–Raphson for rapid refinement.
3. Interpolation and finite differences
Interpolation estimates a function’s value between known data points, commonly via polynomials that pass exactly through the given points (Lagrange or Newton’s divided-difference forms). Finite differences (forward, backward, or central differences between successive tabulated values) provide the building blocks both for constructing these interpolating polynomials efficiently and for numerical differentiation (below).
4. Numerical differentiation and integration
Numerical differentiation approximates a derivative using finite differences (e.g., the central difference formula, [f(x+h) − f(x−h)]/2h, which is generally more accurate than a one-sided forward or backward difference for the same step size). Numerical integration approximates a definite integral by summing the areas of simple shapes fitted to the function: the trapezoidal rule uses straight-line segments between points; Simpson’s rule uses parabolic segments (fitted through pairs of intervals), generally achieving significantly better accuracy than the trapezoidal rule for the same number of points, because a parabola captures curvature that a straight line cannot.
5. Numerical solution of linear systems
Large systems of linear equations (as arise, for example, from discretising a PDE by finite differences) are solved numerically either by direct methods (like Gaussian elimination, which finds an exact solution, subject to round-off error, in a predictable, finite number of steps) or iterative methods (like Gauss–Seidel, which repeatedly refines an initial guess and is often more practical for very large, sparse systems, though it is not guaranteed to converge for every system).
6. Numerical solution of ODEs
Euler’s method takes small steps along the direction given by the ODE’s derivative at each point (yₙ₊₁ = yₙ + h·f(xₙ, yₙ)) — simple to implement but with relatively large truncation error per step, so it requires very small step sizes for accuracy. Runge–Kutta methods (particularly the widely used fourth-order version) achieve substantially better accuracy per step by evaluating the derivative at several cleverly chosen intermediate points within each step and combining them in a weighted average, at the cost of more computation per step — a trade-off (accuracy per step vs. computational cost per step) that recurs throughout this entire chapter and is worth explicitly comparing across every method studied.
Key Takeaways
- Round-off error (finite precision) and truncation error (finite approximation of a continuous process) are distinct error sources, and different numerical methods trade off differently between them.
- Bisection is slow but guaranteed to converge; Newton–Raphson is fast (quadratic convergence) but can fail or diverge depending on the starting point and the function’s derivative.
- Simpson’s rule (parabolic fit) generally outperforms the trapezoidal rule (linear fit) for numerical integration of the same function with the same number of points.
- Direct methods (Gaussian elimination) give an exact finite-step solution to linear systems; iterative methods (Gauss–Seidel) are often preferred for very large systems but are not always guaranteed to converge.
- Euler’s method is simple but requires small steps for accuracy; fourth-order Runge–Kutta trades more computation per step for substantially higher accuracy — the general accuracy-versus-cost trade-off underlying essentially every numerical method in this chapter.