Numerical Differentiation | Sahithyan's Notes

Approximate numerical formulae can be derived using Taylor series.

For all the definitions below, $h$ is a small positive number, and is called the step size.

First Order Derivative

Taylor series expansion of $f(x_0+h)$ around $x_0$ , truncated after $1^{\text{st}}$ derivative.

f'(x_0) = \frac{f(x_0+h) - f(x_0)}{h} + \frac{h}{2!} f^{(2)}(\epsilon)

Error is bounded by:

\frac{M \lvert h \rvert}{2}

Forward Difference Formula

f'(x_0) = \frac{f(x_0+h) - f(x_0)}{h} + O(h)

Here $M$ is a bound on $f^{(2)}(x)$ between $x_0$ and $x_0 + h$ .

Backward Difference Formula

f'(x_0) \approx \frac{f(x_0) - f(x_0 - h)}{h} + O(h)

Here $M$ is a bound on $f^{(2)}(x)$ between $x_0$ and $x_0 - h$ .

Centered Difference Formula

Also known as three-point mid point formula. Truncated after $2^{\text{nd}}$ derivative.

f'(x_0) = \frac{f(x_0 + h) - f(x_0 - h)}{2h} + O(h^2)

Error is in order of $O(h^2)$ which is better than $O(h)$ .

Second Order Derivatives

Second Forward Difference Formula

f^{(2)}(x_0) = \frac{1}{h^2} \Big[ f(x_0 + 2h) - 2f(x_0 + h) + f(x_0) \Big] + O(h)

Second Backward Difference Formula

f^{(2)}(x_0) = \frac{1}{h^2} \Big[ f(x_0) - 2f(x_0 - h) + f(x_0 - 2h) \Big] +O(h)

Second Centered Difference Formula

f^{(2)}(x_0) = \frac{1}{h^2} \Big[ f(x_0 + h) - 2f(x_0) + f(x_0 - h) \Big]+ O(h^2)

Higher Order Derivatives

For $n$ -th derivative, Taylor series is truncated after $n^{\text{th}}$ derivative.

n-th Forward Difference Formula

f^{(n)}(x) =\frac{1}{h^n} \sum_{i=0}^{n} (-1)^{n-i} \binom{n}{i} f(x + ih)+ O(h)

n-th Backward Difference Formula

f^{(n)}(x)=\frac{1}{h^n} \sum_{i=0}^{n} (-1)^i \binom{n}{i} f(x - ih)+ O(h)

n-th Centered Difference Formula

f^{(n)}(x)=\frac{1}{h^n} \sum_{i=0}^{n} (-1)^i \binom{n}{i} f\left(x + \left(\frac{n}{2} - i\right)h\right)+ O(h^2)

Accuracy

The accuracy of these divided difference formulas can be increased by including additional terms in the Taylor series. In each step the accuracy is improved by a power of $h$ .

f'(x_0) = \frac{1}{h} \left[ -\frac{3}{2} f(x_0) +2f(x_0+h) -\frac{1}{2}f(x_0+2h) \right] +O(h^2)