Numerical errors

November 09, 2013

Numerical errors are the by-product of approximation. Approximation arises in various forms, some during the mathematical modelling of the system under investigation, and others during the process of manipulations and calculations using the mathematical model. Hence, depending on the type of approximation used, numerical errors may be classified into the following categories:

1. Errors due to approximation in modelling. When we investigate systems that are complex, where various components of the system interact with one another and with the external environment in various complicated ways, it becomes a necessity to simplify the abstraction, i.e., modelling, of the system if we are to derive useful comprehensible information about the system. This means that we must ignore certain aspects of the system in order to facilitate a focused investigation of the system. For instance, when we model the trajectory of a small missile, we might ignore air resistance as a factor. However, when we model the re-entry of a space shuttle, we cannot ignore air resistance.

By ignoring certain aspects of the system, however, we are limiting the amount of data we are dealing with using the model. Hence, conclusions drawn from such models have inherent errors associated with them; and the only to reduce such errors is to build a more comprehensive model that captures several of the many factors that influence the processes inside the actual system being modelled.

Note here that building a complicated model does not necessarily translate to a more accurate representation of the system. The accuracy of a model depends on its capability to capture the most profound processes observed inside the system. A well design simple model may therefore be a more accurate abstraction than an ill-conceived complex model.

2. Error due to approximation in physical or mathematical constants. A mathematical model uses various physical or mathematical constants, such as $$\pi$$ in models of periodic systems. The value of some of these constants are not known accurately. Hence, a model using these constants are bound to inherit this inaccuracy in the from of numerical errors. The only way we can reduce this type of error is to use the most accurate values that are available

3. Error due to inaccurate measurements. In addition to the various physical and mathematical constants, a model establishes relationship between various parameters in the model. These parameters determine the variation of the model, which in turn reflects the dynamic nature of the system being modelled. While measuring the values of these parameters, it is not always possible to measure the values accurately due to physical limitations. Every measuring instrument is bound by the laws of nature and therefore by the Heisenberg's Uncertainty Principle. The only means to reduce numerical errors contributed by inaccurate measurements is to design instruments that are capable of measuring the parameters more accurately; although, numerical errors of this category may never be avoided entirely.

4. Error due to the representation of numbers. In order to study mathematical models computationally, the constants and the parameters of the model must be stored inside a computer. Since there are various ways to build a computer, the numerical accuracy of the calculations depends on the manner in which the numerical values are represented inside the computer. For instance, if a computer can only store integer values between, say 1 and 10, storing values greater than 10 will result in numerical errors. The only way we can reduce such errors is by choosing a representation that can accommodate any numerical value that may arise during the calculations to the required precision. In the above example, for instance, we can choose a system which can store integer values between 0 and 100 if we know that all of the values will fall within this range.

In The Simpsons, episode Treehouse of Horror VI, Homer Simpson shows a counter-example to Fermat's Last Theorem, which seems to work on a calculator with 10 digits of precision.

$1782^{12} + 1841^{12} = 1922^{12}$

He repeats the feat again in the episode The Wizard of Evergreen Terrace with the following counter-example:

$3987^{12} + 4365^{12} = 4472^{12}$
5. Errors due to mathematical operators. It is not always possible to know the range of numerical values that may be produced by a calculation. Furthermore, it is not always possible to acquire a computer that can accommodate all of the anticipated values of a calculation. In these cases, mathematical operations on the stored values could produce values outside of the range that the computer can accommodate. It becomes imperative, therefore, to introduce mechanisms by which numerical errors due to mathematical operators are reduced. By proper arrangement of the mathematical operations, it is sometimes possible to reduce numerical errors of this category.

6. Errors due to human or system inaccuracy. Finally, every study of a model is bound by the accuracy of the computer used to carry out the calculations and the correctness of the data (both the mathematical model and the parameters) entered by the human. If the system is faulty, neither accurate model nor perfect data can produce accurate results. Similarly, if the stored data is inaccurate, even the most perfect of systems will still produce incorrect results.

Human errors may be reduced by reducing human intervention when the data is transferred to the computer. For instance, by feeding the measurements directly from the measuring instruments. Avoiding errors due to faulty systems is slightly more complicated because one must prove that the computer works as intended. Although it is possible to verify certain aspects of the system, it is impossible to verify a system entirely. In order to verify a system with absolute certainty, we must be certain about the system which is doing the verification.

Continued in Accuracy and precision
This post is based on a draft article I wrote in 27~30 January 2009 to reinforce my understanding of the text in First Course in Numerical Analysis [McGraw-Hill, 1978] by Anthony Ralston.