Data methods in optometry part 10: non-linear regression analysis.

IN PREVIOUS articles in this series, (1,2) the use of correlation and regression methods to analyse the linear relationship between two variables was described. Hence, Pearson's product moment correlation coefficient (r) establishes whether there is a significant linear correlation between two variables X and Y. (1) Once a linear correlation between two variables has been established, a regression line can be fitted to the data by the method of least squares to describe the relationship in more detail. (2) Linear regression may be adequate for many purposes but some variables in optometry and vision sciences may not be connected by such a simple relationship. The discovery of the precise relation between two or more variables is a problem of curve fitting known as 'non-linear' or 'curvilinear regression' and the fitting of a straight line to data is the simplest case of this general principle. Curve fitting may be appropriate in a variety of experimental circumstances. For example, an investigator may be interested in estimating the amplitude of accommodation (AoA) from defocus curves, (3) in studying processes involving 'learning' of visual functions, or studying the thresholds of Vernier acuity with age in infants. (4) Non-linear regression is a complex statistical topic and often not included in statistical text books. This introductory article describes only the common types of non-linear relationships likely to be encountered. Three types of curve fitting are described, viz., curves that can be transformed to straight lines, the general polynomial curve, and curves that can only be fitted by more complex methods such as non-linear estimation. (5)

[FIGURE 1 OMITTED]

Common types of curve

There are four common types of curve that most often occur in research and these are illustrated in Figure 1. The first curve is the compound interest law or exponential growth curve and is the type of curve often exhibited by the growth of a bacterial population during an eye infection. In all of the equations describing non-linear relationships, the lower case letters 'a', 'b', 'c' etc., describe parameters to be estimated. Hence, the exponential curve is described by the relation:

Y = [aexp.sup.bx] ........ (1)

The second curve is the exponential decay curve where Y declines to zero from an initial value and represents the way in which quantities often decay or decline with time. For example, Ridell et al (6) investigated two methods of determining visual acuity in infants, viz., the sweep visual evoked potential (sVEP) and Teller acuity cards (TAC). The ratio of the sVEP to TAC decreases exponentially with the age of the infant ie the greatest difference between methods is observed for the youngest infants and is at a minimum at 6 months of age. The exponential curve is given by the relation:

Y = [aexp.sup.bx] ........ (2)

The third type of curve is the asymptotic curve, which increases from a value 'a - b' and then steadily approaches a maximum value 'a' known as the asymptote. Many visual processes involving learning may show an increase to a threshold level and be fitted by an asymptotic curve. The asymptotic curve is given by the relation:

Y = a - [bexp.sup.-cx] ....... (3)

Finally, the fourth type of curve is the logistic growth law, the most common sigmoid type curve and a relationship, which has played a prominent part in the study of the growth of human populations. Hence, the initial stage of growth is approximately exponential but then as saturation approaches, growth slows down and approaches a maximum value. This type of relationship has also been used in research involving neural networks and in modelling the growth of tumours. The logistic curve in its simplest form is given by the relation:

Y = 1/(1 + [e.sup.-x]) ........ (4)

Types of curve fitting

Three approaches to curve fitting will be discussed in this article, viz., the fitting of certain curves that can be reduced to straight lines by transformation of the Y or X variable, fitting a polynomial curve in X which is often a good approximation to a more complex curve, and the asymptotic curve and logistic curve which require more complex procedures.

[FIGURE 2 OMITTED]

The exponential growth curve

Simple growth phenomena in which the rate of increase is proportional to the size already achieved can be described by an exponential curve. To fit such a curve, the data are transformed to logarithms. Hence, logarithms of the Y values are taken to the base 10 and Log Y is plotted against X. If the original relationship is exponential, the graph of the transformed values will be linear and a straight line can be fitted using the methods of linear regression described previously. (2) Fitting such a regression by transformation of a variable makes similar assumptions to those described previously for linear regression.) (2)

The exponential decay curve

In some circumstances, it is useful to be able to test whether a quantity decays or declines exponentially. An example of fitting a negative exponential curve is shown in Figure 2. Retinoblastoma is a malignant eye tumour of children with a proportion of cases being inherited as an autosomal dominant gene. (7) The developing tumour can extend into the vitreous without detaching the retina or it may extend beneath the retina and ultimately detach it from the pigment epithelium. The tumour is composed principally of undifferentiated neuroblastic cells, which have large nuclei but little cytoplasm, and they form distinct 'collars' around prominent blood vessels in the retina. It is possible to test whether the density of malignant ceils declines exponentially with distance from the blood vessels. After sectioning the tumour, a number of prominent blood vessels were chosen at random and under the microscope, the density of cancer cells was measured at different distances from each vessel, the data from all blood vessels being combined for data analysis. The data are plotted on the original scale and on a log scale in Figure 2. A linear regression appears to fit the data of LogY against X well, suggesting that the number of tumour cells does decline exponentially with distance from the blood vessels. This result is not particularly surprising as tumour cells at a distance greater than 100[micro]m from a blood vessel often degenerate and become necrotic as they require nutrients which diffuse from the blood vessels for continued growth, and a diffusional process is likely to result in a negative exponential decline in nutrients from the source. (8)

The second degree polynomial

In some circumstances, an investigator may have no knowledge of the theoretical relationship connecting Y and X but it may still be necessary to fit a curve to the data eg to be able to predict Y for a new value of X. If simple curvature is present, a second-order (quadratic) polynomial is often an adequate fit to the data. For example, the two variables in Figure 3 appear to be negatively correlated and a straight line of negative slope may provide a reasonable fit to the data. In this example, however, the data also exhibit a degree of curvature and the question may arise as to whether a curvilinear regression would fit the data better than a straight line. This question can be answered using analysis of variance (ANOVA) and the method is described in Snedecor & Cochran. (3) Essentially, a straight line is fitted to the data and an ANOVA carried out to obtain the sums of squares (SS) of the deviation from a linear regression (Table 1). (2) A second-order polynomial curve is then fitted to the data and a second ANOVA carried out to obtain the SS of deviations from the curved regression. The difference between the linear and curvilinear SS measures the reduction in SS of the Y values achieved by fitting the curvilinear rather than the linear regression. This difference is then tested against the deviation from the curved regression using an F test. If the F ratio of the mean square reduction in SS to the mean square of the deviation from a curvilinear regression is significant, then the curved relationship is a significantly better fit to the data than a straight line. Parabolic regressions will often work well for estimation and interpolation within the range of the data even if the actual relationship between Y and X is not strictly parabolic. Extrapolation beyond the data for estimation is, however, extremely risky. If several values of Y are available at each X, then the goodness of fit of the line to the data can be tested more rigorously. (5)

Fitting a general polynomial curve

The method of fitting a quadratic polynomial can be extended to polynomials of higher degree. Hence, polynomials of order 1, 2, 3 ... n, can be fitted successively to the data and the addition of each extra term adds a further 'bend' to the curve. Hence, cubic curves are 'S' shaped and quartic curves have three 'bends'. With each fitted polynomial, the regression coefficients, standard errors (SE), [R.sup.2] a measure of how well the regression fits the data points, values of t, and the residual mean square are obtained. From these statistics, a judgement can be made as to whether a polynomial of sufficiently high degree has been fitted to the data.

There are various strategies that can be employed to decide which polynomial curve actually fits the data best. First, as each polynomial is fitted, the reduction in the SS is tested for significance. The analysis is continued by fitting successively higher order polynomials until a non-significant value of F is obtained. The final polynomial giving a significant F is then chosen as the curve most likely to fit the data.

Second, it may be obvious that a simple relationship such as a linear or quadratic polynomial would not fit the data and that a more complex curve is needed. In this case, examination of the value of [R.sup.2] may give an indication of the correct polynomial to fit. Subsequently, F tests can be used to choose the most parsimonious model. An example of this type of analysis is shown in Table 2. Characteristic of the brain pathology of Creutzfeldt-Jakob Disease (CJD) is the development of vacuolation (spongiform change) within the cerebral cortex resulting from the death of neurons. An investigator wished to determine how the vacuolation varied across the primary visual cortex (area V1) from pia mater to white matter. The specific objective was to determine which cortical laminae were significantly affected in area V1 and therefore, which aspects of visual processing were likely to be impaired. (9) To obtain the data, five traverses from the pia mater to the edge of the white matter were located at random within area V1. The vacuoles were counted in 50[micro]m x 250[micro]m sample fields, the larger dimension of the field being located parallel with the surface of the pia mater. An eyepiece micrometer was used as the sample field and was moved down each traverse one step at a time from the pia mater to the white matter. Histological features of the section were used to correctly position the field. Counts from the five traverses were added together to study the vertical distribution of lesions within area V1. The values of [R.sup.2] suggested that a quartic curve might give the best fit to the data. A linear, quadratic, cubic, and quartic polynomial was then fitted successively to the data. At each stage, the goodness of fit of the polynomial to the data was tested using ANOVA. The analysis (Table 2) suggests that the linear, quadratic, and cubic polynomials were not significant. However, the quartic polynomial was significant (F = 11.67, P < 0.01) suggesting a complex curved relationship between the distribution of the vacuoles and distance below the pia mater consistent with the vacuoles affecting specific cortical laminae. Incidentally, the fit to the fifth-order polynomial (not shown) was also not significant confirming the 'quartic' as the best fit.

Third, Gupta et al. (3) calculated AoA from defocus curves also using a general polynomial curve fitting method. In this application, however, the objective was to obtain the best estimates of AoA from the defocus curve. Hence, polynomial curves varying from the 5th to the 10th order were fitted to the data and curve of best fit selected on the basis of visual inspection and the highest possible regression coefficient obtained.

A further problem that can arise in deciding which non-linear regression fits the data is to be aware of 'overfitting'. Fitting many curves to the same data to discover which fits the data best may result in making too many statistical tests. In this case, the probability that a particular F value is significant may be less than P = 0.05 and a correction of the P values may be necessary using Bonferroni's inequalities. (5)

Non-linear estimation methods

There are a number of curves that cannot be reduced to linear relationships by data transformation or are not well fitted by a general polynomial-type curve. Examples of such curves include the asymptotic regression and logistic growth curve described previously. There are many circumstances in which a quantity may increase (or decrease) and gradually approach a threshold or asymptotic quantity. Carkeet et al. (4) for example, studied Vernier acuity and resolution in infants to estimate the age at which thresholds were twice those of asymptotic levels and the asymptotic level itself (TA) by fitting an asymptotic curve. In addition, many visual processes involving learning may show 'increase to threshold' behaviour. For example, saccadic latency, i.e. the time delay between the sudden presentation of a visual target and the start of eye movement to look at the target, is likely to reflect higher-order neural activity. (10) Under most conditions, latency distribution is consistent with a model in which the decision signal (log likelihood) increases from an initial to a threshold level. The initial level is an estimate of the log prior probability that a target requiring a response is present, the average rate of rise, the rate of arrival of information about the target, and the threshold (asymptote) is influenced by the urgency with which a response is required. (11)

[FIGURE 3 OMITTED]

[FIGURE 4 OMITTED]

Asymptotic or logistic regressions are best fitted using statistical software employing non-linear estimation methods. Non-linear estimation is a general curve fitting procedure that can estimate any kind of relationship. As an example of the method, we fitted an asymptotic regression to a dose-response curve using the software Statistica (Statsoft Inc., Tulsa, OK, USA). Patients often exhibit a characteristic response curve to increasing doses of a drug, the response increasing with dose until a saturation level is reached. The data are listed in Table 3. To fit this type of curve using non-linear estimation requires that the equation of the relationship is known (see equation 3). The curve increases from a value 'a - b' to an asymptotic value 'a', the parameter 'c' describing the rate of increase of the curve.

The equation for the curve is entered into the software and the least squares method selected as the non-linear estimation method. There are a number of methods available for non-linear estimation, the least squares method being often the most widely used. Reasonable estimates of the 'starting values' for the parameters 'a - b' and 'a' are also entered. The statistical significance of the parameters 'a', 'b', and 'c' are shown in Table 3 and the resulting fitted asymptotic curve in Figure 4; the value of [R.sup.2] indicating an excellent fit to the data. This method can be used to fit any model whose mathematical equation is known and will provide estimates of the defining constants and test how good a fit the curve is to the data.

Conclusions

The techniques associated with regression, whether linear or nonlinear, are some of the most useful statistical procedures that can be applied in clinical studies in optometry. In some cases, there may be no scientific model of the relationship between X and Y that can be specified in advance and the objective may be to provide a 'curve of best fit' for predictive purposes. In such cases, the fitting of a general polynomial type curve may be the best approach.

An investigator may have a specific model in mind that relates Y to X and the data may provide a test of this hypothesis. Some of these curves can be reduced to a linear regression by transformation, eg the exponential and negative exponential decay curves. In some circumstances, eg the asymptotic curve or logistic growth law, a more complex process of curve fitting involving nonlinear estimation will be required.

References

See www.optometry.co.uk/references

Richard Armstrong BSc, DPhil and Frank Eperjesi BSc, PhD, MCOptom, DOrth, FAAO

Table 1
Analysis of variance (ANOVA) to test the departure of a set
of data from a linear regression. DF = degrees of freedom,
MS = Mean square, F =Variance ratio, [R.sub.2] = Sums of
squares (SS) due to linear regression, [R.sub.2] = SS due to
quadratic regression, Q = Mean square of quadratic term, DO
= Mean square deviation from a quadratic regression

Source       SS                    DF    MS                    F

Linear       [R.sub.1]             1
regression

Quadratic    [R.sub.2]-[R.sub.1]   1     [R.sub.2]-R.sub.1]/   Q/DQ
term                                     1 = Q

Deviations   [Y.sub.2]-[R.sub.2]   N-3   [Y.sub.2]-R.sub.2]/
from                                     3 = Q
quadratic

Table 2
Fitting a general polynomial type curve. Data are the
density of vacuoles across the visual cortex from pia mater
to white matter in a case of Creutzfeldt-Jakob disease
(CJD). DF = degrees of freedom, SS = Sums of squares, MS =
Mean square, F = Variance ratio, [R.sup.2] = multiple
correlation coefficient, ns = not significant,
*** = P < 0.001

Distance below   Number of      Distance below   Number of
pia mater        vacuoles (Y)   pia mater (um)   vacuoles (Y)
([micro]m) (X)                  (X)

50               3              550              22
100              31             600              20
150              29             650              25
200              28             700              21
250              25             750              19
300              22             800              17
350              19             850              25
400              18             900              14
450              16             950              16
500              22             1,000            10

Analysis of variance:

Source                     DF   SS        MS       F          [R.sup.2]

Total variation            19   825.8
Reduction to linear        1    72.782    72.782              0.09
Deviations from linear     18   753.018   41.834   1.74 ns
Reduction to quadratic     1    80.391    80.391              0.16
Deviation from quadratic   17   672.627   39.566   2.03 ns
Reduction to cubic         1    77.004    77.004              0.16
Deviation from cubic       16   595.62    37.23    2.06 ns
Reduction to quartic       1    327.7     327.7               0.49
Deviation form quartic     15   267.92    17.86    18.35 ***

Table 3
The response of a patient to increasing doses of a drug

Dose of drug (X)         Response of patient (Y)
1                        3.4
2                        24
3                        54.7
4                        82.1
5                        94.8
6                        96.2
7                        96.4
8                        96.5
9                        96.6
10                       96.7

Estimation of parameters

Parameter    Estimated      Standard       't'
             value          error

'a'          103.00         5.25           19.60
'b'          165.65         17.59          9.42
'c'          0.45           0.08           5.32