Evaluating near infrared spectroscopy for field prediction of soil properties.
Near infrared diffuse reflectance spectroscopy (NIR-DRS) has been studied extensively for rapid prediction of soil properties. Most studies have been conducted in laboratories, usually finding that NIR can predict several key soil properties with acceptable accuracy, namely total C content, clay content, and CEC (Chang et al. 2001; Viscarra Rossel et al. 2006). Most studies were conducted with the aid of legacy soil samples with associated laboratory-measured data. Spectral libraries have been built using these data (Brown et al. 2006). On the other hand, field studies also were conducted to investigate the feasibility of using NIR technology for field (Kusumo et al. 2008) and on-the-go measurement of soil properties (Mouazen et al. 2007). There is also debate on the need to derive local calibration functions instead of using functions derived from regional or national databases.
NIR-DRS measurements may not work as well in the field as in the laboratory. Sudduth and Hummel (1993) tested a portable near-infrared spectrometer as an on-the-go method to measure soil properties. The low prediction accuracy with this technique was attributed to movement of the soil past the spectrometer. This may be circumvented by keeping the spectrometer scanning device stable, and by holding the soil stationary while scanning.
Another problem with field measurement is the variation in soil moisture content and surface roughness. Waiser et al. (2007) showed that NIR-DRS is an acceptable technique for rapidly measuring soil clay content in situ for various water contents and parent materials. Their results showed that prediction on unground, air-dried soil samples has a higher accuracy than on field (moist) samples. Smearing the soil surface decreases the accuracy of prediction, but soil aggregates and natural heterogeneity did not degrade in situ predictions. Note that study developed separate calibration functions for each of the soil conditions.
Chang et al. (2005) investigated calibration of NIR-DRS using moist, ground soil samples. They found that total C, organic C, inorganic C, total N, moisture, CEC, and clay content can be predicted with reasonable accuracy for moist soils. They added that prediction using moist samples can be achieved as long as diverse soil samples from the same region are included in the calibration. This also means that calibration samples need to be in a moist condition.
Kusumo et al. (2008) developed a field method for measuring NIR-DRS on a flat, sectioned horizontal soil surface of a soil core. They found that they could predict total C and N accurately in the field despite moisture content differences.
[FIGURE 1 OMITTED]
In the laboratory, NIR-DRS measurements have usually been conducted with air-dried, ground soil samples, removing the variability of the environment. Barthes et al. (2006) found that the most accurate predictions of C and N were obtained with oven-dried finely ground samples.
[FIGURE 2 OMITTED]
This paper investigates whether calibration functions developed from a spectral library can be used for rapid characterisation of soil properties in the field, on a fully independent sample set. Moreover, the samples of this independent set have been submitted to different treatments under various moisture contents. How the validation is affected by these changes is also a target of this paper. Finally, we will also put a scheme on how NIR-DRS can be used as part of digital soil mapping.
Materials and methods
The methods used in this study are summarised in Fig. 1. There are 4 steps: soil sampling; developing a spectral calibration library; spectral measurement and prediction of soil properties; and finally digital soil mapping.
Field soil sampling
The study area is located in the Pokolbin district, NSW, Australia, and comprises ~7[km.sup.2] of predominantly agricultural land. The area is mostly undulating plains with some hills, elevation ranges from 90 to 200 m. We conducted soil sampling based on topography (digital elevation model) using the random toposequence model (Odgers et al. 2008). To generate a toposequence, we first generate a random point and take a random path uphill to the top of a hill and downhill to a stream or valley bottom, and the toposequence is located in a first- or second-order streamshed. We selected 24 toposequences (Fig. 2) and conducted soil sampling along these toposequences. Seven sampling locations are located along each toposequence, where each location is separated by an equal altitude increments. At each sampling location, soil samples were collected using a drilling rig down to a depth of 1 m. Soil cores with a diameter of 48 mm were extracted, morphological descriptions were collected on each sampling location, and samples from the depths of 0-0.10 and 0.40-0.50 m were sectioned and collected. So, in total we had 336 soil samples. The samples collected in the field condition (unground and at field moisture content) were then scanned using a Vis-NIR spectrometer.
The suborders mostly found in this area according to the Australian Soil Classification System are Red and Brown Chromosols; and Red and Brown Dermosols. A small number of Kandosols, Kurosols, Anthroposols, and Calcarasols were also identified.
Collection of Vis-NIR soil reflectance spectra
We used an AgriSpec[TM] instrument with a contact probe (Analytical Spectral Devices, Boulder, CO, USA) for collection of the Vis-NIR soil reflectance spectra. The samples were illuminated by a halogen lamp (4.5 W) and the reflected light was transmitted to the spectrometer through a fibre optic bundle. Spectra (350-2500 nm) were recorded with a 1-nm interval. A Spectralon (Labsphere Inc., North Sutton, NII, USA) was used as a highest reflectance standard and employed to convert raw spectral data to reflectance. Each soil spectrum was obtained as the mean of 40 scans. For the field samples, 4 replicate of scans at different spots on the soil core were performed on each sample.
[FIGURE 3 OMITTED]
After the field samples were scanned with the spectrometer, the samples were weighed and placed in an oven at 40[degrees]C to obtain air-dry soil conditions. The samples were considered air-dried when the mass did not change in 2 consecutive measurements after 24h. Moisture content was calculated from the mass difference between the field and the dried sample. The dried samples were scanned using the spectrometer in the laboratory with 4 replicates. The samples were then ground and sieved to pass through a 2-ram sieve. The sieved samples were scanned again using the spectrometer. Thus, we have 3 soil conditions: field, dried unground, and dried ground.
Spectral data preprocessing
Reflectance data collected from the spectrometer were preprocessed to remove unstable measurements. Data at 350-499 nm and 2451-2500 nm were removed due to their low signal-to-noise ratio, and the remaining 1951 wavelengths were transformed from reflectance to absorbance (log 1/Reflectance). The spectra were then sampled to a resolution of 2nm; smoothing and first derivative were performed using the Savitsky-Golay algorithm with a window size of 11 and polynomial of order 2.
Developing spectral calibration library
We used a library of 316 soil samples from the wheatbelt of southern NSW and northern Victoria (Geeves et al. 1995). The samples were from soil horizons taken up to 1 m depth. They represent Red, Brown and Yellow Chromosols; Red and Brown Dermosols; Red Kandosols; and Red, Brown, Yellow and Black Sodosols.
[FIGURE 4 OMITTED]
In addition, 100 soil samples from the larger area of the Pokolbin irrigation district were incorporated. These samples came mostly from the B2 horizon and were taken using a drilling rig as part of another study (Odgers et al. 2008).
In total we had 416 soil samples with laboratory analyses of physical and chemical properties: sand, silt, clay, total C, pH, EC, and sum of cations. The samples had been ground and passed through a 2-mm sieve. Twenty-five per cent of the data were selected randomly and used for a validation of the prediction accuracy; the rest were used for calibration. The partial Least-squares (PLS) algorithm was used to calibrate the spectra to soil properties.
Digital soil mapping
The predicted soil properties for each sampling location in the landscape were then used to make a map of soil properties for the area. The area has a complete cover of environmental data layers: Landsat TM, land-use from aerial photography, geology, and a digital elevation model and its derivatives. A regression kriging approach was used to predict the soil properties spatially. See Hengl et al. (2004) for details.
To independently validate the prediction of soil C content using NIR spectroscopy, we randomly selected 45 of the field soil samples (28 samples from 0-0.10m and 17 samples from 0.40-0.50m) for laboratory analysis. The soil samples were analysed for total C using the dry combustion method.
Vis-NIR Spectral calibration from spectral library
Table 1 shows the performance of PLS for prediction of soil properties from the spectral library. We found that clay content, total C, and sum of basic cations consistently give a good prediction. Other properties, such as pH, only give fair prediction, while EC was not predicted well. Table 1 also shows that using the first derivative of the spectra produces better accuracy for the PLS model.
[FIGURE 5 OMITTED]
Spectra and calibration convex hull
Principal component analysis was performed on the spectra of the calibration data. We calculated the principal components of the absorbance and first derivative. The first 2 principal components (PCs) form a biplot. A convex hull analysis was then performed on the first PCs. The convex hull biplot area is a representation of the soil variation of the calibration data, and also a way to visualise the similarity and dissimilarity among soil samples from different regions or fields (Islam et al. 2005).
The spectra obtained from field and laboratory were then projected to the first 2 PCs of the calibration spectra. Figure 3 shows the biplot of the absorbance spectra. The first 2 PCs of the calibration spectra capture 96% of the variation of the spectra. The polygon (hull) (Fig. 3) represents the calibration data, while the dots represent the soil samples. The first PC mainly corresponds to the baseline shift due to particle and moisture effect. Different treatments to the soil samples affected the absorbance. The dried ground samples showed the closest similarity to the calibration data, followed by dried unground samples and field samples. The area of the convex hull represents the variation of the spectra and samples. Table 2 shows that field samples had the highest variation followed by dried unground and dried ground samples.
Figure 4 shows the difference in absorbance for topsoil and subsoil samples. Dried ground samples had the lowest absorbance, followed by dried unground. Samples at field condition had the highest absorbance, due to moisture. The prominent peaks around 1400 nm and 1900 nm represent absorption regions for hydroxides (Whiting et al. 2004; Dematte et al. 2006; Kusumo et al. 2008).
Taking the first derivative of the absorbance spectra, most samples are now similar to the calibration spectra (Fig. 5). Taking the first derivative removes the baseline shift effect. Several authors (Chang et al. 2001) have advocated the use of derivatives in order to overcome the problem of the particle effect. We can therefore assume that the PC1 of first-derivative spectra will no longer be affected by baseline shift. The first 2 PC of the derivative spectra represent 87% of the total variation of the spectra. The dried ground and unground samples showed similar spectra signatures, while the field samples had more prominent peaks (Fig. 6). Although the dried ground and dried unground samples look similar, they occupy different regions in the biplot (Fig. 6); the spectra of the unground samples appear to contain more noise and have a sharper peak around 1800 nm (Fig. 7). The area of the biplot convex hull (Table 2) also showed highest variation in field samples, followed by dried unground and dried ground samples.
Prediction of soil properties
The influence of soil moisture on the NIR spectra can be seen mainly at 1400 nm and 1900 nm. Thus, we derived a prediction function from the field spectra to predict soil moisture. A simple linear regression correlated gravimetric soil moisture content (w in % mass) with the height of the absorbance spectra at 1400nm and 1900nm (Fig. 8):
w = 2.49 97 h1400+ 101.7 h1900
([R.sup.2] = 0.62, RMSE = 5.1%)
[FIGURE 6 OMITTED]
where h1400 and h1900 are the height of absorbance from its baseline at 1400 nm and 1900 nm, respectively. We also separate the analysis for topsoil (0-0.10 m) and subsoil (0.40-0.50 m):
Topsoil : w = -3.77 - 161 h1400 + 131 h1900 ([R.sup.2] = 0.62)
Subsoil : w = -3.77 - 16 h1400 + 69 h1900 ([R.sup.2] = 0.65)
The coefficients of the model for the topsoil were bigger than those of subsoil, due to the lower mean of peak height of topsoil. This indicates that the absorptions associated with water are also related to soil properties.
We analysed 45 soil samples for total C to independently validate the NIR prediction. Prediction using dried ground samples fitted closest to measured values, with [R.sup.2]=0.60 (Fig. 9). Prediction using dried unground samples had an [R.sup.2] of only 0.38, and using field samples had an [R.sup.2] of 0.28.
Since the field soil samples showed different spectral signatures from the dried samples and also had a range of gravimetric soil moisture content from 5% to 40%, we could not use the calibration functions which are derived from air-dried soil samples. Thus, we only compared predictions using established calibration functions on air-dried samples which are ground or unground.
[FIGURE 7 OMITTED]
[FIGURE 8 OMITTED]
[FIGURE 9 OMITTED]
Figure 10 shows a comparison of prediction of clay content, total C, and the sum of basic cations. For clay content, although there was a good linear relationship between ground and unground samples, there appeared to be a slight bias, the unground samples tending to under-predict at low clay content. For total C, the unground samples tended to under-predict. The sum of cations appeared unaffected by grinding.
Digital soil mapping
Since the spectra from dried and ground samples most closely resembled the calibration data, and were also found to provide a better correlation with laboratory measured values, we used the prediction from ground dried samples for digital soil mapping. Using regression kriging approach, we produced maps of clay content, total C content, and sum of cations for the area at a grid size of 25m by 25m (Fig. 10).
Table 3 shows the linear model fit for spatial prediction of total C and clay content. The model for topsoil total C accounts for only 32% of variation in the data (Table 1). As a result the regression kriging prediction when validated against independent measured total C at 28 sites is not great, with [R.sup.2] = 0.1.
The topsoil model, using environmental variables for predicting C and clay content, accounts for only 30% of variation of the data, while the subsoil model accounts for only 10% of variation. This spatial prediction indicates a large uncertainty. Nevertheless, the maps produced show variation and trends over the area (Fig. 11).
[FIGURE 10 OMITTED]
[FIGURE 11 OMITTED]
Discussion and conclusions
This study shows that NIR-DRS can behave differently according to soil conditions. Although the prominent peaks appear to be the same, the shape and height of the peak can be different according to the soil's specific surface area and moisture content. Soil moisture has a large effect on the absorbance of the infrared signal, particularly at 1400nm and 1900 nm. This also permits a direct relationship between soil moisture content and absorbance at 1400 nm and 1900 nm. The surface texture of the samples also affects the absorbance. The ground samples appear to have higher reflectance than unground samples.
Unlike previous studies which calibrated the NIR spectra under different soil conditions from measured soil properties, we have asked a practical question of whether a, NIR spectra library developed in laboratory can be used for prediction of samples under field condition. The answer is simply, no.
The implications for field application are the need to calibrate the spectra based on samples collected under field conditions. If soil cores are used to extract the sample, then spectral measurements performed on the core need to be calibrated with laboratory-measured samples. Studies have shown that with appropriate calibration, NIR spectra can be used in the field (Kusumo et al. 2008). Sudduth and Hummel (1993) in a laboratory study showed NIR measurements on a wide range of soil moisture tensions. Although they found best organic C results with dry samples, reasonable estimates were obtained across the full range of tensions. These authors suggested that including a wide range of water contents in the calibration set could that take care of the issue of moisture variation.
However, we argue that soil moisture has the largest influence the on the infrared absorbance; thus, for field measurement the moisture content of the soil needs to be standardised, either dried or moist. In the field, wetting the sample is possibly easier than drying the sample. Performing the NIR measurement on moist samples, similar to reading moist soil colour, will probably be more economical.
From a chemometric point of view, several authors proposed various techniques to overcome this problem, including 'spiking' the calibration library with spectra from field samples (Viscarra Rossel et al. 2009). Including spectra from field samples in the calibration library was hypothesised to 'trick' the PLS so that the calibration equation is not influenced by parts of the spectra which are affected by moisture or particle sizes. A better solution would be to build a calibration in a spectral space which is not affected by water content, i.e. orthogonal to the water effect spectrum. This can be carried out, for instance, by External Parameter Orthogonalisation (EPO) (Roger et al. 2003).
From the results we can conclude that:
* The biplot convex hull can be used to evaluate the similarity of spectra from calibration set to field observation.
* Prediction functions developed from spectral library collected using dried ground samples cannot be used for field condition prediction.
* For field condition prediction, samples need to be calibrated using field condition samples. Wetting samples in the field can be an option to standardise the moisture potential, and from a practical viewpoint is more efficient than drying the samples.
* NIR-DRS is a useful part of digital soil mapping.
This work is supported by ARC Discovery project 'High Resolution Digital Soil Mapping'. The authors wish to thank Drs Damien Field and Veronique Bellon for their valuable inputs, and 3 anonymous reviewers for their helpful comments.
Manuscript received 8 January 2009, accepted 24 July 2009
Barthes BG, Brunet D, Ferrer H, Chottea J-L, Feller C (2006) Determination of total carbon and nitrogen content in a range of tropical soils using near infrared spectroscopy: influence of replication and sample grinding and drying. Journal of Near Infrared Spectroscopy 14, 341-348. doi: 10.1255/jnirs.686
Brown DJ, Shepherd KD, Watsh MG, Mays MD, Reinsch TG (2006) Global soil characterization with VNIR diffuse reflectance spectroscopy. Geoderma 132, 273-290. doi: 10.1016/j.geoderma.2005.04.025
Chang CW, Laird DA, Hurburg CRJ (2005) Influence of soil moisture on near-infrared reflectance spectroscopic measurement of soil properties. Soil Science 170, 244-255. doi: 10.1097/00010694-200504000-00003
Chang CW, Laird DA, Mausbach MJ, Hurburg CRJ (2001) Near-infrared reflectance spectroscopy principal component regression analysis of soil properties. Soil Science Society of America Journal 65, 480-490.
Dematte JAM, Sousa AA, Alves MC, Nanni MR, Fiorio PR, Campos RC (2006) Determining soil water status and other soil characteristics by spectral proximal sensing. Geoderma 135, 179-195. doi: 10.1016/ j.geoderma.2005.12.002
Geeves GW, Cresswell HP, Murphy BW, Gessler PE, Chartres CJ, Little IP, Bowman GM (1995) The physical, chemical and morphological properties of soils in the wheat-belt of southern NSW and northern Victoria. NSW Department of Conservation and Land Management/ CSIRO Division of Soils Occasional Report, CSIRO, Australia.
Hengl T, Heuvelink GBM, Stein A (2004) A generic framework for spatial prediction of soil variables based on regression-kriging. Geoderma 120, 75 93. doi: 10.1016/j.geoderma.2003.08.018
Islam K, McBratney AB, Singh B (2005) Rapid estimation of soil variability from the convex hull biplot area of topsoil ultra-violet, visible and near-infrared diffuse reflectance spectra. Geoderma 128, 249 257. doi: 10.1016/j.geoderma.2005.04.007
Kusumo BH, Hedley CB, Hedley MJ, Hueni A, Tuohy MP, Arnold GC (2008) The use of diffuse reflectance spectroscopy for in situ carbon and nitrogen analysis of pastoral soils. Australian Journal of Soil Research 46, 623-635. doi: 10.1071/SR08118
Lin LI (1989) A concordance correlation coefficient to evaluate reproducibility. Biometrics 45, 255-268. doi: 10.2307/2532051
McBratney AB, Mendonca-Santos ML, Minasny B (2003) On digital soil mapping. Geoderma 117, 3-52. doi: 10.1016/S0016-7061(03)00223-4
Mouazen AM, Maleki MR, De Baerdemaeker J, Ramon H (2007) On-line measurement of selected soil properties using a VIS-NIR sensor. Soil & Tillage Research 93, 13-27. doi: 10.1016/j.still.2006.03.009
Odgers NP, McBratney AB, Minasny B (2008) Generation of kth-order random toposequences. Computers & Geosciences 34, 479-490. doi: 10.1016/j.cageo.2007.05.018
Roger JM, Chauchard F, Bellon-Maurel V (2003) EPO-PLS external parameter orthogonalisation of PLS application to temperature-independent measurement of sugar content of intact fruits. Chemometrics and Intelligent Laboratory Systems 66, 191-204. doi: 10.1016/S0169-7439(03)00051-0
Sudduth KA, Hummel JW (1993) Soil organic matter, CEC, and moisture sensing with a prototype NIR spectrometer. Transactions of the American Society of Agricultural Engineers 36, 1571-1582.
Viscarra Rossel RA, Cattle SR, Ortega A, Fouad Y (2009) In situ measurements of soil colour, mineral composition and clay content by vis-NIR spectroscopy. Geoderma 150, 253 266. doi: 10.1016/ j.geoderma.2009.01.025
Viscarra Rossel RA, Walvoort DJJ, McBratney AB, Janik LJ, Skjemstad JO (2006) Visible, near infrared, mid infrared or combined diffuse reflectance spectroscopy for simultaneous assessment of various soil properties. Geoderma 131, 59-75. doi: 10.1016/j.geoderma.2005. 03.007
Waiser TH, Morgan CLS, Brown DJ, Hallmark CT (2007) In situ characterization of soil clay content with visible near-infrared diffuse reflectance spectroscopy. Soil Science Society of America Journal 71, 389-396. doi: 10.2136/sssaj2006.0211
Whiting ML, Lin L, Ustin SL (2004) Predicting water content using Gaussian model on soil spectra. Remote Sensing of Environment 89, 535-552. doi: 10.1016/j.rse.2003.11.009
Budiman Minasny (A,C), Alex B. McBratney (A), Leo Pichon (A), Wei Sun (A,B), and Michael G. Short (A)
(A) Faculty of Agriculture, Food & Natural Resources, The University of Sydney, NSW 2006, Australia.
(B) China Agricultural University, East Campus, Research Center for Precision Agriculture, Qing Hua Dong Lu 17, Beijing, 100083, P.R. China.
(C) Corresponding author. Email: email@example.com
Table 1. Goodness of fit of NIR spectra calibrated against measures soil properties on a validation set RMSE, Root mean squared error Absorbance First derivative n RMSE [R.sup.2] RMSE [R.sup.2] Clay content (% mass) 98 9.04 0.72 7.78 0.79 Total C content (% mass) 98 0.50 0.79 0.52 0.90 Sum of cations (cmol/kg) 94 4.42 0.68 7.78 0.79 pH 98 0.60 0.66 0.71 0.59 Table 2. Area of the biplot convex hull Absorbance First derivative Calibration 67.1 0.0067 Field condition 196.5 0.0058 Laboratory-dried, unground 97.9 0.0032 Laboratory-dried, ground 58.7 0.0026 Table 3. Spatial prediction of total C and clay content over the area from environmental covariates using linear regression Linear model [R.sup.2] Covariates used Total C (0-0.10 m) 0.32 Landuse, elevation, slope, aspect, profile curvature, NDVI Total C (0.40-0.50m) 0.10 Landuse, aspect, profile curvature, NDVI Clay content (0-0.10 m) 0.34 Landuse, aspect, profile curvature, landsat ETM band 2, NDVI Clay content (0.40-0.50 m) 0.11 Elevation, aspect, CTI, landsat ETM band 5
|Printer friendly Cite/link Email Feedback|
|Author:||Minasny, Budiman; McBratney, Alex B.; Pichon, Leo; Sun, Wei; Short, Michael G.|
|Publication:||Australian Journal of Soil Research|
|Date:||Nov 1, 2009|
|Previous Article:||Field level digital soil mapping of cation exchange capacity using electromagnetic induction and a hierarchical spatial regression model.|
|Next Article:||Spatial heterogeneity of soil chemical properties in a lowland tropical moist forest, Panama.|