Oplecorr

Purpose

Optical path-length estimation and correction with closure constraints.

Synopsis

model = oplecorr(x,y,ncomp,options); %identifies model (calibration)

sx = oplecorr(x,model,options); %applies the model

Description

The OPLEC model is similar to EMSC but doesn't require esimates of the pure spectra for filtering. Instead it assumes closure on the chemical analyte contributions and the use of a non-chemical signal basis P defined by the input (options.order). For example, if options.order = 2, then P = [1, (1:n)', (1:n)'.^2] to account for offset, slope and curvature in the baseline.

Inputs

• x = X-block (2-way array class "double" or "dataset"), and

• ncomp = number of components to to be calculated (positive integer scalar).

1) Calibration: model = oplecorr(x,y,ncomp,options);

• x = M by N matrix of spectra (class "double" or "dataset").
• y = M by 1 matrix of known reference values.
• ncomp = number of components to to be used for the basis Z (positive integer scalar).
• options = an optional input structure array described below.

2) Apply: sx = oplecorr(x,model,options);

• x =M by N matrix of spectra to be correctected .
• model = oplecorr model.

Outputs

• model = oplecorr model is a model structure with the following fields (see Standard Model Structure for additional information):

• modeltype: 'OPLECORR',

• datasource: structure array with information about input data,

• date: date of creation,

• time: time of creation, ...

and

• sx = a M by N matrix of filtered ("corrected") spectra.

Options

options = a structure array with the following fields:

• display: [ {'off'}| 'on' ] governs level of display to the command window.

• order: defines the order of polynomial to describe 'non-chemical' signal due to physical artifacts.

Alternatively, (order) can be a N by Kp matrix corresponding to basis vectors to account for non-chemical signal.

This portion of the signal is not included in the closure constraint. See Algorithm for a more complete description.

• center: [ {false} | true] governs mean-centering of the PLS model that regresses the corrections factors (model.b). No centering (the default) results in a force fit through zero.

Algorithm

The OPLEC algorithm is based on the work Z-P Chen, J Morris, E Martin, “Extracting Chemical Information from Spectral Data with Multiplicative Light Scattering Effects by Optical Path-Length Estimation and Correction,” Anal. Chem., 78, 7674-7681 (2006). OPLEC is similar to extended multiplicative scatter correction (EMSC) except that it incorporates closure in the signal due to chemical analytes.

It is assumed that the measured signal, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{x}} can be modeled as

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{x}=a\left( \mathbf{Sc}+\mathbf{Pt} \right)+\mathbf{e}\text{ (1)}}

where Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{x}} is a Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle N\times 1} column vector, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{S}} is a Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle N\times J} matrix with columns corresponding to analyte spectra, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{c}} is a Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle J\times 1} vector of contributions, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{P}} is a matrix with columns corresponding to physical artifacts in the spectra and Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{t}} is a vector corresponding scores (or contributions for the artifacts). The factor Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathit{a}} is a multiplicative factor (e.g. due to changes in path-length) identified by the OPLEC algorithm. The Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle J} analyte contributions are subject to closure such that

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \sum\limits_{j=1}^{J}{{{c}_{j}}}=1\text{ ; }{{c}_{1}}=1-\sum\limits_{j=2}^{J}{{{c}_{j}}}.\text{ (2)}}

Closure also implies that the contributions are non-negative. It is assumed that the contributions to the first analyte are known (i.e., the Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle M\times 1} column vector Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{c}_1} is known). It is also assumed that the matrix Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{P}} can be modeled a priori. Examples for physical artifacts include an offset, slope and curvature of the baseline that can be accounted for by the Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle N\times 3} basis

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{P}=\left[ \begin{matrix} \mathbf{1} & \lambda & {{\lambda }^{2}} \\ \end{matrix} \right]\text{ (3)}}

where Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \lambda} is the wavelength (or frequency) axis. However, it should be clear that Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{P}} is a matrix with columns that span physical artifacts not subject to closure. The Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{m}^{th}} measured signal, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{x}_{\mathit{m}}} , Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathit{m}=1,...,\mathit{M}} , orthogonal to Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{P}} is

${{\mathbf {z} }_{m}}=\left(\mathbf {I} -{{\mathbf {P} }^{\dagger }}{{\mathbf {P} }^{T}}\right){{\mathbf {x} }_{m}}={{a}_{m}}\mathbf {K} {{\mathbf {c} }_{m}}+\mathbf {e} _{m}^{*}{\text{ (4)}}$

where ${{\mathbf {P} }^{\dagger }}=\mathbf {P} {{\left({{\mathbf {P} }^{T}}\mathbf {P} \right)}^{-1}}$, $\mathbf {K} =\left(\mathbf {I} -{{\mathbf {P} }^{\dagger }}{{\mathbf {P} }^{T}}\right)\mathbf {S}$, and ${{\mathbf {e} }^{*}}=\left(\mathbf {I} -{{\mathbf {P} }^{\dagger }}{{\mathbf {P} }^{T}}\right)\mathbf {e}$. The measurements can be collected into a matrix $\mathbf {Z}$ and it is recognized that a basis for the ${\mathit {M}}$ measurements, $\mathbf {Z} _{\mathit {b}}$, can be obtained from a subset of linearly independent measurements. Partitioning $\mathbf {Z}$ into the basis and remaining measurements, $\mathbf {Z} _{\mathit {r}}$, gives

$\mathbf {Z} =\left[{\begin{matrix}{{\mathbf {Z} }_{b}}\\{{\mathbf {Z} }_{r}}\\\end{matrix}}\right]=\left[{\begin{matrix}diag\left({{\mathbf {a} }_{b}}\right){{\mathbf {C} }_{b}}{{\mathbf {K} }^{T}}\\diag\left({{\mathbf {a} }_{r}}\right){{\mathbf {C} }_{r}}{{\mathbf {K} }^{T}}\\\end{matrix}}\right]+{{\mathbf {E} }^{*}}{\text{ (5)}}$

This partitioning implies that the remaining measurements, $\mathbf {Z} _{\mathit {r}}$, are linear combinations of $\mathbf {Z} _{\mathit {b}}$ such that

${{\mathbf {Z} }_{r}}=\mathbf {\Gamma } {{\mathbf {Z} }_{b}}{\text{ (6)}}$

where

$\mathbf {\Gamma } ={{\mathbf {Z} }_{r}}\mathbf {Z} _{b}^{T}{{\left({{\mathbf {Z} }_{b}}\mathbf {Z} _{b}^{T}\right)}^{-1}}.{\text{ (7)}}$

Expanding a single measurement in $\mathbf {Z} _{\mathit {r}}$ gives

${{\mathbf {z} }_{m}}={{a}_{m}}{{c}_{m,1}}{{\mathbf {k} }_{1}}+{{a}_{m}}\sum \limits _{j=2}^{J-1}{{{c}_{m,j}}{{\mathbf {k} }_{j}}}+{{a}_{m}}{{c}_{m,J}}{{\mathbf {k} }_{J}}+\mathbf {e} _{m}^{*}.{\text{ (8)}}$

Substitution of Equation (2) into (8) gives

${\begin{aligned}&{{\mathbf {z} }_{m}}={{a}_{m}}{{c}_{m,1}}{{\mathbf {k} }_{1}}+{{a}_{m}}\sum \limits _{j=2}^{J-1}{{{c}_{m,j}}{{\mathbf {k} }_{j}}}+{{a}_{m}}\left(1-\sum \limits _{j=2}^{J-1}{{c}_{j}}\right){{\mathbf {k} }_{J}}+\mathbf {e} _{m}^{*}\\&{{\mathbf {z} }_{m}}={{a}_{m}}{{c}_{m,1}}\left({{\mathbf {k} }_{1}}-{{\mathbf {k} }_{J}}\right)+{{a}_{m}}\sum \limits _{j=2}^{J-1}{{{c}_{m,j}}\left({{\mathbf {k} }_{j}}-{{\mathbf {k} }_{J}}\right)}+{{a}_{m}}{{\mathbf {k} }_{J}}+\mathbf {e} _{m}^{*}\\&{{\mathbf {z} }_{m}}={{a}_{m}}\left\{{{c}_{m,1}}\Delta {{\mathbf {k} }_{1}}+\sum \limits _{j=2}^{J-1}{{{c}_{m,j}}\Delta {{\mathbf {k} }_{j}}}+{{\mathbf {k} }_{J}}\right\}+\mathbf {e} _{m}^{*}\\\end{aligned}}{\text{ (9)}}$

where $\Delta {{\mathbf {k} }_{j}}={{\mathbf {k} }_{j}}-{{\mathbf {k} }_{J}}$. The partitioned matrices in Equation (5) can now be written using the last expression of Equation (9) to give

${\begin{aligned}&{{\mathbf {Z} }_{b}}=diag\left({{\mathbf {a} }_{b}}\right)\left\{diag\left({{\mathbf {c} }_{b,1}}\right)\Delta \mathbf {k} _{1}^{T}+{{\mathbf {C} }_{b,2:J-1}}\Delta \mathbf {K} _{1:J-1}^{T}+\mathbf {k} _{J}^{T}\right\}+\mathbf {E} _{b}^{*T}\\&{{\mathbf {Z} }_{r}}=diag\left({{\mathbf {a} }_{r}}\right)\left\{diag\left({{\mathbf {c} }_{r,1}}\right)\Delta \mathbf {k} _{1}^{T}+{{\mathbf {C} }_{r,2:J-1}}\Delta \mathbf {K} _{2:J-1}^{T}+\mathbf {k} _{J}^{T}\right\}+\mathbf {E} _{r}^{*T}\\\end{aligned}}.{\text{ (10)}}$

Noting the relationship in Equation (6) gives

${\begin{aligned}&\mathbf {\Gamma } diag\left({{\mathbf {a} }_{b}}\right)\left\{diag\left({{\mathbf {c} }_{b,1}}\right)\Delta \mathbf {k} _{1}^{T}+{{\mathbf {C} }_{b,2:J-1}}\Delta \mathbf {K} _{2:J-1}^{T}+\mathbf {k} _{J}^{T}\right\}\\&=diag\left({{\mathbf {a} }_{r}}\right)\left\{diag\left({{\mathbf {c} }_{r,1}}\right)\Delta \mathbf {k} _{1}^{T}+{{\mathbf {C} }_{r,2:J-1}}\Delta \mathbf {K} _{2:J-1}^{T}+\mathbf {k} _{J}^{T}\right\}\\\end{aligned}}.{\text{ (11)}}$

Equating terms in Equation (11) gives two additional relationships:

$\mathbf {\Gamma } diag\left({{\mathbf {c} }_{b,1}}\right){{\mathbf {a} }_{b}}=diag\left({{\mathbf {c} }_{r,1}}\right){{\mathbf {a} }_{r}}{\text{ (12)}}$, and

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {{\mathbf{a}}_{r}}=\mathbf{\Gamma} {{\mathbf{a}}_{b}}.\text{ (13)}}

Substitution of Equation (13) into (12) gives

${\begin{aligned}&\mathbf {\Gamma } diag\left({{\mathbf {c} }_{b,1}}\right){{\mathbf {a} }_{b}}=diag\left({{\mathbf {c} }_{r,1}}\right)\mathbf {\Gamma } {{\mathbf {a} }_{b}}\\&{{\left[\mathbf {\Gamma } \odot \left(\mathbf {1c} _{b,1}^{T}\right)\right]}_{{{M}_{r}}\times J}}{{\left({{\mathbf {a} }_{b}}\right)}_{J\times 1}}={{\left[\left({{\mathbf {c} }_{r,1}}{{\mathbf {1} }^{T}}\right)\odot \Gamma \right]}_{{{M}_{r}}\times J}}{{\left({{\mathbf {a} }_{b}}\right)}_{J\times 1}}\\\end{aligned}}.{\text{ (14)}}$

Recall that $\mathbf {\Gamma }$, ${{\mathbf {c} }_{r,1}}$ and ${{\mathbf {c} }_{b,1}}$ are known but ${{\mathbf {a} }_{b}}$ is unknown. However, as with MSC where the reference used for correction is arbitrary (e.g., the mean of the calibration set is often used as the spectrum to “correct to”), any element of ${{\mathbf {a} }_{b}}$ can be set to one. Setting the first element of ${{\mathbf {a} }_{b}}$ to one and rearranging Equation (14) yields

${\begin{aligned}&{{c}_{b,\left(1,1\right)}}{{\mathbf {\Gamma } }_{:,1}}+\left[{{\mathbf {\Gamma } }_{:,2:J}}\odot \left(\mathbf {1c} _{b,\left(2:J,1\right)}^{T}\right)\right]{{\mathbf {a} }_{b,\left(2:end,2\right)}}={{\mathbf {c} }_{r,\left(:,1\right)}}\odot {{\mathbf {\Gamma } }_{:,1}}+\left[\left({{\mathbf {c} }_{r,\left(:,1\right)}}{{\mathbf {1} }^{T}}\right)\odot {{\mathbf {\Gamma } }_{:,2:J}}\right]{{\mathbf {a} }_{b,\left(2:end,2\right)}}\\&\left[{{\mathbf {\Gamma } }_{:,2:J}}\odot \left(\mathbf {1c} _{b,\left(2:J,1\right)}^{T}\right)-\left({{\mathbf {c} }_{r,\left(:,1\right)}}{{\mathbf {1} }^{T}}\right)\odot {{\mathbf {\Gamma } }_{:,2:J}}\right]{{\mathbf {a} }_{b,\left(2:end,2\right)}}=\left({{\mathbf {c} }_{r,\left(:,1\right)}}-\mathbf {1} {{c}_{b,\left(1,1\right)}}\right)\odot {{\mathbf {\Gamma } }_{:,1}}\\\end{aligned}}.{\text{ (15)}}$

Recognizing that the corrections, $\mathbf {a}$, must be non-negative implies that the remaining correction factors ${{\mathbf {a} }_{b2:end}}$ should be obtained by solving Equation (15) using non-negative least squares. The result is correction factors for all the basis vectors $\mathbf {a} _{b}^{T}=\left[{\begin{matrix}1&\mathbf {a} _{b,\left(2:end\right)}^{T}\\\end{matrix}}\right]$. that can be substituted into the sum of Equations (12) to give

${\begin{aligned}&\mathbf {\Gamma } \left(diag\left({{\mathbf {c} }_{b,1}}\right)+\mathbf {I} \right){{\mathbf {a} }_{b}}=\left(diag\left({{\mathbf {c} }_{r,1}}\right)+\mathbf {I} \right){{\mathbf {a} }_{r}}\\&{{\mathbf {a} }_{r}}={{\left(diag\left({{\mathbf {c} }_{r,1}}\right)+\mathbf {I} \right)}^{-1}}\mathbf {\Gamma } \left(diag\left({{\mathbf {c} }_{b,1}}\right)+\mathbf {I} \right){{\mathbf {a} }_{b}}\\\end{aligned}}$

The correction factors can be collected into a single vector given by ${{\mathbf {a} }^{T}}=\left[{\begin{matrix}\mathbf {a} _{b}^{T}&\mathbf {a} _{r}^{T}\\\end{matrix}}\right]{\text{ (16)}}$.

Next, a regression model is obtained to allow estimation of correction factors for future test samples using the following

$\mathbf {a} =\mathbf {Zb} +\mathbf {e} _{mathit{p}}{\text{ (17)}}$

where the regression vector, $\mathbf {b}$, is estimated using PLS. Change options.center to true to use mean-centering for the PLS model. The correction factors for test samples are calculated using the following steps

${{\mathbf {z} }_{test}}=\left(\mathbf {I} -{{\mathbf {P} }^{\dagger }}{{\mathbf {P} }^{T}}\right){{\mathbf {x} }_{test}}{\text{ (18)}}$, and

${{a}_{test}}=\mathbf {z} _{test}^{T}\mathbf {b} {\text{ (19)}}$

where the corrected spectrum is then given by

${{\mathbf {z} }_{corr}}={\frac {1}{{a}_{test}}}{{\mathbf {z} }_{test}}.{\text{ (20)}}$

See Also

emscorr, mscorr, stdfir