Oplecorr

Purpose

Optical path-length estimation and correction with closure constraints.

Synopsis

model = oplecorr(x,y,ncomp,options); %identifies model (calibration)

sx = oplecorr(x,model,options); %applies the model

Description

The OPLEC model is similar to EMSC but doesn't require esimates of the pure spectra for filtering. Instead it assumes closure on the chemical analyte contributions and the use of a non-chemical signal basis P defined by the input (options.order). For example, if options.order = 2, then P = [1, (1:n)', (1:n)'.^2] to account for offset, slope and curvature in the baseline.

Inputs

• x = X-block (2-way array class "double" or "dataset"), and

• ncomp = number of components to to be calculated (positive integer scalar).

1) Calibration: model = oplecorr(x,y,ncomp,options);

• x = M by N matrix of spectra (class "double" or "dataset").
• y = M by 1 matrix of known reference values.
• ncomp = number of components to to be used for the basis Z (positive integer scalar).
• options = an optional input structure array described below.

2) Apply: sx = oplecorr(x,model,options);

• x =M by N matrix of spectra to be correctected .
• model = oplecorr model.

Outputs

• model = oplecorr model is a model structure with the following fields (see Standard Model Structure for additional information):

• modeltype: 'OPLECORR',

• datasource: structure array with information about input data,

• date: date of creation,

• time: time of creation, ...

and

• sx = a M by N matrix of filtered ("corrected") spectra.

Options

options = a structure array with the following fields:

• display: [ {'off'}| 'on' ] governs level of display to the command window.

• order: defines the order of polynomial to describe 'non-chemical' signal due to physical artifacts.

Alternatively, (order) can be a N by Kp matrix corresponding to basis vectors to account for non-chemical signal.

This portion of the signal is not included in the closure constraint. See Algorithm for a more complete description.

• center: [ {false} | true] governs mean-centering of the PLS model that regresses the corrections factors (model.b). No centering (the default) results in a force fit through zero.

Algorithm

The OPLEC algorithm is based on the work Z-P Chen, J Morris, E Martin, “Extracting Chemical Information from Spectral Data with Multiplicative Light Scattering Effects by Optical Path-Length Estimation and Correction,” Anal. Chem., 78, 7674-7681 (2006). OPLEC is similar to extended multiplicative scatter correction (EMSC) except that it incorporates closure in the signal due to chemical analytes.

It is assumed that the measured signal, $\mathbf {x}$ can be modeled as

$\mathbf {x} =a\left(\mathbf {Sc} +\mathbf {Pt} \right)+\mathbf {e} {\text{ (1)}}$

where $\mathbf {x}$ is a $N\times 1$ column vector, $\mathbf {S}$ is a $N\times J$ matrix with columns corresponding to analyte spectra, $\mathbf {c}$ is a $J\times 1$ vector of contributions, $\mathbf {P}$ is a matrix with columns corresponding to physical artifacts in the spectra and $\mathbf {t}$ is a vector corresponding scores (or contributions for the artifacts). The factor ${\mathit {a}}$ is a multiplicative factor (e.g. due to changes in path-length) identified by the OPLEC algorithm. The $J$ analyte contributions are subject to closure such that

$\sum \limits _{j=1}^{J}{{c}_{j}}=1{\text{ ; }}{{c}_{1}}=1-\sum \limits _{j=2}^{J}{{c}_{j}}.{\text{ (2)}}$

Closure also implies that the contributions are non-negative. It is assumed that the contributions to the first analyte are known (i.e., the $M\times 1$ column vector $\mathbf {c} _{1}$ is known). It is also assumed that the matrix $\mathbf {P}$ can be modeled a priori. Examples for physical artifacts include an offset, slope and curvature of the baseline that can be accounted for by the $N\times 3$ basis

$\mathbf {P} =\left[{\begin{matrix}\mathbf {1} &\lambda &{{\lambda }^{2}}\\\end{matrix}}\right]{\text{ (3)}}$

where $\lambda$ is the wavelength (or frequency) axis. However, it should be clear that $\mathbf {P}$ is a matrix with columns that span physical artifacts not subject to closure. The $\mathbf {m} ^{th}$ measured signal, $\mathbf {x} _{\mathit {m}}$, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathit{m}=1,...,\mathit{M}} , orthogonal to $\mathbf {P}$ is

${{\mathbf {z} }_{m}}=\left(\mathbf {I} -{{\mathbf {P} }^{\dagger }}{{\mathbf {P} }^{T}}\right){{\mathbf {x} }_{m}}={{a}_{m}}\mathbf {K} {{\mathbf {c} }_{m}}+\mathbf {e} _{m}^{*}{\text{ (4)}}$

where ${{\mathbf {P} }^{\dagger }}=\mathbf {P} {{\left({{\mathbf {P} }^{T}}\mathbf {P} \right)}^{-1}}$, $\mathbf {K} =\left(\mathbf {I} -{{\mathbf {P} }^{\dagger }}{{\mathbf {P} }^{T}}\right)\mathbf {S}$, and ${{\mathbf {e} }^{*}}=\left(\mathbf {I} -{{\mathbf {P} }^{\dagger }}{{\mathbf {P} }^{T}}\right)\mathbf {e}$. The measurements can be collected into a matrix $\mathbf {Z}$ and it is recognized that a basis for the ${\mathit {M}}$ measurements, $\mathbf {Z} _{\mathit {b}}$, can be obtained from a subset of linearly independent measurements. Partitioning $\mathbf {Z}$ into the basis and remaining measurements, $\mathbf {Z} _{\mathit {r}}$, gives

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{Z}=\left[ \begin{matrix} {{\mathbf{Z}}_{b}} \\ {{\mathbf{Z}}_{r}} \\ \end{matrix} \right]=\left[ \begin{matrix} diag\left( {{\mathbf{a}}_{b}} \right){{\mathbf{C}}_{b}}{{\mathbf{K}}^{T}} \\ diag\left( {{\mathbf{a}}_{r}} \right){{\mathbf{C}}_{r}}{{\mathbf{K}}^{T}} \\ \end{matrix} \right]+{{\mathbf{E}}^{*}}\text{ (5)}}

This partitioning implies that the remaining measurements, $\mathbf {Z} _{\mathit {r}}$, are linear combinations of $\mathbf {Z} _{\mathit {b}}$ such that

${{\mathbf {Z} }_{r}}=\mathbf {\Gamma } {{\mathbf {Z} }_{b}}{\text{ (6)}}$

where

$\mathbf {\Gamma } ={{\mathbf {Z} }_{r}}\mathbf {Z} _{b}^{T}{{\left({{\mathbf {Z} }_{b}}\mathbf {Z} _{b}^{T}\right)}^{-1}}.{\text{ (7)}}$

Expanding a single measurement in $\mathbf {Z} _{\mathit {r}}$ gives

${{\mathbf {z} }_{m}}={{a}_{m}}{{c}_{m,1}}{{\mathbf {k} }_{1}}+{{a}_{m}}\sum \limits _{j=2}^{J-1}{{{c}_{m,j}}{{\mathbf {k} }_{j}}}+{{a}_{m}}{{c}_{m,J}}{{\mathbf {k} }_{J}}+\mathbf {e} _{m}^{*}.{\text{ (8)}}$

Substitution of Equation (2) into (8) gives

${\begin{aligned}&{{\mathbf {z} }_{m}}={{a}_{m}}{{c}_{m,1}}{{\mathbf {k} }_{1}}+{{a}_{m}}\sum \limits _{j=2}^{J-1}{{{c}_{m,j}}{{\mathbf {k} }_{j}}}+{{a}_{m}}\left(1-\sum \limits _{j=2}^{J-1}{{c}_{j}}\right){{\mathbf {k} }_{J}}+\mathbf {e} _{m}^{*}\\&{{\mathbf {z} }_{m}}={{a}_{m}}{{c}_{m,1}}\left({{\mathbf {k} }_{1}}-{{\mathbf {k} }_{J}}\right)+{{a}_{m}}\sum \limits _{j=2}^{J-1}{{{c}_{m,j}}\left({{\mathbf {k} }_{j}}-{{\mathbf {k} }_{J}}\right)}+{{a}_{m}}{{\mathbf {k} }_{J}}+\mathbf {e} _{m}^{*}\\&{{\mathbf {z} }_{m}}={{a}_{m}}\left\{{{c}_{m,1}}\Delta {{\mathbf {k} }_{1}}+\sum \limits _{j=2}^{J-1}{{{c}_{m,j}}\Delta {{\mathbf {k} }_{j}}}+{{\mathbf {k} }_{J}}\right\}+\mathbf {e} _{m}^{*}\\\end{aligned}}{\text{ (9)}}$

where $\Delta {{\mathbf {k} }_{j}}={{\mathbf {k} }_{j}}-{{\mathbf {k} }_{J}}$. The partitioned matrices in Equation (5) can now be written using the last expression of Equation (9) to give

${\begin{aligned}&{{\mathbf {Z} }_{b}}=diag\left({{\mathbf {a} }_{b}}\right)\left\{diag\left({{\mathbf {c} }_{b,1}}\right)\Delta \mathbf {k} _{1}^{T}+{{\mathbf {C} }_{b,2:J-1}}\Delta \mathbf {K} _{1:J-1}^{T}+\mathbf {k} _{J}^{T}\right\}+\mathbf {E} _{b}^{*T}\\&{{\mathbf {Z} }_{r}}=diag\left({{\mathbf {a} }_{r}}\right)\left\{diag\left({{\mathbf {c} }_{r,1}}\right)\Delta \mathbf {k} _{1}^{T}+{{\mathbf {C} }_{r,2:J-1}}\Delta \mathbf {K} _{2:J-1}^{T}+\mathbf {k} _{J}^{T}\right\}+\mathbf {E} _{r}^{*T}\\\end{aligned}}.{\text{ (10)}}$

Noting the relationship in Equation (6) gives

${\begin{aligned}&\mathbf {\Gamma } diag\left({{\mathbf {a} }_{b}}\right)\left\{diag\left({{\mathbf {c} }_{b,1}}\right)\Delta \mathbf {k} _{1}^{T}+{{\mathbf {C} }_{b,2:J-1}}\Delta \mathbf {K} _{2:J-1}^{T}+\mathbf {k} _{J}^{T}\right\}\\&=diag\left({{\mathbf {a} }_{r}}\right)\left\{diag\left({{\mathbf {c} }_{r,1}}\right)\Delta \mathbf {k} _{1}^{T}+{{\mathbf {C} }_{r,2:J-1}}\Delta \mathbf {K} _{2:J-1}^{T}+\mathbf {k} _{J}^{T}\right\}\\\end{aligned}}.{\text{ (11)}}$

Equating terms in Equation (11) gives two additional relationships:

$\mathbf {\Gamma } diag\left({{\mathbf {c} }_{b,1}}\right){{\mathbf {a} }_{b}}=diag\left({{\mathbf {c} }_{r,1}}\right){{\mathbf {a} }_{r}}{\text{ (12)}}$, and

${{\mathbf {a} }_{r}}=\mathbf {\Gamma } {{\mathbf {a} }_{b}}.{\text{ (13)}}$

Substitution of Equation (13) into (12) gives

${\begin{aligned}&\mathbf {\Gamma } diag\left({{\mathbf {c} }_{b,1}}\right){{\mathbf {a} }_{b}}=diag\left({{\mathbf {c} }_{r,1}}\right)\mathbf {\Gamma } {{\mathbf {a} }_{b}}\\&{{\left[\mathbf {\Gamma } \odot \left(\mathbf {1c} _{b,1}^{T}\right)\right]}_{{{M}_{r}}\times J}}{{\left({{\mathbf {a} }_{b}}\right)}_{J\times 1}}={{\left[\left({{\mathbf {c} }_{r,1}}{{\mathbf {1} }^{T}}\right)\odot \Gamma \right]}_{{{M}_{r}}\times J}}{{\left({{\mathbf {a} }_{b}}\right)}_{J\times 1}}\\\end{aligned}}.{\text{ (14)}}$

Recall that $\mathbf {\Gamma }$, ${{\mathbf {c} }_{r,1}}$ and ${{\mathbf {c} }_{b,1}}$ are known but ${{\mathbf {a} }_{b}}$ is unknown. However, as with MSC where the reference used for correction is arbitrary (e.g., the mean of the calibration set is often used as the spectrum to “correct to”), any element of ${{\mathbf {a} }_{b}}$ can be set to one. Setting the first element of ${{\mathbf {a} }_{b}}$ to one and rearranging Equation (14) yields

${\begin{aligned}&{{c}_{b,\left(1,1\right)}}{{\mathbf {\Gamma } }_{:,1}}+\left[{{\mathbf {\Gamma } }_{:,2:J}}\odot \left(\mathbf {1c} _{b,\left(2:J,1\right)}^{T}\right)\right]{{\mathbf {a} }_{b,\left(2:end,2\right)}}={{\mathbf {c} }_{r,\left(:,1\right)}}\odot {{\mathbf {\Gamma } }_{:,1}}+\left[\left({{\mathbf {c} }_{r,\left(:,1\right)}}{{\mathbf {1} }^{T}}\right)\odot {{\mathbf {\Gamma } }_{:,2:J}}\right]{{\mathbf {a} }_{b,\left(2:end,2\right)}}\\&\left[{{\mathbf {\Gamma } }_{:,2:J}}\odot \left(\mathbf {1c} _{b,\left(2:J,1\right)}^{T}\right)-\left({{\mathbf {c} }_{r,\left(:,1\right)}}{{\mathbf {1} }^{T}}\right)\odot {{\mathbf {\Gamma } }_{:,2:J}}\right]{{\mathbf {a} }_{b,\left(2:end,2\right)}}=\left({{\mathbf {c} }_{r,\left(:,1\right)}}-\mathbf {1} {{c}_{b,\left(1,1\right)}}\right)\odot {{\mathbf {\Gamma } }_{:,1}}\\\end{aligned}}.{\text{ (15)}}$

Recognizing that the corrections, $\mathbf {a}$, must be non-negative implies that the remaining correction factors ${{\mathbf {a} }_{b2:end}}$ should be obtained by solving Equation (15) using non-negative least squares. The result is correction factors for all the basis vectors $\mathbf {a} _{b}^{T}=\left[{\begin{matrix}1&\mathbf {a} _{b,\left(2:end\right)}^{T}\\\end{matrix}}\right]$. that can be substituted into the sum of Equations (12) to give

${\begin{aligned}&\mathbf {\Gamma } \left(diag\left({{\mathbf {c} }_{b,1}}\right)+\mathbf {I} \right){{\mathbf {a} }_{b}}=\left(diag\left({{\mathbf {c} }_{r,1}}\right)+\mathbf {I} \right){{\mathbf {a} }_{r}}\\&{{\mathbf {a} }_{r}}={{\left(diag\left({{\mathbf {c} }_{r,1}}\right)+\mathbf {I} \right)}^{-1}}\mathbf {\Gamma } \left(diag\left({{\mathbf {c} }_{b,1}}\right)+\mathbf {I} \right){{\mathbf {a} }_{b}}\\\end{aligned}}$

The correction factors can be collected into a single vector given by ${{\mathbf {a} }^{T}}=\left[{\begin{matrix}\mathbf {a} _{b}^{T}&\mathbf {a} _{r}^{T}\\\end{matrix}}\right]{\text{ (16)}}$.

Next, a regression model is obtained to allow estimation of correction factors for future test samples using the following

$\mathbf {a} =\mathbf {Zb} +\mathbf {e} _{mathit{p}}{\text{ (17)}}$

where the regression vector, $\mathbf {b}$, is estimated using PLS. Change options.center to true to use mean-centering for the PLS model. The correction factors for test samples are calculated using the following steps

${{\mathbf {z} }_{test}}=\left(\mathbf {I} -{{\mathbf {P} }^{\dagger }}{{\mathbf {P} }^{T}}\right){{\mathbf {x} }_{test}}{\text{ (18)}}$, and

${{a}_{test}}=\mathbf {z} _{test}^{T}\mathbf {b} {\text{ (19)}}$

where the corrected spectrum is then given by

${{\mathbf {z} }_{corr}}={\frac {1}{{a}_{test}}}{{\mathbf {z} }_{test}}.{\text{ (20)}}$

See Also

emscorr, mscorr, stdfir