Oplecorr

Purpose

Optical path-length estimation and correction with closure constraints.

Synopsis

model = oplecorr(x,y,ncomp,options); %identifies model (calibration)

sx = oplecorr(x,model,options); %applies the model

Description

The OPLEC model is similar to EMSC but doesn't require esimates of the pure spectra for filtering. Instead it assumes closure on the chemical analyte contributions and the use of a non-chemical signal basis P defined by the input (options.order). For example, if options.order = 2, then P = [1, (1:n)', (1:n)'.^2] to account for offset, slope and curvature in the baseline.

Inputs

• x = X-block (2-way array class "double" or "dataset"), and

• ncomp = number of components to to be calculated (positive integer scalar).

1) Calibration: model = oplecorr(x,y,ncomp,options);

• x = M by N matrix of spectra (class "double" or "dataset").
• y = M by 1 matrix of known reference values.
• ncomp = number of components to to be used for the basis Z (positive integer scalar).
• options = an optional input structure array described below.

2) Apply: sx = oplecorr(x,model,options);

• x =M by N matrix of spectra to be correctected .
• model = oplecorr model.

Outputs

• model = oplecorr model is a model structure with the following fields (see Standard Model Structure for additional information):

• modeltype: 'OPLECORR',

• datasource: structure array with information about input data,

• date: date of creation,

• time: time of creation, ...

and

• sx = a M by N matrix of filtered ("corrected") spectra.

Options

options = a structure array with the following fields:

• display: [ {'off'}| 'on' ] governs level of display to the command window.

• order: defines the order of polynomial to describe 'non-chemical' signal due to physical artifacts.

Alternatively, (order) can be a N by Kp matrix corresponding to basis vectors to account for non-chemical signal.

This portion of the signal is not included in the closure constraint. See Algorithm for a more complete description.

• center: [ {false} | true] governs mean-centering of the PLS model that regresses the corrections factors (model.b). No centering (the default) results in a force fit through zero.

Algorithm

The OPLEC algorithm is based on the work Z-P Chen, J Morris, E Martin, “Extracting Chemical Information from Spectral Data with Multiplicative Light Scattering Effects by Optical Path-Length Estimation and Correction,” Anal. Chem., 78, 7674-7681 (2006). OPLEC is similar to extended multiplicative scatter correction (EMSC) except that it incorporates closure in the signal due to chemical analytes.

It is assumed that the measured signal, $\mathbf {x}$ can be modeled as

$\mathbf {x} =a\left(\mathbf {Sc} +\mathbf {Pt} \right)+\mathbf {e} {\text{ (1)}}$

where $\mathbf {x}$ is a $N\times 1$ column vector, $\mathbf {S}$ is a $N\times J$ matrix with columns corresponding to analyte spectra, $\mathbf {c}$ is a $J\times 1$ vector of contributions, $\mathbf {P}$ is a matrix with columns corresponding to physical artifacts in the spectra and $\mathbf {t}$ is a vector corresponding scores (or contributions for the artifacts). The factor ${\mathit {a}}$ is a multiplicative factor (e.g. due to changes in path-length) identified by the OPLEC algorithm. The $J$ analyte contributions are subject to closure such that

$\sum \limits _{j=1}^{J}{{c}_{j}}=1{\text{ ; }}{{c}_{1}}=1-\sum \limits _{j=2}^{J}{{c}_{j}}.{\text{ (2)}}$

Closure also implies that the contributions are non-negative. It is assumed that the contributions to the first analyte are known (i.e., the $M\times 1$ column vector $\mathbf {c} _{1}$ is known). It is also assumed that the matrix $\mathbf {P}$ can be modeled a priori. Examples for physical artifacts include an offset, slope and curvature of the baseline that can be accounted for by the $N\times 3$ basis

$\mathbf {P} =\left[{\begin{matrix}\mathbf {1} &\lambda &{{\lambda }^{2}}\\\end{matrix}}\right]{\text{ (3)}}$

where $\lambda$ is the wavelength (or frequency) axis. However, it should be clear that $\mathbf {P}$ is a matrix with columns that span physical artifacts not subject to closure. The $\mathbf {m} ^{th}$ measured signal, $\mathbf {x} _{\mathit {m}}$, ${\mathit {m}}=1,...,{\mathit {M}}$, orthogonal to $\mathbf {P}$ is

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {{\mathbf{z}}_{m}}=\left( \mathbf{I}-{{\mathbf{P}}^{\dagger }}{{\mathbf{P}}^{T}} \right){{\mathbf{x}}_{m}}={{a}_{m}}\mathbf{K}{{\mathbf{c}}_{m}}+\mathbf{e}_{m}^{*}\text{ (4)}}

where Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {{\mathbf{P}}^{\dagger }}=\mathbf{P}{{\left( {{\mathbf{P}}^{T}}\mathbf{P} \right)}^{-1}}} , Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{K}=\left( \mathbf{I}-{{\mathbf{P}}^{\dagger }}{{\mathbf{P}}^{T}} \right)\mathbf{S}} , and Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {{\mathbf{e}}^{*}}=\left( \mathbf{I}-{{\mathbf{P}}^{\dagger }}{{\mathbf{P}}^{T}} \right)\mathbf{e}} . The measurements can be collected into a matrix Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{Z}} and it is recognized that a basis for the Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathit{M}} measurements, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{Z}_{\mathit{b}}} , can be obtained from a subset of linearly independent measurements. Partitioning Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{Z}} into the basis and remaining measurements, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{Z}_{\mathit{r}}} , gives

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{Z}=\left[ \begin{matrix} {{\mathbf{Z}}_{b}} \\ {{\mathbf{Z}}_{r}} \\ \end{matrix} \right]=\left[ \begin{matrix} diag\left( {{\mathbf{a}}_{b}} \right){{\mathbf{C}}_{b}}{{\mathbf{K}}^{T}} \\ diag\left( {{\mathbf{a}}_{r}} \right){{\mathbf{C}}_{r}}{{\mathbf{K}}^{T}} \\ \end{matrix} \right]+{{\mathbf{E}}^{*}}\text{ (5)}}

This partitioning implies that the remaining measurements, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{Z}_{\mathit{r}}} , are linear combinations of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{Z}_{\mathit{b}}} such that

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {{\mathbf{Z}}_{r}}=\mathbf{\Gamma} {{\mathbf{Z}}_{b}}\text{ (6)}}

where

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{\Gamma} ={{\mathbf{Z}}_{r}}\mathbf{Z}_{b}^{T}{{\left( {{\mathbf{Z}}_{b}}\mathbf{Z}_{b}^{T} \right)}^{-1}}.\text{ (7)}}

Expanding a single measurement in Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{Z}_{\mathit{r}}} gives

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {{\mathbf{z}}_{m}}={{a}_{m}}{{c}_{m,1}}{{\mathbf{k}}_{1}}+{{a}_{m}}\sum\limits_{j=2}^{J-1}{{{c}_{m,j}}{{\mathbf{k}}_{j}}}+{{a}_{m}}{{c}_{m,J}}{{\mathbf{k}}_{J}}+\mathbf{e}_{m}^{*}.\text{ (8)}}

Substitution of Equation (2) into (8) gives

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \begin{align} & {{\mathbf{z}}_{m}}={{a}_{m}}{{c}_{m,1}}{{\mathbf{k}}_{1}}+{{a}_{m}}\sum\limits_{j=2}^{J-1}{{{c}_{m,j}}{{\mathbf{k}}_{j}}}+{{a}_{m}}\left( 1-\sum\limits_{j=2}^{J-1}{{{c}_{j}}} \right){{\mathbf{k}}_{J}}+\mathbf{e}_{m}^{*} \\ & {{\mathbf{z}}_{m}}={{a}_{m}}{{c}_{m,1}}\left( {{\mathbf{k}}_{1}}-{{\mathbf{k}}_{J}} \right)+{{a}_{m}}\sum\limits_{j=2}^{J-1}{{{c}_{m,j}}\left( {{\mathbf{k}}_{j}}-{{\mathbf{k}}_{J}} \right)}+{{a}_{m}}{{\mathbf{k}}_{J}}+\mathbf{e}_{m}^{*} \\ & {{\mathbf{z}}_{m}}={{a}_{m}}\left\{ {{c}_{m,1}}\Delta {{\mathbf{k}}_{1}}+\sum\limits_{j=2}^{J-1}{{{c}_{m,j}}\Delta {{\mathbf{k}}_{j}}}+{{\mathbf{k}}_{J}} \right\}+\mathbf{e}_{m}^{*} \\ \end{align}\text{ (9)}}

where Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \Delta {{\mathbf{k}}_{j}}={{\mathbf{k}}_{j}}-{{\mathbf{k}}_{J}}} . The partitioned matrices in Equation (5) can now be written using the last expression of Equation (9) to give

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \begin{align} & {{\mathbf{Z}}_{b}}=diag\left( {{\mathbf{a}}_{b}} \right)\left\{ diag\left( {{\mathbf{c}}_{b,1}} \right)\Delta \mathbf{k}_{1}^{T}+{{\mathbf{C}}_{b,2:J-1}}\Delta \mathbf{K}_{1:J-1}^{T}+\mathbf{k}_{J}^{T} \right\}+\mathbf{E}_{b}^{*T} \\ & {{\mathbf{Z}}_{r}}=diag\left( {{\mathbf{a}}_{r}} \right)\left\{ diag\left( {{\mathbf{c}}_{r,1}} \right)\Delta \mathbf{k}_{1}^{T}+{{\mathbf{C}}_{r,2:J-1}}\Delta \mathbf{K}_{2:J-1}^{T}+\mathbf{k}_{J}^{T} \right\}+\mathbf{E}_{r}^{*T} \\ \end{align}.\text{ (10)}}

Noting the relationship in Equation (6) gives

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \begin{align} & \mathbf{\Gamma} diag\left( {{\mathbf{a}}_{b}} \right)\left\{ diag\left( {{\mathbf{c}}_{b,1}} \right)\Delta \mathbf{k}_{1}^{T}+{{\mathbf{C}}_{b,2:J-1}}\Delta \mathbf{K}_{2:J-1}^{T}+\mathbf{k}_{J}^{T} \right\} \\ & =diag\left( {{\mathbf{a}}_{r}} \right)\left\{ diag\left( {{\mathbf{c}}_{r,1}} \right)\Delta \mathbf{k}_{1}^{T}+{{\mathbf{C}}_{r,2:J-1}}\Delta \mathbf{K}_{2:J-1}^{T}+\mathbf{k}_{J}^{T} \right\} \\ \end{align}.\text{ (11)}}

Equating terms in Equation (11) gives two additional relationships:

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{\Gamma} diag\left( {{\mathbf{c}}_{b,1}} \right){{\mathbf{a}}_{b}}=diag\left( {{\mathbf{c}}_{r,1}} \right){{\mathbf{a}}_{r}}\text{ (12)}} , and

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {{\mathbf{a}}_{r}}=\mathbf{\Gamma} {{\mathbf{a}}_{b}}.\text{ (13)}}

Substitution of Equation (13) into (12) gives

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \begin{align} & \mathbf{\Gamma} diag\left( {{\mathbf{c}}_{b,1}} \right){{\mathbf{a}}_{b}}=diag\left( {{\mathbf{c}}_{r,1}} \right)\mathbf{\Gamma} {{\mathbf{a}}_{b}} \\ & {{\left[ \mathbf{\Gamma} \odot \left( \mathbf{1c}_{b,1}^{T} \right) \right]}_{{{M}_{r}}\times J}}{{\left( {{\mathbf{a}}_{b}} \right)}_{J\times 1}}={{\left[ \left( {{\mathbf{c}}_{r,1}}{{\mathbf{1}}^{T}} \right)\odot \Gamma \right]}_{{{M}_{r}}\times J}}{{\left( {{\mathbf{a}}_{b}} \right)}_{J\times 1}} \\ \end{align}.\text{ (14)}}

Recall that Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{\Gamma} } , Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {{\mathbf{c}}_{r,1}}} and Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {{\mathbf{c}}_{b,1}}} are known but Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {{\mathbf{a}}_{b}}} is unknown. However, as with MSC where the reference used for correction is arbitrary (e.g., the mean of the calibration set is often used as the spectrum to “correct to”), any element of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {{\mathbf{a}}_{b}}} can be set to one. Setting the first element of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {{\mathbf{a}}_{b}}} to one and rearranging Equation (14) yields

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \begin{align} & {{c}_{b,\left( 1,1 \right)}}{{\mathbf{\Gamma} }_{:,1}}+\left[ {{\mathbf{\Gamma} }_{:,2:J}}\odot \left( \mathbf{1c}_{b,\left( 2:J,1 \right)}^{T} \right) \right]{{\mathbf{a}}_{b,\left( 2:end,2 \right)}}={{\mathbf{c}}_{r,\left( :,1 \right)}}\odot {{\mathbf{\Gamma} }_{:,1}}+\left[ \left( {{\mathbf{c}}_{r,\left( :,1 \right)}}{{\mathbf{1}}^{T}} \right)\odot {{\mathbf{\Gamma} }_{:,2:J}} \right]{{\mathbf{a}}_{b,\left( 2:end,2 \right)}} \\ & \left[ {{\mathbf{\Gamma} }_{:,2:J}}\odot \left( \mathbf{1c}_{b,\left( 2:J,1 \right)}^{T} \right)-\left( {{\mathbf{c}}_{r,\left( :,1 \right)}}{{\mathbf{1}}^{T}} \right)\odot {{\mathbf{\Gamma} }_{:,2:J}} \right]{{\mathbf{a}}_{b,\left( 2:end,2 \right)}}=\left( {{\mathbf{c}}_{r,\left( :,1 \right)}}-\mathbf{1}{{c}_{b,\left( 1,1 \right)}} \right)\odot {{\mathbf{\Gamma} }_{:,1}} \\ \end{align}.\text{ (15)}}

Recognizing that the corrections, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{a}} , must be non-negative implies that the remaining correction factors Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {{\mathbf{a}}_{b2:end}}} should be obtained by solving Equation (15) using non-negative least squares. The result is correction factors for all the basis vectors Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{a}_{b}^{T}=\left[ \begin{matrix} 1 & \mathbf{a}_{b,\left( 2:end \right)}^{T} \\ \end{matrix} \right]} . that can be substituted into the sum of Equations (12) to give

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \begin{align} & \mathbf{\Gamma} \left( diag\left( {{\mathbf{c}}_{b,1}} \right)+\mathbf{I} \right){{\mathbf{a}}_{b}}=\left( diag\left( {{\mathbf{c}}_{r,1}} \right)+\mathbf{I} \right){{\mathbf{a}}_{r}} \\ & {{\mathbf{a}}_{r}}={{\left( diag\left( {{\mathbf{c}}_{r,1}} \right)+\mathbf{I} \right)}^{-1}}\mathbf{\Gamma} \left( diag\left( {{\mathbf{c}}_{b,1}} \right)+\mathbf{I} \right){{\mathbf{a}}_{b}} \\ \end{align}}

The correction factors can be collected into a single vector given by Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {{\mathbf{a}}^{T}}=\left[ \begin{matrix} \mathbf{a}_{b}^{T} & \mathbf{a}_{r}^{T} \\ \end{matrix} \right]\text{ (16)}} .

Next, a regression model is obtained to allow estimation of correction factors for future test samples using the following

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{a}=\mathbf{Zb} +\mathbf{e}_{mathit{p}} \text{ (17)}}

where the regression vector, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf{b} } , is estimated using PLS. Change options.center to true to use mean-centering for the PLS model. The correction factors for test samples are calculated using the following steps

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {{\mathbf{z}}_{test}}=\left( \mathbf{I}-{{\mathbf{P}}^{\dagger }}{{\mathbf{P}}^{T}} \right){{\mathbf{x}}_{test}}\text{ (18)}} , and

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {{a}_{test}}=\mathbf{z}_{test}^{T}\mathbf{b} \text{ (19)} }

where the corrected spectrum is then given by

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {{\mathbf{z}}_{corr}}=\frac{1}{{{a}_{test}}}{{\mathbf{z}}_{test}}.\text{ (20)}}

See Also

emscorr, mscorr, stdfir