Software Development Kit (SDK)

From Eigenvector Research Documentation Wiki
Revision as of 09:59, 14 September 2020 by Donal (talk | contribs) (→‎Requirements)
Jump to navigation Jump to search

Solo_Predictor Software Development Kit (SDK) Overview

In order to facilitate communication with and operations in Solo_Predictor from external environments, Eigenvector Research provides a software development kit (SDK) for common application languages. At this time the SDK is available in Python. Matlab, Java, and C# ports are planned for later release. Contact the help desk at helpdesk@eigenvector.com to obtain the SDK package.

The SDK includes a number of methods which cover a signification portion of common usage for deploying an existing model with new data. Description of the methods - inputs, outputs, options - may be found in the above table. These methods will, for the most part, be common across all platform and exceptions will be clearly noted.


SDK Methods

method function arguments returns
getLastResponse() last response returned by Solo/Solo_Predictor, typically in XML format none string(plain or XML)
getLastError() last error generated in operations none string(plain)
clearVariables() clear all workspace variables none Boolean
listVariables() list of workspace variables none Python list
applyModel() apply workspace variable mdl to workspace variable data none Boolean
setDataFile(pathString) load specified file (method argument) and convert to workspace variable data string - path to data file Boolean
setModelFile(pathString) load specified file (method argument) and convert to workspace variable mdl string - path to model file (.mat extension required) Boolean
setOutputFormat(formatString) specify output format for prediction results - choice of Python dict or XML string - choice of "dict" or "xml" (case insensitive) Boolean
setPort(portValue) specify communication port with Solo/Solo_Predictor integer or string (which can be converted to integer) in the range of 1024:65535; default value = 2211 Boolean
setIPAddress(IPAddressString) specify IP address to communicate with Solo/Solo_Predictor string with valid value for IP address; default value is 127.0.0.1 Boolean
getDataFile() return data file set by setDataFile none string(plain)
getModelFile() return model file set by setModelFile none string(plain)
getPort() return port value set by setPort/default value none integer
getIPAddress() return IP address set by setIPAddress/default value none string(plain)
getOutputFormat() return output format for model predictions as set by setOutputFormat/default value none string(plain)
getPredictionResults() return model prediction values as either Python dict or XML formatted dataset object none string(XML) or Python dict; empty string if error encountered
getModelInfo() return info from loaded model none string(plain)
getPredictionResultsVarNames() names of Python dict keys for prediction outputs none Python list
getVersion(modeString) returns version information for Solo_Predictor string with value of "terse" or "full"; default value (no input) is "terse" string(plain) or Python dict
runIncludeFile(pathString) execute content of text file pathString containing valid Solo scripting commands string to text file containing valid Solo scripting commands string(XML)
getVersionSDK() return current version of SDK none string(plain)

A number of the methods return Boolean values indicating success or failure at completing the desired operation. When a return value of False is obtained, detail surrounding the nature of the error may be found from the .getLastError() method. It's important to note that communication errors with Solo_Predictor are not handled by the SDK. As such, your code for communicating with Solo_Predictor should include the platform appropriate error trapping procedures for such instances.

Supported Models

The following model types are supported by the SDK:

PCA
MCR
Regression models
PLS
PCR
LWR
SVM-R
MLR
CLS
ANN
XGB
Classification models
PLSDA
SIMCA
SVM-C
ANNDA
XGBDA
LREGDA
PARAFAC

Python Implementation

Requirements

The Python version of the SDK has been tested using Python version 3.7. The following libraries - appropriate to the installed version of Python - need to be installed for proper operation:

  • requests
  • beautifulsoup4
  • numpy
  • lxml

The tested versions were requests = 2.24.0, beautifulsoup4 = 4.9.1, numpy = 1.19.2, lxml = 4.5.2. Another XML processing library could possibly be used instead of lxml.

Example

A working example is provided below with comments for many of the steps.

Configuration
from evrisdk import EvriSdk
curInstance = EvriSdk()
curInstance.setIPAddress("127.0.0.1")
curInstance.setPort(2211)

After creating an instance of the EvriSdk class, the next two lines set the IP address of the computer running Solo_Predictor (here using the localhost address) and the port. The latter may be configured with the argument as either an integer or a string. Note that these lines are somewhat redundant as the values provided are the default ones.

Solo_Predictor Workspace
retVal = curInstance.clearVariables()
variableList = curInstance.listVariables()

The .clearVariables() method will clear the Solo_Predictor workspace with a Boolean return indicating success or failure of the operation. Verification of this step is accomplished from the .listVariables() method.

Loading Data and Model

The following code segment will a) load a data file, b) load a model file, c) get a list of the prediction outuputs from the model, and d) return information on the model (model type, date constructed, etc.): 

retVal        = curInstance.setDataFile(fullPathToDataFile)
retVal        = curInstance.setModelFile(fullPathToModelFile)
predVarList   = curInstance.getPredictionResultsVarNames()
modelInfo     = curInstance.getModelInfo()

A few comments are in order:

  • data may be imported from any of the files supported by Solo_Predictor; see this page for importing data into Solo_Predictor
  • if data is imported from a Matlab file, the file may contain only one variable (at this time the SDK does not support importing specified variables from a Matlab file)
  • a Matlab file (file extension: .mat) is expected for loading a model file. Any other extension will result in an error
    • currently all EVRI model types are supported by the .setModelFile method except for calibration transfer and hierarchical models
  • the Python list output will contain the variable sgenerated from from model.plotscores(psops), where psops is a structure created from
psops = plotscores('options');
psops.reducedstats = {'q' 't2'};

In a later section the difference between prediction output formats - Python dict <=> XML - is addressed.

Apply Model and Return Results

Applying the model to the data and reviewing the outputs with some error trapping in the event the model application fails: 

retVal   = curInstance.applyModel()
if retVal:
    curInstance.setOutputFormat("dict")
    predResults = curInstance.getPredictionResults()
    for key in predResults:
        print(key, "=>", res[key])
else:
    print curInstance.returnLastError()

In the above each value of res[key] will be a numpy array containing as many elements as there are samples in the data which has been loaded.

To look at the contents of the workspace:

variableList = curInstance.listVariables()

IF XML output format is specified - curInstance.setOutputFormat("xml") - then the variable predResults will be an XML formatted string of the dataset object output. As an example, when applying a PCA model built in the arch demo dataset using 3 PCs and applying it to a test sample, specifying XML format results gives: 

<response>
  <result class="dataset">
    <name class="string" />
    <type class="string">data</type>
    <author class="string" />
    <date class="numeric" size="[1,6]">2020,9,3,14,47,53.553401</date>
    <moddate class="numeric" size="[1,6]">2020,9,3,14,47,53.593668</moddate>
    <imagesize class="numeric" size="[0,0]" />
    <imagemode class="numeric" size="[0,0]" />
    <data class="numeric" size="[1,8]">-0.00223280606154,-0.00145155625614,-0.00357717299812,9.42744403283e-05,1.63676200186e-05,5.71331078063e-05,1.91433195948e-06,0.847852727433</data>
    <label class="cell" size="[2,1]">
      <tr>
        <td class="string" />
      </tr>
      <tr>
        <td class="string">
          <sr>Scores on PC 1       </sr>
          <sr>Scores on PC 2       </sr>
          <sr>Scores on PC 3       </sr>
          <sr>Q Residuals          </sr>
          <sr>Hotelling T^2        </sr>
          <sr>Q Residuals Reduced  </sr>
          <sr>Hotelling T^2 Reduced</sr>
          <sr>KNN Score Distance   </sr>
        </td>
      </tr>
    </label>
    <labelname class="cell" size="[2,1]">
      <tr>
        <td class="string" />
      </tr>
      <tr>
        <td class="string" />
      </tr>
    </labelname>
    <axisscale class="cell" size="[2,1]"> 
    . . .

By contrast, specifying Python dict returns the following: 

    {'KNN Score Distance': array([0.84785273]), 'Hotelling T^2 Reduced': array([1.91433196e-06]), 
    'Q Residuals': array([9.42744403e-05]), 'Scores on PC 1': array([-0.00223281]), 
    'Scores on PC 3': array([-0.00357717]), 'Scores on PC 2': array([-0.00145156]), 
    'Q Residuals Reduced': array([5.71331078e-05]), 'Hotelling T^2': array([1.636762e-05])}

The latter format provides only numerical results and the appropriate tags, in this case the keys of the Python dict. The advantage of the former is that any metadata associated with mode 1 of the loaded data - labels, classes, axis scales - are copied to the prediction results output. However, parsing of the XML output falls to the end user.

.runIncludeFile Method

For more advanced operations, a text file containing valid Solo_Predictor scripting commands may be used with the .runIncludeMethod. As an example, 

retVal = curInstance.runIncludeFile(FullPathToIncludeScript)

returns 

<response>
  <result class="numeric" size="[1,3]">-0.00223280606154,-0.00145155625614,-0.00357717299812</result>
  <error class="string" />
  <date class="string">Thu 03 Sep 2020 15:25:52</date>
</response>

for the following commands contained within the the script file: 

:clear
myMdl='FullPathToMOdelFile';
myData='FullPathToDataFile';
myPred=myMdl|myData;
myPred.scores;

As previously, parsing of the XML output is the responsibility of the end user.

Retrieved from "https://www.wiki.eigenvector.com/index.php?title=Software_Development_Kit_(SDK)&oldid=11168"