Data Importing Formats: Difference between revisions

From Eigenvector Research Documentation Wiki
Jump to navigation Jump to search
imported>Scott
No edit summary
No edit summary
 
(43 intermediate revisions by 7 users not shown)
Line 1: Line 1:
Data can be imported into PLS_Toolbox or Solo from a variety of sources using '''File>Import Data''' (or similar) in the [[Workspace Browser]], [[Analysis GUI|Analysis]], and [[DataSet Editor]] windows. Sources include:
Data can be imported into PLS_Toolbox or Solo from a variety of sources using '''File>Import Data''' (or similar) menu itesm in the [[Workspace Browser]], [[Analysis GUI|Analysis]], and [[DataSet Editor]] windows.
== Unsupported File Formats ==
'''Note''': If you do not see the file type list below try the following:
# Export your data to a supported file type. Often this is delimited text, Excel, or SPC.
#* If data can be exported as text, some customization can be done via our text importing tools to accommodate different formatting:
#:* For more information on using the textreadr function see [[Textreadr]]
#:* Create a custom import function [[Editds_userimport]]
# Locate a community developed importer on the Matlab [http://www.mathworks.com/matlabcentral/fileexchange/ file exchange].
#* Other sources exist as well, for example: [http://cdf.gsfc.nasa.gov/html/dttools.html CDF Tools].
# Obtain a file format (specification) from the vendor. We're always happy to develop (or help develop) file importers.
#* An actual specification document is great but often vendors have internally developed Matlab importers that can be used outright or be modified to work. It's also very helpful to have example files that can be used to test with.


== Common Formats ==
* Workspace/MAT file - ''see [[WorkspaceBrowser_ImportingData#To_import_a_.mat_file|Importing .mat Files]]''
* Workspace/MAT file - ''see [[WorkspaceBrowser_ImportingData#To_import_a_.mat_file|Importing .mat Files]]''
* Delimited Text File (CSV,TXT) - ''see [[Text Import Settings]]''
* [[xclreadr|Delimited Text File (CSV,TXT)]] - ''see [[Text Import Settings]]''
* XY... Delimited Text Files (TXT,XY) - ''see [[Text Import Settings]]''
* [[xyreadr|XY... Delimited Text Files (TXT,XY)]] - ''see [[Text Import Settings]]''
* Excel File (XLS,XLSX,CSV,TXT)
* [[xlsreadr|Excel File (XLS,XLSX,CSV,TXT)]]
* Experiment File (EXP,CSV,XLS,TXT)
* [[Experimentreadr |Experiment File (EXP,CSV,XLS,TXT)]]
* [[table2dataset]] - Convert Matlab Table Array to [[Dataset]].
* Text from Clipboard (CSV,TXT,XML)
* Text from Clipboard (CSV,TXT,XML)
* Extensible Markup Language (XML) - ''see [[DataSet XML Format]]''
* Extensible Markup Language (XML) - ''see [[DataSet XML Format]]''


 
== Commercial Formats ==
* [[mtfreadr|AdventaCT MTF File (MTF)]]
* [[mtfreadr|AdventaCT MTF File (MTF)]]
* [[asfreadr|AIT ASF File (ASF, AIF, BKH)]]
* [[pdfreadr|AIT PIONIR File (PDF)]]
* [[asdreadr|Analytical Spectral Devices (ASD) Indico (V6 and V7)]]
* [[asdreadr|Analytical Spectral Devices (ASD) Indico (V6 and V7)]]
* [[envireadr|ENVI Image File (HDR/IMG) Image DSO if MIA_Toolbox installed.]]
* [[opusreadr|Bruker OPUS]]
* [[brukerxrpdreadr|Bruker XRPD (RAW)]]
* [[cytospecreadr| CytoSpec CYT File (CYT)]]
* [[spcreadr|Grams Thermo Galactic File (SPC, DHB)]]
* [[spcreadr|Grams Thermo Galactic File (SPC, DHB)]]
* [[asfreadr|Hamilton Sundstrand ASF File (ASF, AIF, BKH)]]
* [[gwscanreadr|Guided Wave File (SCAN, AUTOSCAN)]]
* [[pdfreadr|Hamilton Sundstrand PIONIR File (PDF)]]
* [[hitachieemreadr| Hitachi EEM File (TXT)]]
* [[hjyreadr|Horiba JY File (NGS, NGC)]]
* [[hjyreadr|Horiba JY Raman File (NGS, NGC)]]
* [[aqualogreadr|Horiba JY Aqualog ASCII File (DAT)]]
* [[jascoeemreadr|Jasco EEM Files (CSV)]]
* [[jcampreadr|JCAMP (DX,JDX)]]
* [[jcampreadr|JCAMP (DX,JDX)]]
* [[rdareadr|Siemens RDA file (RDA)]]
* [[spareadr|Omnic SPA File (SPA)]]
* [[spareadr|Omnic SPA File (SPA)]]
* [[absreadr|Stellarnet ABS File (ABS)]]
* [[spereadr|Princeton Instruments SPE File (SPE)]]
* [[pereadr|PerkinElmer (FSM, SP, VIS, IMP)]]
* [[snabsreadr|Stellarnet ABS File (ABS)]]
* [[shimadzueemreadr|Shimadzu EEM File (TXT)]]
* [[visionairxmlreadr|Vision Air XML Data File (XML)]]
* Other... - Call Matlab importer.
* Other... - Call Matlab importer.
----
Additional import methods may be available depending on the specific purchased options and the distributor of your copy of PLS_Toolbox or Solo. See other notes below.
Additional import methods may be available depending on the specific purchased options and the distributor of your copy of PLS_Toolbox or Solo. See other notes below.


===Notes===
==Notes==
* Our image processing products MIA_Toolbox and Solo+MIA include additional image-specifc importers. See [[Constructing_Image_DataSets]] for more information.
* Our image processing products MIA_Toolbox and Solo+MIA include additional image-specifc importers. See [[Image_Importing_Formats]] and [[Constructing_Image_DataSets]] for more information.


* The Delimited Text File format provides a flexible way to import lots of different text file formats. For details on the format and the options available, see: [[Text Import Settings]]
* The Delimited Text File format provides a flexible way to import lots of different text file formats. For details on the format and the options available, see: [[Text Import Settings]]
Line 34: Line 61:


* In cases involving n-way (multidimensional) data you man need to "build up" you dataset. More information about how to do that can be found [http://software.eigenvector.com/faq/index.php?id=110 here].
* In cases involving n-way (multidimensional) data you man need to "build up" you dataset. More information about how to do that can be found [http://software.eigenvector.com/faq/index.php?id=110 here].
* Convert Matlab Table Object to DatasetObject using [[table2dataset]] and from DataSet to a Table object using [[dataset2table]].
===AutoImport Keywords===
The following table lists keywords (aka method names) that can be used to import the corresponding file types when using [[Solo_Predictor_Script_Construction#Importing_From_a_File|Solo Scripting import commands]] or when using the [[autoimport]] PLS_Toolbox function.
{| class="wikitable" border="1"
|+
! Keyword !! File Type
|-
| workspace || Workspace/MAT file
|-
| text || Delimited Text File (CSV,TXT)
|-
| xy || XY... Delimited Text Files (TXT,XY)
|-
| excel || Excel File (XLS,XLSX,CSV,TXT)
|-
| experimentreadr || Experiment File (EXP,CSV,XLS,TXT)
|-
| clipboard || Text from Clipboard (CSV,TXT,XML)
|-
| xml || XML file (XML)
|-
| ------ || --------------------------------
|-
| editds_mtfimport || AdventaCT MTF File (MTF)
|-
| asf || AIT ASF File (ASF, AIF, BKH)
|-
| pdfreadr || AIT PIONIR File (PDF)
|-
| asdreadr || Analytical Spectral Devices (ASD) Indico (V6 and V7)
|-
| opusreadr || Bruker OPUS File
|-
| brukerxrpdreadr || Bruker XRPD File (RAW)
|-
| cytospecreadr || CytoSpec CYT File (CYT)
|-
| envireadr||ENVI File (HDR/IMG)
|-
| spc || Grams Thermo Galactic File (SPC, DHB)
|-
| gwscanreadr || Guided Wave File (SCAN, AUTOSCAN)
|-
| hjyreadr || Horiba JY Raman File (NGS, NGC)
|-
| aqualogreadr || Horiba JY Aqualog File (DAT)
|-
| hitachieemreadr|| Hitachi EEM File (TXT)
|-
| shimadzueemreadr|| Shimadzu EEM File (TXT)
|-
| jascoeemreadr || Jasco EEM File (CSV)
|-
| jcamp || JCAMP (DX,JDX,JCAMP)
|-
| spareadr || Omnic SPA File (SPA)
|-
| spereadr || Princeton Instruments SPE File (SPE)
|-
| spgreadr || Omnic SPG File (SPG)
|-
| pereadr || PerkinElmer File (FSM, SP, VIS)
|-
| snabsreadr || Stellarnet ABS File (ABS)
|-
| ------ || --------------------------------
|-
| camecard || Cameca Ion-Tof BIF/BIF6 Image (BIF,BIF6)
|-
| opotekenvirdr || ENVI Image Format (HDR)
|-
| lispixrawreadr || Lispix Raw Formatted Image (RAW)
|-
| opotektiffrdr || Multi-layer TIFF files (TIFF)
|-
| rawread || Physical Electronics RAW Image (RAW)
|-
| specimreadr || Spectral Imaging SPECIM (HDR/RAW)
|-
| img || Image (Workspace/MAT file)
|-
| imgother || Image standard (JPG, TIFF, GIF, BMP, PNG)
|}

Latest revision as of 12:21, 27 December 2022

Data can be imported into PLS_Toolbox or Solo from a variety of sources using File>Import Data (or similar) menu itesm in the Workspace Browser, Analysis, and DataSet Editor windows.

Unsupported File Formats

Note: If you do not see the file type list below try the following:

  1. Export your data to a supported file type. Often this is delimited text, Excel, or SPC.
    • If data can be exported as text, some customization can be done via our text importing tools to accommodate different formatting:
  2. Locate a community developed importer on the Matlab file exchange.
    • Other sources exist as well, for example: CDF Tools.
  3. Obtain a file format (specification) from the vendor. We're always happy to develop (or help develop) file importers.
    • An actual specification document is great but often vendors have internally developed Matlab importers that can be used outright or be modified to work. It's also very helpful to have example files that can be used to test with.

Common Formats

Commercial Formats


Additional import methods may be available depending on the specific purchased options and the distributor of your copy of PLS_Toolbox or Solo. See other notes below.

Notes

  • The Delimited Text File format provides a flexible way to import lots of different text file formats. For details on the format and the options available, see: Text Import Settings
  • The Experiment File import method is used to import multiple data files at once along with corresponding reference values. For more information on the Experiment File format and features, see the reference page for experimentreadr.
  • In some cases you may have to use the DataSet Editor to add additional information to your dataset. For instance, you may add labels and or axis scales after you've imported the raw data. See Key_GUIs for more information on using the DataSet Editor.
  • In cases involving n-way (multidimensional) data you man need to "build up" you dataset. More information about how to do that can be found here.

AutoImport Keywords

The following table lists keywords (aka method names) that can be used to import the corresponding file types when using Solo Scripting import commands or when using the autoimport PLS_Toolbox function.


Keyword File Type
workspace Workspace/MAT file
text Delimited Text File (CSV,TXT)
xy XY... Delimited Text Files (TXT,XY)
excel Excel File (XLS,XLSX,CSV,TXT)
experimentreadr Experiment File (EXP,CSV,XLS,TXT)
clipboard Text from Clipboard (CSV,TXT,XML)
xml XML file (XML)
------ --------------------------------
editds_mtfimport AdventaCT MTF File (MTF)
asf AIT ASF File (ASF, AIF, BKH)
pdfreadr AIT PIONIR File (PDF)
asdreadr Analytical Spectral Devices (ASD) Indico (V6 and V7)
opusreadr Bruker OPUS File
brukerxrpdreadr Bruker XRPD File (RAW)
cytospecreadr CytoSpec CYT File (CYT)
envireadr ENVI File (HDR/IMG)
spc Grams Thermo Galactic File (SPC, DHB)
gwscanreadr Guided Wave File (SCAN, AUTOSCAN)
hjyreadr Horiba JY Raman File (NGS, NGC)
aqualogreadr Horiba JY Aqualog File (DAT)
hitachieemreadr Hitachi EEM File (TXT)
shimadzueemreadr Shimadzu EEM File (TXT)
jascoeemreadr Jasco EEM File (CSV)
jcamp JCAMP (DX,JDX,JCAMP)
spareadr Omnic SPA File (SPA)
spereadr Princeton Instruments SPE File (SPE)
spgreadr Omnic SPG File (SPG)
pereadr PerkinElmer File (FSM, SP, VIS)
snabsreadr Stellarnet ABS File (ABS)
------ --------------------------------
camecard Cameca Ion-Tof BIF/BIF6 Image (BIF,BIF6)
opotekenvirdr ENVI Image Format (HDR)
lispixrawreadr Lispix Raw Formatted Image (RAW)
opotektiffrdr Multi-layer TIFF files (TIFF)
rawread Physical Electronics RAW Image (RAW)
specimreadr Spectral Imaging SPECIM (HDR/RAW)
img Image (Workspace/MAT file)
imgother Image standard (JPG, TIFF, GIF, BMP, PNG)