Data Importing Formats: Difference between revisions

From Eigenvector Research Documentation Wiki
Jump to navigation Jump to search
imported>Jeremy
No edit summary
No edit summary
 
(49 intermediate revisions by 7 users not shown)
Line 1: Line 1:
Data can be imported into PLS_Toolbox or Solo from a variety of sources using '''File>Import Data''' (or similar) menu itesm in the [[Workspace Browser]], [[Analysis GUI|Analysis]], and [[DataSet Editor]] windows.
== Unsupported File Formats ==
'''Note''': If you do not see the file type list below try the following:
# Export your data to a supported file type. Often this is delimited text, Excel, or SPC.
#* If data can be exported as text, some customization can be done via our text importing tools to accommodate different formatting:
#:* For more information on using the textreadr function see [[Textreadr]]
#:* Create a custom import function [[Editds_userimport]]
# Locate a community developed importer on the Matlab [http://www.mathworks.com/matlabcentral/fileexchange/ file exchange].
#* Other sources exist as well, for example: [http://cdf.gsfc.nasa.gov/html/dttools.html CDF Tools].
# Obtain a file format (specification) from the vendor. We're always happy to develop (or help develop) file importers.
#* An actual specification document is great but often vendors have internally developed Matlab importers that can be used outright or be modified to work. It's also very helpful to have example files that can be used to test with.


== Common Formats ==
* Workspace/MAT file - ''see [[WorkspaceBrowser_ImportingData#To_import_a_.mat_file|Importing .mat Files]]''
* [[xclreadr|Delimited Text File (CSV,TXT)]] - ''see [[Text Import Settings]]''
* [[xyreadr|XY... Delimited Text Files (TXT,XY)]] - ''see [[Text Import Settings]]''
* [[xlsreadr|Excel File (XLS,XLSX,CSV,TXT)]]
* [[Experimentreadr |Experiment File (EXP,CSV,XLS,TXT)]]
* [[table2dataset]] - Convert Matlab Table Array to [[Dataset]].
* Text from Clipboard (CSV,TXT,XML)
* Extensible Markup Language (XML) - ''see [[DataSet XML Format]]''


Data can be imported into PLS_Toolbox or Solo from a variety of sources using '''File>Import Data''' (or similar) in the [[Workspace Browser]], [[Analysis GUI|Analysis]], and [[DataSet Editor]] windows. Sources include:
== Commercial Formats ==
* [[mtfreadr|AdventaCT MTF File (MTF)]]
* [[asfreadr|AIT ASF File (ASF, AIF, BKH)]]
* [[pdfreadr|AIT PIONIR File (PDF)]]
* [[asdreadr|Analytical Spectral Devices (ASD) Indico (V6 and V7)]]


* Workspace/MAT file
* [[envireadr|ENVI Image File (HDR/IMG) Image DSO if MIA_Toolbox installed.]]
* Delimited Text File (CSV,TXT)
* [[opusreadr|Bruker OPUS]]
* XY... Delimited Text Files (TXT,XY)
* [[brukerxrpdreadr|Bruker XRPD (RAW)]]
* Excel File (XLS,CSV,TXT)
* [[cytospecreadr| CytoSpec CYT File (CYT)]]
* Experiment File (CSV, TXT, XLS)
* [[spcreadr|Grams Thermo Galactic File (SPC, DHB)]]
* Hamilton Sundstrand ASF File (ASF, AIF, BKH)
* [[gwscanreadr|Guided Wave File (SCAN, AUTOSCAN)]]
* Hamilton Sundstrand PDF File (PDF)
* [[hitachieemreadr| Hitachi EEM File (TXT)]]
* Horiba JY Files (NGS, NGC)
* [[hjyreadr|Horiba JY Raman File (NGS, NGC)]]
* Thermo Galactic File (SPC)
* [[aqualogreadr|Horiba JY Aqualog ASCII File (DAT)]]
* JCAMP [general] (DX,JDX)
* [[jascoeemreadr|Jasco EEM Files (CSV)]]
* Extensible Markup Language (XML)
* [[jcampreadr|JCAMP (DX,JDX)]]
* Paste XML from Clipboard
* [[rdareadr|Siemens RDA file (RDA)]]
* [[spareadr|Omnic SPA File (SPA)]]
* [[spereadr|Princeton Instruments SPE File (SPE)]]
* [[pereadr|PerkinElmer (FSM, SP, VIS, IMP)]]
* [[snabsreadr|Stellarnet ABS File (ABS)]]
* [[shimadzueemreadr|Shimadzu EEM File (TXT)]]
* [[visionairxmlreadr|Vision Air XML Data File (XML)]]
* Other... - Call Matlab importer.
----
Additional import methods may be available depending on the specific purchased options and the distributor of your copy of PLS_Toolbox or Solo. See other notes below.


Additional import methods may be available depending on the specific purchased options and the distributor of your copy of PLS_Toolbox or Solo. See other notes below.
==Notes==
* Our image processing products MIA_Toolbox and Solo+MIA include additional image-specifc importers. See [[Image_Importing_Formats]] and [[Constructing_Image_DataSets]] for more information.


===Notes===
* The Delimited Text File format provides a flexible way to import lots of different text file formats. For details on the format and the options available, see: [[Text Import Settings]]
* Our image processing products MIA_Toolbox and Solo+MIA include additional image-specifc importers. See [[Constructing_Image_DataSets]] for more information.


* The [[experimentreadr|'''Experiment File''' import method]] is used to import multiple data files at once along with corresponding reference values. For more information on the Experiment File format and features, see the reference page for [[experimentreadr]].
* The [[experimentreadr|'''Experiment File''' import method]] is used to import multiple data files at once along with corresponding reference values. For more information on the Experiment File format and features, see the reference page for [[experimentreadr]].
Line 28: Line 61:


* In cases involving n-way (multidimensional) data you man need to "build up" you dataset. More information about how to do that can be found [http://software.eigenvector.com/faq/index.php?id=110 here].
* In cases involving n-way (multidimensional) data you man need to "build up" you dataset. More information about how to do that can be found [http://software.eigenvector.com/faq/index.php?id=110 here].
* Convert Matlab Table Object to DatasetObject using [[table2dataset]] and from DataSet to a Table object using [[dataset2table]].
===AutoImport Keywords===
The following table lists keywords (aka method names) that can be used to import the corresponding file types when using [[Solo_Predictor_Script_Construction#Importing_From_a_File|Solo Scripting import commands]] or when using the [[autoimport]] PLS_Toolbox function.
{| class="wikitable" border="1"
|+
! Keyword !! File Type
|-
| workspace || Workspace/MAT file
|-
| text || Delimited Text File (CSV,TXT)
|-
| xy || XY... Delimited Text Files (TXT,XY)
|-
| excel || Excel File (XLS,XLSX,CSV,TXT)
|-
| experimentreadr || Experiment File (EXP,CSV,XLS,TXT)
|-
| clipboard || Text from Clipboard (CSV,TXT,XML)
|-
| xml || XML file (XML)
|-
| ------ || --------------------------------
|-
| editds_mtfimport || AdventaCT MTF File (MTF)
|-
| asf || AIT ASF File (ASF, AIF, BKH)
|-
| pdfreadr || AIT PIONIR File (PDF)
|-
| asdreadr || Analytical Spectral Devices (ASD) Indico (V6 and V7)
|-
| opusreadr || Bruker OPUS File
|-
| brukerxrpdreadr || Bruker XRPD File (RAW)
|-
| cytospecreadr || CytoSpec CYT File (CYT)
|-
| envireadr||ENVI File (HDR/IMG)
|-
| spc || Grams Thermo Galactic File (SPC, DHB)
|-
| gwscanreadr || Guided Wave File (SCAN, AUTOSCAN)
|-
| hjyreadr || Horiba JY Raman File (NGS, NGC)
|-
| aqualogreadr || Horiba JY Aqualog File (DAT)
|-
| hitachieemreadr|| Hitachi EEM File (TXT)
|-
| shimadzueemreadr|| Shimadzu EEM File (TXT)
|-
| jascoeemreadr || Jasco EEM File (CSV)
|-
| jcamp || JCAMP (DX,JDX,JCAMP)
|-
| spareadr || Omnic SPA File (SPA)
|-
| spereadr || Princeton Instruments SPE File (SPE)
|-
| spgreadr || Omnic SPG File (SPG)
|-
| pereadr || PerkinElmer File (FSM, SP, VIS)
|-
| snabsreadr || Stellarnet ABS File (ABS)
|-
| ------ || --------------------------------
|-
| camecard || Cameca Ion-Tof BIF/BIF6 Image (BIF,BIF6)
|-
| opotekenvirdr || ENVI Image Format (HDR)
|-
| lispixrawreadr || Lispix Raw Formatted Image (RAW)
|-
| opotektiffrdr || Multi-layer TIFF files (TIFF)
|-
| rawread || Physical Electronics RAW Image (RAW)
|-
| specimreadr || Spectral Imaging SPECIM (HDR/RAW)
|-
| img || Image (Workspace/MAT file)
|-
| imgother || Image standard (JPG, TIFF, GIF, BMP, PNG)
|}

Latest revision as of 12:21, 27 December 2022

Data can be imported into PLS_Toolbox or Solo from a variety of sources using File>Import Data (or similar) menu itesm in the Workspace Browser, Analysis, and DataSet Editor windows.

Unsupported File Formats

Note: If you do not see the file type list below try the following:

  1. Export your data to a supported file type. Often this is delimited text, Excel, or SPC.
    • If data can be exported as text, some customization can be done via our text importing tools to accommodate different formatting:
  2. Locate a community developed importer on the Matlab file exchange.
    • Other sources exist as well, for example: CDF Tools.
  3. Obtain a file format (specification) from the vendor. We're always happy to develop (or help develop) file importers.
    • An actual specification document is great but often vendors have internally developed Matlab importers that can be used outright or be modified to work. It's also very helpful to have example files that can be used to test with.

Common Formats

Commercial Formats


Additional import methods may be available depending on the specific purchased options and the distributor of your copy of PLS_Toolbox or Solo. See other notes below.

Notes

  • The Delimited Text File format provides a flexible way to import lots of different text file formats. For details on the format and the options available, see: Text Import Settings
  • The Experiment File import method is used to import multiple data files at once along with corresponding reference values. For more information on the Experiment File format and features, see the reference page for experimentreadr.
  • In some cases you may have to use the DataSet Editor to add additional information to your dataset. For instance, you may add labels and or axis scales after you've imported the raw data. See Key_GUIs for more information on using the DataSet Editor.
  • In cases involving n-way (multidimensional) data you man need to "build up" you dataset. More information about how to do that can be found here.

AutoImport Keywords

The following table lists keywords (aka method names) that can be used to import the corresponding file types when using Solo Scripting import commands or when using the autoimport PLS_Toolbox function.


Keyword File Type
workspace Workspace/MAT file
text Delimited Text File (CSV,TXT)
xy XY... Delimited Text Files (TXT,XY)
excel Excel File (XLS,XLSX,CSV,TXT)
experimentreadr Experiment File (EXP,CSV,XLS,TXT)
clipboard Text from Clipboard (CSV,TXT,XML)
xml XML file (XML)
------ --------------------------------
editds_mtfimport AdventaCT MTF File (MTF)
asf AIT ASF File (ASF, AIF, BKH)
pdfreadr AIT PIONIR File (PDF)
asdreadr Analytical Spectral Devices (ASD) Indico (V6 and V7)
opusreadr Bruker OPUS File
brukerxrpdreadr Bruker XRPD File (RAW)
cytospecreadr CytoSpec CYT File (CYT)
envireadr ENVI File (HDR/IMG)
spc Grams Thermo Galactic File (SPC, DHB)
gwscanreadr Guided Wave File (SCAN, AUTOSCAN)
hjyreadr Horiba JY Raman File (NGS, NGC)
aqualogreadr Horiba JY Aqualog File (DAT)
hitachieemreadr Hitachi EEM File (TXT)
shimadzueemreadr Shimadzu EEM File (TXT)
jascoeemreadr Jasco EEM File (CSV)
jcamp JCAMP (DX,JDX,JCAMP)
spareadr Omnic SPA File (SPA)
spereadr Princeton Instruments SPE File (SPE)
spgreadr Omnic SPG File (SPG)
pereadr PerkinElmer File (FSM, SP, VIS)
snabsreadr Stellarnet ABS File (ABS)
------ --------------------------------
camecard Cameca Ion-Tof BIF/BIF6 Image (BIF,BIF6)
opotekenvirdr ENVI Image Format (HDR)
lispixrawreadr Lispix Raw Formatted Image (RAW)
opotektiffrdr Multi-layer TIFF files (TIFF)
rawread Physical Electronics RAW Image (RAW)
specimreadr Spectral Imaging SPECIM (HDR/RAW)
img Image (Workspace/MAT file)
imgother Image standard (JPG, TIFF, GIF, BMP, PNG)