DataSet Object Examples
DataSet Object Tour
Perhaps the best way to understand how DSOs work is to examine a couple of them. Several data sets are included with PLS_Toolbox, and all of them are in the form of DSOs. We will start with the smallest one, the Wine data set. Clear the MATLAB workspace (save anything important first!) and at the command line type:
>> load wine >> whos Name Size Bytes Class wine 10x5 6050 dataset object Grand total is 920 elements using 6050 bytes
We have now loaded the Wine data set. When the whos command is used, we see that there is a single variable in the workspace, wine, of Class dataset object, with a data field that is 10 by 5. We can look at the contents of wine by typing:
>> wine
wine =
name: Wine
type: data
author: B.M. Wise
date: 14-May-2001 13:47:54
moddate: 06-Jun-2001 10:27:24
data: 10x5 [double]
label: {2x1} [array (char)]
Mode 1 [Country: 10x6]
Mode 2 [Variable: 5x6]
axisscale: {2x1} [vector (real)]
Mode 1 [: ]
Mode 2 [: ]
title: {2x1} [vector (char)]
Mode 1 [: 'Country']
Mode 2 [: 'Variable']
class: {2x1} [vector (integer)]
Mode 1 [: ]
Mode 2 [: ]
include: {2x1} [vector (integer)]
Mode 1 [: 1x10]
Mode 2 [: 1x5]
description: Wine, beer, and liquor consumption (gal/yr), life
life expectancy (years), and heart disease rate
(cases/100,000/yr) for 10 countries.
history: {1x1 cell} [array (char)]
userdata:
From this we see that the name of the data is Wine and that the type is “data.” Other types are also possible, such as “image” and “batch.” The author is listed, followed by the creation date and last-modified date. The next field, data, contains the actual data table. These data, or the data from any DSO field, can be extracted from the DSO just as they would be from a conventional structure array (type help struct or refer to the Examining a Structure Array section of Chapter 2 for help) using DSOname.fieldname syntax. For instance:
>> wine.data
ans =
2.5000 63.5000 40.1000 78.0000 61.1000
0.9000 58.0000 25.1000 78.0000 94.1000
1.7000 46.0000 65.0000 78.0000 106.4000
1.2000 15.7000 102.1000 78.0000 173.0000
1.5000 12.2000 100.0000 77.0000 199.7000
2.0000 8.9000 87.8000 76.0000 176.0000
3.8000 2.7000 17.1000 69.0000 373.6000
1.0000 1.7000 140.0000 73.0000 283.7000
2.1000 1.0000 55.0000 79.0000 34.7000
0.8000 0.2000 50.4000 73.0000 36.4000
The labels can be extracted in a similar manner:
>> wine.label ans = [10x6] char [ 5x6] char
Note that ans is a cell array, i.e., the labels for each mode of the array are stored in a cell that is indexed to that mode. Thus, the labels for mode 1, the data set rows, can be extracted with:
>> wine.label{1}
ans =
France
Italy
Switz
Austra
Brit
U.S.A.
Russia
Czech
Japan
Mexico
Note that curly brackets, {}, are used to index into cell arrays (type help cell for more information on cell arrays). In a similar way the labels for mode 2, the data set columns, can be extracted by executing:
>> wine.label{2}
ans =
Liquor
Wine
Beer
LifeEx
HeartD
Other fields in the DSO include .axisscale (e.g., time or wavelength scale), .title (titles for the axes), and .class (e.g., class variables for samples). Note that a typical data set will not have all of the available fields filled. The Wine data set does not have axis scales, for instance, nor class variables.
DSOs also allow for multiple sets of many of these fields; for instance, you may store more than one set of labels for a particular mode. Most GUI tools including Analysis, PlotGUI and the DataSet Editor support multiple sets but there are some rare situations where their use has not yet been fully implemented. GUIs allow limited support of multiple sets; axis scales and titles are not yet completely supported in the main Analysis GUI.
The user is encouraged to explore more of the DSOs included with PLS_Toolbox. For an example with axis scales, please see spec1 in the nir_data.mat file. For an example with class variables, please see arch in the arch.mat file.
Creating A DataSet Object
The following shows an example using the 'wine' data set in the PLS_Toolbox. Other examples can be found in the datasetdemo.m script.
The first step in the example is to load the 'wine' data set and examine the variables. The MATLAB commands are:
»load wine_raw »whos Name Size Bytes Class dat 10x5 400 double array names 10x6 120 char array vars 5x6 60 char array
The variable 'dat' contains the data array corresponding to the 5 variables wine, beer, and liquor consumption, life expectancy, and heart disease for 10 samples (countries).
The country names are contained in the variable 'names' and the variable names are contained in 'vars'. The next step creates a DataSet object, gives it a name, authorship, and description.
»wined = dataset(dat);
»wined.name = 'Wine';
»wined.author = 'A.E. Newman';
»wined.description= ...
{'Wine, beer, and liquor consumption (gal/yr)',...
'life expectancy (years), and heart disease rate', ...
'(cases/100,/yr) for 10 countries.'};
»wined.label{1} = names;
»wined.label{2} = vars;
Additional assignments can also be made. Here the label for the first mode (rows) is shown explicitly next to the data array (like sample labels). Also, titles, axis, and titles are assigned.
»wined.labelname{1} = 'Countries';
»wined.label{1} = ...
{'France' ...
'Italy', ...
'Switz', ...
'Austra', ...
...
'Mexico'};
»wined.title{1} = 'Country';
»wined.class{1} = [1 1 1 2 3];
»wined.classname{1} = 'Continent';
»wined.axisscale{1} = 1:5;
»wined.axisscalename{1} = 'Country Number';
Additional assignments can also be made for mode 2. Here the label for the second mode (columns) is shown explicitly above the data array (like column headings). Also, titles, axis, and titles are assigned.
»wined.labelname{2} = 'Variables';
»wined.label{2} = ...
{'Liquor','Wine','Beer','LifeExp','HeartD'};
If the data matrix is N-way the assignment process can be extended to Mode 3, Mode 4, ... Mode N. It can also be extended to using multiple sets of labels and axis scales e.g.
»wined.labelname{2,2} = 'Alcohol Content and Quality';
»wined.label{2,2} = {'high','medium','low','good','bad'};
An individual label can be replaced by further indexing into a given label set using curly braces followed by the string replacement:
»wined.label{2,2}{4} = 'excellent';
Indexing Into DataSets
Sub-portions of the DataSet can be retrieved by indexing into the main DataSet object. For example, here the first three columns ('Liquor', 'Wine', and 'Beer') are extracted into a new DataSet named "alcohol":
»alcohol = wined(:,1:3);
Additionally, any field in the DataSet can also be indexed into directly. Here the second country name is pulled out of the labels by extracting the entire second row of the mode 1 labels:
»country2 = wined.label{1}(2,:);
Indexing using Labels and Classes
A shortcut to extract a subset of a DataSet is to index into the main DataSet object using labels and/or classes for the requested item(s). For example, to extract a DataSet containing only the Liquor values, you could use:
»alcohol = wined.liquor;
Note that the upper-case characters in the label do not matter. If the label or class starts with a number of contains any non-alphanumeric characters, you must enclose the label in parenthesis and quotes:
»alcohol = wined.('liquor');
Using the Class Lookup Table
Classes in a DataSet are stored as numeric values (in the .class field). Each numeric value and be associated with a text value using the classlookup field. This field contains a simple nx2 table with numeric values in the first column and string values in the second. For example, the arch dataset has a classlookkup table for elements:
>> load arch
>> arch.classlookup{1,1}
ans =
[0] 'Class 0'
[1] 'K'
[2] 'BL'
[3] 'SH'
[4] 'AN'
There are two basic ways to alter a classlookup table, by extracting the table or directly accessing the table.
Extracting the Table
Extracting the table can be useful if you need to perform several changes:
a = arch.classlookup{1};
a{3,2} = 'YYY';
a{4,2} = 'ZZZ';
arch.classlookup{1} = a;
arch.classlookup{1}
ans =
[0] 'Class 0'
[1] 'K'
[2] 'YYY'
[3] 'ZZZ'
[4] 'AN'
Direct Access
If you just need to change a single value (or a small number of values) you can directly access the lookup table using .assignstr and .assignval fields. For example, the change class 0 of arch from "Class 0" to "Unknown":
arch.classlookup{1}.assignstr = {0 'Unknown'}
>> arch.classlookup{1}
ans =
[0] 'Unknown'
[1] 'K'
[2] 'BL'
[3] 'SH'
[4] 'AN'
Then, to change the numeric value of "Unknown" from 0 to 6:
>> arch.classlookup{1}
arch.classlookup{1}.assignval = {6 'Unknown'}
ans =
[0] 'Class 0'
[1] 'K'
[2] 'BL'
[3] 'SH'
[4] 'AN'
[6] 'Unknown'
Using Image DataSets
The DataSet Object contains functionality for handling image data. In image DataSet objects, the .data field contains "unfolded" image data. Image data is usually contained in a 2nd or higher-order matrix in which several modes are used to describe a spatial relationship between pieces of information. For example, many standard JPEG images are three-way images of size M x N x 3. The first two dimensions are the spatial dimensions in that the actual image is M pixels high by N pixels wide. The third dimension is the wavelength dimension and, in this case, contains 3 slabs – one for each of the Red, Green, Blue image components.
Working with such image data in DataSet objects is made easier by unfolding such multi-way images so that all the spatial modes are stacked on top of each other in a single mode. Unfolding is done so that all the spatial information is contained in a single mode and can be handled together – often so that each pixel can be analyzed as an individual sample (or even sometimes as variables). Individual pixels in an unfolded image are independent and can be individually included and excluded (see .include field) or assigned particular classes (see .class field), for example. In the case of the JPEG mentioned above, the unfolded image would be stored as an (MN) x 3 matrix where the first mode was M times N elements (i.e. pixels) in size.
The .imagemode field contains a scalar value indicating which mode of the .data field contains the spatial information. In the JPEG example, .imagemode would be 1 (one). Similarly the .imagesize field contains a vector describing the original size of the spatial mode before unfolding. The JPEG example would contain the two-element vector: [M N] Note: that .imagesize contains only the size of the image mode, not the entire data matrix. Note that the product of the .imagesize field must be equal to the size of the .imagemode mode of the .data field. That is, the number of pixels contained in the spatial mode of the data must be appropriate that it can be reshaped into a matrix of size .imagesize. See the .foldedsize field for the size of entire folded matrix.
The .imagedata field is a special read-only field which returns the contents of the .data field refolded back into the original image-sized matrix. In the JPEG example, the .data field would return a matrix of size (MN) x 3 (the unfolded image) but the .imagedata field would return the original M x N x 3 matrix. Any changes you make to the contents of the .data field will automatically be reflected in the contents returned by .imagedata. .imagedata, however, can not be written to.
The following example uses functions found in MIA_Toolbox.
Build an image dataset from the demonstration data "EchoRidgeClouds.jpeg" included with MIA_Toolbox.
>> dat = imread('EchoRidgeClouds.jpeg','jpeg');
size(dat)
ans =
768 512 3
>> imgdso = buildimage(dat, [1 2], 1)
imgdso =
name:
type: image
author:
date: 08-Sep-2009 21:21:13
moddate: 08-Sep-2009 21:21:13
data: 393216x3 [double]
imagesize: 768x512
imagemode: 1
label: {2x1} [array (char)]
Mode 1 [: ]
Mode 2 [: ]
axisscale: {2x1} [vector (real)]
Mode 1 [: ] (none)
Mode 2 [: ] (none)
title: {2x1} [vector (char)]
Mode 1 [: ]
Mode 2 [: ]
class: {2x1} [vector (double)]
Mode 1 [: ]
Mode 2 [: ]
classid: {2x1} [cell of strings]
include: {2x1} [vector (integer)]
Mode 1 [: 1x393216]
Mode 2 [: 1x3]
description:
history: {1x1 cell} [array (char)]
userdata:
Now we can look at the size of data returned by each field:
>> mydata = imgdso.data; >> myimage = imgdso.imagedata; >> whos Name Size Bytes Class Attributes dat 768x512x3 1179648 uint8 imgdso 393216x3 12591234 dataset mydata 393216x3 9437184 double myimage 768x512x3 9437184 double
Notice the image data is returned as double. If we were to use the imagesc function in Matlab, we'd need to convert the data to uint8 before plotting. There are several image reading functions that return data in an image type DSO. When plotting image DSOs, plotgui will default to using the image (folded data) when appropriate.