Data Asset node

You can use the Data Asset import node to pull in data from remote data sources using connections or from your local computer. First, you must create the connection.

Note for connections to a Planning Analytics database, you must choose a view (not a cube).

You can also pull in data from a local data file (.csv, .txt, .json, .xls, .xlsx, and .sav are supported). Only the first sheet is imported from spreadsheets. In the node's properties, under DATA, select one or more data files to upload. You can also simply drag-and-drop the data file from your local file system onto your canvas.

Note: You can import a stream (.str) into Watson Studio that was created in SPSS Modeler Subscription or SPSS Modeler client. If the imported stream contains one or more import or export nodes, you'll be prompted to convert the nodes. See Importing an SPSS Modeler stream.

Setting the field delimiter, quote character, and decimal symbol

Different countries use different symbols to separate the integer part from the fractional part of a number and to separate fields in data. For example, you might use a comma instead of a period to separate the integer part from the fractional part of numbers. And, rather than using commas to separate fields in your data, you might use colons or tabs. With a Data Asset import or export node, you can specify these symbols for field delimiter and decimal. Double-click the node to open its properties and select the Field delimiter and the Decimal symbol as desired. Available delimiters are Comma, Tab, Colon, or Other. Select Other if you need to specify your own custom delimiter.
Figure 1. Field delimiter and decimal symbol options
Field delimiter and decimal symbol options

Importing data from an SPSS Statistics file

If you import data from an SPSS Statistics file (.sav), the following options are available:

Variable names. Select a method of handling variable names and labels upon import from an SPSS Statistics .sav file. Metadata that you choose to include here persists throughout your work inSPSS Modeler and may be exported again for use in IBM SPSS Statistics.
  • Read names and labels. Select to read both variable names and labels into SPSS Modeler. This is the default option, and variable names are displayed in the Type node. Labels may be displayed in charts, model browsers, and other types of output. By default, the display of labels in output is disabled.
  • Read labels as names. Select to read the descriptive variable labels from the SPSS Statistics .sav file rather than the short field names, and use these labels as variable names in SPSS Modeler.
Values. Select a method of handling values and labels upon import from an SPSS Statistics .sav file. Metadata that you choose to include here persists throughout your work in SPSS Modeler and may be exported again for use in SPSS Statistics.
  • Read data and labels. Select to read both actual values and value labels into SPSS Modeler. This is the default option, and values themselves are displayed in the Type node. Value labels may be displayed in the Expression Builder, charts, model browsers, and other types of output.
  • Read labels as data. Select if you want to use the value labels from the .sav file rather than the numerical or symbolic codes used to represent the values. For example, selecting this option for data with a gender field whose values of 1 and 2 actually represent male and female, respectively, will convert the field to a string and import male and female as the actual values.

    It's important to consider missing values in your SPSS Statistics data before selecting this option. For example, if a numeric field uses labels only for missing values (0 = No Answer, –99 = Unknown), then selecting the Read labels as data option will import only the value labels No Answer and Unknown and will convert the field to a string. In such cases, you should import the values themselves and set missing values in a Type node.

Use field format information to determine storage. If you deselect this option, field values that are formatted in the .sav file as integers (i.e., fields specified as Fn.0 in the Variable View in IBM SPSS Statistics) are imported using integer storage. All other field values except strings are imported as real numbers.

If you select this option (default), all field values except strings are imported as real numbers, whether formatted in the .sav file as integers or not.

Read timestamp as date. By default, all timestamp values are shown as dates. Deselect this option to override this behavior.

Using SQL to pull in data

In the Data Asset import node properties, under Mode, you can select SQL Query if you want to use custom SQL to import data from a database. Use a SQL SELECT statement to pull in rows or columns of data from a database. Note that the Source path field doesn't apply if you're using the SQL Query mode.
Figure 2. Custom SQL query
Custom SQL query
The following example pulls in certain rows of data from a database table:
select * from GOSALES.ORDER_DETAILS
where  UNIT_COST > 40,000 LIMIT 4
And this example pulls in certain columns of data from a database table:
select QUANTITY, UNIT_COST, UNIT_PRICE from GOSALES.ORDER_DETAILS
Note that the SQL syntax you use can vary depending on database platform. For example, if pulling in data from an Informix database, Informix requires field names to be surround by double quotes. For example:
select "Age", "Sex" from testuser.canvas_drug

This SQL feature should only be used to pull in data. Use caution so as not to manipulate the data in your database.

The following databases currently support this custom SQL feature:
  • Amazon Redshift
  • Apache Hive
  • Cloudera Impala
  • Compose for PostgreSQL
  • Db2 on Cloud
  • Db2 Warehouse
  • Google BigQuery
  • Informix
  • Microsoft SQL Server
  • MySQL
  • Netezza
  • Oracle
  • Pivotal Greenplum
  • Salesforce.com
  • Snowflake
  • Sybase
  • Sybase IQ
  • Teradata

Oracle

When using the Data Asset node to pull in data from an Oracle database, you may encounter errors. To work around this issue, perform the following steps:
  1. Add or modify the following line in sqlnet.ora of your Oracle database.
    SQLNET.ALLOWED_LOGON_VERSION_SERVER=8
  2. Restart the database.
  3. Create a new user and set a password for it, or change the password of an existing user.
  4. Use the following query to verify that the user has 10G in its PASSWORD_VERSIONS. If it doesn't, add it.
    select USERNAME,ACCOUNT_STATUS,PASSWORD_VERSIONS from dba_users;
  5. In sqlnet.ora, change the value from 8 to 11 in the following line.
    SQLNET.ALLOWED_LOGON_VERSION_SERVER=11
  6. Verify that the user can still connect to the database.