Conditionally Importing Data from Databases

At times you may not want to import all the data found in a particular database query, but only a subset of it.

Rulex makes this possible through the Conditional Import task.

This task is the only Rulex import task that accepts input data. This data contains the filters you want to apply in the import operation.


Prerequisites

Additional tabs

The following additional tabs are provided:

  • Query tab, where the query created in the Options tab is displayed. If required a pre-existing query can be copied and pasted to this tab, instead of building the query in the Rulex interface.

  • Documentation tab where you can document your task,

  • Parametric options tab where you can configure process variables instead of fixed values. Parametric equivalents are expressed in italics in this page (PO).


Procedure

  1. Drag and drop the Conditional Import task onto the central stage.

  2. Double click the task.

  3. In the Database parameters section, configure the required parameters as described in the table below.

  4. Click Connect: a connection is established with the database and the Table list box is filled with the list of all the available tables which are presented in the database. 

  5. Configure the general options as described in the table below.

  6. Configure the advanced options, as described in the table below, to specify any filters, or define which attributes will be imported.

  7. Save and compute the task.

Import from database options

Database parameters

Parameter

PO

Description

Platform

type

The database platform type. The following platforms are currently supported:

  • Azure Data Warehouse

  • IBM DB2 AS400 FROM VERSION 4.7

  • IBM_DB2

  • Impala

  • Microsoft Access FROM VERSION 4.7

  • MySQL

  • Oracle

  • PostegreSQL

  • SAP Hana

  • Spark

  • SQL Server

  • SQLBase

  • SQLite

  • Teradata

Unicode is also supported, if its corresponding driver is installed.

Server

server

The target database server IP address.

Port

port

The number of the specific process to connect to.

Database name

name

The name of the database you want to use (or the database file name if using SQLite).

Authentication

winauth

The method of authentication used to access the database.

Possible values are:

  • O (User Name/Password)

  • 1  (Windows Authentication)

User name

usr

The login user name.

This parameter is mandatory only when User Name/Password has been selected as the authentication mode

Password

pwd

The login password.

This parameter is mandatory only when User Name/Password has been selected as the authentication mode.

Schema

schema

The database schema.

Driver

driver

ODBC driver.

Service name

service

Database service name of the Oracle database.

Bulk size for prefetch

bulksize

Retrieves blocks of data of the size specified.

Additional options

connparam

Additional parameters which may be required by ODBC drivers (according to database used) can be expressed through this edit box with the following format: option1=value1;option2=value2.

The DSN value can also be inserted here, if present, with the following format: DSN=value.

General options

Repeat query execution every

checkdata, sleeptime, nres

Select to repeat the query until the required number of records are retrieved (checkdata).

Subsequently set the frequency in seconds (sleeptime) with which the query must be performed and the number of records (nres) that must be retrieved. 

Strip spaces

stripspaces

Select to remove spaces from strings. For example, the string ” age ” is then imported as “age”.

Compress white spaces

onlyonespace

Select this option to remove extra consecutive spaces from within strings. For example the string "university    program" would be imported as "university program".

Use old computation data if the source table is not available

useolddata

Select to use data from the previous computation if the source table is not available.

Table list

N/A

Select the tables you want to load from this list: they will be displayed in the Load data from table box.

The table selected corresponds to the parametric option tablename.

Search table

N/A

If your selected database contains many tables, you can use the search function to find the required table, by entering its name in the edit box.

Case sensitive

N/A

If selected, the search operation will be performed in case sensitive mode. 

Load data from table

tablename

The name of the table from which want to select data. You can either select the table from the table list, or manually enter the name.

Extract query

N/A

Click to view the query that you have constructed in the interface in the Query tab, where you can easily modify it if required.

The SQL query itself corresponds to the parametric option Query.

Advanced options

Available attributes


The left-hand list displays all the available attributes, which can be used to filter results. These attributes can be ordered in the corresponding option below to narrow down the list when there are many attributes. It is also possible to search for specific attributes. 

Filtering attributes

wherenames

Drag and drop the attributes you want to use to filter the imported results onto this pane from the Available Attributes list on the left.

The attribute names in Rulex must perfectly correspond to the attribute names in the resulting query.

Imported attributes

valuenames

Drag and drop the attributes you want to import onto this pane from the Available Attributes list on the left.

If no attributes are specified, all attributes will be imported.