Importing Data from Json Files

You can import data directly from a Json file, specifying a specific file.

You can import Json files in two different ways:

  • By dragging and dropping the Json source file directly onto the stage.

  • By using the Import from Json file task. The advantage of using the task is that you can configure the import options and also specify whether you want to import single or multiple files:

    • Single file: only one file is imported, specifying the file from which information will be taken.

    • Multiple files: in this case the files are concatenated to form a single table. Consequently, all files imported together must have the same structure.

Wildcards can be used in a filename/file list. For example, entering C:\Software*.csv during a file import, will result in the bulk import all the files in the Software folder, whose extension is .csv. Wildcards can be used in file names, but not in folder names, or in file extensions. Remember that only wildcard (*) can be used in the file path, and any consequent asterisks will be considered an effective part of the path, and not a wildcard. If the file extension is not actually specified when using a wildcard, files with extensions appropriate to the task in use will be searched for. For example, the extensions .xlsx, .xlsm for Import from Excel File, and .csv, .txt, .tab, etc. for Import from Text File.


Prerequisites

Additional tabs

The following additional tabs are provided:


Procedure

  1. Drag and drop the Import from Json File task onto the central stage.

  2. Double click the task.

  3. If you want to import data via a remote connection, such as HTTP API, a SharePoint site or FTP, select the corresponding source from the Source file URI (uri) drop-down list and configure the connection in the Remote Connections tab.

  4. To import a single Json file:

    • In the Options tab either click Select file (filename) to browse to the Json file you want to import, and click Open, or manually enter the name of the file in the corresponding edit box: the Table preview pane displays the data that will be imported into Rulex, and is dynamically updated each time you change any of the available options.

    • Configure the options as explained in the Single file options below.

  5. To import multiple files click on the Advanced tab and configure the options as explained in the Multiple file options below.

  6. Save and compute the task.

Single file options:

Parameter Name

PO

Description

Number separators

thousep, decsep

Select the separators you want to use for Thousands (thousep) and Decimals (decsep) from the corresponding drop-down lists.

Missing string

missingstr

Specify a character (or string of characters), which can be used to indicate missing values.

Key for types

typestr

Enter the name of the attribute that defines the data type of the columns in the JSON file.

Remove empty rows

delemptyrows

If selected empty rows are automatically deleted.

Remove empty columns

delemptycols

If selected empty columns are automatically deleted.

Strip spaces

stripspaces

If selected, all spaces are removed from strings. For example, the string ” age ” is then imported as “age”.

Compress white spaces

onlyonespace

Select this option to remove extra consecutive spaces from within strings. For example the string "university    program" would be imported as "university program".

Add an attribute containing filename

addfilename

If selected an attribute column is added to the table with the name of the file.

Use old computation data if source file is not available

useolddata

If selected data from the previous computations will be used if the source table is not available.

Continue the execution if the file is missing

continue

If selected, computation of the task continues, even if the selected source files are not available.

Turn off smart type recognition
NEW FOR RULEX 4.7

nosmarttype

If selected, the the data types of attributes is not automatically recognized, leaving the generic nominal type. This option is useful when manual identification is preferable, for example when there is the risk of a code being misinterpreted as a date.

Add group columns

addgroups

If selected, a column is added containing additional information on the group tags belong to within the dataset (for example, 1,1,1,2,2,3,3 if there are 3 possible groups).
This additional information enables Rulex to recreate the nested structure during export operations.

Wait until the target file is present

checkfile, sleeptime

If selected, Rulex polls the target file with the frequency specified (sleeptime) until it is available.

Number of records to preview

nrecords

Specify how many records the table preview will display.

Multiple file options:

Drag and drop files to concatenate

filelist

Drag and drop the required files here. All imported files must have the same structure.

The list of selected files correspond to the filelist parametric option.

Select files/directories

filename

If you are importing data from a remote connection, click this option to select the files and directories from which you want to import data.

Concatenation type

cattype

Select the required concatenation type, which may be:

  • Inner (0), where only attributes present in both tables are included in the final merged table

  • Outer (1), where all attributes are copied, filling in any missing values if necessary.

Match columns by

byname

Select whether you want to match columns by:

  • Position (0), to select the required column according to the position in the sheet

  • Name (1), to select the required column according to its name.