Parameterizing DQC Components

Reported for version 10

DQC Components can be parameterized either by mapping certain values to parameters using plans, workflows or via a console. Parameters accept default values, which are used whenever the parameter is not passed. 

Parameterizing of a DQC component

To map a parameter, right-click on a mark such as the expression or the name of a column of the step you use, as seen in the image below and select Map as Parameter. This will open a dialog to fill in parameter options such as the name of the parameter or description. The same procedure applies for parameterizing a file name within the Text File Reader step and is shown in the figure describing using workflows.

Sending parameters to the component from outside

Using plans

Parameters can be mapped when using a component in a plan. When opening a component as a step, the tab Parameters shows all active parameters and their values. Hovering over the field shows the description of the parameter.

Using workflows

To parameterize a component using workflows create a component along a workflow. In the example below, a file name parameterizing is shown. For such case, you need to create a component that will read an input file and write the output to the output file. The input and output file names in the example will be provided later on as parameters.

The same instructions apply for Text File Writer setup.

Note that when running your component within a plan, you need to provide a file name in the Text File Reader and Text File Writer steps which will behave as a default file names when no parameter is passed. When running your component separately or in a workflow, the default file names are not mandatory (the File Name field of Text File Reader/Writer can be left empty) as long as a parameter is given.

To define file names, create a Run DQC workflow task. In the task, go to the Configuration tab. Under Expression, define what pattern should your file names follow, and under Name specify the variable name which will be used instead of your file name. In the example below, "names_"+toString(today(),"YYYY-MM-d")+".txt" expression is used. The today() sub-expression returns the current date of DAY type. It is designed to return the same value for all records (DQC application start date), even if DQC runs past midnight. The date is then converted by toString() expression to a STRING value. As a DAY value is converted, you need to propose the format to which the DAY value will be altered. For more information about expressions, open the Help tab in your DQC GUI > Help contents > Ataccama Data Quality Center > DQC Basics > Expressions

 

Using console

For independent parameterized launches (direct execution, i.e. components not being used within a plan) you can use the -params.<param_name>=<value> syntax to pass parameter-value pairs for correct plan setup.

Sample usage
./runcif.sh -params.inputFile=in.csv -params.outputFile=out.csv example.comp