Inferring remaining columns

<< Click to Display Table of Contents >>

ETL > 14.0 > Implementation Guide > Tutorial and implementation guide > Steps > Mapping > Advanced topics 

Inferring remaining columns

Map step requires, that you define the list of columns to be mapped. If you do not know the exact names ahead-of-time, or if just a few columns need to be transformed and the others should be taken as they are, it is possible to use the automatic inferring.

To do this, set the value of optional parameter mapRemaining on the step level to true, for example:
 

    {

      "id": 1,

      "name""Map with remaining",

      "columns": {

        "Column1""source1",

        "Column2""source2"

      },

      "target""OutputTable",

      "source""InputTable",

      "mapRemaining":  true 

    }

 

Example

Let's assume that the input table InputTable has the following structure:

col1

col2

col3

Marcin

Otorowski

Renault

John

Smith

Chrysler

Kate

Binks

Peugeot

When the ETL process is started for the sample JSON file, the following table OutputTable is returned:

Column1

Column2

col3

Marcin

Otorowski

Renault

John

Smith

Chrysler

Kate

Binks

Peugeot

If the parameter mapRemaining was set to false or omitted, the following would be produced:

Column1

Column2

Marcin

Otorowski

John

Smith

Kate

Binks

Note that the col3 was not defined in JSON, and yet it made to the OutputTable due to the switch mapRemaining set to true. On the other hand, columns col1 and col2 are not present anymore, because the engine has detected they were referenced by the standard mapping, and thus are potentially not needed anymore. If you want to include them, you must add a manual definition for them.

Once mapRemaining is set to true, the ETL engine will automatically restore all columns with the following exceptions:

Columns referenced as source columns for simple mappings (see simple mappings).

Columns referenced as source columns for transformation mappings (see transforming of values).

Columns referenced as one of the sources for aggregated mapping (see columns aggregation).

Columns referenced by custom SQL mappings are not taken into account when evaluating the mapRemaining switch.