For a quick start with the console, an MS SQL database with some source tables is required. The samples below show how to hand-write the JSON definition. After saving the file in JSON format, you can run it using the console:
Raynet.Etl.Console.exe -f "<path-to-json>" -c "<connection-string-to-mssql-database>"
For example:
Raynet.Etl.Console.exe -f "c:\temp\def.json" -c "Server=.\SQLEXPRESS;Database=RayVentoryDataHubResult;Trusted_Connection=True;"
The console application outputs information about the performed job, for example:
Partial samples and multiple files
|
Note: These options are available in ETL module version 1.1.271 and newer. |
It is possible to split the definition across many JSON files, which are then joined by the ETL runtime. Partial definitions may reference steps from other files, and they will be correctly resolved and processed in parallel by the engine.
To use partial JSON definition you can provide multiple arguments to the -f switch, for example:
Raynet.Etl.Console.exe -f "c:\temp\def-1.json" "c:\temp\def-2.json" -c "Server=.\SQLEXPRESS;Database=RayVentoryDataHubResult;Trusted_Connection=True;"
Alternatively, if all JSON files lie in the same folder and have .json file extension, then instead of the path to the file, the full path to the directory may be provided instead:
Raynet.Etl.Console.exe -f "c:\temp\" -c "Server=.\SQLEXPRESS;Database=RayVentoryDataHubResult;Trusted_Connection=True;"
These options can be joined, for example a folder and specific files.
Beware the following limitations:
•You must ensure the files can be merged by having unique step IDs across all files.
•ETL engine does not de-duplicate your resources - if a file is referenced twice (whether by file path or a folder path) it will be merged twice.
•If using more than one input file, there is no automatic SQL and PY file detection. In case of a single file, they are resolved automatically by taking the file name with a proper extension. In multi-file scenario, you should specify the value of command line parameter to control which script and macro definition gets loaded.