Validators offer a straightforward yet highly effective means of ensuring that data adheres to predefined criteria and rules. They play a pivotal role in maintaining data integrity and reliability by confirming that data meets expected standards and requirements.
To create a new validation rule, go to the Validators tab and click the + Add button.
The following default validation rules are available:
•Within range
This rule validates incoming data based on a specified range of rows relative to the previous data count. It is particularly useful for tasks where the expected number of items remains relatively stable. Any deviation beyond the defined range is considered an anomaly and results in the rejection of the entire data set.
•Maximum count of rows
This rule restricts incoming data from exceeding the number of rows you already have. It's valuable for tasks where data should never surpass the previously collected amount. Any occurrence of data exceeding this limit is treated as an anomaly, leading to the rejection of the entire data set.
•Minimum count of rows
This rule ensures that incoming data contains at least as many rows as the data you already possess. It is applicable to tasks where data should never decrease in quantity compared to previous records. If the incoming data falls below this minimum threshold, it is considered an anomaly, resulting in the rejection of the entire data set.
By default, these rules are applied universally to all tables generated by a particular task. However, in scenarios where a single task produces multiple tables, you have the flexibility to refine the rule application process:
•Using TABLE NAMES: You can employ the TABLE NAMES input field to specify which tables the rules should be applied to. This allows you to target specific tables for validation while excluding others.
•Combining Rules: To further tailor the validation process, you can combine rules for different tables. Simply click the "Add" button and continue adding rules for additional tables as needed.
This customization feature empowers you to fine-tune validation rules, ensuring that they are appropriately applied to the relevant tables within your data, enhancing the precision and control of your data validation processes.