Skip to main content

rule

Definition​

In different cases we want to rewrite the VQL Veezoo generates from a user's question. We might want to add filters, change which function Veezoo uses in specific situations, correct potential errors, or insert our own definitions of concepts. With Measures and default filters Veezoo provides frameworks to rewrite VQL in a simple and straightforward fashion. rules are a more general method. They detect patterns in the generated VQL and modify them in a desired fashion.

For almost all cases there is a way to achieve the desired result using other tools than rules, in which case we strongly recommend to use the alternative instead.

Structure​

Rules follow this structure:

rule only_show_active_customers {
    rule_pattern: """
        // Matching Pattern
        var customer: kb.Customer

        // Modification Pattern
        @add_if_not_exists {
            customer.is_active = true
        }
    """
    
    // specify which user role should have this logic applied
    for_user_role: "Marketing"
    
    // specify the kind of rule
    rule_tag: meta.SemanticLogicTag
    
    comment: "This is a description of the rule and its motivations."
}

rule_pattern​

The rule pattern consists of two parts: the Matching Pattern and the Modification Pattern.

The Matching Pattern can be a variety of VQL statements and serves two functions:

  • Variable declaration: var customer: kb.Customers declares the variable customer, which then can be referred to later on.
  • Rule condition: Veezoo checks the Matching Pattern to see whether a rule should be applied. This rule comes into play only if everything listed in the Matching Pattern is also found in the VQL of the Question.

Additionally we can use these functions to decide if the rule should be applied or not:

  • @missing { ... }: the rule should not be used, if the pattern in the brackets is present
  • { ... } @or { ... }: the rule should be applied when either pattern is present

The Modification Pattern defines the actual behavior of the rule. It takes its arguments from within its brackets. The following are the available Modification Patterns:

  • @add_if_not_exists { ... }: adds a pattern to the VQL
  • @remove { ... }: removes a pattern from the VQL, if present
  • @remove_connected { ... }: similar to @remove, but also removes all classes connected to the pattern as well
  • @replace { ... } with { ... }: replaces a specific pattern with another

@add_if_not_exists { ... }​

This Modification Pattern is mostly used to add additional constraints or filters, the user didn't explicitly ask for, but we, as the data team, still want to be there. Let's look at the example given above again:

rule only_show_active_customers {
    rule_pattern: """
        var customer: kb.Customer

        @add_if_not_exists {
            customer.is_active = true
        }
    """

    for_user_role: "Marketing" 
}

In this example, we want to ensure that users with the user_role Marketing can only see active customers.

  • In the first line we define the variable customer and simultaneously state that this rule should be triggered if the pattern kb.Customer is present in the VQL.
  • We then use the Modification Pattern: if the filter customer.is_active = true is not present, we add it, such that all customers shown in the result are active.
  • We add the for_user_role such that this rule only applies to the marketing users (as explained later)

A similar effect to this can be achieved by using default filters. There are two differences: default filters can be removed by business users, where as this rule will always be applied, and default filters can't be set for specific user roles.

@remove { ... }​

With this Modification Pattern we can remove filters that Veezoo might add, but we don't want in certain cases.

Consider the following scenario: we have two datasets, Employees containing information about employees, including their status (active or inactive), and Transactions containing records of sales transactions, each associated with an employee. We want to calculate the total sales amount without filtering out inactive employees, as we need to consider sales made by all employees, regardless of their current status.

To this end, we use the following rule:

rule total_sales_all_employees {
    rule_pattern: 
    """
        var transactions: kb.Transactions
        var employee: kb.Employee
        sum(transactions.amount)
        transactions.with_Employee = employee
        @remove {
             # Remove the filter that only includes active employees
            employee.is_Active = true
        }
        """
}

In the Matching Pattern we first define the variables transactions and employee and ensure that the rule is only applied when calculating the sum total of transactions. We specify that transactions is linked with employee using transactions.with_Employee = employee.

Using the @remove pattern, we then explicitly remove the filter that restricts the data to only active employees (employee.is_Active = true).

By applying this rule, we ensure that the total sales are calculated without excluding any employees based on their status. This way, sales made by inactive employees are also considered in the total, providing a complete picture of sales performance.

@replace { ... } with { ... }​

This Modification Patterns is used to replace lines in VQL. A typical example of this is, when Veezoo would normally use a certain function, but we want to use a different one in a specific case. In the following example, we will show two uses of this rule pattern working together.

Consider the following situation. We have an Orders dataset where each order has an Order_Qty column. The Order_Qty values are adjusted based on business logic, such as 1 for regular orders, 0 for orders that shouldn't be counted, and -1 for orders that should decrease the count. This should be used when counting the orders instead of a COUNT DISTINCT on an order id.

This rule does this: it first checks if the question contains kb.Orders, and if so replaces all instances of the function count(orders) with sum(orders.Order_Qty).

rule Substitute_Count_For_Sum {            
    rule_pattern: 
     """
        var orders: kb.Orders
        @replace {
            count(orders)
        } with {
            sum(orders.Order_Qty)
        }
        """
}

In a next step, we would like to adapt the average function. Let's say we would like to figure out the average value of an order last month. The default average function might just calculate the average Value of all Orders. This, again, would be false. We therefore replace the default average(orders.Value) with sum(orders.Value) / count(orders):

rule Adapt_Average {
    rule_pattern: 
     """
        var orders: kb.Orders
        @replace {
            average(orders.Value)
        } with {
            sum(orders.Value) / count(orders)
        }
        """
}

Note that this rule uses count(orders). So, when asking Average value of an order last month? Veezoo would first replace average(orders.Value) with sum(orders.Value) / count(orders), and then in a second step this with sum(orders.Value) / sum(orders.Order_Qty). If we would only ever be interested in the average, we could of course only have one rule do both.

for_user_role (optional)​

If there are user roles defined, you can use this tag to specify to whom the rule should apply. This can be useful, for example, if different roles have differing definitions of KPIs. For a more detailed explanation, see here.

rule_tag (optional)​

This tag specifies when the rule is applied. There are two options:

  • use meta.SemanticLogicTag if the rule should be applied in the Semantic Rewriting stage
  • use meta.LogicRewritingTag if the rule should be applied in the Logic Rewriting stage

Semantic Rewriting stage​

This Rewriting stage is concerned with rewriting rules that need to be communicated to the user for clarity when answering.

Classical examples are default values, e.g. "how many customers" means "how many active customers" by default and the user should be made aware of that, in case that's not what he/she wants.

Logic Rewriting stage​

In contrast to Semantic Rewriting, rules in Logic Rewriting are rather technical and have no meaning for the end user.

A common case is when the database does not have the sought concept raw stored, but it should be computed from other concepts. We call these higher-order concepts.

These are usually KPI definitions (measure), e.g. to compute the profit we need to add revenue x with revenue y and subtract costs z, but not costs w.

The main difference for users will be if the rule affects the title of answers or not. If no rule_tag is specified, meta.SemanticLogicTag will be used by default.

comment (optional)​

You can add descriptions to your rules to explain what it does. Comments have to use the comment tag to be saved. // comments using slashes will not cause any errors when compiling, but will be lost upon reloading the page.