Envision 5 user-defined functions

Starting from the revision five of the language, the language can define user functions, allowing to factor some repeated code and to extend the expressiveness of what was possible before.

We have two function classes right now:

Generality

All function classes share common syntax and concepts that we are covering in this section, most of the examples are possibly least expressive, so most of the time, we use pure functions.

Local variables and typing

The user-defined pure functions can be used like any other built-in Envision functions, in any context where they are allowed. Within the user-defined, local variables are assigned with the = operator (even if the variables are scalars). Introducing a temporary variable just means assigning a new unused variable:

def pure withTemporary(arg : number) with
    temp = arg * arg
    return temp + 3

The type of the local variable is deduced from its initialization expression, and the same variable can be assigned multiple times:

def pure icon(newValue: number, oldValue: number) with
  r = "❌"
  where oldValue > newValue
    r = "✔️"
  else where abs(oldValue - newValue) < 0.01
    r = ""
  return r

An error will be emitted if we mix various variable types together. You cannot assign a number to a previously text variable. For function calls, the usual Envision typing rule apply.

def pure icon(newValue: number, oldValue: number) with
  r = "❌"
  where oldValue > newValue
    r = "✔️"
  else where abs(oldValue - newValue) < 0.01
    r = 0   // an error will trigger here
  return r

The value returned by the function is what is passed to the return instruction. You can have multiple returns within the same function, the first executed return instruction will stop the execution of the function.

Conditionals

User-defined functions have conditionals like other imperative languages, with the form where .. else:

def pure qualify(value: number) with
    toReturn = ""
    where value < 0
        toReturn = "Negative"
    else where value > 0
        toReturn = "Positive"
    else
        toReturn = "Zero"

    return toReturn

QuantityQualified = qualify(Quantity)

You can mix conditionals with return statements to avoid the use of some local variables:

def pure qualify(value: number) with
    where value < 0
        return "Negative"

    where value > 0
        return "Positive"

    return "Zero"

QuantityQualified = qualify(Quantity)

Tuples

User-defined functions can return multiple values, calculated from the same inputs:

def pure min.max.pure(a : number, b : number) with
    where a < b
        return (a, b)

    return (b, a)

Minimal, Maximal = min.max.pure(ValueA, ValueB)

Functions producing tuples cannot be composed at will. You must “deconstruct” the tuple before using its content:

def pure min.max.pure(a : number, b : number) with
    where a < b
        return (a, b)

    return (b, a)

def pure validate(a: number, b : number) with
    mini, maxi = min.max.pure(a, b)
    return (mini * 2, maxi * 2)

The following definition will be rejected:

def pure min.max.pure(a : number, b : number) with
    where a < b
        return (a, b)

    return (b, a)

def pure add(a : number, b : number) with
    return a + b

def pure test(a : number, b : number) with
    return add(min.max.pure(a, b)) // error, cannot pass tuple as arguments.

The same rule applies at the top level. You must deconstruct tuples within the Envision script. Nothing stops you from returned variables of different types within tuples:

def pure something(a : number) with
    return a + 1, "\{a}"

Orders.NextQuantity, Orders.QuantityText = something(Orders.Quantity)

Declaration order and self reference

User-defined functions are defined with their order of appearance in the script. A given user function can call any function declared above it, but it cannot call itself:

def pure min.max.pure(a : number, b : number) with
    where a < b
        return (a, b)

    return (b, a)

def pure test(a : number, b : number) with
    mini, maxi = min.max.pure(a, b)
    return a * 2 + b

Trying to call an undefined function will provoke an error:

def pure test(a : number, b : number) with
    mini, maxi = min.max.pure(a, b)  // Error: min.max.pure undefined at this position.
    return a * 2 + b

def pure min.max.pure(a : number, b : number) with
    where a < b
        return (a, b)

    return (b, a)

Overloading

The Envision standard library is being built with the overloading concept. We allow to define functions with the same name as long as the following rules are respected:

def pure min.max.pure(a : number, b : number) with
    return (a < b ? a : b, a > b ? a : b)

// allowed because all the types are different
def pure min.max.pure(a: text, b : text) with
    return (a < b ? a : b, a > b ? a : b)

// allowed because all the number of arguments are different
def pure min.max.pure(a: text) with
    return (a, a)

// rejected, because `min()` is a function of the standard
// library.
def pure min(a : number, b : number) with
    return a < b ? a : b

Other restrictions

User-defined function must have a return statement as last instruction

We will reject some valid functions with the following rule:

def my.abs(n : number) with
    where n < 0
        return -n
    else
        return n
    // error no else statement as the last instruction

but you can accept them with a slight rewrite:

def my.abs(n : number) with
    where n < 0
        return -n
    return n

User-defined function can only use arguments or constants declared within its body

This means that the following definition is rejected:

blackListedSupplier := "SupplierName"

def pure checkSupplier(quantity: number, supplier: text) with
    where supplier == blackListedSupplier // we are trying to access
        return 0                          // a constant variable not
                                          // available at this position
    return quantity

This could be accepted with a simple rewrite:

def pure checkSupplier(quantity: number, supplier: text) with
    blackListedSupplier = "SupplierName"
    where supplier == blackListedSupplier
        return 0

    return quantity

pure functions

Pure functions (or map functions in some contexts) are functions that are only working with their scalar inputs and constants defined within it. When called, all their arguments are from the same table or broadcasted from a compatible table:

def pure square(arg: number) with
    return arg * arg

Orders.Squared = square(Orders.Quantity)

Pure function only calls other pure/map functions

This means that you cannot call aggregation/scan functions or solvers:

def pure withSum(quantity: number, supplier) with
    where supplier == "my.favorite.supplier"
        return sum(quantity) // rejected, sum is a process

    return quantity

process functions

Process is a map function with an internal scalar state that is kept across the calls of the function. With the presence of state, it allows you to call other stateful functions like the aggregators and scan functions of the standard library along with other user-defined processes.

Process functions can be called in two contexts, namely when performing an aggregation (using the by and sort keywords) or when performing a scan (using the by and scan keywords)

Local state and process phase

We present the following example:

def process my.sum(a : number) with
    keep accumulator = 0
    accumulator = accumulator + a
    return accumulator

AllQuantity = my.sum(Orders.Quantity) by [Id]

The local state is introduced with the keep keyword, followed by a variable assignation. In the present example, the variable accumulator is created as a state. The function my.sum is called on group of [Id], this directly affects the life cycle of the process.

The process consists of three distinct phases:

The Envision compiler transforms the user-defined function into a specialized variation for the three phases.

Process parameters

Having process that handles the parameters is not sufficient if we want to initialize states with starting values. We can them introduce process parameters:

def process my.sum(a : number; inital: number) with
    keep accumulator = initial
    accumulator = accumulator + a
    return accumulator

AllQuantity = my.sum(Orders.Quantity; InitialQuantity) by [Id]

In the declaration, parameters are introduced after the argument, and are introduced after the ; token. When calling function parameters, a ; must be used to separate arguments from parameters. When aggregating, the parameters come from the output table, so in the given example the arguments come from the Orders table, and we aggregate into the Items table, thus the parameters must come from the Items table.

You have one parameter value for every group, you can use parameters for state initialization or to pass parameters to nested process call:

def process my.sum(a : number; inital: number) with
    keep accumulator = initial
    accumulator = accumulator + a
    return accumulator

def process clamped.sum(a: number; initial: number) with
    return my.sum(a; max(0, initial)) // Initialization is free as long as it
                                      // depends only on constants and parameters

AllQuantity = clamped.sum(Orders.Quantity; InitialQuantity) by [Id]

Trying to initialize a value or calculate a parameter from an input will give an error at compilation time:

def process my.sum(a : number; inital: number) with
    keep accumulator = a        // Error: can't initialize with argument
    accumulator = accumulator + a
    return accumulator

def process clamped.sum(a: number; initial: number) with
    return my.sum(a; max(0, a)) // Error: calculating parameter from argument

Nested process call

def process sum.cond(a : number, cond : boolean) with
    keep result = 0
    keep skipped = 0
    where cond
        result = sum(a + 1)
    else
        skipped = skipped + 1
    return(result, skipped)

AllIncomingQuantity, SoldLineCount =
    sum.cond(Orders.Quantity, Orders.FromSupplier)

Using the following table:

Id Quantity FromSupplier
A 10 true
A 5 true
A 160 false
A 2 true
B 4 false
B 4 true
B 2 true

We can simulate what happen during the call the function sum.cond

a cond result skipped
RESET RESET 0 0
10 true 11 0
5 true 17 0
160 false 17 1
2 true 20 1
EMIT EMIT => 20 => 1
RESET RESET 0 0
4 false 0 1
4 true 5 1
2 true 8 1
EMIT EMIT => 8 => 1

We can see that the value of the result does not change when cond is false, it is only updated when we call the conditional reaches the sum call. The value returned by the sum call stays consistent with the values it has been given.

The semantic of the nested process/aggregation/scan functions call is the following:

Process aggregation

When using a process like an aggregator (using the by/sort keywords), the process will be called with the following sequence:

You can still use the if filtering capability, but to avoid maintenance headaches, the in-process conditional is recommended.

Process scan

The use of a defined process (or any available aggregator) as a scan requires the use of the scan keyword to the call:

def process sum.cond(a : number, cond : boolean) with
    keep result = 0
    keep skipped = 0
    where cond
        result = sum(a + 1)
    else
        skipped = skipped + 1
    return(result, skipped)

Orders.AllIncomingQuantity, Orders.SoldLineCount =
    sum.cond(Orders.Quantity, Orders.FromSupplier) by [Orders.Id] scan [Orders.1]

The scan keyword specifies an order like the sort keyword, but also tells us that the process will have the same table as input and output.

The process is called with the following sequence when used as a scan function:

The state is still propagated.

Process parameters in scan

Contrary to the aggregation, the parameters in the scan function cannot come from the output table, but from the middle table, which is the intermediate one given by the by keyword. Thus, re-taking the parameter example we have the following:

def process my.sum(a : number; initial: number) with
    keep accumulator = initial
    accumulator = accumulator + a
    return accumulator

Orders.AllQuantity = my.sum(Orders.Quantity; InitialQuantity) by [Id] scan [Orders.1]

The middle table is of an [Id] kind. The only table we know of this kind is the Items table. Thus, the data must come from the Items table.

Shared processes

All the examples show nested processes where all the phases happen at the same time, at the call position. It can happen that we want to update the same process multiple times and read its values before updating it in order to make a decision. In order to refer to the same process instance multiple times we must use the shared process declaration.

def process StockEvolution(quantity: number, leadTime: number) with
    keep lagged as sum.lagged(number,number) // shared process declaration
    keep totalQuantity = 0

    totalQuantity = lagged
    updated = lagged(quantity, leadTime)

    return (totalQuantity, updated)

Shared processes are introduced with the keep keyword, followed by their name introduced with the keyword as. The rest is a function signature. The signature must be able to determine a unique function to be used as a shared process.

You must provide the name, the types of the arguments. If the process has parameters, you must provide them like for any other process:

def process my(a : number, b: number; init : number) with
    // omitted

def process MyShared(a : number, b : text; init : number) with
    keep shared as my(number, number; init)
    // ...
    updated = shared(a, parsenumber(b, "'", "."))

We provide the parameters after the argument types. When calling the shared process to update its state, we do not provide them. Shared processes is a powerful construct that can backfire, so you must use it wisely.