# Envision 5 user-defined functions

Starting from the revision five of the language, the language can define user functions, allowing to factor some repeated code and to extend the expressiveness of what was possible before.

We have two function classes right now:

• pure for simple scenario, only depending on their arguments
• process for aggregation/scan scenario, depending on their arguments and an internal state.

## Generality

All function classes share common syntax and concepts that we are covering in this section, most of the examples are possibly least expressive, so most of the time, we use pure functions.

### Local variables and typing

The user-defined pure functions can be used like any other built-in Envision functions, in any context where they are allowed. Within the user-defined, local variables are assigned with the = operator (even if the variables are scalars). Introducing a temporary variable just means assigning a new unused variable:

def pure withTemporary(arg : number) with
temp = arg * arg
return temp + 3

The type of the local variable is deduced from its initialization expression, and the same variable can be assigned multiple times:

def pure icon(newValue: number, oldValue: number) with
r = "❌"
where oldValue > newValue
r = "✔️"
else where abs(oldValue - newValue) < 0.01
r = ""
return r

An error will be emitted if we mix various variable types together. You cannot assign a number to a previously text variable. For function calls, the usual Envision typing rule apply.

def pure icon(newValue: number, oldValue: number) with
r = "❌"
where oldValue > newValue
r = "✔️"
else where abs(oldValue - newValue) < 0.01
r = 0   // an error will trigger here
return r

The value returned by the function is what is passed to the return instruction. You can have multiple returns within the same function, the first executed return instruction will stop the execution of the function.

### Conditionals

User-defined functions have conditionals like other imperative languages, with the form where .. else:

def pure qualify(value: number) with
toReturn = ""
where value < 0
toReturn = "Negative"
else where value > 0
toReturn = "Positive"
else
toReturn = "Zero"

return toReturn

QuantityQualified = qualify(Quantity)

You can mix conditionals with return statements to avoid the use of some local variables:

def pure qualify(value: number) with
where value < 0
return "Negative"

where value > 0
return "Positive"

return "Zero"

QuantityQualified = qualify(Quantity)

### Tuples

User-defined functions can return multiple values, calculated from the same inputs:

def pure min.max.pure(a : number, b : number) with
where a < b
return (a, b)

return (b, a)

Minimal, Maximal = min.max.pure(ValueA, ValueB)

Functions producing tuples cannot be composed at will. You must “deconstruct” the tuple before using its content:

def pure min.max.pure(a : number, b : number) with
where a < b
return (a, b)

return (b, a)

def pure validate(a: number, b : number) with
mini, maxi = min.max.pure(a, b)
return (mini * 2, maxi * 2)

The following definition will be rejected:

def pure min.max.pure(a : number, b : number) with
where a < b
return (a, b)

return (b, a)

def pure add(a : number, b : number) with
return a + b

def pure test(a : number, b : number) with
return add(min.max.pure(a, b)) // error, cannot pass tuple as arguments.

The same rule applies at the top level. You must deconstruct tuples within the Envision script. Nothing stops you from returned variables of different types within tuples:

def pure something(a : number) with
return a + 1, "\{a}"

Orders.NextQuantity, Orders.QuantityText = something(Orders.Quantity)

### Declaration order and self reference

User-defined functions are defined with their order of appearance in the script. A given user function can call any function declared above it, but it cannot call itself:

def pure min.max.pure(a : number, b : number) with
where a < b
return (a, b)

return (b, a)

def pure test(a : number, b : number) with
mini, maxi = min.max.pure(a, b)
return a * 2 + b

Trying to call an undefined function will provoke an error:

def pure test(a : number, b : number) with
mini, maxi = min.max.pure(a, b)  // Error: min.max.pure undefined at this position.
return a * 2 + b

def pure min.max.pure(a : number, b : number) with
where a < b
return (a, b)

return (b, a)

### Overloading

The Envision standard library is being built with the overloading concept. We allow to define functions with the same name as long as the following rules are respected:

• The name does not match the name of a built-in function.
• The number of arguments are different.
• If the number is the same, the argument types must be different.
def pure min.max.pure(a : number, b : number) with
return (a < b ? a : b, a > b ? a : b)

// allowed because all the types are different
def pure min.max.pure(a: text, b : text) with
return (a < b ? a : b, a > b ? a : b)

// allowed because all the number of arguments are different
def pure min.max.pure(a: text) with
return (a, a)

// rejected, because min() is a function of the standard
// library.
def pure min(a : number, b : number) with
return a < b ? a : b

### Other restrictions

#### User-defined function must have a return statement as last instruction

We will reject some valid functions with the following rule:

def my.abs(n : number) with
where n < 0
return -n
else
return n
// error no else statement as the last instruction

but you can accept them with a slight rewrite:

def my.abs(n : number) with
where n < 0
return -n
return n

#### User-defined function can only use arguments or constants declared within its body

This means that the following definition is rejected:

blackListedSupplier := "SupplierName"

def pure checkSupplier(quantity: number, supplier: text) with
where supplier == blackListedSupplier // we are trying to access
return 0                          // a constant variable not
// available at this position
return quantity

This could be accepted with a simple rewrite:

def pure checkSupplier(quantity: number, supplier: text) with
blackListedSupplier = "SupplierName"
where supplier == blackListedSupplier
return 0

return quantity

## pure functions

Pure functions (or map functions in some contexts) are functions that are only working with their scalar inputs and constants defined within it. When called, all their arguments are from the same table or broadcasted from a compatible table:

def pure square(arg: number) with
return arg * arg

Orders.Squared = square(Orders.Quantity)

### Pure function only calls other pure/map functions

This means that you cannot call aggregation/scan functions or solvers:

def pure withSum(quantity: number, supplier) with
where supplier == "my.favorite.supplier"
return sum(quantity) // rejected, sum is a process

return quantity

## process functions

Process is a map function with an internal scalar state that is kept across the calls of the function. With the presence of state, it allows you to call other stateful functions like the aggregators and scan functions of the standard library along with other user-defined processes.

Process functions can be called in two contexts, namely when performing an aggregation (using the by and sort keywords) or when performing a scan (using the by and scan keywords)

### Local state and process phase

We present the following example:

def process my.sum(a : number) with
keep accumulator = 0
accumulator = accumulator + a
return accumulator

AllQuantity = my.sum(Orders.Quantity) by [Id]

The local state is introduced with the keep keyword, followed by a variable assignation. In the present example, the variable accumulator is created as a state. The function my.sum is called on group of [Id], this directly affects the life cycle of the process.

The process consists of three distinct phases:

• Reset: All states are reinitialized, using the values from the parameters or internal constants
• In the previous example, accumulator is set to 0
• Update: All states are updated with the value from the arguments
• In the example, only accumulator = accumulator + a is executed
• Emit: We emit the value(s) and give them to the function caller
• We only execute return accumulator.

The Envision compiler transforms the user-defined function into a specialized variation for the three phases.

### Process parameters

Having process that handles the parameters is not sufficient if we want to initialize states with starting values. We can them introduce process parameters:

def process my.sum(a : number; inital: number) with
keep accumulator = initial
accumulator = accumulator + a
return accumulator

AllQuantity = my.sum(Orders.Quantity; InitialQuantity) by [Id]

In the declaration, parameters are introduced after the argument, and are introduced after the ; token. When calling function parameters, a ; must be used to separate arguments from parameters. When aggregating, the parameters come from the output table, so in the given example the arguments come from the Orders table, and we aggregate into the Items table, thus the parameters must come from the Items table.

You have one parameter value for every group, you can use parameters for state initialization or to pass parameters to nested process call:

def process my.sum(a : number; inital: number) with
keep accumulator = initial
accumulator = accumulator + a
return accumulator

def process clamped.sum(a: number; initial: number) with
return my.sum(a; max(0, initial)) // Initialization is free as long as it
// depends only on constants and parameters

AllQuantity = clamped.sum(Orders.Quantity; InitialQuantity) by [Id]

Trying to initialize a value or calculate a parameter from an input will give an error at compilation time:

def process my.sum(a : number; inital: number) with
keep accumulator = a        // Error: can't initialize with argument
accumulator = accumulator + a
return accumulator

def process clamped.sum(a: number; initial: number) with
return my.sum(a; max(0, a)) // Error: calculating parameter from argument

### Nested process call

def process sum.cond(a : number, cond : boolean) with
keep result = 0
keep skipped = 0
where cond
result = sum(a + 1)
else
skipped = skipped + 1
return(result, skipped)

AllIncomingQuantity, SoldLineCount =
sum.cond(Orders.Quantity, Orders.FromSupplier)

Using the following table:

Id Quantity FromSupplier
A 10 true
A 5 true
A 160 false
A 2 true
B 4 false
B 4 true
B 2 true

We can simulate what happen during the call the function sum.cond

a cond result skipped
RESET RESET 0 0
10 true 11 0
5 true 17 0
160 false 17 1
2 true 20 1
EMIT EMIT => 20 => 1
RESET RESET 0 0
4 false 0 1
4 true 5 1
2 true 8 1
EMIT EMIT => 8 => 1

We can see that the value of the result does not change when cond is false, it is only updated when we call the conditional reaches the sum call. The value returned by the sum call stays consistent with the values it has been given.

The semantic of the nested process/aggregation/scan functions call is the following:

• Reset phase is synchronized with the caller, when the calling function enter the reset phase all called process are also reset.
• Update is only done when execution reach the call.
• Emit is done immediately after an update, whenever the execution reach the process.

### Process aggregation

When using a process like an aggregator (using the by/sort keywords), the process will be called with the following sequence:

• Reset for the current group.
• Append the first value of the group.
• Append the last value of the group.
• Emit the aggregated value.

You can still use the if filtering capability, but to avoid maintenance headaches, the in-process conditional is recommended.

### Process scan

The use of a defined process (or any available aggregator) as a scan requires the use of the scan keyword to the call:

def process sum.cond(a : number, cond : boolean) with
keep result = 0
keep skipped = 0
where cond
result = sum(a + 1)
else
skipped = skipped + 1
return(result, skipped)

Orders.AllIncomingQuantity, Orders.SoldLineCount =
sum.cond(Orders.Quantity, Orders.FromSupplier) by [Orders.Id] scan [Orders.1]

The scan keyword specifies an order like the sort keyword, but also tells us that the process will have the same table as input and output.

The process is called with the following sequence when used as a scan function:

• Reset for the current group.
• Append the first value of the group.
• Emit the first value for the group
• Append the second value of the group.
• Emit the second value for the group
• Append the last value of the group.
• Emit the last value.

The state is still propagated.

#### Process parameters in scan

Contrary to the aggregation, the parameters in the scan function cannot come from the output table, but from the middle table, which is the intermediate one given by the by keyword. Thus, re-taking the parameter example we have the following:

def process my.sum(a : number; initial: number) with
keep accumulator = initial
accumulator = accumulator + a
return accumulator

Orders.AllQuantity = my.sum(Orders.Quantity; InitialQuantity) by [Id] scan [Orders.1]

The middle table is of an [Id] kind. The only table we know of this kind is the Items table. Thus, the data must come from the Items table.

### Shared processes

All the examples show nested processes where all the phases happen at the same time, at the call position. It can happen that we want to update the same process multiple times and read its values before updating it in order to make a decision. In order to refer to the same process instance multiple times we must use the shared process declaration.

def process StockEvolution(quantity: number, leadTime: number) with
keep lagged as sum.lagged(number,number) // shared process declaration
keep totalQuantity = 0

totalQuantity = lagged
updated = lagged(quantity, leadTime)

return (totalQuantity, updated)

Shared processes are introduced with the keep keyword, followed by their name introduced with the keyword as. The rest is a function signature. The signature must be able to determine a unique function to be used as a shared process.

You must provide the name, the types of the arguments. If the process has parameters, you must provide them like for any other process:

def process my(a : number, b: number; init : number) with
// omitted

def process MyShared(a : number, b : text; init : number) with
keep shared as my(number, number; init)
// ...
updated = shared(a, parsenumber(b, "'", "."))

We provide the parameters after the argument types. When calling the shared process to update its state, we do not provide them. Shared processes is a powerful construct that can backfire, so you must use it wisely.