permalinkOptimize performance

This section aims to describe the most common performance issues that users experience with Functions, and document ways to optimize your code to avoid bottlenecks.

permalinkUse the Performance tab to improve Function performance

The performance tab provides you with a tool to analyze and identify performance issues with your Functions. This can be found in the Functions helper after a Function is run.

The waterfall graph represents operations as horizontal bars stretched out across time on the X-axis. There are markers for each operation to indicate how time is spent.

• Execute Function indicates CPU time spent executing the Function code.
• Load objects from arguments and Load objects from links indicate the time spent calling the underlying ontology backend database service (OSS).

To improve performance, you can:

• Use the Objects API to aggregate and traverse links more quickly than in Typescript (as described below).
• Ensure your ontology backend service calls are done in parallel to avoid sequential loads. If you have multiple async/await calls, use Promise.all to await all the calls in parallel.
  • For example, a common pattern is for each object in a list to use .map() to map the calls to their Promises, and then using Promise.all on the resulting list.
• Avoid unnecessary nested loops, which can increase execution time.

permalinkPrefer using the Objects API where possible

A common paradigm when using Workshop's derived properties is to calculate the property value by aggregating over each object's links (e.g, counting the number of related objects).

Although the code below works, the Function itself must retrieve all linked objects, and then perform an aggregation (in this case, calculating the length):

Copied!
1
2
3
4
5
6
7
8
9
10
@Function()
public async getEmployeeProjectCount(employees: Employee[]): Promise<FunctionsMap<Employee, Integer>> {
    const promises = employees.map(employee => employee.workHistory.allAsync());
    const allEmployeeProjects = await Promise.all(promises);
    let functionsMap = new FunctionsMap();
    for (let i = 0; i < employees.length; i++) {
        functionsMap.set(employees[i], allEmployeeProjects[i].length);
    }
    return functionsMap;
}

While the above takes advantage of the async API and asynchronous functions (see Optimizing link traversals), it's often beneficial to use the aggregation methods provided by the Objects API:

Copied!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
@Function()
public async getEmployeeProjectCount(employees: Employee[]): Promise<FunctionsMap<Employee, Integer>> {
    const result: FunctionsMap<Employee, Integer> = new FunctionsMap();

    // Get all projects that have an employeeId matching from the employees parameter, then count how many projects are mapped to each employeeId
    const aggregation = await Objects.search().project()
            .filter(project => project.employeeId.exactMatch(...employees.map(employee => employee.id)))
            .groupBy(project => project.employeeId.byFixedWidths(1))
            .count();

    const map = new Map();
    aggregation.buckets.forEach(bucket => {
        // bucket.key.min represents the employeeId as each bucket size is 1.
        map.set(bucket.key.min, bucket.value);
    });
    employees.forEach(employee => {
        const value = map.get(employee.primaryKey);
        if (value === undefined) {
            return;
        }
        result.set(employee, value);
    });

    return result;
}

In this way, you can perform the aggregation in a single step without needing to pull in all linked projects first.

permalinkOptimizing link traversals

The most common source of performance issues in Functions comes from traversing links in an inefficient manner. Often, this occurs when you write code that loops over many objects and calls an API to load related objects on every iteration of the loop.

Copied!
1
2
3
for (const employee of employees) {
    const pastProjects = employee.workHistory.all();
}

In this example, each iteration of the loop will load an individual employee's past projects, causing a round-trip to the database. To avoid this slowdown, you can use the asynchronous link traversal APIs (getAsync() and allAsync()) when traversing many links at once. Below is an example of a Function that is written to load links asynchronously:

Copied!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
@Function()
public async findEmployeeWithMostProjects(employees: Employee[]): Promise<Employee> {
    // Create a Promise to load projects for each employee
    const promises = employees.map(employee => employee.workHistory.allAsync());

    // Dispatch all the promises, which will load all links in parallel
    const allEmployeeProjects = await Promise.all(promises);

    // Iterate through the results to find the employee who has the greatest number of projects
    let result;
    let maxProjectsLength;
    for (let i = 0; i < employees.length; i++) {
        const employee = employees[i];
        const projects = allEmployeeProjects[i];

        if (!result || projects.length > maxProjectsLength) {
            result = employee;
            maxProjectsLength = projects.length;
        }
    }

    return result;
}

This example uses an ES6 async function ↗, which makes it convenient to handle the Promise return values that are returned from the .getAsync() and .allAsync() methods.

permalinkOptimize derived column generation

Workshop supports computing derived properties using Functions on objects (FOO). Workshop applications typically call these functions with a few dozen rows of content from an object table. The function then returns a map where each object is mapped to the display value in the derived column. Below is an example of a non-optimized implementation:

Copied!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
import { Function, FunctionsMap, Double } from "@foundry/functions-api";
// Replace "objectTypeA" with the object type you want to process.
import { Objects, ObjectSet, objectTypeA } from "@foundry/ontology-api";

export class MyFunctions {

    /*
     * This Function takes an ObjectSet as input, and generates a derived column as output.
     * This derived column maps each object instance to the numeric value that will populate the column.
     * This implementation is a trivial for-loop that multiplies an object property by a constant value.
     * This serves as the base case that we will improve below.
     */
    @Function()
    public getDerivedColumn_noOptimization(objects: ObjectSet<objectTypeA>, scalar: Double): FunctionsMap<objectTypeA, Double> {
        // Define the result map to return
        const resultMap = new FunctionsMap<objectTypeA, Double>();

        /* There is a limit to the number of objects that can be loaded in memory.
         * See enforced limit documentation for current object set load limits.
         */
        const allObjs: objectTypeA[] = objects.all();

        // For each loaded object, perform the computation. If the result is defined, store it in the result map.
        allObjs.forEach(o => {
            const result = this.computeForThisObject(o, scalar);
            if (result) {
                resultMap.set(o, result);
            }
        });

        return resultMap;
    }

    // An example of a function that computes the required value for the provided object.
    private computeForThisObject(obj: objectTypeA, scalar: Double): Double | undefined {
        if (scalar === 0) {
            // Division by zero error
            return undefined;
        }
        // Checks if exampleProperty is defined, and divides if so. If not, it returns undefined.
        return obj.exampleProperty ? obj.exampleProperty / scalar : undefined;
    }

}

If the computation is simple, the function should execute quickly. If the computation is complex, it is possible to reduce compute time by using asynchronous execution. This way, computations for each object are executed in parallel. Below is an example:

Copied!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
import { Function, FunctionsMap, Double, TwoDimensionalAggregation } from "@foundry/functions-api";
// Replace "objectTypeA" with the object type that will be processed.
import { Objects, ObjectSet, objectTypeA, objectTypeB } from "@foundry/ontology-api";

    /*
     * This Function takes a list of strings that are object primaryKeys as input, and generates a derived column as output.
     */
    @Function()
    public async getDerivedColumn_parallel(objects: ObjectSet<objectTypeA>, scalar: Double): Promise<FunctionsMap<objectTypeA, Double>> {
        // Define the result map
        const resultMap = new FunctionsMap<objectTypeA, Double>();

        /* There is a limit to the number of objects that can be loaded in memory.
         * See enforced limit documentation for current object set load limits.
         * This should not be a problem as Workshop can lazy-load as users are scrolling.
         */
        const allObjs: objectTypeA[] = objects.all();

        // Launch parallel computations for each object in the array. See alternative examples with computeForThisObject_filterOntology below.
        const promises = allObjs.map(currObject => this.computeForThisObject(currObject, scalar));

        // Use Promise.all to parallelize async execution of helper function
        const allResolvedPromises = await Promise.all(promises);

        // Populate resultMap with buckets and calculate their values.
        for (let i = 0; i < allObjs.length; i++) {
            resultMap.set(allObjs[i], allResolvedPromises[i]);
        }

        return resultMap;
    }



    // An example of a function that computes the required value for the provided object.
    private async computeForThisObject(obj: objectTypeA, scalar: Double): Promise<Double | undefined> {
        if (scalar === 0) {
            // Division by zero error
            return undefined;
        }
        // Checks if exampleProperty is defined, and divides if so. If not, it returns undefined.
        return obj.exampleProperty ? obj.exampleProperty / scalar : undefined;
    }


    /*
     * An example of a function that computes the required value for the provided object.
     * For a given object, query the Ontology (filter for other objects, search-around to another object set, etc.)
     */
    @Function()
    private async computeForThisObject_filterOntology(obj: objectTypeA): Promise<Double> {
        // Create an object set by filtering on some properties
        const currObjectSet = await Objects.search().objectTypeB().filter(o => o.property.exactMatch(obj.exampleProperty));
        // Note: If there is an existing link between the ObjectTypes, an alternative would be:
        // const currObjectSet = await Objects.search().objectTypeA([obj]).searchAroundObjectTypeB();
        // Compute the aggregation for this object set
        return await this.computeMetric_B(currObjectSet);
    }

    @Function()
    public async computeMetric_B(objs: ObjectSet<objectTypeB>): Promise<Double> {
        // Set up calls to a different part of the equation.
        const promises = [this.sumValue(objs), this.sumValueIfPresent(objs)];

        // Execute all promises
        const allResolvedPromises = await Promise.all(promises);

        // Get values from the promises
        const sum = allResolvedPromises[0];
        const sumIfPresent = allResolvedPromises[1];

        // Perform calculus
        return sum / sumIfPresent;
    }


    @Function()
    public async sumValue(objs: ObjectSet<objectTypeB>): Promise<Double> {
        // Sum the values of the objects, whichever is the metric of those objects
        const aggregation = await objs.sum(o => o.propertyToAggregateB);
        const firstBucketValue = aggregation.primaryKeys[0].value;
        return firstBucketValue;
    }


    @Function()
    public async sumValueIfPresent(objs: ObjectSet<objectTypeB>): Promise<Double> {
        // Sum the object values if they are not null.
        const aggregation = await objs.filter(o => o.metric.hasProperty()).sum(o => o.propertyToAggregateA);
        const firstBucketValue = aggregation.primaryKeys[0].value;
        return firstBucketValue;
    }

Note: The same applies with a TwoDimensionalAggregation that would populate a Chart XY widget in Workshop. You can pass a list of category strings (buckets) to compute, instead of a list of object instances. Below is an example:

Copied!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
    // ==== Utils - convert to a TwoDimensionalAggregation for the Chart XY widget in Workshop ====
    @Function()
    public async getDerivedColumn_parallel_asTwoDimensional(objects: ObjectSet<objectTypeA>, scalar: Double): Promise<TwoDimensionalAggregation<string>> {
        const resultMap: FunctionsMap<objectTypeA, Double> = await this.getDerivedColumn_parallel(objects, scalar);

        // Create a TwoDimensionalAggregation from the resultMap
        const aggregation: TwoDimensionalAggregation<string> = {
            // Map the entries (object -> Double) of resultMap (string -> Double)
            buckets: Array.from(resultMap.entries()).map(([key, value]) => ({
                key: key.pkProperty, // Destructure key to get its id property
                value
            })),
        };

        return aggregation;
    }