Quickbase Help / Pipelines / Build pipelines / Transform data

Search

Articles in this section

Using Jinja in Quickbase pipelines

Quickbase Pipelines uses Jinja templating to enable dynamic expressions, filters, and access to variables, by providing Python-like expressions while
ensuring that the templates are evaluated in a sandbox.

Jinja is a modern and designer-friendly templating language for Python. It is a text-based language and thus can be used to generate any markup as
well as source code.

Pipelines is currently supporting Jinja2 2.11.3, for various formatting within pipeline steps.

IN THIS ARTICLE

Resources for Jinja

Variables
Logic Conditions
Currently unsupported Jinja features
Filters
List of Built-in Tests
Working with JSON
Function Calls
Example - audit trail creation process
Example - searching for a field value
Example - workaround for address fields with spaces
Example - Sum of two numeric field values in another field
Working around reserved words in Jinja
Jinja tips

Resources for Jinja

Test your expression in realtime:

http://jinja.quantprogramming.com/

For syntax and semantics:

https://jinja.palletsprojects.com/en/2.11.x/templates/

Statements
For statements use {% ... %}

Expressions
For expressions use {{ }} to print to the template output. For example {{ customer['first_name'] }} , prints the content in between the
curly brackets to the template output.

Runtime metadata object

If you want to leverage metadata about your Pipeline, and a specific Run, you can use the Runtime object that is created each time your pipeline runs.
Syntax and specific properties

{{runtime.propertyname}} for example, you could use {{runtime.pipeline_id}}

This table details syntax elements and examples of output.

Syntax Description and examples

run_id =
The alphanumeric identifier for a single run of a Pipeline from start to finish.

Example output = efc4145d637f4ac09f1e7be6ed8ef499

activity_url =
This URL contains realm/domain and run ID information. This can be used to navigate directly
to a single run’s activities within a given domain (when authorized).

Example output =
https://team.pipelines.quickbase.com/activity/pipeline/efc4145d637f4ac09f1e7be6ed8ef499

triggered_at =
The ISO8601 timestamp for the date and time that an individual run of a Pipeline was
triggered. This is relative to UTC.

Example output = 2023-05-18 16:37:09.761914+00:00

pipeline_id =
This is the numeric identifier corresponding to an individual Pipeline.

Example output = 4803524587847680

user_id =
The Pipelines user ID. The example below is specific to the Pipelines portion of the platform.
This output can be helpful when you request assistance from Tech Support.

Example output = 6413832189116416

pipeline_url =
This URL contains the realm/domain and Pipeline ID information. This can be used to
navigate directly to a Pipeline within a given domain (when authorized).

Example output = https://team.pipelines.quickbase.com/pipelines/4803524587847680

pipeline_name =
The user specified Pipeline name.

Example output = Runtime Object Test

Use-cases
If you want to dynamically get the Pipeline ID in a custom alert message (rather than hardcoding a Pipeline ID in your alert) you could use the Runtime
metadata object to help.

Comments
For comments not included in the template output use {# your comment #} For example:

{# note: commented-out template because we no longer use this

{% for user in users %}
...
{% endfor %}
#}

Variables
What attributes a variable has depends heavily on the application providing that variable.

Dot

You can use a dot ( . ) to access the attributes of a variable. The following examples do the same thing, a is an object with hello as a property:

{{ a.hello}}
{{ a['hello'] }}
{{a.get('hello')}}

For example, when you drag the variable from the popup menu, Pipelines uses variables with ( . ) operator

Custom Variables

You can set a custom variable using the expression:

{% set iterated = false %}

This expression will allow you to store dummy variables.

Data Types
Boolean - represents the values true and false
Integer - Integers are whole numbers without a decimal part.
Floating point - numbers can be written using a ‘.’ as a decimal mark
Sequence [‘a’] everything between the two brackets is a list.
String -a sequence of characters, either as a literal constant or as some kind of variable
Tuple - can contain two values of different data types
Mappings - {'dict': 'of', 'key': 'and', 'value': 'pairs'}

Logic Conditions

IF

Example

{% if some_condition %} {{do something }}

{% elif some_other_condition %} {{do something }}
{% else %} {{do something }}
{% endif %}

LOOP

Example
 {% for item in ['a', 'b', 'c']%}
{{ loop.index }} {{item}} “\n”
{% endfor %} output: 1 a \n 2 b \n 3 c \n

Loop over each item in a sequence

Example

To display a list of users provided in a variable called users:

{% for user in users %}

{{ user.username }}
{% endfor %}

Test if a value is empty

{% if some_url %}
<found logic in html>
{% else %}
<Not found logic in html>
{% endif %}

Currently unsupported Jinja features

Dicts are not ordered
Unsupported extensions. Filter(see below) or use the loop.index instead
{% continue %}
{% break %}

Inside of a for-loop block, you can access some special variables:

Variable Description

loop.index
The current iteration of the loop. (1 indexed)

loop.index0
The current iteration of the loop. (0 indexed)

loop.revindex
The number of iterations from the end of the loop (1 indexed)

loop.revindex0
The number of iterations from the end of the loop (0 indexed)

loop.first
True if first iteration.

loop.last
True if last iteration.

loop.length
The number of items in the sequence.

loop.cycle
A helper function to cycle between a list of sequences. See the explanation below.

loop.depth
Indicates how deep in a recursive loop the rendering currently is. Starts at level 1
 loop.depth0
Indicates how deep in a recursive loop the rendering currently is. Starts at level 0

loop.previtem
The item from the previous iteration of the loop. Undefined during the first iteration.

loop.nextitem
The item from the following iteration of the loop. Undefined during the last iteration.

loop.changed(*val)
True if previously called with a different value (or not called at all).

Loop Cycle

{% for row in [1,2] %}

{{ loop.cycle('odd', 'even') }} - {{row}} row
{% endfor %}
output: odd - 1 row even - 2 row

Filters
You can modify variables by using filters. Filters are separated from the variable by the pipe symbol ( | ) and may have optional arguments in
parentheses. Since multiple filters can be chained, the output of one filter is applied to the next. You can also filter like this for the entire block:

{% filter upper%}This text becomes uppercase{% endfilter%}

Some of the filters Quickbase currently supports:

Term Definition

abs
Return the absolute value of the argument.
(x, /)

base64_encode (value) Encode a string using Base64 and return the encoded bytes.

Example of usage: {{‘Test string’ | base64_encode}} output: VGVzdCBzdHJpbmc=

base64_decode (value)
Decode the Base64 encoded string and return the decoded bytes.

Use the parameter output_encoding='binary' to select the format of the decoded output.
Outputs can be formatted as a string (default) or as bytes.

Example of usage: {{‘VGVzdCBzdHJpbmc=’ | base64_decode}} output: Test string

capitalize
(s) Capitalize a value. The first character is uppercase, all others lowercase.

default
If the value is undefined it will return the passed default value, otherwise the value of the variable:
(value, default_value='',
boolean=False)
{{ my_variable|default('my_variable is not defined') }}

{{ QB_VARIABLE | default(42) }}

Note: If the variable ‘QB_VARIABLE’ is not defined, the value used will be 42.

escape
Convert the characters &, <, >, ‘, and ” in strings to HTML-safe sequences. Use this if you need to display text that might contain suc
(s)
value as markup string.
 first
(seq) Return the first item of a sequence.

{{ [1,2,3] | first}} output: 1

float
(value, default=0.0) Convert the value into a floating point number. If the conversion doesn’t work it will return 0.0. You can override this default using th

For example, the float and int are just added like so: {{a.value|float}} and {{a.value|int}}. Replace would be {{a.value|replace("$","") for
without leaving a space.

format
Apply the given values to a printf-style format string, like string % values.
(value, *args, **kwargs)

From JSON

hmac
Use this filter to wrap sensitive data with a specific key and algorithm, so it can later be used for additional verification or authentica

Use the parameter output_encoding='binary' to return the output in binary representation.

int
Convert the value into an integer. If the conversion doesn’t work it will return 0. You can override this default using the first paramete
(value, default=0,
base (10) in the second parameter, which handles input with prefixes such as 0b, 0o and 0x for bases 2, 8 and 16 respectively. The b
base=10)
and non-string values.

For example, the float and int are just added like so: {{a.value|float}} and {{a.value|int}}. Replace would be {{a.value|replace("$","") for
without leaving a space.

join
Return a string which is the concatenation of the strings in the sequence. The separator between elements is an empty string per de
(value, d='',
optional parameter:
attribute=None)

{{ [1, 2, 3]|join('|') }} output: 1|2|3

{{ [1, 2, 3]|join }} output: 123

It is also possible to join certain attributes of an object:

{{ users|join(', ', attribute='username') }}

last Return the last item of a sequence

(seq)

length
(obj, /) Return the number of items in a container.

list
Convert the value into a list. If it was a string the returned list will be a list of characters.
(value)

lower
(s) Convert a value to lowercase.

map
Applies a filter on a sequence of objects or looks up an attribute
(*args, **kwargs)
{{ titles|map('lower')|join(', ') }}

max
Return the largest item from the sequence.
(value,
case_sensitive=False, {{ [1, 2, 3]|max }} output: 3
attribute=None) only in
the latest

md5
Returns MD5 hex-digest string generated from the text input.
(text)
Example usage:
 {{ "some text" | md5 }}
output: 552e21cd4cd9918678e3c1a0df491bc3

min
(value, Return the smallest item from the sequence.
case_sensitive=False, {{ [1, 2, 3]|min }} output: 1
attribute=None)
only in the latest

random
Return a random item from the sequence.
(seq)

reject Filters a sequence of objects by applying a test to each object, and rejecting the objects with the test succeeding.
(*args, **kwargs)
If no test is specified, each object will be evaluated as a boolean.

Example usage:

{{ numbers|reject("odd") }}

replace
Return a copy of the value with all occurrences of a substring replaced with a new one. The first argument is the substring that shou
(s, old, new,
replacement string. If the optional third argument count is given, only the first count occurrences are replaced:
count=None)
{{ "Hello World"|replace("Hello", "Goodbye") }} output: Goodbye World

round
(value, precision=0, Round the number to a given precision. The first parameter specifies the precision (default is 0), the second the rounding method:
method='common') 'common' rounds either up or down
'ceil' always rounds up
'floor' always rounds down
If you don’t specify a method 'common' is used.

{{ 42.55|round }} output: 43.0

select
(*args, **kwargs) Filters a sequence of objects by applying a test to each object, and only selecting the objects with the test succeeding. If no test is s
evaluated as a boolean.

Example usage:

{{ numbers|select("odd") }}
{{ numbers|select("odd") }}
{{ numbers|select("divisibleby", 3) }}
{{ numbers|select("lessthan", 42) }}
{{ strings|select("equalto", "mystring")

{%- for column in [1, 2, 3]| select("odd")%}

{{column}}
{%- endfor %}
output: 1 and 3

sha1
Returns SHA1 hex-digest string generated from the text input.
(text)

Example usage:

{{ "some text" | sha1 }}

output: 37aa63c77398d954473262e1a0057c1e632eda77
 sha224
(text) Returns SHA224 hex-digest string generated from the text input.

Example usage:

{{ "some text" | sha224 }}

output: 9e13fe8526578d5792074d5bcaa9f222df928c7c158804ac70e904f8

sha256
Returns SHA256 hex-digest string generated from the text input.
(text)
Example usage:

{{ "some text" | sha256 }}

output: b94f6f125c79e3a5ffaa826f584c10d52ada669e6762051b826b55776d05aed2

sha384
(text) Returns SHA384 hex-digest string generated from the text input.

Example usage:

{{ "some text" | sha384 }}

output: cc94ec3e9873c0b9a72486442958f671067cdf77b9427416d031440cc62041e2ee1344498447ec0ced9f7043461bd1f3

sha512
Returns SHA512 hex-digest string generated from the text input.
(text)
Example usage:

{{ "some text" | sha512 }}

output: e2732baedca3eac1407828637de1dbca702c3fc9ece16cf536ddb8d6139cd85dfe7464b8235b29826f608ccf4ac643e29b19

slice
Slice an iterator and return a list of lists containing those items. Useful if you want to create a div containing three ul tags that repre
(value, slices,
fill_with=None)
{%- for column in [1,2,3,4]|slice(2)%}
{{column}}
{%- endfor %}
output: [1, 2] [3, 4]

sort
(value, reverse=False, Sort an iterable

case_sensitive=False,
attribute=None) {%- for column in [4,2,3,4]|sort%}
{{column}}
{%- endfor %}
output: 2 3 4 4

{% for user users|sort(attribute="age,name") %} ...{% endfor %}

string Make a string unicode if it isn’t already. In this way, a markup string will not be converted back to unicode.
(object)

striptags
(value) Strip SGML/XML tags and replace adjacent whitespace by one space.

{{ "<html>foo"| striptags}} output: foo

sum
Returns the sum of a sequence of numbers plus the value of parameter ‘start’ (which defaults to 0). When the sequence is empty it
(iterable,
 attribute=None, start=0) {{ [1,2,3]|sum }} output 6

It is also possible to sum up only certain attributes:

Total: {{ items|sum(attribute='price') }}

timestamp_to_time
Converts UNIX timestamp to UTC Date Time format.
(value)

Example of usage:

input : {{1608112432| timestamp_to_time}}

output: 12-16-2020 01:53 AM

title
Return a titlecased version of the value. This means that words start with uppercase letters, all remaining characters are lowercase
(s)
{{ "hello world" | title }} output: text:Hello World

tojson
Dumps the structure to JSON so that it’s safe to use in <script> tags. It accepts the same arguments and returns a JSON string. Not
(value, indent=None)
through the |tojson filter which will also mark the result as safe. Due to how this function escapes certain characters this is safe eve
only in the latest
The following characters are escaped in strings:

<
>
&
'

trim
Strip leading and trailing characters, by default whitespace.
(value, chars=None)

truncate
Return a truncated copy of the string.
(s, length=255,
killwords=False, end='...', {{ "foo bar baz qux"|truncate(9) }} output: foo...
leeway=None)

unique
(value, Returns a list of unique items from the given iterable.
case_sensitive=False, {{ ['hello', 'world', 'helloworld', 'HelloWorld']|unique|list }} output: ['hello', 'world', 'helloworld']
attribute=None)
only in the latest

upper
Convert a value to uppercase.
(s)

urlencode
(value) Quote data for use in a URL path or query using UTF-8.

wordcount
Count the words in that string.
(s)

List of Built-in Tests

{% if variable is defined %}
{{variable}}
{%else %}
UNDEFINED
{% endif %}

Here are the tests that we support

 TEST Description

Boolean
Return true if the object is a boolean value

Defined Return true if the variable is defined

Divisibleby Checks if a variable is divisible by a number

Even Return true if the variable is even

Float Return true if the object is a float

Ge Whether or not the number is greater than or equal to

Gt Whether or not the number is greater than

In In a sequence

Integer If an integer

Le Less than or equal to

Lt Less than

Mapping Return true if the object is a mapping (dict etc.)

Ne Not equal

None True if the variable is none

Number True if the variable is a number

Odd True if the variable is odd

Sequence Whether or not the variable is a sequence

String

True

Working with JSON

from_json

If you have a string that you need to load as JSON...

field {"foo":[1,2]}$ore

...then use the from_json expression

{% set json_field=a.field| from_json %}{{json_field['foo']}}

To_json(value, indent=None)

Dumps a structure to JSON so that it’s safe to use in <script> tags. It accepts the same arguments and returns a JSON string. Note that this is
available in templates through the |_ filter which will also mark the result as safe. Due to how this function escapes certain characters this is safe even
if used outside of <script> tags.

The following characters are escaped in strings:

 < > & '

This makes it safe to embed such strings in any place in HTML with the notable exception of double-quoted attributes. In that case, you should single
quote your attributes or HTML escape it in addition.

You can use the indent parameter to enable pretty printing. Set it to the number of spaces that the structures should be indented with.

Note that this filter is for use in HTML contexts only.

Formatting and Escaping

It is sometimes desirable – even necessary – to have Jinja ignore parts it would otherwise handle as variables or blocks. For example, if, with the
default syntax, you want to use {{ as a raw string in a template and not start a variable, you have to use a trick.

The easiest way to output a literal variable delimiter ( {{ ) is by using a variable expression:

{{ '{{' }}

{% raw %}
{% for item in seq %}
{{ item }}
{% endfor %}
{% endraw %}

Function Calls
You can use macros to avoid repeated tasks

{% macro render_dialog(title, description='dialog') -%}

{{title}} - {{description}}
{{ caller() }}
{%- endmacro %}

{% call render_dialog('Hello World') %}

{% endcall %}
output: Hello World

Example - audit trail creation process

Here are two examples of using $prev to monitor changes to any specific fields on a Projects table. In these examples we check to see which of the
fields are different from their previous value, then create an Audit Log record that’s related to that project:

Audit trail creation process example 2 August 2024.png

Example - searching for a field value:
In our example, we have a field name: {{a.value}} . In your search Jinja, you would re-write it as {{a.$prev.value}} or if the field is
{{b.address_field}} it would be {{b.$prev.address_field}} .

Note: In our example $prev goes after the initial letter, so a. or b. etc.

Example - workaround for address fields with spaces

Address fields in Quickbase using the ‘parent’ label have the safe field name applied when creating the keys for address sub-fields in the pipelines
schema. This means that any time you try to access a sub-field of an address field that contains spaces in the name, the Jinja expressions become
invalid due to the spaces, or may get interpreted as a Jinja expression that differs from the user’s intent.

To work with this behavior in a Jinja expression, we suggest you follow these guidelines:

Properties of objects in Jinja can be accessed either with the formats of {{a.my_field}} or through {{a['my_field']}} , which supports spaces
in the property name.

Example - Sum of two numeric field values in another field

In this example, we perform a sum of two numeric field values in another field:

{{a.field_1 + a.field_2}}

Whether a search step returns a single record or many, it is returned as an array. If you use the for loop, it handles this for you. If not, you have to put in
the index you want to reference. So instead of using b.jan_enhancing you would use b[0].jan_enhancing if you know that your search step will
find results.

Working around reserved words in Jinja

If you need to write code that includes a word that is reserved in Jinja, enclose the word in quotes. For example, since items is a reserved word, change
this:

{{b.json.items}}

to this

b.json['items']
Jinja tips
Pipelines ignore fields with empty values in pipelines mapping, that is, it does not send them to the remote service, to avoid clearing a remote field
accidentally. This, however, is a problem when you want to explicitly clear this field in an Update pipe.
Solution: use template value {{CLEAR}} to clear a field, this is not bold in the rendered value.

Was this helpful?

Give feedback on this help article

Return to top

Return to top

Return to top

Related articles

Transforming your data

API_DoQuery

Common pipeline errors

Creating Data Change Logs Using Pipelines

Using CRON to schedule pipelines

© 2024 Quickbase. All Rights reserved. Quickbase is a registered trademark of Quickbase, Inc. Terms and conditions, features, support, pricing, and service options subject to
change without notice.