CREATIVE IMPACT OF AN EVENT-DRIVEN VISUAL SCRIPTING

TOOL

PRANEET MATHUR1
1
Founder, ARPM Design and Research LLP, Gurugram, India
1
Co-founder, iiterate Technologies GmbH, Adenau, Germany
1
[email protected], [email protected]
ORCID https://orcid.org/0000-0001-7152-2541

Abstract. Computational design is gaining global prominence. With

the increase in demand for technologically capable designers, we find
more designers understanding computers better, learning programming
languages and adapting technologies to fit their needs. This has led to
multidisciplinary communities forming around visual scripting tools
(VSTs) like Grasshopper3D, Dynamo, etc. These communities consist
of many users from creative fields who find it easier to learn a visual
scripting language than a programming language. However, function-
driven programming and various quirks of these tools delimit their
application to a closed spectrum of use-cases. This further limits the
users’ capabilities and forces many to hack their way around basic
programming language paradigms like loops, event handling, etc. VSTs
seem to promote a creative affinity to programming, while also making
it more approachable and accessible. To understand the creative impact
of a more powerful VST, this paper outlines the development and use
of an agnostic event-driven VST - one based on MVVM software
architecture and linked list data structures, written entirely in C# (WPF)
with minimal dependencies. With features like plugin extensibility and
interoperability with 3D software (e.g., Rhinoceros), this new tool is
built to aid creative programming driven by events and data. This
implies enhanced capabilities for the user and enables interactive
computation of data in real-time. User experience inferences are
derived from diverse user studies, with a focus on students and
professionals in the design and AEC industries. Various parameters and
test scenarios are used to objectively assess the impact of enabling
event-driven programming for creative use.

Keywords. Event-Driven Programming, Visual Programming,

Computational Design Tools

1. Introduction
Tools built for creative use are commonly developed by the users themselves. For
instance, an Architect’s toolbox would consist of instruments that aid his/her creative
process. Some architects may prefer to design with sketches, while others may prefer

HUMAN-CENTRIC, Proceedings of the 28th International Conference of the Association for Computer-Aided
Architectural
– LEAVEDesign Research
THIS WHITE BOXinON
Asia (CAADRIA)
PAGE01!!– 2023,
If it has Volume
moved, you can 2,
cut331-340.
and paste it© 2023
back and1,published
to page right click onbythethe
Association for Computer-Aided Architectural Design Research in Asia (CAADRIA), Hong Kong.
boundary and choose 'More Layout Options...' and then under 'Vertical', choose 'Absolute position' - 24 cm (below Page).
332 P. MATHUR

to build massing models with foam-board. Different projects would also demand
different types of tools and workflows. Antoni Gaudi's hanging chain model
(Dragicevic, 2012) is a popular example used to illustrate one of the earliest parametric
tools used for architectural design.
Visual Scripting Tools (VSTs) like Grasshopper3D (GH) and Dynamo have
enabled parametric design and positively impacted users' creative agency (Husein,
2020) (Lee, Gu, & P.Williams). Community-built software plugins have further
expanded the capabilities and flexibility offered by such VSTs over time. Furthermore,
for most creative professionals, learning how to use visual tools like GH seems more
approachable than learning programming languages (Victor, Learnable Programming,
2012). It empowers the user to program geometry in a way that aids their design
process. However, most VSTs come with various limitations.
The solution presented in this paper aims to address some of these limitations and
evaluate their impacts.

1.1. SIGNIFICANCE OF VISUAL PROGRAMMING LANGUAGES

Visual interfaces to computers have a strong positive impact on the user experience
(Dennis Koelma, 1994) (Jackson). This is something the field of computer science has
been aware of since the late 1970s. However, not many attempts have been made to
create a powerful low-level Visual Programming Language (VPL) that is comparable
to traditional text and syntax-based programming languages.
Traditionally, an Integrated Development Environment (IDE) provides a means to
access the features of the programming language via a powerful text editor interface.
Programmers use this interface to describe instructions for the computer to follow in
an English-imitating language like C# or JavaScript.
In the digital era, we have access to powerful visual technologies that allow us to
interface with computers more intuitively. GH is a good example of such technologies
- it enables users to program a 3D modelling software and customize its behaviour as
per their requirements (Sheng-Fen Chien, 2014).
In theory and practice, visual programming has a significant impact on the ability
to interact with computers. Encapsulating several syntactic and semantic rules into one
image empowers a programmer with more information at a glance (Howard, 1994).

1.2. SIGNIFICANCE OF EVENT-DRIVEN PROGRAMMING

Event-driven programming (EDP) and functional programming are disparate
programming paradigms (contributors, Event-driven programming, 2022).
High-level functional programming often implies Total Functional Programming
(TFP), which dictates that the program will be provably terminating (contributors,
Total functional programming, 2022) (D.A.Turner, 2004). This also means
programming recursive computation is not possible. Functional programming
languages are not considered Turing Complete (Wikipedia contributors, 2022).
Examples illustrating functional programming:
● Formulae in Microsoft Office Excel (without lambda functions)
 CREATIVE IMPACT OF AN EVENT-DRIVEN VISUAL 333
SCRIPTING TOOL

● Mathematical operations
● “Prepare an ice cream with the parameters specified by the customer.”
● “Calculate the sum of the provided data.”
Event-driven programs are capable of waiting for an event to occur
asynchronously, before they perform a function or trigger another event. This has many
major implications on how we write programs:
● Recursive programs are made simpler to develop, as the execution of the loop body
can be triggered again when the loop body finishes execution.
● Functions can be performed asynchronously - a program can await the completion
of a task before performing another.
● Events can execute tasks simultaneously and parallelly - Two events can call two
completely separate scripts that would return completely different execution results.
A single event can trigger multiple scripts simultaneously.
● Real-time interactions with the program are possible. These may be human
interactions (e.g., mouse click, text input, button press, etc.), sensor events (e.g.,
temperature change, light detection, knob rotated, etc.) or external software events
(e.g., interoperability communication, API calls, CLI commands, etc.).
Examples illustrating event-driven programming:
● Transitions and Animations in Microsoft Office PowerPoint (Wildenhain, 2022)
● Computer Instruction Codes
● “When a new customer enters, ask them to specify the parameters of the ice cream
they want.”
● “Once the server is finished computing, calculate the sum of the output data from
the server.”

1.3. STUDY OF EXISTING VSTS

Visual Parent TFP/EDP Turing Extensibility Intended Purpose
Scripting Tool Software Complete
by Design
Blender Blender TFP No C++ Parametric Design
GeometryNodes
/ ShaderNodes
Dynamo Autodesk TFP+ No .NET C#, Python Parametric and
Revit Computational BIM
Design
Grasshopper3D McNeel TFP+ No .NET C# or VB, Parametric and
Rhinoceros3D GhPython Computational Design

Houdini SideFX TFP+ No C++, HScript, Parametric Design

Networks Houdini Python
334 P. MATHUR

NodeRED None EDP Yes JavaScript and Cloud IoT

HTML Programming
TouchDesigner None EDP Yes C++ Real-time media
processing
Unity3D Visual Unity3D EDP Yes C# Game engine scripting
Scripting (Bolt)

Table 1. Comparison of popular VSTs

Based on the findings in Table 1, it can be concluded that none of the more popular
VSTs fulfil all of the following conditions:
● No parent software package
● EDP capabilities
● Turing-complete by design
● Extensible with popular modern programming languages
● Built for general purpose programming

2. Proposed Solution Specifications

2.1. VPL SPECIFICATIONS

Following typical programming language design principles (Margaret M. Burnett,
1998) (Victor, Magic Ink, 2006), a VPL should offer the following basic features of a
programming language:
● Identities and metaphors that allow the programmer to relate to the computer’s
context.
● Composability that allows the programmer to break down or build up the script.
● Articulability and comprehensibility for both the programmer and the computer.
In order to provide value beyond what most programming languages offer, the
following features should also be incorporated:
Context Agnostic: The VPL should be a general-purpose language - it should not
be dependent on or restricted to use with a single parent application or context. It should
allow interoperability with multiple applications locally via an SDK, and also over a
network via an API. It should work across multiple platforms, and should allow scripts
to be safely executed on cloud servers (i.e., "Headless" execution).
Culture Agnostic: The VPL should not be derivative of the English language, as
most programming languages are. It should be possible to easily translate the VPL
programs to any other language for the programmer, computing consistent and
expected results.
Homoiconicity: A language is homoiconic if a program written in it can be
manipulated as data using the language (contributors, Homoiconicity, 2022). Programs
 CREATIVE IMPACT OF AN EVENT-DRIVEN VISUAL 335
SCRIPTING TOOL

written in the VPL should be capable of modifying their own script at runtime.
Low-Level Capabilities: The VPL should act as a wrapper around a powerful
language like C#, capable of replacing such a language for many common uses.
Reactive Programming: Programming in the VPL should allow reactive
programming; computing the flow of data and event streams in real-time.
REPL GUI and CLI: Programs written in the VPL should be capable of execution
in a Read–Eval–Print Loop (REPL) cycle with stable exception handling.
Turing-Complete: The VPL should function as a typical Turing-complete
programming language.

2.2. VPL-IDE / VST SPECIFICATIONS

In order to enable access to all the features of the VPL as a VPL-IDE, the VST should
allow the programmer to:
● Read and understand the VPL vocabulary.
● Follow execution and data flow at runtime.
● Observe and manipulate states and data in the script.
● Build scripts by abstraction, implementation and reaction.
A powerful VST should offer the following features:
Standalone IDE: The script editor interface should not depend on any parent
application, but should be capable of communicating with external applications locally
or over a network for real-time interoperability.
Reactive UI: Interface elements should react to events and data flowing through
the script in real-time. User interactions with the elements should trigger events or
compute the script.
Error Handling: Errors thrown by the script components should be reflected to
the user at runtime, breaking only further downstream execution/computation till the
error is resolved.
Breakpoints: The user should be able to define temporary breakpoints in the script.
Execution of all computations should be paused when a breakpoint is hit.
Maximum Information at a Glance: The editor UI should give the user as much
information about the script as possible, while editing as well as at runtime. It should
behave as a typical modern IDE for building and debugging scripts.
Maximum Interaction in a Gesture: The user should be able to modify the script
and interact with it at runtime via UI elements. Actions like manual data entry, script
refactoring, manual event triggers and interactive computation parameters should be
possible while the script is running.

3. Solution Implementation
The solution implemented for this paper is a standalone, EDP VPL that is Turing
complete by design, natively extensible (using .NET C#) and built for general purpose
programming. The project was given the name "Verse3".
336 P. MATHUR

The implementation is limited to a minimum viable working model. The editor

canvas is based on the Windows Presentation Framework (WPF) written using C#.
WPF dictates following the Model-View-ViewModel (MVVM) software architecture
pattern. Doing so decouples the visual rendering of the script from its computation
logic. Based on the outlined VPL specifications, the implemented VPL was built
around a visual syntax as described in Figure 1.

Figure 1. VPL Syntax Abstraction Diagram

Figure 2. Comp Architecture Diagram

Following the MVVM pattern, each Comp contains pre-compiled instructions that can
be accessed by the Core Pipelines. Figure 2 illustrates the parts of a Comp and their
various functions & purposes. These different parts (computation, rendering and VST
interaction) are accessed by the VST core to manipulate or compute each comp.
 CREATIVE IMPACT OF AN EVENT-DRIVEN VISUAL 337
SCRIPTING TOOL

Figure 3. Core Pipelines Architecture Diagram

Each comp in a script contains information about its computation and rendering order.
This information is accessed by the core pipelines and VST interface to compute and
render the comp (as shown in Figure 3).

3.1. DATA STRUCTURES

The data structure architecture implemented is based on multi-level linked lists. This
allows a dynamic pointer and reference-based definition of data structures, which
increases the flexibility of data structures. All data is manipulated internally as
instances of the DataStructure class. All objects can be implicitly cast to and from
DataStructure<T>. This acts as a wrapper around the data and its methods can be used
to ascertain the type, structure and contents of the data. It can also contain child
DataStructure instances, which can be done to construct cascading trees of data. A non-
generic typed DataStructure can also contain DataStructures of more than one data
type, which can be used to define dynamic objects. Each DataStructure contains
internal data and data members. While internal data can contain a raw data object, it is
where the metadata for the DataStructure is stored as a DSMetadata object. This object
reflects information about the DataStructure, such as the number of items stored, the
type of structure (item, list, tree or object), timestamps, etc. The Data property of
DataStructures can be used to get the stored data directly as a raw single object or an
array of objects.
This is drastically different from GH’s implementation of Data Trees. Data in GH
is structured explicitly as individual items, lists or trees, wherein trees are instances of
the proprietary DataTree<T> class.

3.2. EXTENSIBILITY WITH LIBRARY REFERENCES

Similar to GH’s implementation, pre-compiled assemblies can be loaded to extend the
toolset. These assemblies are loaded by default from a subdirectory in the user’s
AppData on Windows systems, but unlike GH the plugins can be hot-loaded at runtime
as well. Each library can contain multiple Comp classes that implement the BaseComp
abstract class. These comps are added to the ribbon panel once the plugin has been
loaded. Any dependencies of the plugins are also loaded from accompanying .dll files.
338 P. MATHUR

The following toolkits were implemented as 14 libraries with 117 comps, for testing
purposes: Boolean, Code, Color, DateTime, Events, Interop, Data, Math, Rhino3DM
(Brep, Curve, Surface and Vector), Text and Type.

3.3. VST EDITOR INTERFACE

Figure 4 depicts a screenshot of the implemented script editor interface. The layout of
the interface is inspired by GH's toolbox ribbon UI and scripting canvas interface.

Figure 4. A screenshot of the Verse3 interface

4. User Studies
User studies were conducted with a diverse group of 26 users. The following
summarized test-case studies are based on a selection of the most typical examples of
scripts built by them during observed testing.

4.1. CASE A
Script description: A debouncing script for an input number slider (also shown in
Figure 4).
Script author description: Experienced AEC technology professional.
Script features: The script is written in a way that checks whether the input value
lies within a selected range, before passing it to the debounce logic. This demonstrates
the user's ability to kitbash various logic and processes with the VPL to describe their
intended behaviour.
Script critique: It does not have any obvious usefulness as an AECO tool, but
could be a part of a larger script for processing real-time analogue or network data. The
script's function could also be consolidated as a single comp in the Data toolkit.
Author feedback: The libraries provided were missing many comps that the
author was looking for (e.g., Range/domain functions, event stream functions like
delay, etc).
 CREATIVE IMPACT OF AN EVENT-DRIVEN VISUAL 339
SCRIPTING TOOL

4.2. CASE B
Script description: Live countdown timer for multiple dates with a time-zone selector.
Script author description: Post-graduate Architecture student.
Script features: It can show the user simultaneous countdowns for multiple
timestamps. Changing the time-zone input also adjusts the countdowns accordingly.
Script critique: It's a simple script which could be useful as a project management
utility or a personal event tracking tool.
Author feedback: The author praised the user interface for selecting the date inputs
and the intuitiveness in programming real-time interactions with data.

4.3. CASE C
Script description: Recursively drawn randomized lines made using an event loop.
Script author description: Computational designer.
Script features: The event loop is used to generate lines recursively, each starting
from the point where the previous line ends. This creates a chain of lines that grows
from a starting point.
Script critique: Adding a 1 second delay before drawing the next line animates the
output geometry, revealing how they are drawn. While the output is not very visually
interesting, the author was successfully able to program simple recursive geometry.
Author feedback: The author was unable to explore other types of recursive
geometry due to the lack of geometry modification comps (e.g., geometry
transformations, deconstruction, etc).

4.4. CASE D
Script description: Simple geometric explorations
Script author description: Software engineer
Script features: It produces a simple scene in Rhino using basic 2D and 3D
geometry constructors.
Script critique: There is very little computational logic in the script. It comprises
of various geometry constructors and input parameters to adjust the output.
Author feedback: The author thoroughly enjoyed the Rhino interoperability
feature, but did not show much interest in the other capabilities of the application, only
wishing for more geometry-related features.

5. Conclusion
The tested implementation lacked many crucial utilities and functionalities for more
powerful test-cases with a focus on creative or AEC applications. However, it led to
some useful insights.
Users with an AEC or creative professional background (little to no prior
programming experience) engaged with the application as a programming tool, using
events and non-visual data types more frequently than working with geometry.
On the other hand, users with prior programming experience showed more interest
340 P. MATHUR

in the visual aspects of the application - the graphical UI and the ability to
programmatically generate 3D geometry.
Some users were unable to intuitively use the application without assistance. This
was a result of multiple bugs and missing features, which hindered the user experience.
The ability to script event-based logic was useful to most users (80%), and did not
seem unintuitive or difficult to work with. These users also agreed that event-driven
programming has a positive impact on the creative agency offered by the application.
The survey also indicated that after using the VST, 76% of them would prefer to
use a VPL as powerful and capable as a typical CLI language.
Further development and refinement of the implemented VST is needed for
drawing a more definitive conclusion on the value imparted by an event-driven VPL.
The project source-code for the VST (Verse3) is available publicly on GitHub:
https://github.com/iiterate-de/Verse3

References
D.A.Turner. (2004). Total Functional Programming. Journal of Universal Computer Science.
Dennis Koelma, A. S. (1994). A visual programming interface for an image processing
environment. Pattern Recognition Letters.
Dragicevic, P. a. (2012). List of Physical Visualizations. Retrieved from:
http://dataphys.org/list/gaudis-hanging-chain-models/.
Event-driven programming. (2022, June 2). In Wikipedia.
https://wikipedia.org/w/index.php?title=Eventdriven_programming&oldid=1091160231
Homoiconicity. (2022). In Wikipedia.
https://en.wikipedia.org/w/index.php?title=Homoiconicity&oldid=1109757100
Howard, E. (1994). Visual Programming: Concepts and Implementations. Computer Science
and Systems Analysis, Technical Reports.
Husein, S. H. (2020). The Influence of Parametric Design Tools on Increasing Creativity in
the Furniture Design Process. Eurasian Journal of Science & Engineering.
Jackson, M. B. (n.d.). Visualisation as an Aid to Low-Level Programming.
Lee, J., Gu, N., & P.Williams, A. (n.d.). Parametric design strateies for the generation of
creative designs. International Journal of Architectural Computing.
https://doi.org/10.1260/1478-0771.12.3.263
Margaret M. Burnett, J. W. (1998). Implementing Level 4 Liveness in Declarative Visual
Programming Languages.
Sheng-Fen Chien, Y.-T. Y. (2014). On Creativity and Parametric Design, A preliminary study
of designer’s behaviour when employing parametric. Digital Aids to Design Creativity,
eCAAADe.
Total functional programming. (2022). In Wikipedia.
https://en.wikipedia.org/wiki/Total_functional_programming
Turing completeness. (2022). In Wikipedia.
https://en.wikipedia.org/wiki/Turing_completeness
Victor, B. (2006). Magic Ink.
Victor, B. (2012). Learnable Programming.
Wildenhain, T. (2022). On the Turing Completeness of MS PowerPoint. Retrieved from:
https://www.andrew.cmu.edu/user/twildenh/PowerPointTM/Paper.pdf.