Towards Sparse Synchronous Programming in Lua
Towards Sparse Synchronous Programming in Lua
brary that provides the benefits of SSM in a more accessible setting. 9 function ssm.fib(n)
Relying on Lua’s incremental garbage collector and support 10 if n < 2 then
for coroutines, lua-ssm is both easier to use and was simpler to 11 ssm.pause(1)
implement than its C counterpart. It provides both a flexible way 12 return n
for users to construct SSM systems and a way for us to more quickly 13 end
experiment with new features. 14 local r1 = ssm.fib:spawn(n - 1)
15 local r2 = ssm.fib:spawn(n - 2)
CCS CONCEPTS 16 local rp = ssm.pause:spawn(n)
17 ssm.wait { r1, r2, rp }
• Computer systems organization → Real-time languages;
18 return r1[1] + r2[1]
Real-time system specification.
19 end
20
KEYWORDS
21 local n = 10
real time systems, concurrency control, computer languages, timing 22
ACM Reference Format: 23 ssm.start(function()
John Hui and Stephen A. Edwards. 2023. Towards Sparse Synchronous 24 local v = ssm.fib(n)
Programming in Lua. In Cyber-Physical Systems and Internet of Things Week 25
2023 (CPS-IoT Week Workshops ’23), May 09–12, 2023, San Antonio, TX, USA. 26 print(("fib(%d) => %d"):format(n, v))
ACM, New York, NY, USA, 6 pages. https://doi.org/10.1145/3576914.3587502
27 −− prints “fib(10) => 55”
28
1 INTRODUCTION 29 local t = ssm.as_msec(ssm.now())
Earlier, we introduced the Sparse Synchronous Model (SSM) [6, 30 print(("Completed in %.2fms"):format(t))
9], an imperative programming model featuring precise timing 31 −− prints “Completed in 10.00ms”
prescriptions and deterministic concurrency. We intended for SSM 32 end)
to be the basis of a compiled language that runs on microcontrollers.
However, implementing a new programming language from
Figure 1: A synchronous Fibonacci example in lua-ssm with
scratch is difficult. Writing a compiler is laborious and error-prone,
delayed table assignment (after), waiting for table writes
and new languages suffer from a lack of libraries and tooling that
(wait), and concurrent function calls (spawn). Library primi-
established languages enjoy. Building and maintaining a custom
tives are highlighted in blue; −− starts comments.
language runtime is further complicated by our desire to support a
wide range of microcontroller platforms.
In this work, we implement SSM as a library for Lua, a light-
This work was supported by the NIH under grant RF1MH120034-01.
weight scripting language that can be easily embedded in other
Permission to make digital or hard copies of all or part of this work for personal or applications [10]. Our library, lua-ssm1 , extends Lua with synchro-
classroom use is granted without fee provided that copies are not made or distributed nous concurrency primitives à la SSM. Lua-ssm builds on existing
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for components of this work owned by others than the Lua features like stackful coroutines [4] and tables (what Lua calls
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or associative arrays [3]) to implement SSM concepts like processes
republish, to post on servers or to redistribute to lists, requires prior specific permission and scheduled variables. Lua-ssm is implemented in <1000 LoC of
and/or a fee. Request permissions from [email protected].
CPS-IoT Week Workshops ’23, May 09–12, 2023, San Antonio, TX, USA pure Lua, does not require modifications to its host language or its
© 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM. runtime, and is compatible with Lua 5.1 to 5.4.
ACM ISBN 979-8-4007-0049-1/23/05. . . $15.00
https://doi.org/10.1145/3576914.3587502 1 Source code available at https://github.com/ssm-lang/lua-ssm
361
CPS-IoT Week Workshops ’23, May 09–12, 2023, San Antonio, TX, USA John Hui and Stephen A. Edwards
1.1 Example: Synchronous Fibonacci In lines 23–32, the program uses lua-ssm’s start() function to
Figure 1 shows an adaptation of the Fibonacci example from Ed- call fib() within a synchronous context. This execution context is
wards & Hui [6], implemented using lua-ssm (imported in line 1). managed by lua-ssm and provides processes, priorities, and instants.
This deterministic synchronous program logically terminates in Library primitives like wait() and spawn() only work within such
10 ms. Like SSM, lua-ssm programs reason about logical time (in- a context.
stead of physical “wall-clock” time), which only advances when The program in Figure 1 specifies that it executes in exactly
the program explicitly requests it to. Most statements execute and 10 ms of logical time because it prescribes that fib(n) produces a
terminate in the same logical time instant, and the future is always result at 𝑛 ms. start() executes its first argument, a Lua closure,
referred to relative to the current instant. Isolating logical from starting at instant 0. The synchronous call to fib() in line 24 blocks
physical time like this allows our library to give deterministic guar- until fib() returns, so the call to now() in line 29 tells us the amount
antees about programs’ logical temporal behavior, independent of of logical time elapsed since fib() was called. Here, all 177 pro-
platform speed. The lua-ssm runtime strives to keep logical time cesses are spawned at instant 0, so the overall execution time is
synchronized to physical time, but may lag behind. set by the largest n passed to fib(). Though spawning so many
Our program defines a helper function, pause(d) (lines 3–7), processes like this is contrived and inefficient, lua-ssm supports a
which suspends execution for d ms using a local channel table t practically unbounded number of processes. Lua’s garbage collector
created by the Channel constructor in line 4. The after() method automatically reclaims unused memory after processes terminate.
on line 5 schedules a delayed assignment of t.go = true for d ms in
the future (here, the choice to assign the key "go" the value true is 1.2 Overview
arbitrary). The call of after() is non-blocking, so pause() in line 7 In this paper, we discuss the design and implementation of lua-
immediately calls wait() to suspend the process until t is written ssm. In Section 2, we describe our library’s API and semantics; in
d ms later. Section 3, we discuss our library’s implementation; in Section 4, we
The recursive fib() function (lines 9–19) uses pause() in two conclude with a discussion on related and future work.
ways. In line 11, fib() calls pause() synchronously, meaning the
caller, fib(), will block until the callee, pause(), returns. So, fib() 2 SEMANTICS
will pause for 1 ms when called with n < 2 before returning n. Lua-ssm adapts SSM to better suit the idioms of Lua. In this section,
By contrast, fib() calls pause() concurrently in line 16 using we describe the semantics of our library and compare it to the SSM
our library’s spawn() function. This spawns a new process that toy language of Edwards & Hui [6]. We will focus our discussion on
immediately executes the first instant of pause(). Once the newly library-provided primitives like after() and wait(). Utilities like
spawned pause() process reaches the wait() at line 6, fib() will pause() can be implemented using these primitives, and can be
resume execution at line 17, in the same instant it called spawn(). made available to lua-ssm programs in a separate library.
The two recursive fib() calls in lines 15 and 16 are similarly
concurrent. A spawned process is placed at a priority just above the 2.1 Channel Tables
process that spawned it and below any existing higher-priority pro- Lua-ssm’s channel tables replace SSM’s scheduled variables, and
cesses, so processes spawned from the same process are prioritized support delayed assignments (using after()) and blocking (using
in the order they are created with highest first. So, after line 17, the wait()). They are implemented as an extension of Lua’s native
processes are prioritized as follows (here, ≺ means “is at a higher tables, which consist of entries that map non-nil keys to non-nil
priority than”): values. Their fields are initialized by passing a table literal to the
fib(n - 1) ≺ fib(n - 2) ≺ pause(n) ≺ fib(n) Channel constructor, and are accessed using Lua’s dot- and index-
Lua-ssm uses tag-range relabeling [2, 5] to dynamically allocate notation:
priority numbers; see Section 3.1. local tbl = ssm.Channel { key = 42 }
Functions called synchronously behave as usual: calling a func- assert(tbl.key == 42) −− dot notation
tion synchronously blocks its calling processes until the result is assert(tbl["key"] == 42) −− equivalent index notation
calculated and returned. For example, fib()’s return value is bound Assigning to keys in a channel table is an instantaneous assign-
to local variable v in line 24 once fib() terminates. By contrast, ment: in contrast to assignments scheduled using after(), instanta-
spawning a process returns a return channel that will be written neous assignments take effect immediately like regular table writes:
with the return value when the spawned process terminates. The
tbl.key = 420 −− write existing entry
return channels from the processes spawned in lines 14–16 are
tbl.newkey = 0 −− define new entry
bound to variables r1, r2, and rp. When fib() terminates, its return
assert(tbl.key == 420 and tbl.newkey == 0)
value is written to key 1 of its return channel; e.g., as read in line 18.
Processes can wait() on return channels just like they can on As in SSM, instantaneous assignments unblock all lower priority
regular channel tables. For example, at line 17, fib() waits for the processes waiting on that channel table. However, lua-ssm’s chan-
spawned fib() and pause() processes to complete. When invoked nel tables are written at a finer granularity than SSM’s scheduled
with braces, such as in line 17, wait() is conjunctive—it will only variables, whose scheduled variables can only be written in their en-
unblock once r1, r2, and rp have all been written; see Section 2.2. tirety. In fact, there is no direct lua-ssm equivalent for updating an
After wait() at line 17, fib() returns the sum of the return values entire table: tbl = different_tbl sets tbl to refer to different_tbl,
from its two concurrent recursive calls. rather than writing values into the channel table referred by tbl.
362
Towards Sparse Synchronous Programming in Lua CPS-IoT Week Workshops ’23, May 09–12, 2023, San Antonio, TX, USA
Lua-ssm’s per-key semantics extend to its after() primitive, Since Lua syntax allows us to omit parentheses for function calls
which schedules delayed assignments only to the specified keys: when the only argument is a table literal, we can write purely
tbl:after(ssm.msec(10), { key = 24 }) conjunctive wait() statements like wait {r1, r2, rp}, from line 17
ssm.wait(tbl) −− wait for the write; only tbl.key is written of Figure 1. wait() returns an array of Booleans indicating which
assert(tbl.key == 24 and tbl.newkey == 0) wait specifications were met.
Further work is needed to determine which condition—both buttons For convenience, lua-ssm also supports concurrent function calls
being pressed or timing out—caused the statement to unblock. using method call syntax (e.g., ssm.fib:spawn(), as seen in Figure 1)
To address this limitation, lua-ssm’s wait() provides both dis- when synchronous functions are defined on keys of the ssm module:
junctive and conjunctive semantics, with the unblocking condi- function ssm.wait_for(chan, timeout)
tion given in disjunctive normal form (DNF). Its general form is −− same as wait_for()
wait(𝑤1 , . . ., 𝑤𝑛 ), where each wait specification 𝑤𝑘 may be a sin-
end
gle channel table 𝑐 or a Lua array of channel tables {𝑐 1 , . . ., 𝑐𝑘 }.
wait() is disjunctive over all wait specifications, but conjunctive When a synchronous function is defined this way, concurrent calls
within each wait specification, so the above unblocking condition may be made with method call syntax:
can be succinctly expressed as:
ssm.wait_for:spawn(some_chan, some_timeout)
ssm.wait({a, b}, t) −− i.e., (a & b) | t
363
CPS-IoT Week Workshops ’23, May 09–12, 2023, San Antonio, TX, USA John Hui and Stephen A. Edwards
for p in scheduled_processes(ctx.run_q) do
Figure 2: Initialization of lua-ssm’s synchronous context.
process_resume(p)
end
2.4 Synchronization and Return Channels end
The assigned priorities are relative to those of processes that exist
at the time of creation. So, calling spawn() multiple times will create Figure 3: lua-ssm’s “tick” function, which executes the syn-
processes with successively lower priorities: chronous context for an instant. Each backend drives ex-
ssm.foo:spawn() −− highest priority
ecution by calling run_instant() while synchronizing with
ssm.foo:spawn() −− next highest priority
a platform-specific clock. channel_do_update() (not shown)
−− (parent) lowest priority
copies scheduled updates from each channel’s later field
to its shadow field and adds sensitive processes to ctx.run_q.
and vice versa for defer():
ssm.foo:defer() −− lowest priority
ssm.foo:defer() −− next lowest priority local function process_resume(next_proc)
−− (parent) highest priority local prev_proc
prev_proc, ctx.proc = ctx.proc, next_proc
Lua-ssm’s concurrent function calls differ from SSM’s blocking coroutine.resume(ctx.proc.tid)
fork in that they do not suspend the execution of the parent. As ctx.proc = prev_proc
such, they allow more flexibility than nested fork/join. For instance, end
lua-ssm permits calling spawn() in a loop to create a variable number
of processes in one instant. function ssm.wait(...)
A lua-ssm process is created with a return channel that allows local wait_specs = { ... }
a parent to wait for its children and receive their return values. sensitize(ctx.proc, wait_specs)
Return channels behave like regular channel tables, and are (in- while not specs_satisfied(wait_specs) do
stantaneously) assigned the return values of a process when it −− keep waiting until at least one wait specification is satisfied
terminates. As such, they function as a future for the child process, coroutine.yield()
which can be queried by the parent to check for return values. end
Return channels support Lua functions that return multiple val- return desensitize(ctx.proc, wait_specs)
ues at once (e.g., return true, "mesg"). These are assigned to return end
channels positionally, so each return value is written to a successive
key, starting from 1:
Figure 4: Implementation of process_resume() and wait(), us-
local rc = ssm.foo:spawn() ing Lua’s built-in coroutine.
−− foo(): return true, “mesg”
assert(rc[1] == true and rc[2] == "mesg")
3.1 Scheduling Processes
3 IMPLEMENTATION Like the runtime proposed by Edwards & Hui [6], lua-ssm uses two
As a Lua library, lua-ssm takes advantage of its host language’s priority queues implemented as binary heaps: an event queue of
existing features, libraries, and ecosystem. For instance, Lua’s in- delayed assignments on channel tables and a run queue of active
cremental garbage collector relieves a major burden from the im- processes scheduled in the current instant. Each instant is executed
plementation of SSM, while its standard library, module system, by the “tick” function shown in Figure 3.
and FFI capabilities allow lua-ssm to integrate with existing code. Lua’s built-in coroutines [4] greatly simplify managing activa-
However, SSM’s scheduled variables and synchronous execution tion records of suspended threads. process_resume(), shown in
model are foreign concepts to Lua, as they rely on the synchronous Figure 4, uses Lua’s built-in coroutine.resume() to resume a sus-
context maintained by lua-ssm. The synchronous context, shown pended process; wait() calls coroutine.yield() to return to the
in Figure 2, keeps track of the timestamp of the current instant, the coroutine.resume() call site. Since Lua’s coroutines are stackful,
current running process, and two queues used by the scheduler. coroutine.yield() (hence wait()) works at arbitrarily deep levels
In this section, we discuss how lua-ssm uses Lua’s coroutines and of the Lua call stack.
metatables to embed a synchronous programming model within a Where possible, lua-ssm also uses Lua’s own call stack to avoid
procedural scripting language. adding to the run queue unnecessarily. Synchronous function calls
364
Towards Sparse Synchronous Programming in Lua CPS-IoT Week Workshops ’23, May 09–12, 2023, San Antonio, TX, USA
365
CPS-IoT Week Workshops ’23, May 09–12, 2023, San Antonio, TX, USA John Hui and Stephen A. Edwards
1 local ssm = require("ssm") { backend = "luv" } Edwards & Hui’s ssm-runtime library and runs on embedded hard-
2 ware. While lua-ssm is based on the same programming model, it is
3 function ssm.pause(d) implemented entirely in Lua. It overcomes many of ssm-runtime’s
4 −− same as before, see Figure 1 limitations that stem from its low-level implementation in C.
5 end Like Copilot [13], Haski [14], and Scoria [11], lua-ssm is an em-
6 bedded domain-specific language (eDSL) [8] for synchronous com-
7 ssm.start(function() puting, though it uses Lua rather than Haskell as its host language.
8 local stdin = ssm.io.get_stdin() Unlike the aforementioned eDSLs, which are deep embeddings that
9 local stdout = ssm.io.get_stdout() generate C code, lua-ssm is a shallow embedding whose execution
10 takes place within its host language. While a shallow embedding
11 while ssm.wait(stdin) do precludes compiling and optimizing lua-ssm programs within its
12 if not stdin.data then −− stdin was closed host language, it enables easier integration with the Lua ecosystem.
13 break Like SSM, Lingua Franca (LF) [12]’s reactor model is also inspired
14 end by discrete-event systems, but LF takes the opposite implementation
15 local str = stdin.data −− buffer data from stdin strategy of lua-ssm. Rather than embedding the reactor model in an
16 ssm.pause(250) −− suspend for a bit existing language, LF embeds fragments of existing languages like C
17 stdout.data = str −− write to stdout and TypeScript within its model. While LF emphasizes analyzability
18 end and scalability, lua-ssm prioritizes flexibility and expressiveness.
19 stdout.data = nil −− close stdout We believe that embedded application development with lua-
20 end) ssm will greatly benefit from Lua’s flexibility and FFI capabilities.
Using an interpreted language will likely impact performance, so
we plan to quantify that impact and replace lua-ssm’s hot code
Figure 7: An “echo” program using lua-ssm’s luv backend.
paths with optimized C or Pallene [7]. Finally, we hope to evaluate
the suitability of using lua-ssm for non-performance-critical tasks
physical timing characteristics. Meanwhile, lua-ssm’s scheduler is such as real-time application prototyping, coordination, and testing.
“driven” by a backend that is responsible for synchronizing the pro-
gram with physical time. A backend has access to platform-specific REFERENCES
timing and I/O capabilities (e.g., device registers, system calls), and [1] 2019. IEEE Standard for Floating-Point Arithmetic. IEEE Std 754-2019 (Revision
of IEEE 754-2008) (2019), 1–84. https://doi.org/10.1109/IEEESTD.2019.8766229
repeatedly invokes the core scheduler by calling run_instant() [2] Michael A. Bender, Richard Cole, Erik D. Demaine, Martin Farach-Colton, and Jack
(Figure 3). A backend may expose asynchronous input sources and Zito. 2002. Two Simplified Algorithms for Maintaining Order in a List. In European
Symposium on Algorithms. 152–164. https://doi.org/10.1007/3-540-45749-6_17
output destinations to lua-ssm programs as channel tables. [3] Jon Bentley. 1985. Programming Pearls: Associative Arrays. Commun. ACM 28,
Lua-ssm currently supports two backends. The simulation back- 6 (June 1985), 570–576. https://doi.org/10.1145/3812.315108
end simulates the execution of lua-ssm programs without synchro- [4] Ana Lúcia de Moura and Roberto Ierusalimschy. 2009. Revisiting coroutines.
ACM Transactions on Programming Languages and Systems 31 (2009), 6:1–6:31.
nizing to a physical clock. It lacks external dependencies, so it is [5] P. Dietz and D. Sleator. 1987. Two Algorithms for Maintaining Order in a List.
useful for platform-agnostic prototyping. Due to SSM’s determin- In Proceedings of the Symposium on Theory of Computing (STOC). 365—-372.
ism, simulation can be used as an oracle for logical behavior. https://doi.org/10.1145/28395.28434
[6] Stephen A. Edwards and John Hui. 2020. The Sparse Synchronous Model. In
The second backend, luv, uses Lua bindings to libuv2 , a multi- Forum on Specification and Design Languages (FDL). Kiel, Germany. https://doi.
platform asynchronous I/O library. For example, the interactive org/10.1109/FDL50818.2020.9232938
[7] Hugo Musso Gualandi and Roberto Ierusalimschy. 2020. Pallene: A companion
terminal application in Figure 7 echoes its standard input to stan- language for Lua. Science of Computer Programming 189 (apr 2020), 102393.
dard output after a 250 ms delay. The standard input and output https://doi.org/10.1016/j.scico.2020.102393
streams are modeled using channel tables (bound to stdin and [8] Paul Hudak. 1996. Building Domain-Specific Embedded Languages. Comput.
Surveys 28, 4es (Dec. 1996), 196–es. https://doi.org/10.1145/242224.242477
stdout on lines 8–9) that can be waited on and written to. A libuv [9] John Hui and Stephen A. Edwards. 2022. The Sparse Synchronous Model on
callback for standard input assigns the received input to stdin.data, Real Hardware. ACM Transactions on Embedded Computing Systems (Dec. 2022).
which the echo program reads (line 15), while a low-priority han- https://doi.org/10.1145/3572920 Just Accepted.
[10] Roberto Ierusalimschy, Luiz Henrique de Figueiredo, and Waldemar Celes. 2007.
dler process—created using defer() by io.get_stdout()—forwards The Evolution of Lua. In Proceedings of the History of Programming Languages
assignments to stdout.data (line 17) to standard output. The stream (HOPL III). 2–1–2–26. https://doi.org/10.1145/1238844.1238846
[11] Robert Krook, John Hui, Bo Joel Svensson, Stephen A. Edwards, and Koen
is closed when nil is assigned to the data field (lines 12 and 19). Claessen. 2022. Creating a Language for Writing Real-Time Applications for
the Internet of Things. In Proceedings of the International Conference on Formal
4 RELATED AND FUTURE WORK Methods and Models for Codesign (MEMOCODE). Shanghai, China.
[12] Marten Lohstroh, Christian Menard, Soroush Bateni, and Edward A. Lee. 2021.
Lua-ssm is not the first implementation of the Sparse Synchro- Toward a Lingua Franca for Deterministic Concurrent Systems. ACM Transactions
nous Model [6]. Hui & Edwards [9] are developing a functional on Embedded Computing Systems 20, 4 (July 2021), 1–27. https://doi.org/10.1145/
3448128
language with user-defined algebraic data types, references, and [13] Lee Pike, Alwyn Goodloe, Robin Morisset, and Sebastian Niller. 2010. Copilot:
pattern matching that implements SSM; Krook et al. [11] embed an A Hard Real-Time Runtime Monitor. In Proceedings of the 1st Intl. Conference on
SSM-based imperative language, Scoria, Haskell. Both of these syn- Runtime Verification (LNCS). Springer.
[14] Nachiappan Valliappan, Robert Krook, Alejandro Russo, and Koen Claessen. 2020.
chronous languages compile to portable C code that links against Towards secure IoT programming in Haskell. In Proceedings of the 13th ACM
SIGPLAN International Symposium on Haskell. ACM. https://doi.org/10.1145/
2 See https://libuv.org and https://github.com/luvit/luv 3406088.3409027
366