Wikifunctions:Status updates/2024-11-01

◀

Wikifunctions Status updates

▶

Rewriting the backend

The Abstract Wikipedia team is working toward a rewrite of our backend services in a different programming language, likely Rust. Node/JS has served us well, but we have run up against some limits that would be best dealt with by switching to a different ecosystem.

The immediate work surrounds how we might better interact with WebAssembly (WASM). Almost precisely one year ago, we announced that we would begin running Python and JS code in WASM for its sandboxing characteristics. Since then, we have been interacting with WASM via the WebAssembly System Interface (WASI), which allows WASM commands selective access to the underlying operating system.

While WASI has made our code executors more secure, the use of this tool has caused some bumps. The Node tooling around WASI is not particularly rich. At the time of adoption, we found that our best option was to use WASI command-line interfaces. We decided to run these interfaces in subprocesses. The use of subprocesses has resulted in system stability issues, mainly related to the impossibility of cleaning up subprocesses under certain conditions.

The WASI ecosystem in Rust is much more advanced. Several WASI runtimes provide tools which offer fine levels of control over the binding of WASM commands to syscalls. With these tools, we can streamline our use of WASI. We can run our code executors directly inside of the host Rust program, eliminate subprocesses, and thereby avoid several sources of system instability. As an added bonus, this level of control means that we can also improve our sandboxing–a security win–and, in the far (or not so far?) future, co-opt certain system calls to implement new features for our code executors.

Upcoming Volunteer’s Corner on November 4

Next week, on Monday, 4 November 2024, at 18:30 UTC, we will have our monthly Volunteers’ Corner. Unless you have many questions, we will follow our usual agenda, of giving updates on the upcoming plans and recent activities, having plenty of time and space for your questions, and building a Function together. Looking forward to seeing you on Monday!

Function of the Week: language of lexeme

Function language of lexeme (Z19295) is a simple function: it takes a single argument, a lexeme, and returns the (natural) language of the lexeme. Each Lexeme in Wikidata belongs to exactly one language, e.g. if we take a look at the lexeme L610505, “mkpụrụokwu”, it tells us that it is a word in the Igbo language.

Accordingly, if we choose the Lexeme “mkpụrụokwu” on the function page for language of lexeme, it returns us the language object, displaying the language code ig (which stands for Igbo).

The function has one tester, which takes the Lexeme for the English noun “dog” and returns the object representing the English language.

There is one implementation, a composition: it uses the value by key function, a foundational function to work with objects in Wikifunctions. Every lexeme consists of seven keys, as defined on the type page for lexemes. In order to get the value of a specific key, we can use the value by key function with two arguments: the key we want to look up, and the object itself on which we perform the key lookup.

It makes sense for most types to have a function for each key to decompose objects of that type. Language of Lexeme is such a function: it simply picks one of the keys of the Lexeme object and returns that.