Optimising Runtime.New()

[aggrand]

Hello, thank you for this amazing package!

I am trying to create vms rapidly, run an extremely short script, then tear them down at high throughput. I'm noticing that, with enough throughput, most time seems to be spent in memory management when either creating the runtime or when populating it with initial global objects needed by the script, or in garbage collection later. Increasing GOGC doesn't seem to help beyond a point.

Something I was considering that may make a huge difference would be structuring the runtime to be completely stack-allocatable. I was imagining allocating both the core runtime a well as the initial global objects on the stack, while allowing data allocated during execution to be heap-allocated. I think the changes needed to support this would be pretty pervasive, since heap escapes happen so easily, and would be something that would need to inform future changes. Examples of relevant changes would be removing pointer references to parts of the runtime (e.g. vm.stash = &r.global.stash, or the references between Runtime and vm).

I was wondering if the maintainers would have interest in this? I'd be interested in working on these changes but would prefer to merge them upstream so that I can stay up-to-date more easily.

[dop251]

Hi. I'm not sure what you're proposing is possible with the current escape analysis in Go. And even if it were possible, it would mean relying on unspecified behaviour that could change without warning with any release of Go.

A better way of minimising allocations on runtime creation would be using template-backed standard objects, whereby each property is created from a template when it's accessed the first time. I believe v8 uses something like this. In the most extreme case this could reduce the number of allocations to just a few (the Runtime struct and its immediate fields).

I had this idea a while back, but unfortunately haven't had enough time to try.

[aggrand]

Hi! Thank you for your response! I see what you mean about it being difficult with the tools we're given, and relying too strongly on unspecified behavior is definitely a problem.

Just for my own understanding, is the benefit of your proposed solution more related to the lazy allocations than to the templates? It seems like copying templates would have similar amounts of allocations, but we'd avoid actually doing so unless necessary?

[dop251]

Yes, exactly. The optimisation will come from the fact that most scripts do not use the full extent of the standard library. See also #459

[aggrand]

Gotcha, makes sense. Would you be interested in a PR for this? I wanted to confirm before setting aside time for it, since staying current with upstream is a priority for me.

[dop251]

I will consider a PR, but please bear in mind there is no guarantee that I will merge it. It's a rather complex and time consuming task, a lot of consideration must be taken to ensure it's not made worse for some common cases (for example, creating the templates is also a time consuming task comparable with the current Runtime initialisation, so creating them always might not be the best approach).

Also, I intend to do this eventually anyway, just not sure when.

[andrewzlchen]

Hello @dop251!

I was also looking to get a better idea of how to reduce allocations in GoJa both on startup and during runtime and it'd be great to get your input on this if possible!

Template Objects Vs Lazy Loading Objects

You mentioned above that perhaps a better way of reducing allocations in GoJa would be to create object templates that are responsible for creating objects when they're accessed for the first time at runtime with V8 being cited as the inspiration.

I found the v8 documentation for something that matches the description here and was wondering if this was what you were referring to:

https://v8docs.nodesource.com/node-0.8/db/d5f/classv8_1_1_object_template.html

I was wondering what you had in mind in terms of how this approach is preferable over lazy loading objects like what's already been done in GoJa for certain objects found here:

https://github.com/search?q=repo:dop251/goja%20newLazyObject&type=code

In short, I had the following questions:

1. Why are template objects preferable over lazy-loading objects? is there more literature on how templating objects works?
2. Assuming that lazy-loading is also possible as a way forward, are there any other current "gotchas" with lazy-loading that prevents us from using them as the main mechanism for reducing allocations?

Thank you for all the work that you do on goja. I really appreciate you taking the time to help the community!

[dop251]

In short, the main difference is that templates would work on a property level, rather than on an object level, i.e. it would be possible not to instantiate every single property of say String.prototype if only one method is used in the code.

I don't see a reason to have the both mechanisms, so templates will eventually replace lazy loading. The only problem here is that while templates would be relatively straightforward to implement for plain objects, it wouldn't be for functions (constructors) due to how the code is structured.

[andrewzlchen]

In short, the main difference is that templates would work on a property level, rather than on an object level, i.e. it would be possible not to instantiate every single property of say String.prototype if only one method is used in the code.

Thanks for the explanation! Not wanting to keep 2 mechanisms for reducing allocations a lot of makes sense though I am still a little fuzzy on the details regarding what the template object looks like.

I've done some research and it seems like the idea of "templates" is C++ feature that doesn't seem to translate very well into golang.

My interpretation of what I've seen is that these template objects will hold references to children and will create them whenever they're referenced.

So as a concrete example in goja, we could make the class prototypes (string, regexp, etc) object templates and whenever javascript code calls into them (ie: "mystring".toUpperCase()), we'd invoke the constructors for those methods living on the prototype

I made a naive implementation of this in this go playground and was wondering if I'm on the right track? I'd love to help out and make a contribution to the project. Thanks again for your time!

[dop251]

A "release candidate" implementation has been pushed into a separate branch (https://github.com/dop251/goja/tree/object-templates).

Obviously the difference in benchmarking of just Runtime.New() is pretty dramatic:

cpu: Intel(R) Core(TM) i7-2600S CPU @ 2.80GHz
      │    old.txt    │               new.txt                │
      │    sec/op     │    sec/op     vs base                │
New-8   332.188µ ± 5%   1.331µ ± 10%  -99.60% (p=0.000 n=10)

      │    old.txt     │               new.txt                │
      │      B/op      │     B/op      vs base                │
New-8   142.258Ki ± 0%   1.719Ki ± 0%  -98.79% (p=0.000 n=10)

      │    old.txt    │              new.txt               │
      │   allocs/op   │ allocs/op   vs base                │
New-8   1787.000 ± 0%   8.000 ± 0%  -99.55% (p=0.000 n=10)

I intend to run a few more tests before merging it into master. In the meantime, any feedback would be greatly appreciated.