Math - Random in V8
Math - Random in V8
random()
By Betable CTO, Mike Malone
ike most good TIFUs this didnt happen today. It actually happened about
L two years ago. It is, however, still relevant. More signicantly, its not just
me who screwed up. Math.random() in the V8 Javascript engine is screwed
up, too.
Many random number generators in use today are not very good. There is a
tendency for people to avoid learning anything about such subroutines; quite
often we nd that some old method that is comparatively unsatisfactory has
blindly been passed down from one programmer to another, and todays users
have no understanding of its limitations.
etable is built on random numbers. Aside from other more obvious uses
B we like using randomly generated identiers. Our architecture is
distributed and microservice-y, and random identiers are easier to
implement than sequential identiers in this sort of system.
For example, we generate random request identiers whenever we receive an
API request. We thread these identiers through to sub-requests in headers,
log them, and use them to collate and correlate all of the things that
happened, across all of our services, as a result of a single request.
Generating random identiers isnt rocket science. Theres only one
requirement
It must be really, really unlikely that the same identier will ever be generated
twice, causing a collision.
And there are just two factors that impact the likelihood of a collision
1. The size of the identier space the number of unique identiers that are
possible
2. The method of identier generation how an identier is selected from
the space of all possible identiers
Ideally we want a big identier space from which identiers are selected at
random from a uniform distribution (henceforth, well assume that anything
done at random uses a uniform distribution).
We did the birthday paradox math and settled on making our request
identiers 22 character words with each character drawn from a 64 character
alphabet. They look like EB5iGydiUL0h4bRu1ZyRIi or
HV2ZKGVJJklN0eH35IgNaB. Each character in the word has 64 possible
values, there are 22 characters, so there are 64 such words. That makes the
size of our identier space 64 or ~2.
With 2 possible values, if identiers were randomly generated at the rate of one
million per second for the next 300 years the chance of a collision would be
roughly 1 in six billion.
So weve got a big enough identier space, but how do we generate identiers
at random? The answer is a decent pseudo-random number generator
(PRNG), a common feature of many standard libraries. The top of our API
stack is a Node.js service (we also use a lot of Go, but thats another blog
post). Node.js, in turn, uses the V8 Javascript engine that Google built for its
Chrome web browser. All compliant ECMAScript (Javascript)
implementations must implement Math.random(), which takes no arguments
and returns a random number between 0 and 1. Good start.
So, given a sequence of pseudo-random numbers between 0 and 1 we need to
generate a random word with characters from our 64 character alphabet. This
is a pretty common problem, heres the pretty standard solution we chose
varALPHABET='ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_
2
3
random_base64=functionrandom_base64(length){
varstr="";
for(vari=0;i<length;++i){
varrand=Math.floor(Math.random()*ALPHABET.length);
str+=ALPHABET.substring(rand,rand+1);
returnstr;
10
random_base64.jshostedwithbyGitHub
viewraw
Before you start picking it apart, theres nothing wrong with this code it
does exactly what its supposed to do.
Were in business. A procedure for producing random identiers that is
extremely unlikely to produce a collision, even if were producing a million a
second for 300 years. Test, commit, push, test, deploy.
The above code hit production and we forgot about it until a rather alarming
email came through from a colleague, Nick Forte, telling us that the
impossible had happened
Lets ground our discussion by looking at a simple PRNG and the random
numbers it produces
This should make von Neumanns point clear the sequence of numbers
generated by this algorithm is obviously not random. For most purposes, this
non-randomness doesnt really matter. What we really need is an algorithm
Lets go one step further and analyze the number of distinct random values
that a PRNG can produce through some deterministic transformation on its
output. For instance, consider the problem of generating triples of random
values between 0 and 15, like (2, 13, 4) or (5, 12, 15). There are 16 or 4096
such triples, but our simple PRNG can only produce 16 of them.
The V8 PRNG
would have gone with mersenne twister since it is what everyone else
I uses (python, ruby, etc). This short critique, left by Dean McNamee, is the
only substantive feedback on the code review of V8s PRNG when it was rst
committed on June 15, 2009. Deans recommendation is the same one Ill
eventually get around to making in this post.
V8s PRNG code has been tweaked and moved around over the past six years.
It used to be native code, now its in user-space, but the algorithm has
remained essentially the same. The actual implementation uses internal API
and is a bit obfuscated, so lets look at a more readable implementation of the
same algorithm
varMAX_RAND=Math.pow(2,32);
varstate=[seed(),seed()];
3
4
varmwc1616=functionmwc1616(){
varr0=(18030*(state[0]&0xFFFF))+(state[0]>>>16)|
varr1=(36969*(state[1]&0xFFFF))+(state[1]>>>16)|
state=[r0,r1];
8
9
varx=((r0<<16)+(r1&0xFFFF))|0;
10
if(x<0){
11
x=x+MAX_RAND;
12
13
returnx/MAX_RAND;
14
mwc1616.jshostedwithbyGitHub
viewraw
Before returning to the V8 PRNG, lets look one more time at our random
identier generation code
varALPHABET='ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_
2
3
random_base64=functionrandom_base64(length){
varstr="";
for(vari=0;i<length;++i){
varrand=Math.floor(Math.random()*ALPHABET.length);
str+=ALPHABET.substring(rand,rand+1);
returnstr;
10
random_base64.jshostedwithbyGitHub
viewraw
The scaling method on line 6 is important. This is the method that MDN
recommends for scaling a random number, and its in widespread use in the
wild. Its called the multiply-and-oor method, because thats what it does. Its
also called the take-from-top method, because the lower bits of the random
number are truncated, leaving the left-most or top bits as our scaled integer
result. (Quick note: its subtle, but in general this method is slightly biased if
your scaled range doesnt evenly divide your PRNGs output range. A general
solution should use rejection sampling like this, which is part of the standard
library in other languages.)
Do you see the problem yet? Whats weird about the V8 algorithm is how the
two generators are mixed. It doesnt xor the numbers from the two streams
together. Instead, it simply concatenates the lower 16 bits of output from the
If that were true, we should have started seeing collisions almost immediately.
To understand why we didnt, recall our simple example where we pulled
triples from a 4 bit LCG. Birthday paradox math doesnt apply for this
application the sequence is nowhere near random, so we cant pretend it is. Its
clear that we wont produce a duplicate until the 17th triple. The same thing is
happening with the V8 PRNG and our random identiers under certain
conditions, the PRNGs lack of randomness is making it less likely that well see
a collision.
In this case the generators determinism worked in our favor, but thats not
always true. The general lesson here is that, even for a high quality PRNG, you
cant assume a random distribution unless the generators cycle length is much
larger than the number of random values youre generating. A good general
heuristic is
If you need to use n random values you need a PRNG with a cycle length of at least
n.
The reason is that, within a PRNGs period, excessive regularity can cause poor
performance on some important statistical tests (in particular, collision tests).
To perform well, the sample size n must be proportional to the square root of
the period length. Page 22 of Pierre LEcuyers excellent chapter on random
number generation has more detail.
For a use case like ours, where were trying to generate unique values using
multiple independent sequences from the same generator, were less
concerned about statistical randomness and more concerned that the
sequences not overlap. If we have n sequences of length l from a generator
with period p, the probability of an overlap is [1-(nl)/(p-1)] , or
approximately ln/p for a big enough p (see here and here for details). The
point is we need a long cycle length. Otherwise were making a mistake
pretending our sequence is random.
Long story short, if youre using Math.random() in V8 and you need a
sequence of random numbers thats reasonably high quality, you shouldnt use
more than about 24,000 numbers. If youre generating multiple streams of
any substantial size and dont want any overlap, you shouldnt use
Math.random() at all.
If the algorithm that V8s Math.random() uses is poor quality, you might be
wondering how it was chosen at all. Lets see if we can nd out.
A Brief History of MWC1616
Unfortunately, the Diehard tests he was using in the late 1990s werent that
good, at least by todays standards. If you run MWC1616 through a more
modern empirical testing framework like TestU01's SmallCrush it fails
catastrophically (it does even worse than the MINSTD generator, which was
outdated even in the 1990s, but Marsaglias Diehard tests probably didnt have
the granularity to tell him that).
//January12,1999/V8PRNG:((r0<<16)+(r1^0xFFFF))%2^32
varx=((r0<<16)+(r1&0xFFFF))|0;
3
4
//January20,1999:(r0<<16)+r1)%2^32
varx=((r0<<16)+r1)|0;
mwc1616versions.jshostedwithbyGitHub
viewraw
In general, PRNGs are subtle and you should do your own analysis and
understand the limitations of any algorithm youre implementing or
using
Luckily, the Node.js standard library has another PRNG that meets both
requirements: crypto.randomBytes(), a cryptographically secure PRNG
(CSPRNG) that calls OpenSSLs RAND_bytes (which, according to the docs,
produces a random number by generating the SHA-1 hash of 8184 bits of
internal state, which it regularly reseeds from various entropy sources). If
youre in a web browser crypto.getRandomValues() should do the same job.
This isnt a perfect general solution for three reasons
However
Speed is relative, and CSPRNGs are fast enough for most use cases (I can
get about 100MB/s of random data from crypto.getRandomValues() in
Chrome on my machine)
Were still making some assumptions, but they are evidence-based and
pragmatic. If youre unsure about the quality of your non-cryptographic
alternatives, and unless you need deterministic seeding or require rigorous
proofs of quality measures, using a CSPRNG is your best option. If you dont
trust your standard librarys CSPRNG (and you shouldnt for cryptographic
purposes) the right solution is to use urandom, which is managed by the
kernel (Linux uses a scheme similar to OpenSSLs, OS X uses Bruce Schneiers
Yarrow generator).
I cant tell you the exact cycle length of crypto.randomBytes() because as far as
I know theres no closed form solution to that problem (i.e., no one knows).
All I can say is that with a large state space and a continuous stream of new
entropy coming in, it should be safe. If you trust OpenSSL to generate your
public/private key pairs then it doesnt make much sense not to trust it here.
Empirically, once we swapped our call to Math.random() with a call to
crypto.randomBytes() our collision problem went away.
In fact, Chrome could just have Math.random() call the same CSPRNG theyre
using for crypto.randomBytes(), which appears to be what Webkit is doing.
That said, there are lots of fast, high quality non-cryptographic alternatives,
too. Lets put a nal nail in the MWC1616 con and take a look at some other
options.
V8s PRNG is Comparatively Unsatisfactory
My goal was to convince you that V8s Math.random() is broken, and should
be replaced. So far weve found obvious structural patterns in its output bits,
catastrophic failure on empirical tests, and poor performance in the real
world. If you still want more evidence, here are some pretty pictures that
might sway you
Random noise from Safari (left) and V (right) generated in browser with this code.
A large state space, and a large seed ideally at least 1024 bits, since this
will be an upper bound on other qualities of the generator. A state space
of 2 is enough for 99.9% of use cases, with a signicant safety factor.
A very long period, a full cycle generator is great but anything over 2
should be sucient to avoid cycling. Anything over 2 should let us
safely pull 2 values while continuing to pretend weve got a random
sequence.
There are many PRNG algorithms that meet or exceed these requirements.
Xorshift generators (also discovered by Marsaglia) are fast and do very well
on statistical tests (much better than MWC1616). An xorshift variant called
xorgens4096 has been implemented in Javascript by David Bau. It has a 4096bit state space, a cycle length of ~2, and it runs faster than MWC1616 in
Chrome on my machine. Moreover, it has no systematic failures on BigCrush.
Recently its been shown that taking the output of an xorshift generator and
multiplying by a constant is a sucient non-linear transformation for the
generator to pass BigCrush. This class of generators, called xorshift*, is very
fast, easy to implement, and memory ecient. The xorshift1024* generator
meets or exceeds all of our requirements. If the memory premium turns out to
be a real problem, the xorshift64* generator has the same memory footprint,
a longer cycle length, and is faster than MWC1616, beating it on all counts.
Another new family of linear/non-linear hybrid generator called PCG claims
similar performance and quality characteristics.
So there are lots of good algorithms to chose from. That said, the safest choice
is probably a standard Mersenne Twister. The most popular variant,
MT19937, was introduced in the late 90s. Since then its become the standard
generator in dozens of software packages. Its not perfect, but it has been
battle tested and thoroughly analyzed. Its properties are well understood, and
it does well on empirical tests. With an ostentatiously long cycle length of
2-1 its hard to misuse, but it does have an imposing 2KB state space and is
criticized for its memory footprint and relatively poor performance. A quick
search uncovered an existing Javascript implementation, by Sean
McCullough. Inexplicably, its as fast as the existing Math.random()
implementation in Chrome on my machine.
So my advice is that V8 reconsider Dean McNamees comment from six years
ago and use the Mersenne Twister algorithm. Its fast enough, and robust
enough to be safely used by developers who dont have a deep understanding
of how PRNGs work. A more exotic alternative is ne too. Just get rid of
MWC1616, please!
In Summary
There are options for non-cryptographic PRNG algorithms that are faster
and higher quality than MWC1616. V8 should replace its Math.random()
implementation with one of them. There are no losers. Mersenne Twister
(MT19937) is the most popular, and probably the safest choice.
Ill note, in passing, that Mozillas use of the LCG from Javas util.Random
package isnt much better than MWC1616. So SpiderMonkey should probably
go ahead and upgrade too.
In the meantime, the browser continues to be a confusing and dangerous
place. Be safe out there!
.
Special thanks to Nick Forte, who discovered the collision bug in our production
code, Wade Simmons, who originally tracked the problem down to V8s
Math.random() implementation, and the entire Betable Engineering team, who
put up with me ranting about random numbers for two weeks while I wrote this
post.