Random number generation using TensorFlow

In the field of Machine Learning, Random numbers generation plays an important role by providing stochasticity essential for model training, initialization, and augmentation. We have TensorFlow, a powerful open-source machine learning library, that contains tf.random module. This module helps us for implementing reproducible random number generation.

In this article, we will understand how we can generate random numbers by using the tools provided by TensorFlow. Two major approaches that we will discuss are:

1. tf.random.Generator Class
2. Stateless Random Functions

Let us discuss each of these approaches in detail.

tf.random.Generator Class

TensorFlow provides us with tf.random.Generator which is a powerful tool for managing random number generation. It allows us to create unique random numbers, each of them having their own internal state. This internal state is updated each time a random number is generated. It provides a level of control and reproducibility.

Creating a Generator

A generator can be created by using various methods like Generator.from_seed or Generator.from_non_deterministic_state. You can check the code given below.

The tf.random.Generator class in TensorFlow is used to create random number generators with customizable properties and behaviors. Instances of this class represent individual random number generators that can be used to produce random numbers according to various distributions and configurations.

In the following code, we are using TensorFlow's 'tf.random.Generator' to create a random number generator with specific seed. Then, we are going generate random numbers from a normal (Gaussian) distribution using this generator.

• we first create a random number generator with a specified seed value, the practise of using a fixed seed ensures that the generated random number will be the same every time we run the code.
• then, we generate random numbers from a normal distribution using the generator.normal() method.

import tensorflow as tf

# Create a generator with a specific seed
generator = tf.random.Generator.from_seed(42)

# Generate random numbers using the generator
random_numbers = generator.normal(shape=(4, 2))

print(random_numbers)

Output:

tf.Tensor(
[[-0.7565803  -0.06854702]
 [ 0.07595026 -1.2573844 ]
 [-0.23193763 -1.8107855 ]
 [ 0.09988727 -0.50998646]], shape=(4, 2), dtype=float32)

The output contains random numbers drawn from normal distribution with mean 0 and standard deviation 1.

Device Placement

In TensorFlow, Device placement refers to the assignment of computational operations to specific devices, such as CPUs or GPUs, for execution. In this case device placement is relevent, when we are obtaining the global generator using tf.random.get_global_generator().

The tf.random.get_global_generator() function in TensorFlow is used to obtain the global random number generator instance.

It is important to consider device placement when you are working with generators. When you invoke tf.random.get_global_generator() the first time, the global generator is constructed and set up on the default device.

# Obtain the global generator
global_generator = tf.random.get_global_generator()

# Generate random numbers using the global generator
random_numbers_global = global_generator.normal(shape=())

print(random_numbers_global)

Output:

tf.Tensor(-0.1616769, shape=(), dtype=float32)

As an output we get a single random number. Take a note that the global generator is set up on the default device.

Switching Global Generator

It is possible to switch the global generator with a different one. This is done using tf.random.set_global_generator. However, you must use this function cautiously. A change in the global generator can impact tf.function.

# Create a new generator
new_generator = tf.random.Generator.from_seed(123)

# Switch out the global generator
tf.random.set_global_generator(new_generator)

# Generate random numbers using the new global generator
random_numbers_new_global = tf.random.normal(shape=())

print(random_numbers_new_global)

Output:

tf.Tensor(-0.054140557, shape=(), dtype=float32)

In the above code, we have created a new generator and set it as global generator. Then, by using this new global generator we have generated a random number.

Stateless Random Functions

In TensorFlow, Stateless Random Number Generators (RNGs) are the functions that generate random numbers without maintaining any internal state. They are purely functional as they produce the same output for the same set of input arguments.

Stateless random number generation is achieved using cryptographic hash functions to generate random numbers based on input seeds. This ensures that the same input seeds will always produce the same random numbers, making the process deterministic and reproducible.

The tf.random.stateless module provides functionality for stateless random number generation.

Stateless RNGs are deterministic which makes them useful in cases where we need reproducible results are required. Check the code given below, where we have generated random uniform numbers using tf.random.stateless_uniform function.

• After importing TensorFlow library, we have defined a Stateless RNG Key with seed values. These seed values are used as input to the stateless RNG to produce random numbers.
• Then, we generate random uniform numbers using the Stateless RNG.

import tensorflow as tf

# Define a stateless RNG key
key = tf.constant([1, 2], dtype=tf.int32)

# Generate random uniform numbers using the stateless RNG
random_uniform_numbers = tf.random.stateless_uniform(shape=(4, 2), seed=key)

print("Random Uniform Numbers:")
print(random_uniform_numbers)

Output:

Random Uniform Numbers:
tf.Tensor(
[[0.8440604  0.19204533]
 [0.9962232  0.1603874 ]
 [0.27199018 0.649346  ]
 [0.10694444 0.06516242]], shape=(4, 2), dtype=float32)

In the above code, tf.random.stateless_uniform function is used to generate a 4x2 tensor of random uniform numbers. The function takes as an input, a shape argument and a stateless RNG key (seed). If you use the same key, then the same set of random numbers will be produced always.

Deterministic Behavior

Stateless RNGs maintain deterministic behavior based on their input arguments. If you re-run the same code with same input arguments, you will get identical random numbers.

# Re-run with the same key
random_uniform_numbers_again = tf.random.stateless_uniform(shape=(4, 2), seed=key)

print("Random Uniform Numbers (Again):")
print(random_uniform_numbers_again)

Output:

Random Uniform Numbers (Again):
tf.Tensor(
[[0.8440604  0.19204533]
 [0.9962232  0.1603874 ]
 [0.27199018 0.649346  ]
 [0.10694444 0.06516242]], shape=(4, 2), dtype=float32)

You can see that we got the same numbers as we used the code with the same seed (key).

Difference Between Stateless RNGs and tf.random.Generator

A major difference between stateless RNGs and tf.random.Generator class is that stateless RNGs do not maintain any internal state. Here, the randomness is solely determined by the input seed. Thus, these functions are useful in scenarios where you want to control randomness deterministically. Let's explore more differences:

Conclusion

Thus, TensorFlow provides us with variety of tools that can be used for creating randomness in the machine learning workflows. We studied two approaches to achieve the same. The tf.random.Generator class allows for the creation of distinct random number generators with manageable states, providing control and reproducibility. Stateless RNGs, on the other hand, offer a functional and deterministic approach, ensuring consistent results based solely on input arguments.