Random number generation using TensorFlow
In the field of Machine Learning, Random numbers generation plays an important role by providing stochasticity essential for model training, initialization, and augmentation. We have TensorFlow, a powerful open-source machine learning library, that contains tf.random module. This module helps us for implementing reproducible random number generation.
In this article, we will understand how we can generate random numbers by using the tools provided by TensorFlow. Two major approaches that we will discuss are:
- tf.random.Generator Class
- Stateless Random Functions
Let us discuss each of these approaches in detail.
tf.random.Generator Class
TensorFlow provides us with tf.random.Generator which is a powerful tool for managing random number generation. It allows us to create unique random numbers, each of them having their own internal state. This internal state is updated each time a random number is generated. It provides a level of control and reproducibility.
Creating a Generator
A generator can be created by using various methods like Generator.from_seed or Generator.from_non_deterministic_state. You can check the code given below.
The tf.random.Generator class in TensorFlow is used to create random number generators with customizable properties and behaviors. Instances of this class represent individual random number generators that can be used to produce random numbers according to various distributions and configurations.
In the following code, we are using TensorFlow’s ‘tf.random.Generator’ to create a random number generator with specific seed. Then, we are going generate random numbers from a normal (Gaussian) distribution using this generator.
- we first create a random number generator with a specified seed value, the practise of using a fixed seed ensures that the generated random number will be the same every time we run the code.
- then, we generate random numbers from a normal distribution using the generator.normal() method.
Python3
import tensorflow as tf# Create a generator with a specific seedgenerator = tf.random.Generator.from_seed(42)# Generate random numbers using the generatorrandom_numbers = generator.normal(shape=(4, 2))print(random_numbers) |
Output:
tf.Tensor(
[[-0.7565803 -0.06854702]
[ 0.07595026 -1.2573844 ]
[-0.23193763 -1.8107855 ]
[ 0.09988727 -0.50998646]], shape=(4, 2), dtype=float32)
The output contains random numbers drawn from normal distribution with mean 0 and standard deviation 1.
Device Placement
In TensorFlow, Device placement refers to the assignment of computational operations to specific devices, such as CPUs or GPUs, for execution. In this case device placement is relevent, when we are obtaining the global generator using tf.random.get_global_generator().
The tf.random.get_global_generator() function in TensorFlow is used to obtain the global random number generator instance.
It is important to consider device placement when you are working with generators. When you invoke tf.random.get_global_generator() the first time, the global generator is constructed and set up on the default device.
Python3
# Obtain the global generatorglobal_generator = tf.random.get_global_generator()# Generate random numbers using the global generatorrandom_numbers_global = global_generator.normal(shape=())print(random_numbers_global) |
Output:
tf.Tensor(-0.1616769, shape=(), dtype=float32)
As an output we get a single random number. Take a note that the global generator is set up on the default device.
Switching Global Generator
It is possible to switch the global generator with a different one. This is done using tf.random.set_global_generator. However, you must use this function cautiously. A change in the global generator can impact tf.function.
Python3
# Create a new generatornew_generator = tf.random.Generator.from_seed(123)# Switch out the global generatortf.random.set_global_generator(new_generator)# Generate random numbers using the new global generatorrandom_numbers_new_global = tf.random.normal(shape=())print(random_numbers_new_global) |
Output:
tf.Tensor(-0.054140557, shape=(), dtype=float32)
In the above code, we have created a new generator and set it as global generator. Then, by using this new global generator we have generated a random number.
Stateless Random Functions
In TensorFlow, Stateless Random Number Generators (RNGs) are the functions that generate random numbers without maintaining any internal state. They are purely functional as they produce the same output for the same set of input arguments.
Stateless random number generation is achieved using cryptographic hash functions to generate random numbers based on input seeds. This ensures that the same input seeds will always produce the same random numbers, making the process deterministic and reproducible.
The tf.random.stateless module provides functionality for stateless random number generation.
Stateless RNGs are deterministic which makes them useful in cases where we need reproducible results are required. Check the code given below, where we have generated random uniform numbers using tf.random.stateless_uniform function.
- After importing TensorFlow library, we have defined a Stateless RNG Key with seed values. These seed values are used as input to the stateless RNG to produce random numbers.
- Then, we generate random uniform numbers using the Stateless RNG.
Python3
import tensorflow as tf# Define a stateless RNG keykey = tf.constant([1, 2], dtype=tf.int32)# Generate random uniform numbers using the stateless RNGrandom_uniform_numbers = tf.random.stateless_uniform(shape=(4, 2), seed=key)print("Random Uniform Numbers:")print(random_uniform_numbers) |
Output:
Random Uniform Numbers:
tf.Tensor(
[[0.8440604 0.19204533]
[0.9962232 0.1603874 ]
[0.27199018 0.649346 ]
[0.10694444 0.06516242]], shape=(4, 2), dtype=float32)
In the above code, tf.random.stateless_uniform function is used to generate a 4×2 tensor of random uniform numbers. The function takes as an input, a shape argument and a stateless RNG key (seed). If you use the same key, then the same set of random numbers will be produced always.
Deterministic Behavior
Stateless RNGs maintain deterministic behavior based on their input arguments. If you re-run the same code with same input arguments, you will get identical random numbers.
Python3
# Re-run with the same keyrandom_uniform_numbers_again = tf.random.stateless_uniform(shape=(4, 2), seed=key)print("Random Uniform Numbers (Again):")print(random_uniform_numbers_again) |
Output:
Random Uniform Numbers (Again):
tf.Tensor(
[[0.8440604 0.19204533]
[0.9962232 0.1603874 ]
[0.27199018 0.649346 ]
[0.10694444 0.06516242]], shape=(4, 2), dtype=float32)
You can see that we got the same numbers as we used the code with the same seed (key).
Difference Between Stateless RNGs and tf.random.Generator
A major difference between stateless RNGs and tf.random.Generator class is that stateless RNGs do not maintain any internal state. Here, the randomness is solely determined by the input seed. Thus, these functions are useful in scenarios where you want to control randomness deterministically. Let’s explore more differences:
|
Feature |
Stateful RNGs (e.g., |
Stateless RNGs (e.g., |
|---|---|---|
| Internal State | Maintains internal state that evolves as random numbers are generated | Does not maintain any internal state; random numbers are solely determined by input seeds |
| Reproducibility | Sequence of random numbers may vary depending on internal state, but can be seeded for reproducibility | Produces deterministic and reproducible random numbers based solely on input seeds |
| Parallelism and Independence | May introduce dependencies among parallel random number sequences | Generates independent random number sequences from the same seed values |
| Performance | Optimized for generating large numbers of random samples | Slightly higher computational overhead due to hash function calculations for each random number |
| Use Cases | General-purpose random number generation where exact reproducibility is not required | Scenarios requiring deterministic reproducibility and parallelism, such as distributed computing or model training |
Conclusion
Thus, TensorFlow provides us with variety of tools that can be used for creating randomness in the machine learning workflows. We studied two approaches to achieve the same. The tf.random.Generator class allows for the creation of distinct random number generators with manageable states, providing control and reproducibility. Stateless RNGs, on the other hand, offer a functional and deterministic approach, ensuring consistent results based solely on input arguments.
Contact Us