Random numbers are important when doing all sorts of Artificial Intelligence jobs. Neural networks and evolutionary algorithms rely heavily on the generation of random numbers.
Most programming languages provide a simple random function. In C/C++ and Java it’s called rand() and in Delphi it’s Random. These functions return a random integer between 0 and a set maximum. In Delphi you can specify the maximum by passing a parameter and in the other languages I mentioned you can do something like…
x = rand() % Max
To generate a random real number between 0 and +1 something like the following should be done…
x = Random(100000000) / 100000000;
These functions all return random numbers (actually they’re pseudo-random, but lets not get into that) with a uniform distribution. That means that the probability of any two output values occurring is identical (so an output of 0.001 is just as likely as 0.999 or any other value). The following histogram shows the distribution of outputs (between 0 and 1) from a simple real-valued random function in Delphi. 10,000 random numbers were generated and each one was placed into one of 100 “bins” corresponding to it’s value. Values between 0 and 0.01 go in the first bin, values between 0.01 and 0.02 go in the second bin and so on. The graph shows the number of random values which were placed into each bin.
As you can see, each bin contains roughly the same number of values.
Figure 1: Unifomly distributed random numbers
For the sake of completeness, Delphi code for the uniform random number generation function follows…
function UniformRandom: Double; begin Result := Random(1000000000) / 1000000000; end;
Other Random Distributions
For many AI problems, a good example being the mutation of a synaptic weight in a neural network, it’s not good to use a uniform distribution. When mutating a neural network weight you want small mutations to have a higher probability than large mutations, so that you aren’t too “rough” with the network and it can learn more easily. To do this you can use one of the following two random distributions…
The Normal or Gaussian distribution is well known in most branches of science. It produces a bell shaped curve, half of which can be seen in the following graph. Delphi code for the generation of a random real number with a normal distribution looks like…
function NormalRandom(Sigma : Double): Double; var u1, u2, z : Double; begin u1 := UniformRandom; u2 := UniformRandom; try z := Sqrt(-2.0 * Ln(u1)) * Cos(2 * Pi * u2); except z := 0; end; Result := Sigma * z; end;
The variance parameter, Sigma, controls the width of the distribution. Note that our graph is showing only values above 0, so we only see half of the distribution curve; to see the other half, hold a mirror up to your monitor
Figure 2: Normal (Gaussian) distribution
The Exponential distribution is also useful. It’s worth noting that it favours smaller values to a greater extent than the normal distribution and has a steeper “dropoff”. Programmers will note that it contains fewer calls to maths functions, which is a good indication that it runs more quickly! Here’s a graph…
Figure 3: Exponential distribution
… and a code listing…
function ExponentialRandom(Lambda : Double): Double; begin Result := (-Ln(1.0 - UniformRandom) / Lambda); end;
The parameter, Lambda, controls the width of the distribution.
Symmetric Random Distributions
Here are some more figures, showing symetric versions of the random distributions discussed above. Code follows lower down.
Figure 4: Symetric uniform distribution
Figure 5: Symetric normal distribution
Figure 6: Symetric exponential distribution
The code for this is very simple, but it does the job well enough.
if Random(2) = 0 then Result := X else Result := X * -1;
Note for Delphi users: Check out the RandG function in the Math unit, it generates normally distributed random numbers.