Multinomial Distributions

The tutorial will showcase all the functionality of the multinomial distributions implemented in fglib2.

First, let’s create three discrete random variables to reason over.

from random_events.variables import Symbolic, Integer
animal = Symbolic("animal", {"Cat", "Mouse", "Dog"})
color = Symbolic("color", {"black", "white", "brown", "grey"})
weight = Integer("weight", range(25))
animal, color, weight

(Symbolic(animal), Symbolic(color), Integer(weight))

Next, we create a random, multinomial distribution over those variables. We will normalize the distribution, such that the sum of all probabilities in it is 1. Note that it is not necessary for factor graphs to have normalized factors.

from fglib2.distributions import Multinomial
import numpy as np
np.random.seed(69)

distribution = Multinomial([animal, color, weight], np.random.rand(3, 4, 25)).normalize()
distribution.variables, distribution.probabilities.shape

((Symbolic(animal), Symbolic(color), Integer(weight)), (3, 4, 25))

As we can see, the distribution first sorts the variables by name for consistent access. The shape of the probabilities is determined by the size of the domain of the distributions variables. Since animal has 3 possible values, color has 4 and age has 25 the shape is (3, 4, 25). It is also possible to create a not normalized distribution by setting normalize to false.

Next, let’s have a look at the 4 basic inference methods. For this, we will create an event and then use it for inference.

from random_events.events import Event
event = Event({animal: {"Cat", "Dog"}, weight: range(10, 25)})

The probability of such an event can be determined by using the probability method.

distribution.probability(event)

0.37218590717704436

The most probable state(s) of a distribution is determined by using the mode method. The mode method returns two things. First, a list of all modes that exist in the distributions. If there would be multiple values inside the probabilities that are the maximum, the first return value of the method would return a list of length > 1. Second, the likelihood of the mode is returned. In discrete settings, the likelihood corresponds to the probability.

distribution.mode()

([{Symbolic(animal): ('Cat',), Symbolic(color): ('brown',), Integer(weight): (5,)}],
 0.006809245726270245)

Next, the marginal distribution can be created by using the marginal method. The resulting distribution will lose the variables and their probabilities that are not in the list of given variables.

marginal = distribution.marginal([animal, color])
marginal.variables, marginal.probabilities.shape

((Symbolic(animal), Symbolic(color)), (3, 4))

As we can see, the remaining variables are animal and color. The shape of the remaining probabilities is (3,4), just as the domain sizes of both variables dictate.

Lastly, the conditional method conditions the distribution to an event. Everything outside the event gets impossible. It computes the probability space P( \(\cdot\) | E).

conditional, probability = distribution.conditional(event)
probability

0.37218590717704436

As we can see, the conditional distribution assigns 1. to the probability of the event it conditioned on.

Bonus: Distributions can also be fancy printed on to tables:

print(marginal.to_tabulate())

╒══════════╤═════════╤═══════════╕
│ animal   │ color   │         P │
╞══════════╪═════════╪═══════════╡
│ Cat      │ black   │ 0.0759703 │
├──────────┼─────────┼───────────┤
│ Cat      │ brown   │ 0.0672517 │
├──────────┼─────────┼───────────┤
│ Cat      │ grey    │ 0.101287  │
├──────────┼─────────┼───────────┤
│ Cat      │ white   │ 0.0702398 │
├──────────┼─────────┼───────────┤
│ Dog      │ black   │ 0.0943926 │
├──────────┼─────────┼───────────┤
│ Dog      │ brown   │ 0.0853249 │
├──────────┼─────────┼───────────┤
│ Dog      │ grey    │ 0.0900629 │
├──────────┼─────────┼───────────┤
│ Dog      │ white   │ 0.0684553 │
├──────────┼─────────┼───────────┤
│ Mouse    │ black   │ 0.0874334 │
├──────────┼─────────┼───────────┤
│ Mouse    │ brown   │ 0.0875107 │
├──────────┼─────────┼───────────┤
│ Mouse    │ grey    │ 0.0827272 │
├──────────┼─────────┼───────────┤
│ Mouse    │ white   │ 0.0893445 │
╘══════════╧═════════╧═══════════╛