Infer.NET is a framework for running Bayesian inference in graphical models. Creating a generative model in Infer.NET typically involves defining a probabilistic model that specifies how data is generated. Here’s a high-level overview of the steps you might follow to create a generative model using Infer.NET:

  1. Define the Model: Specify the probabilistic relationships between variables in your model. This involves defining prior distributions for parameters and likelihood functions for observed data.

  2. Create Variables: Define the random variables in your model. These can be observed variables (data) or latent variables (parameters you want to infer).

  3. Specify the Inference Algorithm: Choose an inference algorithm to estimate the posterior distributions of the latent variables. Infer.NET supports various algorithms like Expectation Propagation (EP), Variational Message Passing (VMP), and Gibbs sampling.

  4. Run Inference: Use the chosen inference algorithm to estimate the posterior distributions of the latent variables given the observed data.

  5. Generate New Data: Once you have the posterior distributions, you can sample from them to generate new data.

Here’s a simple example of how you might define and use a generative model in Infer.NET:

using System;
using Microsoft.ML.Probabilistic.Models;

class Program
{
    static void Main()
    {
        // Define the model
        Variable<bool> coinBias = Variable.Bernoulli(0.5); // Prior for the coin bias
        VariableArray<bool> observations = Variable.Array<bool>(new Range(10));
        observations[0] = Variable<bool>.Observed(true); // Observed data
        observations[1] = Variable<bool>.Observed(false);
        // Add more observations as needed

        using (var inferenceEngine = new InferenceEngine())
        {
            // Run inference
            var posterior = inferenceEngine.Infer<Bernoulli>(coinBias);
            Console.WriteLine($"Posterior probability of heads: {posterior.GetProbTrue()}");

            // Generate new data
            var generatedData = inferenceEngine.Infer<Bernoulli[]>(observations);
            Console.WriteLine("Generated data:");
            foreach (var data in generatedData)
            {
                Console.WriteLine(data.GetProbTrue());
            }
        }
    }
}

Explanation:

  • Model Definition: A simple coin flip model is defined with a Bernoulli prior for the coin bias.
  • Observations: Some observed data (coin flips) are provided.
  • Inference: The posterior distribution of the coin bias is inferred using the observed data.
  • Data Generation: New data is generated by sampling from the posterior distribution.

Let’s take this pymc example and try to write it’s infer.net version ideomatically

import pymc as pm

# Taking draws from a normal distribution
seed = 42
x_dist = pm.Normal.dist(shape=(100, 3))
x_data = pm.draw(x_dist, random_seed=seed)

# Independent Variables:
# Sunlight Hours: Number of hours the plant is exposed to sunlight daily.
# Water Amount: Daily water amount given to the plant (in milliliters).
# Soil Nitrogen Content: Percentage of nitrogen content in the soil.


# Dependent Variable:
# Plant Growth (y): Measured as the increase in plant height (in centimeters) over a certain period.


# Define coordinate values for all dimensions of the data
coords={
 "trial": range(100),
 "features": ["sunlight hours", "water amount", "soil nitrogen"],
}

# Define generative model
with pm.Model(coords=coords) as generative_model:
   x = pm.Data("x", x_data, dims=["trial", "features"])

   # Model parameters
   betas = pm.Normal("betas", dims="features")
   sigma = pm.HalfNormal("sigma")

   # Linear model
   mu = x @ betas

   # Likelihood
   # Assuming we measure deviation of each plant from baseline
   plant_growth = pm.Normal("plant growth", mu, sigma, dims="trial")


# Generating data from model by fixing parameters
fixed_parameters = {
 "betas": [5, 20, 2],
 "sigma": 0.5,
}
with pm.do(generative_model, fixed_parameters) as synthetic_model:
   idata = pm.sample_prior_predictive(random_seed=seed) # Sample from prior predictive distribution.
   synthetic_y = idata.prior["plant growth"].sel(draw=0, chain=0)


# Infer parameters conditioned on observed data
with pm.observe(generative_model, {"plant growth": synthetic_y}) as inference_model:
   idata = pm.sample(random_seed=seed)

   summary = pm.stats.summary(idata, var_names=["betas", "sigma"])
   print(summary)

Next let’s take this pymc example and try to write it’s infer.net version ideomatically

To translate the PyMC example into an idiomatic Infer.NET version, we need to:

  1. Define the data structure for the independent and dependent variables.
  2. Specify the probabilistic model using Infer.NET’s syntax for random variables and factors.
  3. Perform inference to estimate the posterior distributions of the parameters.
  4. Generate synthetic data from the model, if needed.

Here’s how you can write an idiomatic Infer.NET version of your PyMC example:

Infer.NET Version

using System;
using Microsoft.ML.Probabilistic.Models;
using Microsoft.ML.Probabilistic.Distributions;
using Microsoft.ML.Probabilistic.Algorithms;

class Program
{
    static void Main()
    {
        // Define the number of trials and features
        int numTrials = 100;
        int numFeatures = 3;

        // Generate synthetic data for independent variables (x_data)
        double[][] x_data = new double[numTrials][];
        Random random = new Random(42);
        for (int i = 0; i < numTrials; i++)
        {
            x_data[i] = new double[numFeatures];
            for (int j = 0; j < numFeatures; j++)
            {
                x_data[i][j] = Gaussian.Sample(random, 0, 1);
            }
        }

        // Define the model
        Range trial = new Range(numTrials).Named("trial");
        Range feature = new Range(numFeatures).Named("feature");

        // Define variables for the model
        VariableArray<double> x = Variable.Array<double>(trial, feature).Named("x");
        VariableArray<double> betas = Variable.Array<double>(feature).Named("betas");
        Variable<double> sigma = Variable.GammaFromShapeAndScale(1, 1).Named("sigma");

        // Set priors for betas and sigma
        betas[feature] = Variable.GaussianFromMeanAndVariance(0, 1).ForEach(feature);
        sigma.SetTo(Variable.GammaFromShapeAndScale(1, 1));

        // Define the linear model
        VariableArray<double> mu = Variable.Array<double>(trial).Named("mu");
        using (Variable.ForEach(trial))
        {
            mu[trial] = Variable.SumProduct(x[trial, feature], betas[feature]);
        }

        // Define the likelihood
        VariableArray<double> plantGrowth = Variable.Array<double>(trial).Named("plantGrowth");
        plantGrowth[trial] = Variable.GaussianFromMeanAndVariance(mu[trial], sigma * sigma);

        // Observe the synthetic data
        double[] synthetic_y = new double[numTrials];
        for (int i = 0; i < numTrials; i++)
        {
            synthetic_y[i] = 5 * x_data[i][0] + 20 * x_data[i][1] + 2 * x_data[i][2] + Gaussian.Sample(random, 0, 0.5);
        }
        plantGrowth.ObservedValue = synthetic_y;

        // Set the observed data for x
        x.ObservedValue = x_data;

        // Run inference
        InferenceEngine engine = new InferenceEngine();
        engine.Algorithm = new VariationalMessagePassing();

        // Infer the posterior distributions
        var betasPosterior = engine.Infer<Gaussian[]>(betas);
        var sigmaPosterior = engine.Infer<Gamma>(sigma);

        // Print the inferred parameters
        Console.WriteLine("Inferred betas:");
        for (int i = 0; i < numFeatures; i++)
        {
            Console.WriteLine($"beta[{i}]: Mean = {betasPosterior[i].GetMean()}, Variance = {betasPosterior[i].GetVariance()}");
        }
        Console.WriteLine($"Inferred sigma: Shape = {sigmaPosterior.Shape}, Scale = {sigmaPosterior.Scale}");
    }
}

Key Points

  1. Data Generation:
    • The independent variables (x_data) are generated from a normal distribution, similar to the PyMC example.
  2. Model Definition:
    • The model is defined using Infer.NET’s Variable and Range classes.
    • Priors for betas and sigma are set using GaussianFromMeanAndVariance and GammaFromShapeAndScale.
  3. Linear Model:
    • The linear model (mu) is defined as the sum product of x and betas.
  4. Likelihood:
    • The likelihood (plantGrowth) is defined as a Gaussian distribution with mean mu and variance sigma^2.
  5. Inference:
    • The VariationalMessagePassing algorithm is used for inference.
    • The posterior distributions of betas and sigma are inferred and printed.
  6. Synthetic Data:
    • Synthetic data for plantGrowth is generated using fixed parameters, similar to the PyMC example.

Notes

  • Infer.NET uses C# and requires defining variables and ranges explicitly.
  • The VariationalMessagePassing algorithm is used for inference, but you can also use ExpectationPropagation or other algorithms supported by Infer.NET.
  • The syntax for defining probabilistic models and performing inference is different from PyMC, but the underlying concepts are the same.

More from Infer.NET