Bayesian Approaches: Exercise

import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

1. Apply Bayesian linear regression with gaussian basis functions to a data sampled from a nonlionear function of your interest and see how \(\alpha\), \(\beta\), \(M\) and \(N\) affect the performance.

2. Compute the log evidence for the above models and see how that is related to the predictin errors.

3. Try forward-backward algorithm with different chains and inputs.

4. In the above chain example, we assumed that only one of the \(K\) bits is one. If we allow simultaneous activation, the number of states increases to \(2^K\), such that the conditional probability has \(2^{K_n}2^{K_{n+1}}\) parameters.

By using a stochastic neural network,

\[ p(x_{n+1,j}|x_n) = f( \sum_{k=1}^{K_n} w_{jk} x_{nk}) \]

where \(f(x)=\frac{1}{1+e^{-x}}\), the conditional probabilty can be defined by \(K_nK_{n+1}\) parameters.

Try implementing data sampling and inference in such a chain.