Turing-like `@model` macro for specifying BUGS models

[yebai]

Logistic Regression with Random Effects using current syntax

model_def = JuliaBUGS.@bugs begin
    for i in 1:N
        r[i] ~ dbin(p[i], n[i])
        b[i] ~ dnorm(0.0, tau)
        p[i] = logistic(alpha0 + alpha1 * x1[i] + alpha2 * x2[i] + alpha12 * x1[i] * x2[i] + b[i])
    end
    alpha0 ~ dnorm(0.0, 1.0E-6)
    alpha1 ~ dnorm(0.0, 1.0E-6)
    alpha2 ~ dnorm(0.0, 1.0E-6)
    alpha12 ~ dnorm(0.0, 1.0E-6)
    tau ~ dgamma(0.001, 0.001)
    sigma = 1 / sqrt(tau)
end

Desired Turing-like modelling syntax

JuliaBUGS.@parameters struct Tp; r; b; alpha0; alpha1; alpha2; alpha12; tau; end

JuliaBUGS.@model function model((;r, b, alpha0, alpha1, alpha2, alpha12, tau)::Tp, x1, x2, N, n)
    for i in 1:N
        r[i] ~ dbin(p[i], n[i])
        b[i] ~ dnorm(0.0, tau)
        p[i] = logistic(alpha0 + alpha1 * x1[i] + alpha2 * x2[i] + alpha12 * x1[i] * x2[i] + b[i])
    end
    alpha0 ~ dnorm(0.0, 1.0E-6)
    alpha1 ~ dnorm(0.0, 1.0E-6)
    alpha2 ~ dnorm(0.0, 1.0E-6)
    alpha12 ~ dnorm(0.0, 1.0E-6)
    tau ~ dgamma(0.001, 0.001)
    sigma = 1 / sqrt(tau)
end

Note that LHS variables of tilde are passed as the first argument via destructuring syntax ((;r, b, alpha0, alpha1, alpha2, alpha12, tau), ...) while constants and other variables are passed as (..., x1, x2, N, n).

Then, we could do

posterior = model(Tp(), x1, x2, N, n) | data
AbstractMCMC.sample(posterior, 
                        NUTS(0.8),
                        n_samples;
                        chain_type = Chains,
                        n_adapts = n_adapts,
                        init_params = initializations,
                        discard_initial = n_adapts
)

instead of compile(model_def, data, initializations). Similarly, one could use all these other model operations like (condition / )fix / predict / to_distribution.

[sunxd3]

A supplement from offline discussion:

• We'll support default values for arguments
• For now, we haven't give keyward arguments any meaning, so disallowed
• The intermediate variables like p and sigma don't need to appear in arguments (both as regular arguments or parameters)

An idea for the third point: if we want to allow declaring these too, we can use trace instead of parameters, but for prototyping, this is just a nuance now.

[yebai]

Can we re-use Turing.Gibbs with this new unified syntax, given that now Turing.Gibbs is based on condition/decondition? That would be a nice feature.

Related #185

[sunxd3]

Not right now, unfortunately. Some reasons I can think of now:

• Turing.Gibbs is largely implemented using GibbsContext, so not entirely using condition/deconsition
• BUGS' condition right now actually creates "sub-model", not just conditioning. Not that this is a total block, but it needs some thoughts.

[yebai]

@sunxd3, can you open an issue for Turing.jl on how to improve Turing.Gibb for JuliaBUGS's usecase.

[sunxd3]

open an issue for Turing.jl

I haven't forgotten this, but would need a bit more time to clearly think about a proper way to present it.

We had an effort TuringLang/AbstractMCMC.jl#144 trying to unify the two.

Current implementation of Gibbs in Turing.jl is still very DynamicPPL-centric and for good reasons. But I feel that we are moving towards better modularity both in VarInfo and Context, also people are certainly thinking about it (TuringLang/DynamicPPL.jl#892 (review)).

I'll try to give Turing.Gibbs another read and formulate some actionable items for at least discussion points.

[sunxd3]

#291