開発終了したオワコンtheanoを使っていたpymc3が、時代の寵児tensorflowを使うPyMC4として生まれ変わったらしい。

PyMC4¶

このサイトを参考にしてpymc4を使ってみる。

!pip3 install git+https://github.com/pymc-devs/pymc4.git#egg=pymc4

Collecting pymc4 from git+https://github.com/pymc-devs/pymc4.git#egg=pymc4
  Cloning https://github.com/pymc-devs/pymc4.git to /tmp/pip-install-f_r9c4h6/pymc4
Collecting xarray==0.10.4 (from pymc4)
  Downloading https://files.pythonhosted.org/packages/8b/36/084834fb7b5f67afa4788c302ec876f8fffceea4f9d39dc10db9f7308c01/xarray-0.10.4-py2.py3-none-any.whl (431kB)
    100% |################################| 440kB 9.7MB/s eta 0:00:01
Collecting numpy==1.14.3 (from pymc4)
  Downloading https://files.pythonhosted.org/packages/71/90/ca61e203e0080a8cef7ac21eca199829fa8d997f7c4da3e985b49d0a107d/numpy-1.14.3-cp36-cp36m-manylinux1_x86_64.whl (12.2MB)
    100% |################################| 12.2MB 5.9MB/s  eta 0:00:01
Collecting tqdm==4.23.3 (from pymc4)
  Downloading https://files.pythonhosted.org/packages/d8/ca/6524dfba7a0e850d3fda223693779035ddc8bf5c242acd9ee4eb9e52711a/tqdm-4.23.3-py2.py3-none-any.whl (42kB)
    100% |################################| 51kB 25.9MB/s ta 0:00:01
Collecting biwrap (from pymc4)
  Downloading https://files.pythonhosted.org/packages/ac/54/78d4d5459d7385f90908185641647ff2e8ed4a4b5c239976f78e26888679/biwrap-0.1.6.tar.gz
Collecting tf-nightly==1.9.0.dev20180607 (from pymc4)
  Downloading https://files.pythonhosted.org/packages/3d/79/3a29de809de4b6a8b6d57f176a5fcb7f3e42ac047d48132a572ee20d7f24/tf_nightly-1.9.0.dev20180607-cp36-cp36m-manylinux1_x86_64.whl (51.6MB)
    100% |################################| 51.6MB 1.6MB/s  eta 0:00:01
Collecting tfp-nightly==0.3.0.dev20180725 (from pymc4)
  Downloading https://files.pythonhosted.org/packages/09/d4/55b301ddfee7da1391c7def2bd055ee61945910dfda1e86217e72670c60a/tfp_nightly-0.3.0.dev20180725-py2.py3-none-any.whl (496kB)
    100% |################################| 501kB 14.1MB/s ta 0:00:01
Collecting tb-nightly==1.9.0a20180613 (from pymc4)
  Downloading https://files.pythonhosted.org/packages/31/0e/2ede9a6f7dee7774f1214a4ce3e71be819e9e2c7b680d896921ff1410121/tb_nightly-1.9.0a20180613-py3-none-any.whl (3.3MB)
    100% |################################| 3.3MB 29.1MB/s eta 0:00:01
Collecting pandas>=0.18.0 (from xarray==0.10.4->pymc4)
  Downloading https://files.pythonhosted.org/packages/e1/d8/feeb346d41f181e83fba45224ab14a8d8af019b48af742e047f3845d8cff/pandas-0.23.4-cp36-cp36m-manylinux1_x86_64.whl (8.9MB)
    100% |################################| 8.9MB 11.5MB/s eta 0:00:01
Requirement already satisfied: termcolor>=1.1.0 in /root/.pyenv/versions/3.6.6/envs/py36/lib/python3.6/site-packages (from tf-nightly==1.9.0.dev20180607->pymc4) (1.1.0)
Requirement already satisfied: grpcio>=1.8.6 in /root/.pyenv/versions/3.6.6/envs/py36/lib/python3.6/site-packages (from tf-nightly==1.9.0.dev20180607->pymc4) (1.15.0)
Requirement already satisfied: wheel>=0.26 in /root/.pyenv/versions/3.6.6/envs/py36/lib/python3.6/site-packages (from tf-nightly==1.9.0.dev20180607->pymc4) (0.31.1)
Requirement already satisfied: protobuf>=3.4.0 in /root/.pyenv/versions/3.6.6/envs/py36/lib/python3.6/site-packages (from tf-nightly==1.9.0.dev20180607->pymc4) (3.6.1)
Requirement already satisfied: six>=1.10.0 in /root/.pyenv/versions/3.6.6/envs/py36/lib/python3.6/site-packages (from tf-nightly==1.9.0.dev20180607->pymc4) (1.11.0)
Requirement already satisfied: gast>=0.2.0 in /root/.pyenv/versions/3.6.6/envs/py36/lib/python3.6/site-packages (from tf-nightly==1.9.0.dev20180607->pymc4) (0.2.0)
Requirement already satisfied: absl-py>=0.1.6 in /root/.pyenv/versions/3.6.6/envs/py36/lib/python3.6/site-packages (from tf-nightly==1.9.0.dev20180607->pymc4) (0.5.0)
Requirement already satisfied: astor>=0.6.0 in /root/.pyenv/versions/3.6.6/envs/py36/lib/python3.6/site-packages (from tf-nightly==1.9.0.dev20180607->pymc4) (0.7.1)
Requirement already satisfied: markdown>=2.6.8 in /root/.pyenv/versions/3.6.6/envs/py36/lib/python3.6/site-packages (from tb-nightly==1.9.0a20180613->pymc4) (3.0.1)
Requirement already satisfied: werkzeug>=0.11.10 in /root/.pyenv/versions/3.6.6/envs/py36/lib/python3.6/site-packages (from tb-nightly==1.9.0a20180613->pymc4) (0.14.1)
Requirement already satisfied: python-dateutil>=2.5.0 in /root/.pyenv/versions/3.6.6/envs/py36/lib/python3.6/site-packages (from pandas>=0.18.0->xarray==0.10.4->pymc4) (2.7.3)
Collecting pytz>=2011k (from pandas>=0.18.0->xarray==0.10.4->pymc4)
  Downloading https://files.pythonhosted.org/packages/30/4e/27c34b62430286c6d59177a0842ed90dc789ce5d1ed740887653b898779a/pytz-2018.5-py2.py3-none-any.whl (510kB)
    100% |################################| 512kB 54.5MB/s eta 0:00:01
Requirement already satisfied: setuptools in /root/.pyenv/versions/3.6.6/envs/py36/lib/python3.6/site-packages (from protobuf>=3.4.0->tf-nightly==1.9.0.dev20180607->pymc4) (39.0.1)
Building wheels for collected packages: pymc4, biwrap
  Running setup.py bdist_wheel for pymc4 ... done
  Stored in directory: /tmp/pip-ephem-wheel-cache-lh3cvsb2/wheels/0b/4b/70/0db4af73d245304b307881f640eed518015f901d999e5b08aa
  Running setup.py bdist_wheel for biwrap ... done
  Stored in directory: /root/.cache/pip/wheels/e5/d9/4a/0c3162411bad4df21befe385cea5bb26458ebb1aa9b2e1c13e
Successfully built pymc4 biwrap
keras 2.2.2 has requirement keras-applications==1.0.4, but you'll have keras-applications 1.0.5 which is incompatible.
keras 2.2.2 has requirement keras-preprocessing==1.0.2, but you'll have keras-preprocessing 1.0.3 which is incompatible.
Installing collected packages: numpy, pytz, pandas, xarray, tqdm, biwrap, tb-nightly, tf-nightly, tfp-nightly, pymc4
  Found existing installation: numpy 1.15.2
    Uninstalling numpy-1.15.2:
      Successfully uninstalled numpy-1.15.2
Successfully installed biwrap-0.1.6 numpy-1.14.3 pandas-0.23.4 pymc4-0.0.1 pytz-2018.5 tb-nightly-1.9.0a20180613 tf-nightly-1.9.0.dev20180607 tfp-nightly-0.3.0.dev20180725 tqdm-4.23.3 xarray-0.10.4

import pymc4 as pm
from tensorflow_probability import edward2 as ed

model = pm.Model()

@model.define
def simple(cfg):
    normal = ed.Normal(loc=0., scale=1., name='normal')

trace = pm.sample(model)

Acceptance rate: 0.9894

# See https://github.com/arviz-devs/arviz
# pip install git+git://github.com/arviz-devs/arviz.git
import arviz as az

posterior_data = az.convert_to_xarray(trace, chains=1)
az.posteriorplot(posterior_data, figsize=(8, 4), textsize=15, round_to=2);

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-26-8eafa5941007> in <module>
      3 import arviz as az
      4 
----> 5 posterior_data = az.convert_to_xarray(trace, chains=1)
      6 az.posteriorplot(posterior_data, figsize=(8, 4), textsize=15, round_to=2);

AttributeError: module 'arviz' has no attribute 'convert_to_xarray'

# See https://github.com/arviz-devs/arviz
# pip install git+git://github.com/arviz-devs/arviz.git
import arviz as az

posterior_data = az.convert_to_inference_data(trace, chains=1)
az.posteriorplot(posterior_data, figsize=(8, 4), textsize=15, round_to=2);

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-30-0602ed35f0ab> in <module>
      4 
      5 posterior_data = az.convert_to_inference_data(trace, chains=1)
----> 6 az.posteriorplot(posterior_data, figsize=(8, 4), textsize=15, round_to=2);

AttributeError: module 'arviz' has no attribute 'posteriorplot'

# See https://github.com/arviz-devs/arviz
# pip install git+git://github.com/arviz-devs/arviz.git
import arviz as az

posterior_data = az.convert_to_inference_data(trace, chains=1)
az.plot_posterior(posterior_data, figsize=(8, 4), textsize=15, round_to=2);

Non-Centered Eight Schools Model with PyMC4¶

このサイトのコードを試してみる。

import pymc4 as pm4
import numpy as np
import tensorflow as tf # For Random Variable operation
from tensorflow_probability import edward2 as ed # For defining random variables

# Common
J = 8 # No. of schools
y = np.array([28.,  8., -3.,  7., -1.,  1., 18., 12.])
sigma = np.array([15., 10., 16., 11.,  9., 11., 10., 18.])

# PyMC4 specific
pymc4_non_centered_eight = pm4.Model(num_schools=J, y=y, sigma=sigma)

# PyMC4 specific
@pymc4_non_centered_eight.define
def process(cfg):
    mu = ed.Normal(loc=0., scale=5., name="mu")
    log_tau = ed.Normal(
        loc=5., scale=1., name="log_tau")
    theta_prime = ed.Normal(
        loc=tf.zeros(cfg.num_schools),
        scale=tf.ones(cfg.num_schools),
        name="theta_prime")
    theta = mu + tf.exp(
        log_tau) * theta_prime
    y = ed.Normal(
        loc=theta,
        scale=np.float32(cfg.sigma),
        name="y")
    return y

# In PyMC4
pymc4_non_centered_eight.observe(y=y)

<pymc4.model.base.Model at 0x7f60691eb5f8>

# PyMC4 specific
pymc4_trace = pm4.sample(pymc4_non_centered_eight)

Acceptance rate: 0.6256

pymc4_theta = pymc4_trace['mu'][:, np.newaxis] + np.exp(pymc4_trace['log_tau'])[:, np.newaxis] * pymc4_trace['theta_prime']

import pandas as pd

pd.DataFrame({"PyMC4_mu": pymc4_theta.mean(axis=0), "PyMC4_std": pymc4_theta.std(axis=0)})

	PyMC4_mu	PyMC4_std
0	13.659657	11.061406
1	6.392598	7.839753
2	1.661329	10.338994
3	5.512524	8.293460
4	1.136670	7.021422
5	2.540748	8.236799
6	11.781768	7.933476
7	6.705863	10.677010

theanoは開発が終了してはいるが、有志達によってメンテは続けられてはいる。サイトを見てみると、cuda10にも対応する予定らしいので、完全に死んではいないようだ。とは言っても、オワコンであることには何ら変わりはない。