ipythonのline magicと2to3モジュールの使用例

ipythonのline magicを使ってネットからコードやデータを簡単にダウンロードする方法(!wgetや!curlでも可)とダウンロードしたコードを簡単にjupyter notebook cellにloadする方法、さらにpython2用に書かれたcodeをpython3用に変換する方法を公開。

スポンサーリンク

codeをgithubからdownloadする

%download https://raw.githubusercontent.com/neubig/nn4nlp-code/master/05-cnn/cnn-activation.py
UsageError: Line magic function `%download` not found.

line magicをインストールして使えるように設定する。

!pip3 install metakernel
Collecting metakernel
  Using cached https://files.pythonhosted.org/packages/49/90/2ef457aa76ca3a7cd2b7f1f1baf97cbd12817e7430394ade52d143105e98/metakernel-0.20.14-py2.py3-none-any.whl
Requirement already satisfied: pexpect>=4.2 in /root/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages (from metakernel) (4.6.0)
Requirement already satisfied: ipykernel<6.0 in /root/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages (from metakernel) (4.8.2)
Requirement already satisfied: ptyprocess>=0.5 in /root/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages (from pexpect>=4.2->metakernel) (0.5.2)
Requirement already satisfied: jupyter-client in /root/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages (from ipykernel<6.0->metakernel) (5.2.3)
Requirement already satisfied: traitlets>=4.1.0 in /root/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages (from ipykernel<6.0->metakernel) (4.3.2)
Requirement already satisfied: tornado>=4.0 in /root/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages (from ipykernel<6.0->metakernel) (5.0.2)
Requirement already satisfied: ipython>=4.0.0 in /root/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages (from ipykernel<6.0->metakernel) (6.4.0)
Requirement already satisfied: python-dateutil>=2.1 in /root/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages (from jupyter-client->ipykernel<6.0->metakernel) (2.7.3)
Requirement already satisfied: pyzmq>=13 in /root/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages (from jupyter-client->ipykernel<6.0->metakernel) (17.0.0)
Requirement already satisfied: jupyter-core in /root/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages (from jupyter-client->ipykernel<6.0->metakernel) (4.4.0)
Requirement already satisfied: ipython-genutils in /root/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages (from traitlets>=4.1.0->ipykernel<6.0->metakernel) (0.2.0)
Requirement already satisfied: six in /root/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages (from traitlets>=4.1.0->ipykernel<6.0->metakernel) (1.11.0)
Requirement already satisfied: decorator in /root/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages (from traitlets>=4.1.0->ipykernel<6.0->metakernel) (4.3.0)
Requirement already satisfied: simplegeneric>0.8 in /root/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages (from ipython>=4.0.0->ipykernel<6.0->metakernel) (0.8.1)
Requirement already satisfied: pygments in /root/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages (from ipython>=4.0.0->ipykernel<6.0->metakernel) (2.2.0)
Requirement already satisfied: pickleshare in /root/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages (from ipython>=4.0.0->ipykernel<6.0->metakernel) (0.7.4)
Requirement already satisfied: setuptools>=18.5 in /root/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages (from ipython>=4.0.0->ipykernel<6.0->metakernel) (39.0.1)
Requirement already satisfied: prompt-toolkit<2.0.0,>=1.0.15 in /root/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages (from ipython>=4.0.0->ipykernel<6.0->metakernel) (1.0.15)
Requirement already satisfied: backcall in /root/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages (from ipython>=4.0.0->ipykernel<6.0->metakernel) (0.1.0)
Requirement already satisfied: jedi>=0.10 in /root/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages (from ipython>=4.0.0->ipykernel<6.0->metakernel) (0.12.0)
Requirement already satisfied: wcwidth in /root/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages (from prompt-toolkit<2.0.0,>=1.0.15->ipython>=4.0.0->ipykernel<6.0->metakernel) (0.1.7)
Requirement already satisfied: parso>=0.2.0 in /root/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages (from jedi>=0.10->ipython>=4.0.0->ipykernel<6.0->metakernel) (0.2.1)
Installing collected packages: metakernel
Successfully installed metakernel-0.20.14
from metakernel import register_ipython_magics
register_ipython_magics()
%download https://raw.githubusercontent.com/neubig/nn4nlp-code/master/05-cnn/cnn-activation.py
Downloaded 'cnn-activation.py'.

python2コードを3に変換

cnn-activation.pyをpython3コードに変換する。

!2to3 -w -n cnn-activation.py
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
RefactoringTool: Skipping optional fixer: buffer
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
RefactoringTool: Skipping optional fixer: idioms
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
RefactoringTool: Skipping optional fixer: set_literal
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
RefactoringTool: Skipping optional fixer: ws_comma
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
RefactoringTool: Refactored cnn-activation.py
--- cnn-activation.py	(original)
+++ cnn-activation.py	(refactored)
@@ -73,20 +73,20 @@
     pool_out = dy.rectify(pool_out)
 
     scores = (W_sm_express * pool_out + b_sm_express).npvalue()
-    print '%d ||| %s' % (tag, ' '.join(words))
+    print('%d ||| %s' % (tag, ' '.join(words)))
     predict = np.argmax(scores)
-    print display_activations(words, activations)
-    print 'scores=%s, predict: %d' % (scores, predict)
+    print(display_activations(words, activations))
+    print('scores=%s, predict: %d' % (scores, predict))
     features = pool_out.npvalue()
     W = W_sm_express.npvalue()
     bias = b_sm_express.npvalue()
-    print '  bias=%s' % bias
+    print('  bias=%s' % bias)
     contributions = W * features
-    print ' very bad (%.4f): %s' % (scores[0], contributions[0])
-    print '      bad (%.4f): %s' % (scores[1], contributions[1])
-    print '  neutral (%.4f): %s' % (scores[2], contributions[2])
-    print '     good (%.4f): %s' % (scores[3], contributions[3])
-    print 'very good (%.4f): %s' % (scores[4], contributions[4])
+    print(' very bad (%.4f): %s' % (scores[0], contributions[0]))
+    print('      bad (%.4f): %s' % (scores[1], contributions[1]))
+    print('  neutral (%.4f): %s' % (scores[2], contributions[2]))
+    print('     good (%.4f): %s' % (scores[3], contributions[3]))
+    print('very good (%.4f): %s' % (scores[4], contributions[4]))
 
 
 def display_activations(words, activations):
@@ -116,7 +116,7 @@
         train_loss += my_loss.value()
         my_loss.backward()
         trainer.update()
-    print("iter %r: train loss/sent=%.4f, acc=%.4f, time=%.2fs" % (ITER, train_loss/len(train), train_correct/len(train), time.time()-start))
+    print(("iter %r: train loss/sent=%.4f, acc=%.4f, time=%.2fs" % (ITER, train_loss/len(train), train_correct/len(train), time.time()-start)))
     # Perform testing
     test_correct = 0.0
     for _, wids, tag in dev:
@@ -124,9 +124,9 @@
         predict = np.argmax(scores)
         if predict == tag:
             test_correct += 1
-    print("iter %r: test acc=%.4f" % (ITER, test_correct/len(dev)))
+    print(("iter %r: test acc=%.4f" % (ITER, test_correct/len(dev))))
 
 
 for words, wids, tag in dev:
     calc_predict_and_activations(wids, tag, words)
-    raw_input()
+    input()
RefactoringTool: Files that were modified:
RefactoringTool: cnn-activation.py

cnn-activation.pyをロードして実行する。

# %load cnn-activation.py
from collections import defaultdict
import time
import random
import dynet as dy
import numpy as np

np.set_printoptions(linewidth=np.nan, threshold=np.nan)

# Functions to read in the corpus
w2i = defaultdict(lambda: len(w2i))
UNK = w2i["<unk>"]
def read_dataset(filename):
    with open(filename, "r") as f:
        for line in f:
            tag, words = line.lower().strip().split(" ||| ")
            words = words.split(" ")
            yield (words, [w2i[x] for x in words], int(tag))

# Read in the data
train = list(read_dataset("../data/classes/train.txt"))
w2i = defaultdict(lambda: UNK, w2i)
dev = list(read_dataset("../data/classes/test.txt"))
nwords = len(w2i)
ntags = 5

# Start DyNet and define trainer
model = dy.Model()
trainer = dy.AdamTrainer(model)

# Define the model
EMB_SIZE = 10
W_emb = model.add_lookup_parameters((nwords, 1, 1, EMB_SIZE)) # Word embeddings
WIN_SIZE = 3
FILTER_SIZE = 8
W_cnn = model.add_parameters((1, WIN_SIZE, EMB_SIZE, FILTER_SIZE)) # cnn weights
b_cnn = model.add_parameters((FILTER_SIZE)) # cnn bias

W_sm = model.add_parameters((ntags, FILTER_SIZE))          # Softmax weights
b_sm = model.add_parameters((ntags))                      # Softmax bias

def calc_scores(wids):
    dy.renew_cg()
    W_cnn_express = dy.parameter(W_cnn)
    b_cnn_express = dy.parameter(b_cnn)
    W_sm_express = dy.parameter(W_sm)
    b_sm_express = dy.parameter(b_sm)
    if len(wids) < WIN_SIZE:
        wids += [0] * (WIN_SIZE-len(wids))

    cnn_in = dy.concatenate([dy.lookup(W_emb, x) for x in wids], d=1)
    cnn_out = dy.conv2d_bias(cnn_in, W_cnn_express, b_cnn_express, stride=(1, 1), is_valid=False)
    pool_out = dy.max_dim(cnn_out, d=1)
    pool_out = dy.reshape(pool_out, (FILTER_SIZE,))
    pool_out = dy.rectify(pool_out)
    return W_sm_express * pool_out + b_sm_express

def calc_predict_and_activations(wids, tag, words):
    dy.renew_cg()
    W_cnn_express = dy.parameter(W_cnn)
    b_cnn_express = dy.parameter(b_cnn)
    W_sm_express = dy.parameter(W_sm)
    b_sm_express = dy.parameter(b_sm)
    if len(wids) < WIN_SIZE:
        wids += [0] * (WIN_SIZE-len(wids))

    cnn_in = dy.concatenate([dy.lookup(W_emb, x) for x in wids], d=1)
    cnn_out = dy.conv2d_bias(cnn_in, W_cnn_express, b_cnn_express, stride=(1, 1), is_valid=False)
    filters = (dy.reshape(cnn_out, (len(wids), FILTER_SIZE))).npvalue()
    activations = filters.argmax(axis=0)

    pool_out = dy.max_dim(cnn_out, d=1)
    pool_out = dy.reshape(pool_out, (FILTER_SIZE,))
    pool_out = dy.rectify(pool_out)

    scores = (W_sm_express * pool_out + b_sm_express).npvalue()
    print('%d ||| %s' % (tag, ' '.join(words)))
    predict = np.argmax(scores)
    print(display_activations(words, activations))
    print('scores=%s, predict: %d' % (scores, predict))
    features = pool_out.npvalue()
    W = W_sm_express.npvalue()
    bias = b_sm_express.npvalue()
    print('  bias=%s' % bias)
    contributions = W * features
    print(' very bad (%.4f): %s' % (scores[0], contributions[0]))
    print('      bad (%.4f): %s' % (scores[1], contributions[1]))
    print('  neutral (%.4f): %s' % (scores[2], contributions[2]))
    print('     good (%.4f): %s' % (scores[3], contributions[3]))
    print('very good (%.4f): %s' % (scores[4], contributions[4]))

def display_activations(words, activations):
    pad_begin = (WIN_SIZE - 1) / 2
    pad_end = WIN_SIZE - 1 - pad_begin
    words_padded = ['pad' for i in range(pad_begin)] + words + ['pad' for i in range(pad_end)]

    ngrams = []
    for act in activations:
        ngrams.append('[' + ', '.join(words_padded[act:act+WIN_SIZE]) + ']')

    return ngrams

for ITER in range(10):
    # Perform training
    random.shuffle(train)
    train_loss = 0.0
    train_correct = 0.0
    start = time.time()
    for _, wids, tag in train:
        scores = calc_scores(wids)
        predict = np.argmax(scores.npvalue())
        if predict == tag:
            train_correct += 1

        my_loss = dy.pickneglogsoftmax(scores, tag)
        train_loss += my_loss.value()
        my_loss.backward()
        trainer.update()
    print(("iter %r: train loss/sent=%.4f, acc=%.4f, time=%.2fs" % (ITER, train_loss/len(train), train_correct/len(train), time.time()-start)))
    # Perform testing
    test_correct = 0.0
    for _, wids, tag in dev:
        scores = calc_scores(wids).npvalue()
        predict = np.argmax(scores)
        if predict == tag:
            test_correct += 1
    print(("iter %r: test acc=%.4f" % (ITER, test_correct/len(dev))))

for words, wids, tag in dev:
    calc_predict_and_activations(wids, tag, words)
    input()
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-7-dc223ceda859> in <module>()
     19 
     20 # Read in the data
---> 21 train = list(read_dataset("../data/classes/train.txt"))
     22 w2i = defaultdict(lambda: UNK, w2i)
     23 dev = list(read_dataset("../data/classes/test.txt"))

<ipython-input-7-dc223ceda859> in read_dataset(filename)
     12 UNK = w2i["<unk>"]
     13 def read_dataset(filename):
---> 14     with open(filename, "r") as f:
     15         for line in f:
     16             tag, words = line.lower().strip().split(" ||| ")

FileNotFoundError: [Errno 2] No such file or directory: '../data/classes/train.txt'

dataをダウンロードする

train.txtとtest.txtをダウンロードしてパスを変更する。

%download https://raw.githubusercontent.com/neubig/nn4nlp-code/master/data/classes/train.txt
Downloaded 'train.txt'.
%download https://raw.githubusercontent.com/neubig/nn4nlp-code/master/data/classes/test.txt
Downloaded 'test.txt'.
# %load cnn-activation.py
from collections import defaultdict
import time
import random
import dynet as dy
import numpy as np

np.set_printoptions(linewidth=np.nan, threshold=np.nan)

# Functions to read in the corpus
w2i = defaultdict(lambda: len(w2i))
UNK = w2i["<unk>"]
def read_dataset(filename):
    with open(filename, "r", encoding="utf8") as f:
        for line in f:
            tag, words = line.lower().strip().split(" ||| ")
            words = words.split(" ")
            yield (words, [w2i[x] for x in words], int(tag))

# Read in the data
train = list(read_dataset("train.txt"))
w2i = defaultdict(lambda: UNK, w2i)
dev = list(read_dataset("test.txt"))
nwords = len(w2i)
ntags = 5

# Start DyNet and define trainer
model = dy.Model()
trainer = dy.AdamTrainer(model)

# Define the model
EMB_SIZE = 10
W_emb = model.add_lookup_parameters((nwords, 1, 1, EMB_SIZE)) # Word embeddings
WIN_SIZE = 3
FILTER_SIZE = 8
W_cnn = model.add_parameters((1, WIN_SIZE, EMB_SIZE, FILTER_SIZE)) # cnn weights
b_cnn = model.add_parameters((FILTER_SIZE)) # cnn bias

W_sm = model.add_parameters((ntags, FILTER_SIZE))          # Softmax weights
b_sm = model.add_parameters((ntags))                      # Softmax bias

def calc_scores(wids):
    dy.renew_cg()
    W_cnn_express = dy.parameter(W_cnn)
    b_cnn_express = dy.parameter(b_cnn)
    W_sm_express = dy.parameter(W_sm)
    b_sm_express = dy.parameter(b_sm)
    if len(wids) < WIN_SIZE:
        wids += [0] * (WIN_SIZE-len(wids))

    cnn_in = dy.concatenate([dy.lookup(W_emb, x) for x in wids], d=1)
    cnn_out = dy.conv2d_bias(cnn_in, W_cnn_express, b_cnn_express, stride=(1, 1), is_valid=False)
    pool_out = dy.max_dim(cnn_out, d=1)
    pool_out = dy.reshape(pool_out, (FILTER_SIZE,))
    pool_out = dy.rectify(pool_out)
    return W_sm_express * pool_out + b_sm_express

def calc_predict_and_activations(wids, tag, words):
    dy.renew_cg()
    W_cnn_express = dy.parameter(W_cnn)
    b_cnn_express = dy.parameter(b_cnn)
    W_sm_express = dy.parameter(W_sm)
    b_sm_express = dy.parameter(b_sm)
    if len(wids) < WIN_SIZE:
        wids += [0] * (WIN_SIZE-len(wids))

    cnn_in = dy.concatenate([dy.lookup(W_emb, x) for x in wids], d=1)
    cnn_out = dy.conv2d_bias(cnn_in, W_cnn_express, b_cnn_express, stride=(1, 1), is_valid=False)
    filters = (dy.reshape(cnn_out, (len(wids), FILTER_SIZE))).npvalue()
    activations = filters.argmax(axis=0)

    pool_out = dy.max_dim(cnn_out, d=1)
    pool_out = dy.reshape(pool_out, (FILTER_SIZE,))
    pool_out = dy.rectify(pool_out)

    scores = (W_sm_express * pool_out + b_sm_express).npvalue()
    print('%d ||| %s' % (tag, ' '.join(words)))
    predict = np.argmax(scores)
    print(display_activations(words, activations))
    print('scores=%s, predict: %d' % (scores, predict))
    features = pool_out.npvalue()
    W = W_sm_express.npvalue()
    bias = b_sm_express.npvalue()
    print('  bias=%s' % bias)
    contributions = W * features
    print(' very bad (%.4f): %s' % (scores[0], contributions[0]))
    print('      bad (%.4f): %s' % (scores[1], contributions[1]))
    print('  neutral (%.4f): %s' % (scores[2], contributions[2]))
    print('     good (%.4f): %s' % (scores[3], contributions[3]))
    print('very good (%.4f): %s' % (scores[4], contributions[4]))


def display_activations(words, activations):
    pad_begin = (WIN_SIZE - 1) // 2
    pad_end = WIN_SIZE - 1 - pad_begin
    words_padded = ['pad' for i in range(pad_begin)] + words + ['pad' for i in range(pad_end)]

    ngrams = []
    for act in activations:
        ngrams.append('[' + ', '.join(words_padded[act:act+WIN_SIZE]) + ']')

    return ngrams

for ITER in range(10):
    # Perform training
    random.shuffle(train)
    train_loss = 0.0
    train_correct = 0.0
    start = time.time()
    for _, wids, tag in train:
        scores = calc_scores(wids)
        predict = np.argmax(scores.npvalue())
        if predict == tag:
            train_correct += 1

        my_loss = dy.pickneglogsoftmax(scores, tag)
        train_loss += my_loss.value()
        my_loss.backward()
        trainer.update()
    print(("iter %r: train loss/sent=%.4f, acc=%.4f, time=%.2fs" % (ITER, train_loss/len(train), train_correct/len(train), time.time()-start)))
    # Perform testing
    test_correct = 0.0
    for _, wids, tag in dev:
        scores = calc_scores(wids).npvalue()
        predict = np.argmax(scores)
        if predict == tag:
            test_correct += 1
    print(("iter %r: test acc=%.4f" % (ITER, test_correct/len(dev))))

for words, wids, tag in dev:
    calc_predict_and_activations(wids, tag, words)
    input()
iter 0: train loss/sent=1.6997, acc=0.2441, time=1.41s
iter 0: test acc=0.2380
iter 1: train loss/sent=1.5533, acc=0.2907, time=1.40s
iter 1: test acc=0.2842
iter 2: train loss/sent=1.5149, acc=0.3201, time=1.38s
iter 2: test acc=0.2502
iter 3: train loss/sent=1.4739, acc=0.3599, time=1.38s
iter 3: test acc=0.2715
iter 4: train loss/sent=1.4241, acc=0.3859, time=1.37s
iter 4: test acc=0.2710
iter 5: train loss/sent=1.3680, acc=0.4309, time=1.38s
iter 5: test acc=0.2606
iter 6: train loss/sent=1.3133, acc=0.4652, time=1.37s
iter 6: test acc=0.2629
iter 7: train loss/sent=1.2584, acc=0.4989, time=1.40s
iter 7: test acc=0.2968
iter 8: train loss/sent=1.2082, acc=0.5357, time=1.40s
iter 8: test acc=0.2624
iter 9: train loss/sent=1.1512, acc=0.5681, time=1.41s
iter 9: test acc=0.2706
2 ||| effective but too-tepid biopic
['[too-tepid, biopic, pad]', '[pad, effective, but]', '[effective, but, too-tepid]', '[too-tepid, biopic, pad]', '[too-tepid, biopic, pad]', '[too-tepid, biopic, pad]', '[too-tepid, biopic, pad]', '[effective, but, too-tepid]']
scores=[-2.14508677  1.80063832  0.4667888   2.57697201 -1.77939856], predict: 3
  bias=[ 0.45970333  0.21161206  0.0378314  -0.09863383 -0.71208775]
 very bad (-2.1451): [-1.00636244  0.45980044 -0.2624169  -1.81049908 -0.63072133  0.5239285   0.1214805   0.        ]
      bad (1.8006): [-1.53434043 -0.57592413  3.0418034   0.17435898  1.58051758  1.06904326 -2.16643226 -0.        ]
  neutral (0.4668): [-0.07407974  0.57865431 -0.7846805  -1.62850398  1.28606316 -0.28583617  1.33734038 -0.        ]
     good (2.5770): [ 1.48128444 -0.71559598 -0.00967717  3.40248995 -1.08104225  0.9850371  -1.38689029 -0.        ]
very good (-1.7794): [ 2.52291113  0.32042333 -2.5950743   2.36342509  0.0231848  -1.07920695 -2.62297362  0.        ]
movie
3 ||| if you sometimes like to go to the movies to have fun , wasabi is a good place to start .
['[you, sometimes, like]', '[place, to, start]', '[to, go, to]', '[have, fun, ,]', '[a, good, place]', '[good, place, to]', '[you, sometimes, like]', '[good, place, to]']
scores=[0.91884202 1.29624271 1.30941761 2.06649065 0.33678687], predict: 3
  bias=[ 0.45970333  0.21161206  0.0378314  -0.09863383 -0.71208775]
 very bad (0.9188): [-0.93985815  2.18456269 -0.20684992 -2.27969342 -0.92990049  1.52602864  0.10291523  1.00193414]
      bad (1.2962): [-1.43294532 -2.73627917  2.39769914  0.21954445  2.33022733  3.11376578 -1.83534692 -0.97203432]
  neutral (1.3094): [-0.06918426  2.74925059 -0.61852379 -2.05053393  1.89610008 -0.83254524  1.13296114 -0.93593819]
     good (2.0665): [ 1.38339548 -3.3998756  -0.00762802  4.28425179 -1.59382864  2.86908391 -1.17493857 -0.19533598]
very good (0.3368): [ 2.35618747  1.52236666 -2.04556528  2.97591127  0.03418237 -3.14336922 -2.22211728  1.57127832]

4 ||| emerges as something rare , an issue movie that 's so honest and keenly observed that it does n't feel like one .
['[rare, ,, an]', '[keenly, observed, that]', '[feel, like, one]', '[and, keenly, observed]', "[does, n't, feel]", '[keenly, observed, that]', "[it, does, n't]", '[honest, and, keenly]']
scores=[0.49828887 2.52131009 2.42735863 2.10773492 0.26036012], predict: 1
  bias=[ 0.45970333  0.21161206  0.0378314  -0.09863383 -0.71208775]
 very bad (0.4983): [-1.1094739   2.48367164 -0.22463913 -2.88854236 -1.61661887  1.96316314  0.14459492  1.2864302 ]
      bad (2.5213): [-1.69154828 -3.11092879  2.60390273  0.27817926  4.05106733  4.00571132 -2.57864506 -1.24804042]
  neutral (2.4274): [-0.08166991  3.12567625 -0.67171721 -2.59818012  3.29634323 -1.07102979  1.59179968 -1.20169491]
     good (2.1077): [ 1.6330562  -3.86538444 -0.00828403  5.428468   -2.77084859  3.69093975 -1.65077758 -0.25080102]
very good (0.2604): [ 2.78140749  1.73080815 -2.22148514  3.77070254  0.05942557 -4.04379473 -3.12205376  2.01743788]
g
2 ||| the film provides some great insight into the neurotic mindset of all comics -- even those who have reached the absolute top of the game .
['[great, insight, into]', '[comics, --, even]', '[even, those, who]', '[the, absolute, top]', '[some, great, insight]', '[the, game, .]', '[provides, some, great]', '[top, of, the]']
scores=[ 0.58816946  5.10641527  0.03029785  2.33626652 -1.31793129], predict: 1
  bias=[ 0.45970333  0.21161206  0.0378314  -0.09863383 -0.71208775]
 very bad (0.5882): [-0.97797358  2.12086581 -0.48645023 -2.23984993 -0.98750427  1.48076851  0.06675882  1.15185111]
      bad (5.1064): [-1.49105763 -2.65649549  5.63868388  0.21570735  2.47457601  3.02141532 -1.1905487  -1.11747745]
  neutral (0.0303): [-0.07198999  2.66908869 -1.45458621 -2.0146956   2.01355624 -0.80785298  0.73492668 -1.07598035]
     good (2.3363): [ 1.43949832 -3.30074296 -0.01793886  4.20937351 -1.69256023  2.78399041 -0.7621565  -0.22456363]
very good (-1.3179): [ 2.45174136  1.47797791 -4.81056851  2.9238996   0.03629984 -3.05014076 -1.44143803  1.80638489]
a
4 ||| offers that rare combination of entertainment and education .
['[entertainment, and, education]', '[that, rare, combination]', '[rare, combination, of]', '[of, entertainment, and]', '[combination, of, entertainment]', '[offers, that, rare]', '[offers, that, rare]', '[offers, that, rare]']
scores=[-0.31476688  0.0317194   0.72916222  2.33270717  2.78792572], predict: 4
  bias=[ 0.45970333  0.21161206  0.0378314  -0.09863383 -0.71208775]
 very bad (-0.3148): [-1.07674037  1.35118289 -0.1013447  -2.07569544 -0.92323937  0.91451498  0.0769167   1.0599352 ]
      bad (0.0317): [-1.64164144 -1.69242733  1.17473625  0.19989855  2.31353531  1.86601049 -1.37170003 -1.0283045 ]
  neutral (0.7292): [-0.07926035  1.70045034 -0.30304149 -1.86704225  1.88251783 -0.49892582  0.84675154 -0.99011881]
     good (2.3327): [ 1.58487508e+00 -2.10287110e+00 -3.73729635e-03  3.90087624e+00 -1.58241163e+00  1.71937809e+00 -8.78124602e-01 -2.06643804e-01]
very good (2.7879): [ 2.69934584  0.94160529 -1.00221068  2.70961236  0.03393752 -1.88375117 -1.66076414  1.6622382 ]
---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
~/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages/ipykernel/kernelbase.py in _input_request(self, prompt, ident, parent, password)
    728             try:
--> 729                 ident, reply = self.session.recv(self.stdin_socket, 0)
    730             except Exception:

~/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages/jupyter_client/session.py in recv(self, socket, mode, content, copy)
    802         try:
--> 803             msg_list = socket.recv_multipart(mode, copy=copy)
    804         except zmq.ZMQError as e:

~/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages/zmq/sugar/socket.py in recv_multipart(self, flags, copy, track)
    465         """
--> 466         parts = [self.recv(flags, copy=copy, track=track)]
    467         # have first part already, only loop while more to receive

zmq/backend/cython/socket.pyx in zmq.backend.cython.socket.Socket.recv()

zmq/backend/cython/socket.pyx in zmq.backend.cython.socket.Socket.recv()

zmq/backend/cython/socket.pyx in zmq.backend.cython.socket._recv_copy()

~/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages/zmq/backend/cython/checkrc.pxd in zmq.backend.cython.checkrc._check_rc()

KeyboardInterrupt: 

During handling of the above exception, another exception occurred:

KeyboardInterrupt                         Traceback (most recent call last)
<ipython-input-15-d7caee1ae7a5> in <module>()
    130 for words, wids, tag in dev:
    131     calc_predict_and_activations(wids, tag, words)
--> 132     input()

~/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages/ipykernel/kernelbase.py in raw_input(self, prompt)
    702             self._parent_ident,
    703             self._parent_header,
--> 704             password=False,
    705         )
    706 

~/.pyenv/versions/3.6.5/envs/py365/lib/python3.6/site-packages/ipykernel/kernelbase.py in _input_request(self, prompt, ident, parent, password)
    732             except KeyboardInterrupt:
    733                 # re-raise KeyboardInterrupt, to truncate traceback
--> 734                 raise KeyboardInterrupt
    735             else:
    736                 break

KeyboardInterrupt: