# スタンフォード/CS131/宿題5-2 画素レベル特徴(色・位置特徴)

スポンサーリンク

## Pixel-Level Features¶

Before we can use a clustering algorithm to segment an image, we must compute some feature vectore for each pixel. The feature vector for each pixel should encode the qualities that we care about in a good segmentation. More concretely, for a pair of pixels $p_i$ and $p_j$ with corresponding feature vectors $f_i$ and $f_j$, the distance between $f_i$ and $f_j$ should be small if we believe that $p_i$ and $p_j$ should be placed in the same segment and large otherwise.
クラスタ化アルゴリズムを使って画像を分割する前に、各画素のfeature vectoreを算出しなければならない。各画素に対する特徴ベクトルは、分割の出来栄えを良くする質をエンコードしている必要がある。もっと具体的に言うと、対応する特徴ベクトル $f_i$, $f_j$ を持つ画素 $p_i$, $p_j$ のペアに関して、$f_i$, $f_j$間の距離は、$p_i$, $p_j$が同じセグメントに置かれているなら小さくなり、そうでないなら大きくなるはずである。

from time import time
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import rc
from skimage import io,color
import random
from segmentation import hierarchical_clustering,kmeans_fast
from scipy.spatial.distance import squareform, pdist
from skimage.util import img_as_float
from __future__ import print_function

%matplotlib inline
plt.rcParams['figure.figsize'] = (15.0, 10.0) # set default size of plots
plt.rcParams["font.size"] = "17"
plt.rcParams['image.interpolation'] = 'nearest'
plt.rcParams['image.cmap'] = 'gray'

# Load and display image
H, W, C = img.shape

plt.imshow(img)
plt.axis('off')
plt.show()


### Color Features¶

One of the simplest possible feature vectors for a pixel is simply the vector of colors for that pixel. Implement color_features in segmentation.py. Output should look like the following:

def color_features(img):
""" Represents a pixel by its color.
Args:
img - array of shape (H, W, C)
Returns:
features - array of (H * W, C)
"""
H, W, C = img.shape
img = img_as_float(img)
features = np.zeros((H*W, C))
features = img.reshape(H*W, C)
return features

np.random.seed(0)

features = color_features(img)

# Sanity checks
assert features.shape == (H * W, C),\

assert features.dtype == np.float,\
"dtype of color_features should be float."

assignments = kmeans_fast(features, 8)
segments = assignments.reshape((H, W))

# Display segmentation
plt.imshow(segments, cmap='viridis')
plt.axis('off')
plt.show()


In the cell below, we visualize each segment as the mean color of pixels in the segment.

from utils import visualize_mean_color_image
visualize_mean_color_image(img, segments)


### Color and Position Features¶

Another simple feature vector for a pixel is to concatenate its color and position within the image. In other words, for a pixel of color $(r, g, b)$ located at position $(x, y)$ in the image, its feature vector would be $(r, g, b, x, y)$. However, the color and position features may have drastically different ranges; for example each color channel of an image may be in the range $[0, 1)$, while the position of each pixel may have a much wider range. Uneven scaling between different features in the feature vector may cause clustering algorithms to behave poorly.

One way to correct for uneven scaling between different features is to apply some sort of normalization to the feature vector. One of the simplest types of normalization is to force each feature to have zero mean and unit variance.

Implement color_position_features in segmentation.py.
color_position_featuresを実装せよ。

Output segmentation should look like the following:

def color_position_features(img):
""" Represents a pixel by its color and position.
Combine pixel's RGB value and xy coordinates into a feature vector.
i.e. for a pixel of color (r, g, b) located at position (x, y) in the
image. its feature vector would be (r, g, b, x, y).
Don't forget to normalize features.
Hints
- You may find np.mgrid and np.dstack useful
- You may use np.mean and np.std
Args:
img - array of shape (H, W, C)
Returns:
features - array of (H * W, C+2)
"""
H, W, C = img.shape
color = img_as_float(img)
features = np.zeros((H*W, C+2))
position = np.dstack(np.mgrid[0:H,0:W]).reshape((H*W,2))
features[:,0:C] = color.reshape((H*W,C))
features[:,C:C+2] = position
features = (features-np.mean(features,axis=0))\
/np.std(features,axis=0)
return features

np.random.seed(0)

features = color_position_features(img)

# Sanity checks
assert features.shape == (H * W, C + 2),\

assert features.dtype == np.float,\
"dtype of color_features should be float."

assignments = kmeans_fast(features, 8)
segments = assignments.reshape((H, W))

# Display segmentation
plt.imshow(segments, cmap='viridis')
plt.axis('off')
plt.show()

visualize_mean_color_image(img, segments)


### Extra Credit: Implement Your Own Feature¶

For this programming assignment we have asked you to implement a very simple feature transform for each pixel. While it is not required, you should feel free to experiment with other feature transforms. Could your final segmentations be improved by adding gradients, edges, SIFT descriptors, or other information to your feature vectors? Could a different type of normalization give better results?
このプログラミング課題では、各画素に対する非常に単純な特徴変換を実装する。必須ではないが、他の特徴変換を使って自由に実験を試みるべきだろう。最終的なセグメンテーションは、勾配、エッジ、Scale Invariant Feature Transform descriptor(スケール不変特徴変換記述子)、あるいは、他の情報を特徴ベクトルに付け加えることで向上できるか？種類の異なる正規化はより良い結果をもたらすか？

Implement your feature extractor my_features in segmentation.py

def my_features(img):
Args:
img - array of shape (H, W, C)
Returns:
features - array of (H * W, C)
"""
features = None
H, W, C = img.shape
colors = img_as_float(img)
position = np.dstack(np.mgrid[0:H,0:W]).reshape((H*W,2))
gray = color.rgb2gray(img)
features = np.zeros((H*W,C+3))
features[:,0:C] = np.reshape(colors,(H*W,C))
features[:,C:C+2] = position
features = (features-np.mean(features,axis=0))\
/ (np.std(features,axis=0))
return features

# Feel free to experiment with different images
# and varying number of segments
num_segments = 8

H, W, C = img.shape

# Extract pixel-level features
features = my_features(img)

# Run clustering algorithm
assignments = kmeans_fast(features, num_segments)

segments = assignments.reshape((H, W))

# Display segmentation
plt.imshow(segments, cmap='viridis')
plt.axis('off')
plt.show()


スポンサーリンク

フォローする