±à¼ÍƼö: |
±¾ÎÄÀ´Ô´Finley²©¿Í£¬±¾ÎĽ«¼òµ¥½éÉÜÆäÔÀí²¢·ÖÎöTensorflow¹Ù·½ÌṩµÄʾÀý¡£ |
|
¾í»ýÉñ¾ÍøÂç(Convolutional Neural Network, CNN)ÊÇÒ»ÖÖǰÀ¡Éñ¾ÍøÂç, ÔÚ¼ÆËã»úÊÓ¾õµÈÁìÓò±»¹ã·ºÓ¦ÓÃ. ±¾ÎĽ«¼òµ¥½éÉÜÆäÔÀí²¢·ÖÎöTensorflow¹Ù·½ÌṩµÄʾÀý.
¹ØÓÚÉñ¾ÍøÂçÓëÎó²î·´Ïò´«²¥µÄÔÀí¿ÉÒԲο¼×÷ÕßµÄÁíһƪ²©ÎÄBPÉñ¾ÍøÂçÓëPythonʵÏÖ.
Á˽â¾í»ýÉñ¾ÍøÂç
ʲôÊǾí»ý
¾í»ýÊÇͼÏñ´¦ÀíÖÐÒ»ÖÖ»ù±¾·½·¨. ¾í»ýºËÊÇÒ»¸öf*fµÄ¾ØÕó. ͨ³£nÈ¡ÆæÊý,ʹµÃ¾í»ýºËÓÐÖÐÐĵã.
¶ÔͼÏñÖÐÿ¸öµãÈ¡ÒÔÆäΪÖÐÐĵÄf½×·½Õó, ½«¸Ã·½ÕóÖи÷ÖµÓë¾í»ýºËÖжÔӦλÖõÄÖµÏà³Ë, ²¢ÓÃËüÃǵĺÍ×÷Ϊ½á¹û¾ØÕóÖжÔÓ¦µãµÄÖµ.

1*1 + 1*0 + 1*1 + 0*0 + 1*1 + 1*0 + 0*1 + 0*0 + 1*1 = 4
¾í»ýºËÿ´ÎÏòÓÒÒÆ¶¯1ÁÐ, ÓöÐÐÄ©ÏòÏÂÒÆ¶¯1ÁÐÖ±µ½Íê³ÉËùÓмÆËã. ÎÒÃǰÑÿ´ÎÒÆ¶¯µÄ¾àÀë³ÆÎª²½·ùs.

ÉÏÊö²Ù×÷´¦ÀíͼÏñµÃµ½ÐÂͼÏñµÄ²Ù×÷³ÆÎª¾í»ý, ÔÚͼÏñ´¦ÀíÖоí»ýºËÒ²±»³ÆÎª¹ýÂËÆ÷(filter).
¾í»ýµÃµ½µÄ½á¹û¾ØÕóͨ³£ÓÃÓÚ±íʾÔͼµÄijÖÖÌØÕ÷(Èç±ßÔµ), Òò´Ë¾í»ý½á¹û±»³ÆÎªÌØÕ÷ͼ(Feature Map).
ÿ¸ö¾í»ýºË¿ÉÒÔ°üº¬Ò»¸öÆ«ÖòÎÊýb, ¼´¶Ô¾í»ý½á¹ûµÄÿһ¸öÔªËØ¶¼¼Ób×÷ΪÊä³öµÄÌØÕ÷ͼ.
±ßÔµ¼ì²âÊǾí»ýµÄÒ»ÖÖµäÐÍÓ¦ÓÃ, ÈËÑÛËù¼ûµÄ±ßÔµÊÇͼÏñÖв»Í¬ÇøÓòµÄ·Ö½çÏß. ·Ö½çÏßÁ½²àµÄÉ«²Ê»ò»Ò¶Èͨ³£ÓÐ׎ϴóµÄ²»Í¬.

ÏÂÃæÎÒÃÇʹÓÃÒ»¸ö·Ç³£¼òµ¥µÄʾÀýÀ´Õ¹Ê¾±ßÔµ¼ì²â¹ý³Ì. µÚÒ»¸ö6*6µÄ¾ØÕóÊǻҶÈͼ, ÏÔȻͼÏñ×ó²à½ÏÁÁÓÒ²à½Ï°µ, ÖмäÐγÉÁËÒ»ÌõÃ÷ÏԵĴ¹Ö±±ßÔµ.

ÔÚÌØÕ÷ͼÖÐÑëÓÐÒ»Ìõ´¹Ö±ÁÁÏß(µÚ2,3ÁÐ), ¼´ÔͼµÄ´¹Ö±±ßÔµ. ÀàËÆµÄ¿ÉÒÔ¼ì²â×ÝÏò±ßÔµ:

¾í»ýºËµÄÖÐÐÄÎÞ·¨¶Ô×¼ÔͼÏñÖбßÔµµÄÏñËØµã(Óë±ßÔµ¾àÀëСÓÚ¾í»ýºË°ë¾¶), ÈôÒª¶Ô±ßÔµµÄµã½øÐмÆËã±ØÐëÌî³ä(padding)ÍⲿȱÉٵĵãʹ¾í»ýºËµÄÖÐÐÄ¿ÉÒÔ¶Ô×¼ËüÃÇ. ³£ÓõÄÌî³ä²ßÂÔÓÐ:
- SAME: ʹÓø½½üµãµÄÖµ´úÌæÈ±Ê§µÄµã, ¿ÉÒÔ±£Ö¤ÌØÕ÷ͼ²»»á±äС
- VALID: Ö»¶Ô¿ÉÓõÄλÖýøÐоí»ý(²»½øÐÐÌî³ä), µ«ÌØÕ÷ͼ»á±äС
´ËÍ⻹ÓÐ0ÖµÌî³ä, ¾ùÖµÌî³äµÈ·½·¨. ͨ³£ÓÃpÀ´ÃèÊöÌî³äµÄ¿í¶È.
SAMEÌî³äЧ¹û, 4*4¾ØÕó±»Ìî³äΪ6*6¾ØÕó, Ìî³ä¿í¶Èp=1:

¶ÔÓÚn*nµÄ¾ØÕó, ʹÓÃf*fµÄºË½øÐоí»ý, Ìî³ä¿í¶ÈΪp, Èô×ÝÏò²½·ùΪs1, ºáÏò²½·ùΪs2ÔòÌØÕ÷ͼµÄÐÐÁÐÊýΪ:

Èýά¾í»ý
»Ò¶ÈͼËùÄÜÃèÊöµÄÐÅÏ¢µÄ¼«ÎªÓÐÏÞ, ÎÒÃǸü¶àµØ´¦ÀíRGBͼÏñ. RGBͼÏñÐèÒª3¸ö¾ØÕó²ÅÄÜÃèÊöͼƬ, ÎÒÃdzÆÎª3¸öͨµÀ(channel).
ÒÔÏÂͼ6*6µÄRGBͼΪÀý, 3¸ö¾ØÕó·Ö±ðÓë»ÆÉ«¾í»ýºË½øÐоí»ýµÃµ½3¸ö4*4ÌØÕ÷ͼ, ½«3¸öÌØÕ÷ͼͬλÖõÄÖµµþ¼ÓµÃµ½×îÖյľí»ý½á¹û.

ÔÚ±ßÔµ¼ì²âÖÐÎÒÃÇ×¢Òâµ½, Ò»¸ö¾í»ýºËͨ³£Ö»ÄÜÌáȡͼÏñÒ»ÖÖÌØÕ÷Èçˮƽ±ßÔµ»ò´¹Ö±±ßÔµ. ΪÁËÌáȡͼÏñµÄ¶à¸öÌØÕ÷, ÎÒÃÇͨ³£Ê¹Óöà¸ö¾í»ýºË.
ÎÒÃÇʹÓøßά¾ØÕóÀ´ÃèÊöÕâÒ»¹ý³Ì, RGBͼÏñΪ6*6*3¾ØÕó, Á½¸ö¾í»ýºËµþ¼ÓΪ3*3*2¾ØÕó, Á½¸öÌØÕ÷ͼµþ¼ÓΪ4*4*2¾ØÕó. ÊäÈë, Êä³öºÍ¾í»ýºË¾ùʹÓÃÈýά¾ØÕóÀ´±íʾ, ÕâÑùÎÒÃÇ¿ÉÒÔ·½±ãµÄ¼¶Áª¶à¸ö¾í»ý²ã.
ΪʲôʹÓþí»ý
ÔÚÉÏÒ»½ÚÖÐÎÒÃÇÒѾ½éÉÜÁËÒ»¸ö¾í»ý²ãÈçºÎ¹¤×÷µÄ, ÏÖÔÚÎÒÃÇÀ´Ì½ÌÖΪʲôʹÓþí»ýÌáȡͼÏñÌØÕ÷.
Ê×ÏÈ·ÖÎö¾í»ý²ãµÄÊäÈëÊä³ö, ÿ¸ö¾í»ý²ãÊäÈëÁËÒ»¸öw1 * h1 * c1 µÄÈýά¾ØÕó, Êä³öw2 * h2 *c2µÄÈýά¾ØÕó.
ÈôʹÓÃÈ«Á¬½Ó²ãÐèÒª(w1 * h1 * c1) * (w2 * h2 *c2)¸ö²ÎÊý, ¾í»ý²ãÖ»ÐèҪѵÁ·c2¸ö¶þά¾í»ýºËÖеÄf1 * f1 * c2¸ö²ÎÊýºÍc2¸öÆ«ÖÃÖµ, ¿É¼û¾í»ý²ã¼«´óµØ¼õÉÙÁ˲ÎÊýµÄÊýÁ¿.
¸üÉٵIJÎÊý¶ÔÓÚѵÁ·Êý¾ÝºÍ¼ÆËã×ÊÔ´¶¼ÓÐÏÞµÄÈÎÎñ¶øÑÔ, ͨ³£Òâζןü¸ßµÄ¾«¶ÈºÍ¸üºÃµÄѵÁ·Ð§ÂÊ.
¸üÖØÒªµÄÊÇ, ¾í»ýÕë¶ÔС¿éÇøÓò¶ø²»Êǵ¥¸öÏñËØ½øÐд¦Àí, ¸üºÃµØ´Ó¿Õ¼ä·Ö²¼ÖÐÌáÈ¡ÌØÕ÷, ÕâÓëÈËÀàÊÓ¾õÊÇÀàËÆµÄ. ¶øÈ«Á¬½Ó²ãÑÏÖØºöÂÔÁ˿ռä·Ö²¼Ëù°üº¬µÄÐÅÏ¢.
ÌØÕ÷ͼÖÐÒ»¸öÏñËØÖ»ÓëÊäÈë¾ØÕóÖÐf * f¸öÏñËØÓйØ, ÕâÖÖÐÔÖʱ»³ÆÎª¾Ö²¿¸ÐÖª. Ò»¸ö¾í»ýºËÓÃÓÚÉú³ÉÌØÕ÷ͼÖÐËùÓÐÏñËØ, ¸ÃÌØÐÔ±»³ÆÎªÈ¨Öµ¹²Ïí.
³Ø»¯
ͨ¹ý¾í»ýѧϰµ½µÄͼÏñÌØÕ÷ÈÔÈ»ÊýÁ¿¾Þ´ó, ²»±ãÖ±½Ó½øÐзÖÀà. ³Ø»¯²ã±ãÓÃÓÚ¼õÉÙÌØÕ÷ÊýÁ¿.
³Ø»¯²Ù×÷·Ç³£¼òµ¥, ±ÈÈçÎÒÃÇʹÓÃÒ»¸ö¾í»ýºË¶ÔÒ»ÕÅͼƬ½øÐйýÂ˵õ½Ò»¸ö8x8µÄ·½Õó, ÎÒÃÇ¿ÉÒÔ½«·½Õó»®·ÖΪ16¸ö2x2·½Õó, ÿ¸öС·½Õó³ÆÎªÁÚÓò.
ÓÃ16¸öС·½ÕóµÄ¾ùÖµ×é³ÉÒ»¸ö4x4·½Õó±ãÊǾùÖµ³Ø»¯, ÀàËÆµØ»¹ÓÐ×î´óÖµ³Ø»¯µÈ²Ù×÷. ¾ùÖµ³Ø»¯¶Ô±£Áô±³¾°µÈÌØÕ÷½ÏºÃ, ×î´óÖµ³Ø»¯¶ÔÎÆÀíÌáÈ¡¸üºÃ.
Ëæ»ú³Ø»¯ÔòÊǸù¾ÝÏñËØµãÊýÖµ´óС¸³Óè¸ÅÂÊ(Ȩֵ), È»ºó°´Æä¼ÓȨÇóºÍ.

³Ø»¯²Ù×÷ÓÃÓÚ¼õÉÙͼµÄ¿í¶ÈºÍ¸ß¶È, µ«²»ÄܼõÉÙͨµÀÊý.
ÓÃ1*1*c2µÄºË½øÐоí»ý¿ÉÒÔʹw1 * h1 * c1µÄÊäÈë¾ØÕóÓ³Éäµ½w1 * h1 * c2µÄÊä³ö¾ØÕó. ¼´¶Ô¸÷ͨµÀÊä³ö¼ÓȨÇóºÍʵÏÖ¼õÉÙͨµÀÊýµÄЧ¹û.
TensorFlowʵÏÖ
TensorFlowµÄÎĵµDeep MNIST for Experts½éÉÜÁËʹÓÃCNNÔÚMNISTÊý¾Ý¼¯ÉÏʶ±ðÊÖдÊý×ֵķ½·¨., ¸ÃʾÀý²ÉÓÃÁËLeNet5Ä£ÐÍ.
ÍêÕû´úÂë¿ÉÒÔÔÚGitHubÉÏÕÒµ½, ±¾ÎĽ«¶ÔÆä½øÐмòµ¥·ÖÎö. Ô´ÂëÀ´×Ôtensorflow-1.3.0°æ±¾Ê¾Àý.
Ö÷ÒªÓÐ3ÌõÒýÈë:
import tempfile
from tensorflow.examples.tutorials.mnist import input_data
import tensorflow as tf |
main(_)º¯Êý¸ºÔðÍøÂçµÄ¹¹½¨:
def main(_):
# µ¼ÈëMNISTÊý¾Ý¼¯
# FLAGS.data_dirÊDZ¾µØÊý¾ÝµÄ·¾¶, ¿ÉÒÔÓÿÕ×Ö·û´®´úÌæÒÔ×Ô¶¯ÏÂÔØÊý¾Ý¼¯
mnist = input_data.read_data_sets(FLAGS.data_dir, one_hot=True)
# xÊÇÊäÈë²ã, ÿ¸ö28x28µÄͼÏñ±»Õ¹¿ªÎª784½×ÏòÁ¿
x = tf.placeholder(tf.float32, [None, 784])
# y_ÊÇѵÁ·¼¯Ô¤±ê×¢ºÃµÄ½á¹û, ²ÉÓÃone-hotµÄ·½·¨±íʾ10ÖÖ·ÖÀà
y_ = tf.placeholder(tf.float32, [None, 10])
# deepnn·½·¨¹¹½¨ÁËÒ»¸öcnn, y_convÊÇcnnµÄÔ¤²âÊä³ö
# keep_probÊÇdropout²ãµÄ²ÎÊý, ÏÂÎÄÔÙ½²
y_conv, keep_prob = deepnn(x)
# ¼ÆËãÔ¤²ây_convºÍ±êÇ©y_µÄ½»²æìØ×÷ΪËðʧº¯Êý
with tf.name_scope('loss'):
cross_entropy = tf.nn.softmax_cross_entropy_with_logits(labels=y_,
logits=y_conv)
cross_entropy = tf.reduce_mean(cross_entropy)
# ʹÓÃAdamÓÅ»¯Ëã·¨, ÒÔ×îС»¯Ëðʧº¯ÊýΪĿ±ê
with tf.name_scope('adam_optimizer'):
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)
# ¼ÆË㾫ȷ¶È(ÕýÈ··ÖÀàµÄÑù±¾ÊýÕ¼²âÊÔÑù±¾ÊýµÄ±ÈÀý), ÓÃÓÚÆÀ¹ÀÄ£ÐÍЧ¹û
with tf.name_scope('accuracy'):
correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y_, 1))
correct_prediction = tf.cast(correct_prediction, tf.float32)
accuracy = tf.reduce_mean(correct_prediction)
mainº¯ÊýÓëÆäËütensorflowÉñ¾ÍøÂç²¢ÎÞ¶þÖÂ, ¹Ø¼ü·ÖÎödeepnn·½·¨ÈçºÎ¹¹½¨cnn:
def deepnn(x):
# xµÄ½á¹¹Îª[n, 784], ½«ÆäÕ¹¿ªÎª[n, 28, 28]
# »Ò¶ÈͼֻÓÐÒ»¸öͨµÀ, x_imageµÚËÄάΪ1
# x_imageµÄËÄά·Ö±ðÊÇ[n_sample, width, height, channel]
with tf.name_scope('reshape'):
x_image = tf.reshape(x, [-1, 28, 28, 1])
# µÚÒ»¸ö¾í»ý²ã, ½«28x28*1»Ò¶ÈͼʹÓÃ5*5*32ºË½øÐоí»ý
with tf.name_scope('conv1'):
# ³õʼ»¯Á¬½ÓȨֵ, ΪÁ˱ÜÃâÌݶÈÏûʧȨֵʹÓÃÕýÔò·Ö²¼½øÐгõʼ»¯
W_conv1 = weight_variable([5, 5, 1, 32])
# ³õʼ»¯Æ«ÖÃÖµ, ÕâÀïʹÓõÄÊÇ0.1
b_conv1 = bias_variable([32])
# stridesÊǾí»ýºËÒÆ¶¯µÄ²½·ù. ²ÉÓÃSAME²ßÂÔÌî³ä, ¼´Ê¹ÓÃÏàֵͬÌî³ä
# def conv2d(x, W):
# tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')
# h_conv1µÄ½á¹¹Îª[n, 28, 28, 32]
h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)
# µÚÒ»¸ö³Ø»¯²ã, 2*2×î´óÖµ³Ø»¯, µÃµ½14*14¾ØÕó
with tf.name_scope('pool1'):
h_pool1 = max_pool_2x2(h_conv1)
# µÚ¶þ¸ö¾í»ý²ã, ½«28*28*32ÌØÕ÷ͼʹÓÃ5*5*64ºË½øÐоí»ý
with tf.name_scope('conv2'):
W_conv2 = weight_variable([5, 5, 32, 64])
b_conv2 = bias_variable([64])
# h_conv2µÄ½á¹¹Îª[n, 14, 14, 64]
h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
# µÚ¶þ¸ö³Ø»¯²ã, 2*2×î´óÖµ³Ø»¯, µÃµ½7*7¾ØÕó
with tf.name_scope('pool2'):
# h_pool2µÄ½á¹¹Îª[n, 7, 7, 64]
h_pool2 = max_pool_2x2(h_conv2)
# µÚÒ»¸öÈ«Á¬½Ó²ã, ½«7*7*64ÌØÕ÷¾ØÕóÓÃÈ«Á¬½Ó²ãÓ³Éäµ½1024¸öÌØÕ÷
with tf.name_scope('fc1'):
W_fc1 = weight_variable([7 * 7 * 64, 1024])
b_fc1 = bias_variable([1024])
h_pool2_flat = tf.reshape(h_pool2, [-1, 7*7*64])
h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, W_fc1) + b_fc1)
# ʹÓÃdropout²ã±ÜÃâ¹ýÄâºÏ
# ¼´ÔÚѵÁ·¹ý³ÌÖеÄÒ»´Îµü´úÖÐ, Ëæ»úÑ¡ÔñÒ»¶¨±ÈÀýµÄÉñ¾Ôª²»²ÎÓë´Ë´Îµü´ú
# ²ÎÓëµü´úµÄ¸ÅÂÊÖµÓÉkeep_probÖ¸¶¨, keep_prob=1.0ΪʹÓÃÕû¸öÍøÂç
with tf.name_scope('dropout'):
keep_prob = tf.placeholder(tf.float32)
h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)
# µÚ¶þ¸öÈ«Á¬½Ó²ã, ½«1024¸öÌØÕ÷Ó³Éäµ½10¸öÌØÕ÷, ¼´10¸ö·ÖÀàµÄone-hot±àÂë
# one-hot±àÂëÊÇÖ¸Óà `100`´úÌæ1, `010`´úÌæ2, `001`´úÌæ3... µÄ±àÂ뷽ʽ
with tf.name_scope('fc2'):
W_fc2 = weight_variable([1024, 10])
b_fc2 = bias_variable([10])
y_conv = tf.matmul(h_fc1_drop, W_fc2) + b_fc2
return y_conv, keep_prob |
mainº¯ÊýÓëÆäËütensorflowÉñ¾ÍøÂç²¢ÎÞ¶þÖÂ, ¹Ø¼ü·ÖÎödeepnn·½·¨ÈçºÎ¹¹½¨cnn:
graph_location = tempfile.mkdtemp()
print('Saving graph to: %s' % graph_location)
train_writer = tf.summary.FileWriter(graph_location)
train_writer.add_graph(tf.get_default_graph()) |
Õû¸öÍøÂ籩¶µÄ½Ó¿ÚÓÐ3¸ö:
- ÊäÈë²ãx[n, 784]
- Êä³ö²ãy_conv[n, 10]
- dropout±£Áô±ÈÀýkeep_prob[1]
ÏÖÔÚ¿ÉÒÔ¼ÌÐø¹Ø×¢main·½·¨ÁË, Íê³ÉÍøÂç¹¹½¨Ö®ºómainÏȽ«ÍøÂç½á¹¹»º´æµ½Ó²ÅÌ:
with tf.Session() as sess:
# ³õʼ»¯È«¾Ö±äÁ¿
sess.run(tf.global_variables_initializer())
for i in range(10000):
# ÿ´ÎȡѵÁ·Êý¾Ý¼¯ÖÐ50¸öÑù±¾, ·Ö10000´ÎÈ¡³ö
# batch[0]ÎªÌØÕ÷¼¯, ½á¹¹Îª[50, 784]¼´50×é784½×ÏòÁ¿
# batch[1]Ϊ±êÇ©¼¯, ½á¹¹Îª[50, 10]¼´50¸ö²ÉÓÃone-hot±àÂëµÄ±êÇ©
batch = mnist.train.next_batch(50)
# ÿ½øÐÐ100´Îµü´úÆÀ¹ÀÒ»´Î¾«¶È
if i % 100 == 0:
train_accuracy = accuracy.eval(feed_dict={
x: batch[0], y_: batch[1], keep_prob: 1.0})
print('step %d, training accuracy %g' % (i, train_accuracy))
# ½øÐÐѵÁ·, dropout keep probÉèΪ0.5
train_step.run(feed_dict={x: batch[0], y_: batch[1], keep_prob: 0.5})
# ÆÀ¹À×îÖÕ¾«¶È, dropout keep probÉèΪ1.0¼´Ê¹ÓÃÈ«²¿ÍøÂç
print('test accuracy %g' % accuracy.eval(feed_dict={
x: mnist.test.images, y_: mnist.test.labels, keep_prob: 1.0})) |
½ÓÏÂÀ´³õʼ»¯tf.Session()½øÐÐѵÁ·:
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('--data_dir', type=str,
default='/tmp/tensorflow/mnist/input_data',
help='Directory for storing input data')
FLAGS, unparsed = parser.parse_known_args()
tf.app.run(main=main, argv=[sys.argv[0]] + unparsed) |
Keep working, we will find a way out. This is Finley, welcome to join us.
|