ÔÚÁ˽âÈçºÎÀûÓÃTesnsorFlow¹¹½¨ºÍѵÁ·¸÷ÖÖÄ£ÐÍ¡ª¡ª´Ó»ù±¾µÄ»úÆ÷ѧϰģÐ͵½¸´ÔÓµÄÉî¶ÈÑ§Ï°ÍøÂçºó£¬ÎÒÃǾÍÒª¿¼ÂÇÈçºÎ½«ÑµÁ·ºÃµÄÄ£ÐÍͶÈëÓÚ²úÆ·£¬ÒÔʹÆäÄܹ»ÎªÆäËûÓ¦ÓÃËùÓ㬱¾ÎĶԴ˽«½øÐÐÏêϸ½éÉÜ¡£ÎÄÕ½ÚÑ¡×Ô¡¶ÃæÏò»úÆ÷ÖÇÄܵÄTensorFlowʵ¼ù¡·µÚ7Õ¡£
±¾ÎĽ«´´½¨Ò»¸ö¼òµ¥µÄWeb App£¬Ê¹Óû§Äܹ»ÉÏ´«Ò»·ùͼÏñ£¬²¢¶ÔÆäÔËÐÐInceptionÄ£ÐÍ£¬ÊµÏÖͼÏñµÄ×Ô¶¯·ÖÀà¡£
´î½¨TensorFlow·þÎñ¿ª·¢»·¾³
Docker¾µÏñ
TensorFlow·þÎñÊÇÓÃÓÚ¹¹½¨ÔÊÐíÓû§ÔÚ²úÆ·ÖÐʹÓÃÎÒÃÇÌṩµÄÄ£Ð͵ķþÎñÆ÷µÄ¹¤¾ß¡£ÔÚ¿ª·¢¹ý³ÌÖУ¬Ê¹Óøù¤¾ßµÄ·½·¨ÓÐÁ½ÖÖ£ºÊÖ¹¤°²×°ËùÓеÄÒÀÀµÏîºÍ¹¤¾ß£¬²¢´ÓÔ´Â뿪ʼ¹¹½¨£»»òÀûÓÃDocker¾µÏñ¡£ÕâÀï×¼±¸Ê¹ÓúóÕߣ¬ÒòΪËü¸üÈÝÒס¢¸ü¸É¾»£¬Í¬Ê±ÔÊÐíÔÚÆäËû²»Í¬ÓÚLinuxµÄ»·¾³ÖнøÐпª·¢¡£
Èç¹û²»Á˽âDocker¾µÏñ£¬²»·Á½«ÆäÏëÏóΪһ¸öÇáÁ¿¼¶µÄÐéÄâ»ú¾µÏñ£¬µ«ËüÔÚÔËÐÐʱ²»ÐèÒªÒÔÔÚÆäÖÐÔËÐÐÍêÕûµÄ²Ù×÷ϵͳΪ´ú¼Û¡£Èç¹ûÉÐδ°²×°Docker£¬ÇëÔÚ¿ª·¢»úÖа²×°Ëü£¬µã»÷²é¿´¾ßÌå°²×°²½Öè¡£
ΪÁËʹÓÃDocker¾µÏñ£¬»¹¿ÉÀûÓñÊÕßÌṩµÄÎļþ£¬ËüÊÇÒ»¸öÓÃÓÚÔÚ±¾µØ´´½¨¾µÏñµÄÅäÖÃÎļþ¡£ÒªÊ¹ÓøÃÎļþ£¬¿ÉʹÓÃÏÂÁÐÃüÁ
docker build --pull -t $USER/tensorflow
-serving-devel https://raw.githubusercontent.com/
tensorflow/serving/master/ tensorflow_serving/tools/docker/
Dockerfile.devel |
Çë×¢Ò⣬ִÐÐÉÏÊöÃüÁîºó£¬ÏÂÔØËùÓеÄÒÀÀµÏî¿ÉÄÜÐèÒªÒ»¶Î½Ï³¤µÄʱ¼ä¡£
ÉÏÊöÃüÁîÖ´ÐÐÍê±Ïºó£¬ÎªÁËʹÓøþµÏñÔËÐÐÈÝÆ÷£¬¿ÉÊäÈëÏÂÁÐÃüÁ
docker run -v $HOME:/mnt/home -p 9999:
9999 -it $USER/ tensorflow-serving-devel |
¸ÃÃüÁîÖ´Ðкó»á½«ÄãµÄhomeĿ¼¼ÓÔØµ½ÈÝÆ÷µÄ/mnt/home·¾¶ÖУ¬²¢ÔÊÐíÔÚÆäÖеÄÒ»¸öÖÕ¶ËϹ¤×÷¡£ÕâÊǷdz£ÓÐÓõģ¬ÒòΪÄã¿ÉʹÓÃ×Ô¼ºÆ«ºÃµÄIDE»ò±à¼Æ÷Ö±½Ó±à¼´úÂ룬ͬʱÔÚÔËÐй¹½¨¹¤¾ßʱ½öʹÓøÃÈÝÆ÷¡£Ëü»¹»á¿ª·Å¶Ë¿Ú9999£¬Ê¹Äã¿É´Ó×Ô¼ºµÄÖ÷»úÖзÃÎÊËü£¬²¢¹©ÒÔºó½«Òª¹¹½¨µÄ·þÎñÆ÷ʹÓá£
¼üÈëexitÃüÁî¿ÉÍ˳ö¸ÃÈÝÆ÷ÖÕ¶Ë£¬Ê¹ÆäÍ£Ö¹ÔËÐУ¬Ò²¿ÉÀûÓÃÉÏÊöÃüÁîÔÚÐèÒªµÄʱºòÆô¶¯Ëü¡£
Bazel¹¤×÷Çø
ÓÉÓÚTensorFlow·þÎñ³ÌÐòÊÇÓÃC++±àдµÄ£¬Òò´ËÔÚ¹¹½¨Ê±Ó¦Ê¹ÓÃGoogleµÄBazel¹¹½¨¹¤¾ß¡£ÎÒÃǽ«´Ó×î½ü´´½¨µÄÈÝÆ÷ÄÚ²¿ÔËÐÐBazel¡£
BazelÔÚ´úÂë¼¶¹ÜÀí×ŵÚÈý·½ÒÀÀµÏ¶øÇÒÖ»ÒªËüÃÇÒ²ÐèÒªÓÃBazel¹¹½¨£¬Bazel±ã»á×Ô¶¯ÏÂÔØºÍ¹¹½¨ËüÃÇ¡£ÎªÁ˶¨ÒåÎÒÃǵÄÏîÄ¿½«Ö§³ÖÄÄЩµÚÈý·½ÒÀÀµÏ±ØÐëÔÚÏîÄ¿¿âµÄ¸ùĿ¼Ï¶¨ÒåÒ»¸öWORKSPACEÎļþ¡£
ÎÒÃÇÐèÒªµÄÒÀÀµÏîÊÇTensorFlow·þÎñ¿â¡£ÔÚÎÒÃǵÄÀý×ÓÖУ¬TensorFlowÄ£ÐÍ¿â°üº¬ÁËInceptionÄ£Ð͵ĴúÂë¡£
²»ÐÒµÄÊÇ£¬ÔÚ׫д±¾Êéʱ£¬TensorFlow·þÎñÉв»Ö§³Ö×÷ΪGit¿âͨ¹ýBazelÖ±½ÓÒýÓã¬Òò´Ë±ØÐëÔÚÏîÄ¿Öн«Ëü×÷Ϊһ¸öGitµÄ×ÓÄ£¿é°üº¬½øÈ¥£º
# ÔÚ±¾µØ»úÆ÷ÉÏ mkdir ~/serving_example cd ~/serving_example git init git submodule add https://github
.com/tensorflow/serving.git tf_serving git.submodule update - -init - -recursive |
ÏÂÃæÀûÓÃWORKSPACEÎļþÖеÄlocal_repository¹æÔò½«µÚÈý·½ÒÀÀµÏÒåΪÔÚ±¾µØ´æ´¢µÄÎļþ¡£´ËÍ⣬»¹ÐèÀûÓôÓÏîÄ¿Öе¼ÈëµÄtf_workspace¹æÔò¶ÔTensorFlowµÄÒÀÀµÏî³õʼ»¯£º
# Bazel WORKSPACEÎļþ workspace(name = "serving") local_repository( name = "tf_serving", path = _workspace_dir__ + "/tf_serving"£¬ local_repository( name = "org_tensorflow", path = _workspace_dir__ + "/tf_serving/
tensorflow",) load('//tf_serving/tensorflow/
tensorflow:workspace.bzl', 'tf_workspace') tf_workspace("tf_serving/tensorflow/",
"@org_tensorflow") bind( name = "libssl", actual = "@boringssl_git//:ssl", ) bind( name = "zlib", actual = "@zlib_archive//:zlib" £© # ½öµ±µ¼Èëinception Ä£ÐÍʱÐèÒª local_repository( name = "inception_model", path = __workspace_dir__ + "/tf_serving
/tf_models/inception¡±£¬£© ×îºó£¬ÐèÒª´ÓÈÝÆ÷ÄÚΪTensorflowÔËÐÐ
/configure:# ÔÚDockerÈÝÆ÷ÖÐ cd /mnt/home/serving_example/tf_serving
/tensorflow ./configure |
µ¼³öѵÁ·ºÃµÄÄ£ÐÍ
Ò»µ©Ä£ÐÍѵÁ·Íê±Ï²¢×¼±¸½øÐÐÆÀ¹À£¬±ãÐèÒª½«Êý¾ÝÁ÷ͼ¼°Æä±äÁ¿Öµµ¼³ö£¬ÒÔʹÆä¿ÉΪ²úÆ·ËùÓá£
Ä£Ð͵ÄÊý¾ÝÁ÷ͼӦµ±ÓëÆäѵÁ·°æ±¾ÓÐËùÇø·Ö£¬ÒòΪËü±ØÐë´Óռλ·û½ÓÊÕÊäÈ룬²¢¶ÔÆä½øÐе¥²½ÍƶÏÒÔ¼ÆËãÊä³ö¡£¶ÔÓÚInceptionÄ£ÐÍÕâ¸öÀý×Ó£¬ÒÔ¼°¶ÔÓÚÈÎÒâÒ»°ãͼÏñʶ±ðÄ£ÐÍ£¬ÎÒÃÇÏ£ÍûÊäÈëÊÇÒ»¸ö±íʾÁËJPEG±àÂëµÄͼÏñ×Ö·û´®£¬ÕâÑù¾Í¿ÉÇáÒ׵ؽ«Ëü´«Ë͵½Ïû·ÑAppÖС£ÕâÓë´ÓTFRecordÎļþ¶ÁȡѵÁ·ÊäÈëÆÄΪ²»Í¬¡£
¶¨ÒåÊäÈëµÄÒ»°ãÐÎʽÈçÏ£º
def convert_external_inputs (external_x): #½«ÍⲿÊäÈë±ä»»ÎªÍƶÏËùÐèµÄÊäÈë¸ñʽ def inference(x): #´ÓÔʼģÐÍÖС¡ external_x = tf.placeholder(tf.string) x = convert_external_inputs(external_x) y = inference(x) |
ÔÚÉÏÊö´úÂëÖУ¬ÎªÊäÈ붨ÒåÁËռλ·û£¬²¢µ÷ÓÃÁËÒ»¸öº¯Êý½«ÓÃռλ·û±íʾµÄÍⲿÊäÈëת»»ÎªÔÊ¼ÍÆ¶ÏÄ£ÐÍËùÐèµÄÊäÈë¸ñʽ¡£ÀýÈ磬ÎÒÃÇÐèÒª½«JPEG×Ö·û´®×ª»»ÎªInceptionÄ£ÐÍËùÐèµÄͼÏñ¸ñʽ¡£×îºó£¬µ÷ÓÃÔʼģÐÍÍÆ¶Ï·½·¨£¬ÒÀ¾Ýת»»ºóµÄÊäÈëµÃµ½ÍƶϽá¹û¡£
ÀýÈ磬¶ÔÓÚInceptionÄ£ÐÍ£¬Ó¦µ±ÓÐÏÂÁз½·¨£º
import tensorflow as tf from tensorflow_serving.session_bundle
import exporter from inception import inception_model def convert_external_inputs (external_x) # ½«ÍⲿÊäÈë±ä»»ÎªÍƶÏËùÐèµÄÊäÈë¸ñʽ # ½«Í¼Ïñ×Ö·û´®×ª»»ÎªÒ»¸ö¸÷·ÖÁ¿Î»ÓÚ[0,1]
ÄÚµÄÏñËØÕÅÁ¿ image = tf.image.convert_image_dtype(tf.image.
decode_jpeg(external_x, channels=3), tf.float32) # ¶ÔͼÏñ³ß´ç½øÐÐËõ·Å£¬Ê¹Æä·ûºÏÄ£ÐÍÆÚÍû
µÄ¿í¶ÈºÍ¸ß¶È images = tf.image.resize_bilinear(tf.
expand_dims(image, 0),[299,299]) # ½«ÏñËØÖµ±ä»»µ½Ä£ÐÍËùÒªÇóµÄÇø¼ä[-1,1]ÄÚ images =tf.mul(tf.sub(image,0.5),2) return images
def inference(images):
logits, _ = inception_model.inference
(images, 1001)return logits
|
Õâ¸öÍÆ¶Ï·½·¨ÒªÇó¸÷²ÎÊý¶¼±»¸³Öµ¡£ÎÒÃǽ«´ÓÒ»¸öѵÁ·¼ì²éµã»Ö¸´ÕâЩ²ÎÊýÖµ¡£Äã¿ÉÄÜ»¹¼ÇµÃ£¬ÔÚÇ°ÃæµÄÕ½ÚÖУ¬ÎÒÃÇÖÜÆÚÐԵر£´æÄ£Ð͵ÄѵÁ·¼ì²éµãÎļþ¡£ÄÇЩÎļþÖаüº¬Á˵±Ê±Ñ§Ï°µ½µÄ²ÎÊý£¬Òò´Ëµ±³öÏÖÒ쳣ʱ£¬ÑµÁ·½øÕ¹²»»áÊܵ½Ó°Ïì¡£
ѵÁ·½áÊøÊ±£¬×îºóÒ»´Î±£´æµÄѵÁ·¼ì²éµãÎļþÖн«°üº¬×îºó¸üеÄÄ£ÐͲÎÊý£¬ÕâÕýÊÇÎÒÃÇÏ£ÍûÔÚ²úÆ·ÖÐʹÓõİ汾¡£
Òª»Ö¸´¼ì²éµãÎļþ£¬¿ÉʹÓÃÏÂÁдúÂ룺
saver = tf.train.Saver() with tf.Session() as sess: # ´ÓѵÁ·¼ì²éµãÎļþ»Ö¸´¸÷½»Á¿ ckpt = tf.train.get_checkpoint_state
(sys.argv[1]) if ckpt and ckpt.model_checkpoint_path: saver.restore(sess, sys.argv[1])+¡±/¡±+ ckpt.model_checkpoint_path) else: print(¡°Checkpoint file not found¡±) raise SystemExit |
¶ÔÓÚInceptionÄ£ÐÍ£¬¿É´ÓÏÂÁÐÁ´½ÓÏÂÔØÒ»¸öԤѵÁ·µÄ¼ì²éµãÎļþ£º
# ÔÚdockerÈÝÆ÷ÖÐ cd/tmp curl -O http://download.tensorflow.
org/models/image/imagenet/ inception-v3-2016-03-01.tar.gz tar ¨Cxzf inception-v3-2016-03-01.tar.gz |
×îºó£¬ÀûÓÃtensorflow_serving.session_bundle.exporter.ExporterÀཫģÐ͵¼³ö¡£ÎÒÃÇͨ¹ý´«ÈëÒ»¸ö±£´æÆ÷ʵÀý´´½¨ÁËÒ»¸öËüµÄʵÀý¡£È»ºó£¬ÐèÒªÀûÓÃexporter.classification_signature·½·¨´´½¨¸ÃÄ£Ð͵ÄÇ©Ãû¡£¸ÃÇ©ÃûÖ¸¶¨ÁËʲôÊÇinput_tensorÒÔ¼°ÄÄЩÊÇÊä³öÕÅÁ¿¡£Êä³öÓÉclasses_tensor¹¹³É£¬Ëü°üº¬ÁËÊä³öÀàÃû³ÆÁбíÒÔ¼°Ä£ÐÍ·ÖÅ䏸¸÷Àà±ðµÄ·ÖÖµ£¨»ò¸ÅÂÊ£©µÄsocres_tensor¡£Í¨³££¬ÔÚÒ»¸ö°üº¬µÄÀà±ðÊýÏ൱¶àµÄÄ£ÐÍÖУ¬Ó¦µ±Í¨¹ýÅäÖÃÖ¸¶¨½ö·µ»Øtf.nn.top_kËùÑ¡ÔñµÄÄÇЩÀà±ð£¬¼´°´Ä£ÐÍ·ÖÅäµÄ·ÖÊý°´½µÐòÅÅÁкóµÄǰK¸öÀà±ð¡£
×îºóÒ»²½ÊÇÓ¦ÓÃÕâ¸öµ÷ÓÃÁËexporter.Exporter.init·½·¨µÄÇ©Ãû£¬²¢Í¨¹ýexport·½·¨µ¼³öÄ£ÐÍ£¬¸Ã·½·¨½ÓÊÕÒ»¸öÊä³ö·¾¶¡¢Ò»¸öÄ£Ð͵İ汾ºÅºÍ»á»°¶ÔÏó¡£
Scores, class_ids=tf.nn.top_k(y,
NUM_CLASS_TO_RETURN) #ΪÁ˼ò±ãÆð¼û£¬ÎÒÃǽ«½ö·µ»ØÀà±ðID,
Ó¦µ±ÁíÍâ¶ÔËüÃÇÃüÃû classes = tf.contrib.lookup.index_to_string
(tf.to_int64(class_ids) mapping=tf.constant([str(i) for i
in range(1001)]))
model_exporter = exporter.Exporter
(saver)
signature = exporter.classification_
signature(
input_tensor=external_x, classes_
tensor=classes,
scores_tensor=scores) model_exporter.
init(default_graph_
signature=signature,
init_op=tf.initialize_all_tables())
model_exporter.export(sys.argv[1]+
"/export"
tf.constant(time.time()), sess)
|
ÓÉÓÚ¶ÔExporterÀà´úÂëÖÐ×Ô¶¯Éú³ÉµÄ´úÂë´æÔÚÒÀÀµ£¬ËùÒÔÐèÒªÔÚDockerÈÝÆ÷ÄÚ²¿Ê¹ÓÃbazelÔËÐÐÎÒÃǵĵ¼³öÆ÷¡£
Ϊ´Ë£¬ÐèÒª½«´úÂë±£´æµ½Ö®Ç°Æô¶¯µÄbazel¹¤×÷ÇøÄÚµÄexporter.pyÖС£´ËÍ⣬»¹ÐèÒªÒ»¸ö´øÓй¹½¨¹æÔòµÄBUILDÎļþ£¬ÀàËÆÓÚÏÂÁÐÄÚÈÝ£º
# BUILDÎļþ py_binary£¨ name = "export"£¬ srcs =[ ¡°export.py¡±, ], deps = [ ¡°//tensorflow_serving/session_
bundle:exporter¡±, ¡°@org_tensorflow//tensorflow:
tensorflow_py¡±, #½öÔÚµ¼³ö inceptionÄ£ÐÍʱÐè ¡°@inception_model//inception¡±, ], ) |
È»ºó£¬¿ÉÔÚÈÝÆ÷ÖÐͨ¹ýÏÂÁÐÃüÁîÔËÐе¼³öÆ÷£º
# ÔÚDockerÈÝÆ÷ÖÐ cd /mnt/home/serving_example |
Ëü½«ÒÀ¾Ý¿É´Ó/tmp/inception-v3ÖÐÌáÈ¡µ½µÄ¼ì²éµãÎļþÔÚ/tmp/inception-v3/{current_timestamp}/
Öд´½¨µ¼³öÆ÷¡£
×¢Ò⣬Ê×´ÎÔËÐÐËüʱÐèÒª»¨·ÑһЩʱ¼ä£¬ÒòΪËü±ØÐëÒª¶ÔTensorFlow½øÐбàÒë¡£
¶¨Òå·þÎñÆ÷½Ó¿Ú
½ÓÏÂÀ´ÐèҪΪµ¼³öµÄÄ£ÐÍ´´½¨Ò»¸ö·þÎñÆ÷¡£
TensorFlow·þÎñʹÓÃgRPCÐÒ飨gRPCÊÇÒ»ÖÖ»ùÓÚHTTP/2µÄ¶þ½øÖÆÐÒ飩¡£ËüÖ§³ÖÓÃÓÚ´´½¨·þÎñÆ÷ºÍ×Ô¶¯Éú³É¿Í»§¶Ë´æ¸ùµÄ¸÷ÖÖÓïÑÔ¡£ÓÉÓÚTensorFlowÊÇ»ùÓÚC++µÄ£¬ËùÒÔÐèÒªÔÚÆäÖж¨Òå×Ô¼ºµÄ·þÎñÆ÷¡£ÐÒÔ˵ÄÊÇ£¬·þÎñÆ÷¶Ë´úÂë±È½Ï¼ò¶Ì¡£
ΪÁËʹÓÃgRPS£¬±ØÐëÔÚÒ»¸öprotocol bufferÖж¨Òå·þÎñÆõÔ¼£¬ËüÊÇÓÃÓÚgRPCµÄIDL£¨½Ó¿Ú¶¨ÒåÓïÑÔ£©ºÍ¶þ½øÖƱàÂë¡£ÏÂÃæÀ´¶¨ÒåÎÒÃǵķþÎñ¡£Ç°ÃæµÄµ¼³öÒ»½ÚÔøÌáµ½£¬ÎÒÃÇÏ£Íû·þÎñÓÐÒ»¸öÄܹ»½ÓÊÕÒ»¸öJPEG±àÂëµÄ´ý·ÖÀàµÄͼÏñ×Ö·û´®×÷ΪÊäÈ룬²¢¿É·µ»ØÒ»¸öÒÀ¾Ý·ÖÊýÅÅÁеÄÓÉÍÆ¶ÏµÃµ½µÄÀà±ðÁÐ±í¡£
ÕâÑùµÄ·þÎñÓ¦¶¨ÒåÔÚÒ»¸öclassification_service.protoÎļþÖУ¬ÀàËÆÓÚ£º
syntax = "proto3"£» message ClassificationRequest { // JPEG ±àÂëµÄͼÏñ×Ö·û´® bytes input = 1£» }£» message ClassificationResponse{ repeated ClassificationClass
classes = 1; }; message ClassificationClass { string name = 1; float score = 2; } |
¿É¶ÔÄܹ»½ÓÊÕÒ»·ùͼÏñ£¬»òÒ»¸öÒôƵƬ¶Î»òÒ»¶ÎÎÄ×ÖµÄÈÎÒâÀàÐ͵ķþÎñʹÓÃͬһ¸ö½Ó¿Ú¡£
ΪÁËʹÓÃÏñÊý¾Ý¿â¼Ç¼ÕâÑùµÄ½á¹¹»¯ÊäÈ룬ÐèÒªÐÞ¸ÄClassificationRequestÏûÏ¢¡£ÀýÈ磬Èç¹ûÊÔͼΪIrisÊý¾Ý¼¯¹¹½¨·ÖÀà·þÎñ£¬ÔòÐèÒªÈçϱàÂ룺
message ClassificationRequest { float petalWidth = 1; float petaHeight = 2; float petalWidth = 3; float petaHeight = 4; } |
Õâ¸öprotoÎļþ½«ÓÉproto±àÒëÆ÷ת»»Îª¿Í»§¶ËºÍ·þÎñÆ÷ÏàÓ¦µÄÀඨÒ塣ΪÁËʹÓÃprotobuf±àÒëÆ÷£¬±ØÐëΪBUILDÎļþÌí¼ÓÒ»ÌõÐµĹæÔò£¬ÀàËÆÓÚ£º
load("@protobuf//:protobuf.bzl",
"cc_proto_library")cc_proto_library( name="classification_service_proto", srcs=["classification_service.proto"], cc_libs = ["@protobuf//:protobuf"], protoc="@protobuf//:protoc", default_runtime="@protobuf//:protobuf", use_grpc_plugin=1 ) |
Çë×¢ÒâλÓÚÉÏÊö´úÂëÆ¬¶ÎÖÐ×îÉÏ·½µÄload¡£Ëü´ÓÍⲿµ¼ÈëµÄprotobuf¿âÖе¼ÈëÁËcc_proto_library¹æÔò¶¨Ò塣Ȼºó£¬ÀûÓÃËüΪprotoÎļþ¶¨ÒåÁËÒ»¸ö¹¹½¨¹æÔò¡£ÀûÓÃbazel
build :classification_service_proto¿ÉÔËÐиù¹½¨£¬²¢Í¨¹ýbazel-genfiles/classification_service.grpc.pb.h¼ì²é½á¹û£º
¡ class ClassificationService { ... class Service : public ::grpc:
:Service { public: Service(); virtual ~Service(); virtual ::grpc::Status classif
y(::grpc::ServerContext* context, const ::Classification
Request* request, ::ClassificationRespons
e* response); }; |
°´ÕÕÍÆ¶ÏÂß¼£¬ClassificationService::ServiceÊDZØÐëҪʵÏֵĽӿڡ£ÎÒÃÇÒ²¿Éͨ¹ý¼ì²ébazel-genfiles/classification_service.pb.h²é¿´requestºÍresponseÏûÏ¢µÄ¶¨Ò壺
¡ class ClassificationRequest : public ::google::protobuf::
Message { ... const ::std::string& input()
const;void set_input(const :
:std::string& value);...} class ClassificationResponse : public ::google::protobuf::Message
{...const ::ClassificationClass&
classes() const; void set_allocated_classes(::
ClassificationClass* classes); ... } class ClassificationClass : public ::google::protobuf::Message { ... const ::std::string& name() const; void set_name(const ::std::string&
value); float score() const; void set_score(float value); ... } |
¿ÉÒÔ¿´µ½£¬proto¶¨ÒåÏÖÔÚ±ä³ÉÁËÿÖÖÀàÐ͵ÄC++Àà½Ó¿Ú¡£ËüÃǵÄʵÏÖÒ²ÊÇ×Ô¶¯Éú³ÉµÄ£¬ÕâÑù±ã¿ÉÖ±½ÓʹÓÃËüÃÇ¡£
ʵÏÖÍÆ¶Ï·þÎñÆ÷
ΪʵÏÖClassificationService::Service£¬ÐèÒª¼ÓÔØµ¼³öÄ£ÐͲ¢¶ÔÆäµ÷ÓÃÍÆ¶Ï·½·¨¡£Õâ¿Éͨ¹ýÒ»¸öSessionBundle¶ÔÏóÀ´ÊµÏÖ£¬¸Ã¶ÔÏóÊÇ´Óµ¼³öµÄÄ£ÐÍ´´½¨µÄ£¬Ëü°üº¬ÁËÒ»¸ö´øÓÐÍêÈ«¼ÓÔØµÄÊý¾ÝÁ÷ͼµÄTF»á»°¶ÔÏó£¬ÒÔ¼°´øÓж¨ÒåÔÚµ¼³ö¹¤¾ßÉϵķÖÀàÇ©ÃûµÄÔªÊý¾Ý¡£
ΪÁË´Óµ¼³öµÄÎļþ·¾¶´´½¨SessionBundle¶ÔÏ󣬿ɶ¨ÒåÒ»¸ö±ã½Ýº¯Êý£¬ÒÔ´¦ÀíÕâ¸öÑù°åÎļþ£º
#include <iostream> #include <memory> #include <string>
#include <grpc++/grpc++.h>
#include "classification_service.
grpc.pb.h"
#include "tensorflow_serving/
servables/tensorflow/
session_bundle_factory.h"
using namespace std;
using namespace tensorflow::
serving;
using namespace grpc;
unique_ptr<SessionBundle> create
SessionBundle(const string&
pathToExportFiles) {
SessionBundleConfig session_bundl
e_config =
SessionBundleConfig();
unique_ptr<SessionBundleFactory>
bundle_factory;
SessionBundleFactory::Create(session
_bundle_config,
&bundle_factory);
unique_ptr<SessionBundle> session
Bundle;
bundle_factory-
>CreateSessionBundle(pathToExportFiles,
&sessionBundle);
return sessionBundle;
} |
ÔÚÕâ¶Î´úÂëÖУ¬ÎÒÃÇÀûÓÃÁËÒ»¸öSessionBundleFactoryÀà´´½¨ÁËSessionBundle¶ÔÏ󣬲¢½«ÆäÅäÖÃΪ´ÓpathToExportFilesÖ¸¶¨µÄ·¾¶ÖмÓÔØµ¼³öµÄÄ£ÐÍ¡£×îºó·µ»ØÒ»¸öÖ¸ÏòËù´´½¨µÄSessionBundleʵÀýµÄuniqueÖ¸Õë¡£
½ÓÏÂÀ´ÐèÒª¶¨Òå·þÎñµÄʵÏÖ¡ªClassificationServiceImpl£¬¸ÃÀཫ½ÓÊÕSessionBundleʵÀý×÷Ϊ²ÎÊý£¬ÒÔÔÚÍÆ¶ÏÖÐʹÓãº
class ClassificationServiceImpl
final : public ClassificationService::Service {
private:
unique_ptr<SessionBundle> session
Bundle;public:
ClassificationServiceImpl(unique_ptr
<SessionBundle>
sessionBundle) :
sificationServiceImpl(unique_ptr<Sessi
Status classify(ServerContext* context,
constClassificationRequest* request,
ClassificationResponse* response)
override {
// ¼ÓÔØ·ÖÀàÇ©Ãû
ClassificationSignature signature;
const tensorflow::Status signature
Status =GetClassificationSignature
(sessionBundle->meta_graph_def,
&signature);
if (!signatureStatus.ok()) {
return Status(StatusCode::INTERNAL,
signatureStatus.error_message());
}
// ½« protobuf ÊäÈë±ä»»ÎªÍƶÏÊäÈëÕÅÁ¿
tensorflow::Tensor
input(tensorflow::DT_STRING, tensorflo
w::TensorShape());input.scala
r<string>()() = request->input();
vector<tensorflow::Tensor> outputs;
//ÔËÐÐÍÆ¶Ï
const tensorflow::Status inference
Status =sessionBundle->session->Run(
{{signature.input().tensor_name(),
input}},
{signature.classes().tensor_name(),
signature.scores().tensor_name()},
{},
&outputs);
if (!inferenceStatus.ok()) {
return Status(StatusCode::INTERNAL,
inferenceStatus.error_message());
}
//½«ÍƶÏÊä³öÕÅÁ¿±ä»»ÎªprotobufÊä³ö
for (int i = 0; i <
outputs[0].vec<string>().size();
++i) {ClassificationClass
*classificationClass = response->
add_classes();classificationClass-
>set_name(outputs[0].flat<string>
()(i));classificationClass-
>set_score(outputs[1].flat<float
>()(i));}return Status::OK;
}
}; |
classify·½·¨µÄʵÏÖ°üº¬ÁË4¸ö²½Ö裺
1.ÀûÓÃGetClassificationSignatureº¯Êý¼ÓÔØ´æ´¢ÔÚÄ£Ð͵¼³öÔªÊý¾ÝÖеÄClassification-Signature¡£Õâ¸öÇ©ÃûÖ¸¶¨ÁËÊäÈëÕÅÁ¿µÄ£¨Âß¼£©Ãû³Æµ½Ëù½ÓÊÕµÄͼÏñµÄÕæÊµÃû³ÆÒÔ¼°Êý¾ÝÁ÷ͼÖÐÊä³öÕÅÁ¿µÄ£¨Âß¼£©Ãû³Æµ½¶ÔÆä»ñµÃÍÆ¶Ï½á¹ûµÄÓ³Éä¡£
2.½«JPEG±àÂëµÄͼÏñ×Ö·û´®´Órequest²ÎÊý¸´ÖƵ½½«±»½øÐÐÍÆ¶ÏµÄÕÅÁ¿¡£
3.ÔËÐÐÍÆ¶Ï¡£Ëü´ÓsessionBundle»ñµÃTF»á»°¶ÔÏ󣬲¢ÔËÐÐÒ»´Î£¬Í¬Ê±´«ÈëÊäÈëºÍÊä³öÕÅÁ¿µÄÍÆ¶Ï¡£
4.´ÓÊä³öÕÅÁ¿½«½á¹û¸´ÖƵ½ÓÉClassificationResponseÏûÏ¢Ö¸¶¨µÄÐÎ×´ÖеÄresponseÊä³ö²ÎÊý²¢¸ñʽ»¯¡£
×îºóÒ»¶Î´úÂëÊÇÉèÖÃgRPC·þÎñÆ÷²¢´´½¨ClassificationServiceImplʵÀý£¨ÓÃSession-Bundle¶ÔÏó½øÐÐÅäÖ㩵ÄÑù°å´úÂë¡£
int main(int argc, char** argv) { if (argc < 3) { cerr << "Usage: server <port>
/path/to/export/files" << endl; return 1; } const string serverAddres
s(string("0.0.0.0:") + argv[1]); const string pathToExportFile
(argv[2]) ;
unique_ptr<SessionBundle>
sessionBundle =
createSessionBundle(pathToExportFiles);
const string serverAddres
classificationServiceImpl(move
(sessionBundle));
ServerBuilder builder;
builder. AddListeningPort(serverAddress,
grpc::InsecureServerCredentials());
builder.RegisterService(&classifi
cationServiceImpl);
unique_ptr<Server> server = builder
.BuildAndStart();
cout << "Server listening on "
<<
serverAddress << endl;
server->Wait();
return 0;
}
ΪÁ˱àÒëÕâ¶Î´úÂ룬ÐèÒªÔÚBUILDÎļþÖÐ
ΪÆä¶¨ÒåÒ»Ìõ¹æÔò£º
cc_binary(
name = "server",
srcs = [
"server.cc",
],
deps = [
":classification_service_proto",
"@tf_serving//tensorflow_serving/
servables/
tensorflow:session_bundle_factory",
"@grpc//:grpc++",
],
£© |
ΪÁ˱àÒëÕâ¶Î´úÂ룬ÐèÒªÔÚBUILDÎļþÖÐΪÆä¶¨ÒåÒ»Ìõ¹æÔò£º
½èÖúÕâ¶Î´úÂ룬±ã¿Éͨ¹ýÃüÁîbazel run :server 9999
/tmp/inception-v3/export/{timestamp}´ÓÈÝÆ÷ÖÐÔËÐÐÍÆ¶Ï·þÎñÆ÷¡£
¿Í»§¶ËÓ¦ÓÃ
ÓÉÓÚgRPCÊÇ»ùÓÚHTTP/2µÄ£¬½«À´¿ÉÄÜ»áÖ±½Ó´Óä¯ÀÀÆ÷µ÷ÓûùÓÚgRPCµÄ·þÎñ£¬µ«³ý·ÇÖ÷Á÷µÄä¯ÀÀÆ÷Ö§³ÖËùÐèµÄHTTP/2ÌØÐÔ£¬Çҹȸ跢²¼ä¯ÀÀÆ÷¶ËµÄJavaScript
gRPC¿Í»§¶Ë³ÌÐò£¬´Ówebapp·ÃÎÊÍÆ¶Ï·þÎñ¶¼Ó¦µ±Í¨¹ý·þÎñÆ÷¶ËµÄ×é¼þ½øÐС£
½ÓÏÂÀ´½«»ùÓÚBaseHTTPServer´î½¨Ò»¸ö¼òµ¥µÄPython
Web·þÎñÆ÷£¬BaseHTTPServer½«´¦ÀíÉÏÔØµÄͼÏñÎļþ£¬²¢½«Æä·¢Ë͸øÍƶϷþÎñ½øÐд¦Àí£¬ÔÙ½«ÍƶϽá¹ûÒÔ´¿Îı¾ÐÎʽ·µ»Ø¡£
ΪÁ˽«Í¼Ïñ·¢Ë͵½ÍƶϷþÎñÆ÷½øÐзÖÀ࣬·þÎñÆ÷½«ÒÔÒ»¸ö¼òµ¥µÄ±íµ¥¶ÔGETÇëÇó×ö³öÏìÓ¦¡£ËùʹÓõĴúÂëÈçÏ£º
From BaseHTTPServer import HTTPServer
,BaseHTTPRequestHandler import cgi import classification_service_pb2 From grpc.beta import implementations
class ClientApp (BaseHTTPRequestHandler);
def do_GET(self):
self.respond_form()
def respond_form(self, response=""):
form = """
<html><body>
<h1>Image classification service
</h1>
<form enctype="multipart/form-data"
method="post">
<div>Image: <input type="file"
name="file"
accept="image/jpeg"></div>
<div><input type="submit"
value="Upload">
</div>
</form>
%s
</body></html>
"""
response = form % response
self.send_response(200)
self.send_header("Content-type",
"text/html")
self.send_header("Content-length",
len(response))
self.end_headers()
self.wfile.write(response) |
ΪÁË´ÓWeb App·þÎñÆ÷µ÷ÓÃÍÆ¶Ï¹¦ÄÜ£¬ÐèÒªClassificationServiceÏàÓ¦µÄPython
protocol buffer¿Í»§¶Ë¡£ÎªÁËÉú³ÉËü£¬ÐèÒªÔËÐÐPythonµÄprotocol buffer±àÒëÆ÷£º
pip install grpcio cython
grpcio-tools python -m grpc.tools.protoc
-I. --python_out=. -- grpc_python_out=. classification_
service.proto |
Ëü½«Éú³É°üº¬ÁËÓÃÓÚµ÷Ó÷þÎñµÄstubµÄclassification_service_pb2.pyÎļþ¡£
·þÎñÆ÷½ÓÊÕµ½POSTÇëÇóºó£¬½«¶Ô·¢ËÍµÄ±íµ¥½øÐнâÎö£¬²¢ÓÃËü´´½¨Ò»¸öClassification-Request¶ÔÏó¡£È»ºóΪÕâ¸ö·ÖÀà·þÎñÆ÷ÉèÖÃÒ»¸öchannel£¬²¢½«ÇëÇóÌá½»¸øËü¡£×îºó£¬Ëü»á½«·ÖÀàÏìÓ¦äÖȾΪHTML£¬²¢ËͻظøÓû§¡£
def do_POST(self): form = cgi.FieldStorage( fp=self.rfile, headers=self.headers, environ={ 'REQUEST_METHOD': 'POST', 'CONTENT_TYPE': self.headers
['Content-Type'], }) request = classification_service_pb2.Classi
ficationRequest() request.input = form['file'].file.read()
channel =
implementations.insecure_channel
("127.0.0.1", 9999)
stub =
classification_service_pb2.beta_create
_ClassificationService_stub(channel)
response = stub.classify(request, 10) # 10 secs
timeout
self.respond_form("<div>Response:
%s</div>" %
response) |
ΪÁËÔËÐи÷þÎñÆ÷£¬¿É´Ó¸ÃÈÝÆ÷ÍⲿʹÓÃÃüÁîpython client.py¡£È»ºó£¬ÓÃä¯ÀÀÆ÷µ¼º½µ½http://localhost:8080À´·ÃÎÊÆäUI¡£ÇëÉÏ´«Ò»·ùͼÏñ²¢²é¿´ÍƶϽá¹ûÈçºÎ¡£
²úÆ·×¼±¸
ÔÚ½áÊø±¾ÎÄÄÚÈÝ֮ǰ£¬ÎÒÃÇ»¹½«Ñ§Ï°ÈçºÎ½«·ÖÀà·þÎñÆ÷Ó¦ÓÃÓÚ²úÆ·ÖС£
Ê×ÏÈ£¬½«±àÒëºóµÄ·þÎñÆ÷Îļþ¸´ÖƵ½Ò»¸öÈÝÆ÷ÄÚµÄÓÀ¾ÃλÖ㬲¢ÇåÀíËùÓеÄÁÙʱ¹¹½¨Îļþ£º
#ÔÚÈÝÆ÷ÄÚ²¿ mkdir /opt/classification_server cd /mnt/home/serving_example cp -R bazel-bin/. /opt/class
ification_server bazel clean |
ÏÖÔÚ£¬ÔÚÈÝÆ÷Íⲿ£¬ÎÒÃDZØÐ뽫Æä״̬Ìá½»¸øÒ»¸öеÄDocker¾µÏñ£¬»ù±¾º¬ÒåÊÇ´´½¨Ò»¸ö¼Ç¼ÆäÐéÄâÎļþϵͳ±ä»¯µÄ¿ìÕÕ¡£
#ÔÚÈÝÆ÷Íⲿ docker ps #»ñÈ¡ÈÝÆ÷ID docker commit <container id> |
ÕâÑù£¬±ã¿É½«Í¼ÏñÍÆË͵½×Ô¼ºÆ«ºÃµÄdocker·þÎñÔÆÖУ¬²¢¶ÔÆä½øÐзþÎñ¡£ |