Äú¿ÉÒÔ¾èÖú£¬Ö§³ÖÎÒÃǵĹ«ÒæÊÂÒµ¡£

1Ôª 10Ôª 50Ôª





ÈÏÖ¤Â룺  ÑéÖ¤Âë,¿´²»Çå³þ?Çëµã»÷Ë¢ÐÂÑéÖ¤Âë ±ØÌî



  ÇóÖª ÎÄÕ ÎÄ¿â Lib ÊÓÆµ iPerson ¿Î³Ì ÈÏÖ¤ ×Éѯ ¹¤¾ß ½²×ù Model Center   Code  
»áÔ±   
   
 
     
   
 ¶©ÔÄ
  ¾èÖú
TensorFlow ÖÇÄÜ»úÆ÷ÈËÔ­ÀíÓëʵÏÖ
 
×÷ÕߣºÀî¼Îè¯ ·¢²¼ÓÚ£º 2017-8-25
  3051  次浏览      28
 

µÚÒ»²¿·Ö½²½â×ÔÈ»ÓïÑÔ´¦ÀíµÄÔ­Àí£¬µÚ¶þ²¿·Ö½²½âÁÄÌì»úÆ÷È˵ÄʵÏÖÔ­Àí£¬½â¾ö·½°¸¼°ÌôÕ½£¬×îºóÒÔ seq2seq+Attention »úÖÆ½²½âÄ£Ðͽṹ¡£µÚÈý²¿·Ö½²½âÈçºÎ´Ó0¿ªÊ¼ÑµÁ·Ò»¸öÁÄÌì»úÆ÷ÈË¡£

Ò»¡¢×ÔÈ»ÓïÑÔ´¦ÀíµÄÔ­Àí

È˹¤ÖÇÄÜÀí½â×ÔÈ»ÓïÑÔµÄÔ­Àí£¬Òª»Ø´ðÕâ¸öÎÊÌ⣬Ê×ÏÈÐèÒª½ç¶¨ÏÂÕâ¸öÎÊÌâµÄº¬Ò壬һÖÖÊÇÏÁÒåµÄ˵ÈçºÎÓüÆËã»úÀ´´¦ÀíºÍ·ÖÎö×ÔÈ»ÓïÑÔ£»ÁíÒ»ÖÖÊÇÔòÊǹãÒ嵨Àí½â¹ØÓÚ¡¸È˹¤ÖÇÄÜ¡¹¡¢¡¸×ÔÈ»ÓïÑÔ¡¹ºÍ¡¸Àí½â¡¹µÄº¬Òå¡£

ÎÒÃÇÏÈ´Ó¹ãÒå²ãÃæÌ½ÌÖ¡£ÅªÇå³þÕ⼸¸öÃû´Ê¡£

×ÔÈ»ÓïÑÔ£º¾ÍÊÇÈËÀàÉç»áÖз¢Ã÷ºÍÑݱäµÄÓÃÓÚ¹µÍ¨ºÍ½»Á÷µÄÓïÑÔ¡£¶øÈ˹¤ÖÇÄÜÔÚ¶ÔÊÂÎ²»½ö½öÊÇ×ÔÈ»ÓïÑÔ£©µÄÀí½â£¬ÍùÍù°üº¬Á½¸ö²ã´Î£ºÒ»ÊÇÑо¿ÄÚÈÝ£»¶þÊÇ·½·¨ÂÛ¡£

Ñо¿ÄÚÈÝÉÏÖ÷ÒªÊÇÏÖÔÚÁ÷ÐеÄÑо¿¿ÎÌ⣬ÀýÈç֪ʶͼÆ×¡¢CV¡¢ÓïÒôʶ±ð¡¢NLP µÈ¡£

·½·¨ÂÛÊÇָʵÏÖÈ˹¤ÖÇÄܵķ½·¨£¬Ö÷ÒªÓÐÈýÖÖ£º·ûºÅÖ÷Òå¡¢Áª½áÖ÷Òå¡¢ÐÐΪÖ÷Òå¡£

·ûºÅÖ÷ÒåÊÇÓÃÊýÀíÂß¼­µÄÍÆÀíÀ´Ä£ÄâÈ˵Ä˼άÖÇÄÜ£¬ÀýÈçר¼ÒϵͳµÄʵÏÖ¡£Áª½áÖ÷Òå¶ÔÈËÄԵķÂÉúѧÑо¿£¬×î³£¼ûµÄ¾ÍÊÇÉñ¾­ÍøÂçÄ£ÐÍ¡£

ÐÐΪÖ÷ÒåÖØµãÔÚ¿ÉÔ¤²âµÄÈËÀàÐÐΪÉÏ£¬ÈÏΪÈËÀàͨ¹ýÓëÍâ½ç»·¾³µÄ½»»¥¶øµÃµ½×ÔÊÊÓ¦ÐÔ£¬Éæ¼°µÄËã·¨ÓÐÒÅ´«Ëã·¨¡¢Ç¿»¯Ñ§Ï°µÈ¡£

ÏÖÓеÄNLPÖ÷ÒªÊÇÒÔ¹æÔòºÍͳ¼ÆÏà½áºÏÀ´´¦ÀíµÄ¡£ËüµÄ¹æÔòÒ»ÃæÆ«ÏòÓÚ·ûºÅÖ÷ÒåµÄÊӽǣ»

¶øÍ³¼ÆÒ»ÃæÆ«ÏòÓÚÍÚ¾òÒ»°ã¹æÂÉ£¬ÊôÓÚ¹éÄÉ£¬Ä¿Ç°Óõķ½·¨£¬±ÈÈ罫×ÔÈ»ÓïÑÔÓôÊÏòÁ¿µÄ·½·¨±íÕ÷£¬È»ºó½ÓÈëÉñ¾­ÍøÂçÖнøÐÐѵÁ·£¬Ò²¾ÍÊÇÁª½áÖ÷ÒåµÄ˼Ïë¡£

Àí½â£º¹ØÓÚ»úÆ÷ÊÇ·ñÄÜÕæÕýÀí½âÓïÑÔÒ»Ö±ÓÐÕùÂÛ¡£ÏÈÅ׿ªÕâ¸öÎÊÌ⣬ÎÒÃÇ¿´¿´ÈËÀà¶ÔÓïÑÔµÄÀí½âÊÇÔõôÑùµÄ¡£Êµ¼ÊÉÏ£¬ÈËÀà¶ÔÀí½âÕâ¸öÊÂÇéÒ²×öµÄ²»Ò»¶¨ºÃ¡£

±ÈÈ磬Äϱ±·½¶Ô¡¸¶¹¸¯ÄÔ¡¹µÄÈÏÖªÊDz»Í¬µÄ£¬Á½È˽»Ì¸¿ÉÄܾͻá¶ÔͬһÎïÌåµÄÀí½â²»Í¬¡£

Òò´Ë£¬Àí½âÊÇÐèÒªÓÉÏàËÆµÄÉú»î¾­Àú¡¢¹²Í¬»°Ìâ¡¢ÉÏÏÂÎÄ¡¢»á»°µÄ»·¾³¡¢Ë«·½µÄ֪ʶµÈºÜ¶àÒòËØ¾ö¶¨µÄ¡£

¼ÈÈ»¶ÔÓÚÈËÀàÀ´Ëµ£¬ÕæÕýÄÜÀí½â¶Ô·½£¬ÐèÒªÕâô¶àµÄÒþÐÔÒòËØ£¬ÄǶÔÓÚ»úÆ÷À´Ëµ£¬ÎÒÃÇ×îºÃ¾Í²»Òª¹ØÐÄ»úÆ÷ÊÇ·ñÕæÕýÄÜÀí½âÎÊÌâµÄº¬Òå±¾Éí£¬¶øÊǾ¡¿ÉÄܵØÈûúÆ÷¹Ø×¢ÉÏÊöÒòËØ£¬À´Ä£ÄâÈ˵ÄÖÇÄÜ¡£

ÏÁÒåµÄ²ãÃæÊÇÎÒÃǹ¤³ÌʦÑо¿µÄÖ÷Òª·½Ïò¡£

Ò²¾ÍÊǽ«×ÔÈ»ÓïÑÔÀí½â¿´³ÉÊÇÓüÆËã»úÀ´´¦ÀíºÍ·ÖÎö×ÔÈ»ÓïÑÔ£¬ËüÉæ¼°µ½ÓïÑÔѧ£¨´Ê¡¢´ÊÐÔ¡¢Óï·¨£©ºÍ¼ÆËã»úѧ¿Æ£¨Ä£ÐÍ/Ëã·¨£©µÄ·¶³ë¡£

´ÓÓïÑÔѧÉÏÀ´¿´£¬Ñо¿µÄ·½Ïò°üÀ¨´Ê¸ÉÌáÈ¡¡¢´ÊÐÔ»¹Ô­¡¢·Ö´Ê¡¢´ÊÐÔ±ê×¢¡¢ÃüÃûʵÌåʶ±ð¡¢´ÊÐÔÏûÆç¡¢¾ä·¨·ÖÎö¡¢ÆªÕ·ÖÎöµÈµÈ¡£

ÕâÊôÓÚÑо¿µÄ»ù´¡·¶³ë£¬ÔÚÕâЩ»ù´¡µÄÑо¿ÄÚÈÝÖ®ÉÏ£¬ÃæÏòµÄÊǾßÌåµÄÎı¾´¦ÀíÓ¦Óã¬È磬»úÆ÷·­Òë¡¢Îı¾ÕªÒª¡¢Çé¸Ð·ÖÀà¡¢ÎÊ´ðϵͳ¡¢ÁÄÌì»úÆ÷È˵ȡ£

ÔÚ¼ÆËã»úËã·¨µÄÑо¿·½Ã棬һ°ãÊÇÒÔ¹æÔòºÍͳ¼ÆÏà½áºÏµÄ·½·¨£¬Ò²¾ÍÊÇÀíÐÔÖ÷ÒåºÍ¾­ÑéÖ÷ÒåÏà½áºÏ¡£

×ÔÈ»ÓïÑÔ±¾ÖÊÉÏ»¹ÊÇ·ûºÅϵͳ£¬Òò´ËÓÐÒ»¶¨µÄ¹æÔò¿ÉѰ£¬µ«ÊÇËüµÄ¸´ÔÓÐÔÓÖ¾ö¶¨ÁËûÓйæÔò¿ÉÒԼȲ»Ï໥³åÍ»ÓÖÄܸ²¸ÇËùÓеÄÓïÑÔÏÖÏó¡£

ºóÀ´´ó¹æÄ£ÓïÁÏ¿âµÄÍêÉÆºÍͳ¼Æ»úÆ÷ѧϰ·½·¨Á÷ÐÐÆðÀ´ºó£¬¾ÍʡȥÁ˺ܶàÈ˹¤±àÖÆ¹æÔòµÄ¸ºµ££¬Ê¹Ä£ÐÍÉú³É×Ô¶¯Éú³ÉÌØÕ÷¡£

ËùÒÔ£¬ÎÒÃÇÑо¿µÄ NLP ¾ÍÊÇʹÓÃÊýÀíºÍ»úÆ÷ѧϰµÄ·½·¨¶ÔÓïÑÔ½øÐн¨Ä£¡£¿ÉÒÔ˵£¬NLP ²»ÊÇ´ïµ½ÕæÕýµÄ×ÔÈ»ÓïÑÔÀí½â£¬¶øÊǰÑÓïÑÔµ±³ÉÊÇÒ»ÖÖ¼ÆËãÈÎÎñ¡£

¶þ¡¢ÁÄÌì»úÆ÷È˵ÄʵÏÖÔ­Àí£¬½â¾ö·½°¸¼°ÌôÕ½

ÎÒÃÇ´ÓÁÄÌì»úÆ÷È˵ķÖÀàºÍʵÏÖÔ­Àí·Ö±ð˵Æð¡£

ĿǰÁÄÌì»úÆ÷È˸ù¾Ý¶Ô»°µÄ²úÉú·½Ê½£¬¿ÉÒÔ·ÖΪ»ùÓÚ¼ìË÷µÄÄ£ÐÍ£¨Retrieval-Based Models£©ºÍÉú³ÉʽģÐÍ£¨Generative Models£©¡£

»ùÓÚ¼ìË÷µÄÄ£ÐÍÓÐÒ»¸öÔ¤Ïȶ¨ÒåµÄ»Ø´ð¼¯£¬ÎÒ

ÃÇÐèÒªÉè¼ÆÒ»Ð©Æô·¢Ê½¹æÔò£¬ÕâЩ¹æÔòÄܹ»¸ù¾ÝÊäÈëµÄÎʾ估ÉÏÏÂÎÄ£¬ÌôÑ¡³öºÏÊʵĻشð¡£

Éú³ÉʽģÐͲ»ÒÀÀµÔ¤Ïȶ¨ÒåµÄ»Ø´ð¼¯£¬¶øÊǸù¾ÝÊäÈëµÄÎʾ估ÉÏÏÂÎÄ£¬²úÉúÒ»¸öеĻشð¡£

ÁÄÌì»úÆ÷È˵ÄÕâÁ½Ìõ¼¼Êõ·Ïߣ¬´Ó³¤Ô¶µÄ½Ç¶È¿´Ä¿Ç°¼¼Êõ»¹¶¼»¹´¦ÔÚɽµ×£¬Á½ÖÖ¼¼Êõ·ÏßµÄÒìͬºÍÓÅÊÆÈçÏ£º

»ùÓÚ¼ìË÷µÄÄ£Ð͵ÄÓÅÊÆ£º

´ð¾ä¿É¶ÁÐÔºÃ

´ð¾ä¶àÑùÐÔÇ¿

³öÏÖ²»Ïà¹ØµÄ´ð¾ä£¬ÈÝÒ×·ÖÎö¡¢¶¨Î» bug

µ«ÊÇËüµÄÁÓÊÆÔÚÓÚ£ºÐèÒª¶ÔºòÑ¡µÄ½á¹û×öÅÅÐò£¬½øÐÐÑ¡Ôñ

»ùÓÚÉú³ÉʽģÐ͵ÄÓÅÊÆ£º

¶Ëµ½¶ËµÄѵÁ·£¬±È½ÏÈÝÒ×ʵÏÖ

±ÜÃâά»¤Ò»¸ö´óµÄ Q-A Êý¾Ý¼¯

²»ÐèÒª¶Ôÿһ¸öÄ£¿é¶îÍâ½øÐе÷ÓÅ£¬±ÜÃâÁ˸÷¸öÄ£¿éÖ®¼äµÄÎó²î¼¶ÁªÐ§Ó¦

µ«ÊÇËüµÄÁÓÊÆÔÚÓÚ£ºÄÑÒÔ±£Ö¤Éú³ÉµÄ½á¹ûÊǿɶÁµÄ£¬¶àÑùµÄ¡£

Òò´Ë£¬ÉÏÊö·½·¨¹²Í¬ÃæÁÙµÄÌôÕ½ÓУº

ÈçºÎÀûÓÃǰ¼¸ÂÖ¶Ô»°µÄÐÅÏ¢£¬Ó¦Óõ½µ±ÂÖ¶Ô»°µ±ÖÐ

ºÏ²¢ÏÖÓеÄ֪ʶ¿âµÄÄÚÈݽøÀ´

ÄÜ·ñ×öµ½¸öÐÔ»¯£¬Ç§ÈËÇ§Ãæ¡£

ÕâÓеãÀàËÆÓÚÎÒÃǵÄÐÅÏ¢¼ìË÷ϵͳ£¬¼ÈÏ£ÍûÔÚ´¹Ö±ÁìÓò×öµÃ¸üºÃ£»Ò²Ï£Íû¶Ô²»Í¬µÄÈ赀 query Óв»Í¬µÄÅÅÐòÆ«ºÃ¡£

´ÓÓ¦ÓÃÄ¿µÄµÄ½Ç¶ÈÇø·Ö£¬¿ÉÒÔ·ÖΪĿ±êÇý¶¯£¨Goal Driven£©£¬Ó¦ÓÃÓÚ¿Í·þÖúÀíµÈ£¬ÔÚ·â±Õ»°Ìⳡ¾°ÖУ»ÎÞÄ¿±êÇý¶¯£¨Non-Goal Driven£©£¬Ó¦ÓÃÔÚ¿ª·Å»°ÌâµÄ³¡¾°Ï£¬ÕâÊÇ¿É̸ÂÛµÄÖ÷ÌâÊDz»Ï޵쬵«ÊÇÐèÒª»úÆ÷ÈËÓÐÒ»¶¨µÄ»ù´¡³£Ê¶¡£

¾¡¹ÜĿǰ¹¤Òµ½çÓ¦ÓõĴó¶àÊýÊÇ»ùÓÚ¼ìË÷µÄÄ£ÐÍ£¬ÊôÓÚÄ¿±êÇý¶¯µÄ£¬ÀýÈ磺°¢ÀïСÃÛ£¬Ó¦ÓõϰÌâÁìÓò±È½ÏÕ­£¬ÉÔ΢½«»°ÌâÀ©´óÒ»µã£¬Ëü¾Í»á²»×ű߼ʻظ´»òÕßÎIJ»¶ÔÌâ¡£ÈçÏÂͼ£º

Ò»¸ö¿ª·Å»°Ìⳡ¾°ÏµÄÉú³ÉʽģÐÍÓ¦¸ÃÊÇ×îÖÇÄÜ¡¢·ûºÏÎÒÃÇÔ¤ÆÚµÄÁÄÌì»úÆ÷ÈË¡£Òò´Ë×ܽáÀ´¿´:

ÖÇÄÜÁÄÌì»úÆ÷È˵ÄÄ¿±ê£º

ºÍÈËÀàÄܹ»½øÐгÖÐøµÄ¹µÍ¨

¶Ô²»Í¬µÄÌáÎÊÄܹ»¸ø³öºÏÊʵĻشð

¿¼Âǵ½ÈËÀ಻ͬ¸öÐÔ»¯µÄ²îÒìÐÔ£¬¸ø³ö²îÒìÐԵĻشð£¨ÀýÈ磬ͬһ¸öÎÊÌ⣬¶ÔÄÐÅ®ÀÏÉÙ²»Í¬ÈºÌåµÄ»Ø´ðÓ¦¸ÃÂÔÓвîÒ죩

ÄÇô¶ÔÓÚÒ»¸öÖÇÄÜ»úÆ÷ÈËÀ´Ëµ£¬ËüµÄÁÄÌìµÄ¹¦ÄÜÔÚÆäÖÐÓ¦¸Ã´¦ÓÚʲôλÖã¿Ê×ÏÈ£¬ÁÄÌìÓ¦¸ÃÊÇÒ»¸ö»ù´¡Ä£¿é£»Æä´Î£¬ÁÄÌìÓ¦¸ÃºÍÍê³ÉÈÎÎñµÄÄ£¿éÓкܺõÄЭ×÷£»×îºó£¬ÁÄÌìÓ¦¸Ãʹ»úÆ÷ÈË¿´ÉÏÈ¥ÏñÄúµÄÅóÓÑ£¬¶ø²»ÊÇÄúµÄ´úÀí»òÕßÖúÊÖ¡£

´ÓÉÏÊö½Ç¶ÈÀ´Ëµ£¬ÏÖÔÚÓÐһЩ¾­³£ÓëÁÄÌì»úÆ÷ÈË»ìÏýµÄ¸ÅÄҲÊÇһЩÁÄÌìϵͳµÄÖܱ߲úÆ·£º

QA ÎÊ´ðϵͳ£ºÊǻشðÊÂʵÎÊÌ⣨ÀýÈçÖé·åÓжà¸ß£©ÒÔ¼°·ÇÊÂʵÎÊÌ⣨ÀýÈç why, how, opinion µÈ¹ÛµãÐÔÎÊÌ⣩µÄÁìÓò»úÆ÷ÈË¡£

Dialog system ¶Ô»°ÏµÍ³£ºÕâÖÖ´ó¶àÊÇÄ¿±êÇý¶¯µÄ£¬µ«Êǽü¼¸Äê¶¼Ò²ÔÚÂýÂý½ÓÊÜÁÄÌì»úÆ÷È˹¦ÄÜ

Online customer service ÔÚÏ߿ͷþ£ºÀýÈçÌÔ±¦Ð¡ÃÛ£¬ËüÔÚ¶àÊýÇé¿öÏÂÏñÒ»¸ö×Ô¶¯µÄ FAQ¡£

Òò´Ë£¬¾¡¹ÜÁÄÌìϵͳ¶¼ÊÇÕë¶ÔÎı¾Àí½âµÄ´ó·½Ïò£¬µ«Ä¿±ê²»Í¬¾ö¶¨Á˼¼Êõ·Ïß»áÓÐËùÆ«ÖØ£¬µ«ÁÄÌ칦ÄÜÊÇÒ»¸ö»ù´¡¹¦ÄÜ¡£

ÖÇÄÜÁÄÌì»úÆ÷ÈË¿ÉÒÔ´ÓÉÏÃæµÄÖܱßϵͳÑо¿ÁìÓòµÄÊý¾Ý¼¯ÓÐËù½è¼ø£º

·ÇÊÂʵÎÊÌâµÄÎÊ´ð

ÉçÇøÐÍÎÊ´ðϵͳ£¨ÀýÈç°Ù¶ÈÖªµÀµÈ£¬¶ÔÎÊÌâºÍ´ð°¸¼äÓнÏÇ¿µÄÆ¥Å䣻²¢ÇÒÒ»¸öÎÊÌâ¶à¸ö´ð°¸Ê±ÓÐÆÀ·Ö¡¢ÅÅÐò£©

´ÓÔÚÏßϵͳÖÐÍÚ¾òһЩºÃµÄ QA corpus

ÄÇÈçºÎÀ´ÆÀ¼ÛÒ»¸öÁÄÌì»úÆ÷È˵ĺûµ£¿×îÖØÒªµÄÊÇÎʾäºÍ´ð¾äµÄÏà¹ØÐÔ£¬Ò²¾ÍÊDZ¾ÖÊÊÇ£º¶ÌÎı¾Ïà¹Ø¶È¼ÆËã¡£µ«Òª×¢Ò⣺

ÏàËÆÐÔºÍÏà¹ØÐÔÊDz»Í¬µÄ¡£ÓÃÓÚÏàËÆÐÔ¼ÆËãµÄ¸÷ÖÖ·½·¨²¢²»ÊÊÓÃÓÚÏà¹ØÐÔ¡£ÎÒÃÇÐèÒª½¨Á¢Ò»Ì×¶ÌÎı¾Ïà¹ØÐÔ¼ÆËã·½·¨¡£

Ïà¹ØÐÔ¼ÆËãÓÐһЩÔÚÔçÆÚµÄÁÄÌì»úÆ÷È˵Ĺ¹½¨ÖÐÑÓÐøÏÂÀ´µÄ·½·¨£º

´ÊÓï¹²ÏÖµÄͳ¼Æ

»ùÓÚ»úÆ÷·­ÒëµÄÏà¹Ø¶È¼ÆËã

Ö÷ÌâÄ£ÐÍ£¨LDA)µÄÏàËÆ¶È¼ÆËã

ĿǰÔÚÁÄÌì»úÆ÷ÈËÉÏʹÓõÄÉî¶Èѧϰ·½·¨ÓÐÈçÏÂÕâЩ£º

Word2vec, Glove
CNN, LSTM, GRU
Seq2Seq
Attention mechanism
Deep Reinforcement Learning

ÆäÖУ¬Éî¶ÈÇ¿»¯Ñ§Ï°Êǰѡ°ÈçºÎµÃµ½Ò»¸ö×ã¹»ÈÃÈËÂúÒâµÄÉú³Éʽ½á¹û¡±Á¿»¯³ÉReward¡£ÕâЩÒòËØ°üÀ¨£º

ÊÇ·ñÈÝÒ×±»»Ø´ð£¨Ò»¸öºÃµÄÉú³ÉʽµÄ½á¹ûÓ¦¸ÃÊǸüÈÝÒ×ÈÃÈ˽ÓÏÂÈ¥µÄ£©

Ò»¸öºÃµÄ»Ø¸´Ó¦¸ÃÈöԻ°Ç°½ø£¬±ÜÃâÖØ¸´ÐÔ

±£Ö¤ÓïÒåµÄÁ¬¹áÐÔ¡£ÓÃÉú³ÉʽµÄ½á¹û·´ÍÆ»ØqueryµÄ¿ÉÄÜÐÔÓжà´ó£¬Äܱ£Ö¤ÔÚÓïÒåÉÏÊǺÍгµÄ¡¢Á¬¹áµÄ¡£

ÄÇôÈçºÎʵÏÖÔÚÖÇÄÜÁÄÌì»úÆ÷È˵ĵÚÈý¸öÄ¿±ê£¬²îÒ컯»Ø´ðÄØ£¿Ò²¾ÍÊÇÈçºÎÔÚ¶Ô»°ÀïÈçºÎÒýÈë¸öÐÔ»¯£¨personality£©ÐÅÏ¢£¿

Ê×ÏÈԤѵÁ·Ò»¸öÈ˵ÄÏòÁ¿£¬ÈÃÉú³ÉµÄ½á¹ûÌù½üÈ˵ÄÌØµã¡£Í¨¹ý word embedding ½«È˵ÄÐÅÏ¢×÷Ϊһ¸ö´ÊÏòÁ¿£¬ÔÚÊäÈ벿·ÖÔö¼ÓÒ»×éÈ˵ÄÏòÁ¿£»ÔÚÉú³É»Ø´ðµÄʱºò£¬¿¼ÂDz»Í¬µÄÈËÓ¦¸ÃѡʲôÑùµÄ´ÊÓï¡£

Ŀǰ£¬ÎÒÃÇÒª×öÒ»¸öÕæÕýÖÇÄܵÄ×Ô¶¯ÁÄÌì»úÆ÷ÈË£¬ÈÔÈ»ÃæÁÙһЩÌôÕ½£º

ȱ·¦¹«¹²µÄѵÁ·Êý¾Ý¼¯£¬Ä¿Ç°Ê¹ÓùúÍâµÄÊý¾Ý¼¯½Ï¶à

Ubuntu dialog corpus£¨subset of Ubuntu Corpus£©

Reddit dataset£¨¿É¶ÁÐÔºÍÖÊÁ¿¶¼²»´í£©

Corpora from Chinese SNS(΢²©£©

²âÊÔ¼¯»¹²»Í³Ò»

ÆÀ¹À¶ÈÁ¿£º¶ÈÁ¿ºÜÄÑÉè¼Æ£¨Ä¿Ç°ÊÇ´Ó»úÆ÷·­ÒëºÍ×Ô¶¯ÕªÒªµÄBLEU¡¢ROUGEÀï½è¼øµÄ£¬µ«ÃæÁÙÎÊÌâÊÇ·ñÄ̻ܿ­ÁÄÌì»úÆ÷È˵ĺûµ£¬²¢ÇÒÖ¸µ¼ÁÄÌì»úÆ÷È˵ļ¼Êõ³¯×ÅÕýÏòµÄ·½Ïò·¢Õ¹£©

ÁÄÌì»úÆ÷È˵ÄÒ»°ã¶Ô»°ºÍÈÎÎñµ¼ÏòµÄ¶Ô»°Ö®¼äÈçºÎÄܹ»Æ½»¬Çл»

ÕâÖÖÆ½»¬Çл»¶ÔÓû§µÄÌåÑé·Ç³£ÖØÒª

Çл»µÄ¼¼ÊõÐèÒªÒÀÀµÇéÐ÷·ÖÎö¼°ÉÏÏÂÎÄ·ÖÎö

Óû§²»ÐèÒª¸ø³öÃ÷È·µÄ·´À¡¡£ÀýÈ磬ÎÒǰһ¾ä˵¹êϺ㬺óÒ»¾ä˵²»Ï²»¶º«·¶£¬ÐèÒªÁÄÌì»úÆ÷ÈËÄÜÕýȷʶ±ð

ÈÔÈ»´æÔÚµÄÎÊÌâ

¾ä×Ó¼¶¡¢Æ¬¶Î¼¶µÄÓïÒ彨ģ»¹Ã»ÓдÊÓïµÄ½¨Ä££¨word embedding£©ÄÇôºÃ

Éú³ÉʽÈÔÈ»»á²úÉúһЩ°²È«»Ø´ð

ÔÚ֪ʶµÄ±íʾºÍÒýÈëÉÏ»¹ÐèҪŬÁ¦

×îºóÓÐһЩ¿ÉÒÔÆÚ´ýµÄ½áÂÛºÍǰ¾°£º

¿ÉÒÔÈÏΪÖÇÄÜÁÄÌì»úÆ÷ÈËÊÇÒ»¸öALL-NLPÎÊÌ⣬Éî¶ÈѧϰÊÇ´Ù½øÁËchatbotµÄ·¢Õ¹£¬µ«»¹²»¹»£¬ÐèÒªºÍ´«Í³»úÆ÷ѧϰÏà½áºÏ¡£Ä¿Ç°ÎÒÃÇÈÔÔÚ̽Ë÷һЩ·½·¨ÂÛ¼°²âÊÔµÄÆÀ¼ÛÌåϵ¡£

²¢ÇÒÏ£ÍûÁÄÌì»úÆ÷ÈËÄܹ»Ö÷¶¯ÕÒÈË˵»°£¬ÀýÈçËü˵¡°Äã×òÌì¸úÎÒ˵ʲô¡­¡­ÄãÉúÈÕ¿ìµ½À²¡­¡­¡±µÈµÈ¡£

Õâ¸ö¿ÉÒÔÓùæÔòʵÏÖ£¬ÐèÒª°ÑÎÊÌⶨÒåÇå³þ¡£¸öÐÔ»¯µÄÁÄÌì»úÆ÷È˵ÄʵÏÖ£¬×ÅÁ¦µã¿ÉÄܸüÆ«¹¤³Ì¡£ÁÄÌì»úÆ÷È˵ÄÑо¿¿ÉÄÜÔÚ2015Äê×óÓҲſªÊ¼ÅÐËÆð£¬ËùÒÔ´ó¼ÒÓкܴóµÄ»ú»áºÍÌôÕ½¡£

һЩÁÄÌì»úÆ÷È˵IJο¼ÎÄÏ×£º

Neural Responding Machine for Short-Text Conversation (2015-03)

A Neural Conversational Model (2015-06)

A Neural Network Approach to Context-Sensitive Generation of Conversational Responses (2015-06)

The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue - Systems (2015-06)

Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models (2015-07)

A Diversity-Promoting Objective Function for Neural Conversation Models (2015-10)

Attention with Intention for a Neural Network Conversation Model (2015-10)

Improved Deep Learning Baselines for Ubuntu Corpus Dialogs (2015-10)

A Survey of Available Corpora for Building Data-Driven Dialogue Systems (2015-12)

Incorporating Copying Mechanism in Sequence-to-Sequence Learning (2016-03)

A Persona-Based Neural Conversation Model (2016-03)

How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation (2016-03)

ÏÂÃæÊǽ²½âÖÇÄÜÁÄÌì»úÆ÷È赀 Seq2Seq+Attention »úÖÆ¡£

¶øÉú³ÉʽģÐÍÏÖÔÚÖ÷ÒªÑо¿·½·¨ÊÇ Sequence to Sequence+Attention µÄÄ£ÐÍ£¬ÒÔ¼°×îгöµÄ GANSeq Ä£ÐÍ¡££¨²Î¿¼ÂÛÎÄ£ºSequence to Sequence Learning with Neural Networks£©

seq2seq Ä£ÐÍÊÇÒ»¸ö·­ÒëÄ£ÐÍ£¬Ö÷ÒªÊǰÑÒ»¸öÐòÁз­Òë³ÉÁíÒ»¸öÐòÁС£ËüµÄ»ù±¾Ë¼ÏëÊÇÓÃÁ½¸ö RNNLM£¬Ò»¸ö×÷Ϊ±àÂëÆ÷£¬ÁíÒ»¸ö×÷Ϊ½âÂëÆ÷£¬×é³É RNN ±àÂëÆ÷-½âÂëÆ÷¡£

ÔÚÎı¾´¦ÀíÁìÓò£¬ÎÒÃdz£ÓñàÂëÆ÷-½âÂëÆ÷£¨encoder-decoder£©¿ò¼Ü¡£ÈçÏÂͼ£º

ÔÚÕâ¸öÄ£ÐÍÖУ¬Ã¿Ò»Ê±¼äµÄÊäÈëºÍÊä³öÊDz»Í¬µÄ¡£ÎÒÃǵÄÄ¿µÄÊǰÑÏÖÓÐÐòÁÐ ABC ×÷ΪÊäÈ룬ӳÉä³É WXYZ ×÷ΪÊä³ö¡£±àÂëÆ÷¶ÔÊäÈëµÄ ABC ½øÐбàÂ룬Éú³ÉÖмäÓïÒå±àÂë C¡£

È»ºó½âÂëÆ÷¶ÔÖмäÓïÒå±àÂë C ½øÐнâÂ룬ÔÚÿ¸öiʱ¿Ì£¬½áºÏÒѾ­Éú³ÉµÄ WXY µÄÀúÊ·ÐÅÏ¢Éú³É Z¡£

µ«ÊÇ£¬Õâ¸ö¿ò¼ÜÓÐÒ»¸öȱµã£¬¾ÍÊÇÉú³ÉµÄ¾ä×ÓÖÐÿһ¸ö´Ê²ÉÓõÄÖмäÓïÒå±àÂëÊÇÏàͬµÄ£¬¶¼ÊÇ C¡£

Òò´Ë£¬ÔÚ¾ä×ӱȽ϶̵Äʱºò£¬»¹ÄܱȽÏÌùÇУ¬¾ä×Ó³¤Ê±£¬¾ÍÃ÷ÏÔ²»ºÏÓïÒåÁË¡£

ËüµÄ¹ý³Ì·ÖΪÕâÁ½²½£º

£¨1£©±àÂ루Encode£©

ht ÊÇÓɵ±Ç°µÄÊäÈë xt ºÍÉÏÒ»´ÎµÄÒþ²Ø²ãµÄÊä³ö ht-1£¬¾­¹ý·ÇÏßÐԱ任µÃµ½µÄ¡£

ÏòÁ¿cͨ³£ÎªRNNÖеÄ×îºóÒ»¸öÒþ½Úµã£¨h£¬Hidden state£©£¬»òÊǶà¸öÒþ½ÚµãµÄ¼ÓȨ×ܺͣ¨ÔÚ×¢ÒâÁ¦»úÖÆÀ¡£

£¨2£©½âÂ루Decode£©

½âÂëµÄ¹ý³Ì³£³£Ê¹ÓÃ̰ÐÄËã·¨»òÕß¼¯ÊøËÑË÷£¬À´·µ»Ø¶ÔÓ¦¸ÅÂÊ×î´óµÄ´Ê»ã¡£

ÔÚʵ¼ÊʵÏÖÁÄÌìϵͳµÄʱºò£¬Ò»°ã±àÂëÆ÷ºÍ½âÂëÆ÷¶¼²ÉÓÃRNNÄ£ÐÍÒÔ¼°RNNÄ£Ð͵ĸĽøÄ£ÐÍ LSTM¡£

µ±¾ä×Ó³¤¶È³¬¹ý30ÒÔºó£¬LSTMÄ£Ð͵ÄЧ¹û»á¼±¾çϽµ£¬Ò»°ã´Ëʱ»áÒýÈë Attention Ä£ÐÍ£¬¶Ô³¤¾ä×ÓÀ´ËµÄܹ»Ã÷ÏÔÌáÉýϵͳЧ¹û¡£

Attention »úÖÆÊÇÈÏÖªÐÄÀíѧ²ãÃæµÄÒ»¸ö¸ÅÄËüÊÇÖ¸µ±ÈËÔÚ×öÒ»¼þÊÂÇéµÄʱºò£¬»áרעµØ×öÕâ¼þʶøºöÂÔÖÜΧµÄÆäËûÊ¡£

ÀýÈ磬ÈËÔÚרעµØ¿´Õâ±¾Ê飬»áºöÂÔÅÔ±ßÈË˵»°µÄÉùÒô¡£ÕâÖÖ»úÖÆÓ¦ÓÃÔÚÁÄÌì»úÆ÷ÈË¡¢»úÆ÷·­ÒëµÈÁìÓò£¬¾Í°ÑÔ´¾ä×ÓÖжÔÉú³É¾ä×ÓÖØÒªµÄ¹Ø¼ü´ÊµÄÈ¨ÖØÌá¸ß£¬²úÉú³ö¸ü׼ȷµÄÓ¦´ð¡£

Ôö¼ÓÁË Attention Ä£Ð͵ıàÂëÆ÷-½âÂëÆ÷¿ò¼ÜÈçÏÂͼËùʾ¡£Seq2Seq ÖеıàÂëÆ÷±»Ì滻Ϊһ¸öË«ÏòÑ­»·ÍøÂ磨bidirectional RNN£©£¬Ô´ÐòÁÐ x£½(x1,x2,¡­,xt) ·Ö±ð±»ÕýÏòÓë·´ÏòµØÊäÈëÁËÄ£ÐÍÖУ¬½ø¶øµÃµ½ÁËÕý·´Á½²ãÒþ½Úµã£¬Óï¾³ÏòÁ¿cÔòÓÉ RNN ÖеÄÒþ½Úµãhͨ¹ý²»Í¬µÄÈ¨ÖØ a ¼ÓȨ¶ø³É¡£

Èý¡¢¶¯ÊÖʵÏÖÒ»¸öÖÇÄÜ»úÆ÷ÈË

±¾´Î´úÂë»ùÓÚTF 1.1°æ±¾ÊµÏÖ£¬Ê¹ÓÃË«Ïò LSTM+Attention Ä£ÐÍ¡£·ÖΪ6²½£º

1£©¿âÒýÈë¼°³¬²ÎÊý¶¨Òå

import numpy as np #matrix math
import tensorflow as tf #machine learningt
import helpers #for formatting data into batches and generating random sequence data
tf.reset_default_graph() #Clears the default graph stack and resets the global default graph.
sess = tf.InteractiveSession()
PAD = 0
EOS = 1
vocab_size = 10
input_embedding_size = 20 #character length
encoder_hidden_units = 20 #num neurons
decoder_hidden_units = encoder_hidden_units * 2
#input placehodlers
encoder_inputs = tf.placeholder(shape=(None, None), dtype=tf.int32, name='encoder_inputs')
encoder_inputs_length = tf.placeholder(shape=(None,), dtype=tf.int32, name='encoder_inputs_length')
decoder_targets = tf.placeholder(shape=(None, None), dtype=tf.int32, name='decoder_targets')

 

2£©ÊäÈëÎı¾µÄÏòÁ¿±íʾ

embeddings = tf.Variable(tf.random_uniform([vocab_size, input_embedding_size], -1.0, 1.0), dtype=tf.float32)
#this thing could get huge in a real world application
encoder_inputs_embedded = tf.nn.embedding_lookup(embeddings, encoder_inputs)

 

3£©Encoder

from tensorflow.contrib.rnn import LSTMCell, LSTMStateTuple
encoder_cell = LSTMCell(encoder_hidden_units) ((encoder_fw_outputs,
encoder_bw_outputs),
(encoder_fw_final_state,
encoder_bw_final_state)) = (
tf.nn.bidirectional_dynamic_rnn(cell_fw=encoder_cell,
cell_bw=encoder_cell,
inputs=encoder_inputs_embedded,
sequence_length=encoder_inputs_length,
dtype=tf.float64, time_major=True)
)
encoder_outputs = tf.concat((encoder_fw_outputs, encoder_bw_outputs), 2)

encoder_final_state_c = tf.concat(
(encoder_fw_final_state.c, encoder_bw_final_state.c), 1)
encoder_final_state_h = tf.concat(
(encoder_fw_final_state.h, encoder_bw_final_state.h), 1)
encoder_final_state = LSTMStateTuple(
c=encoder_final_state_c,
h=encoder_final_state_h
)

4£©Decoder

decoder_cell = LSTMCell(decoder_hidden_units)
encoder_max_time, batch_size = tf.unstack(tf.shape(encoder_inputs))
decoder_lengths = encoder_inputs_length + 3
#weights
W = tf.Variable(tf.random_uniform([decoder_hidden_units, vocab_size], -1, 1), dtype=tf.float32)
#bias
b = tf.Variable(tf.zeros([vocab_size]), dtype=tf.float32) assert EOS == 1 and PAD == 0
eos_time_slice = tf.ones([batch_size], dtype=tf.int32, name='EOS')
pad_time_slice = tf.zeros([batch_size], dtype=tf.int32, name='PAD')
#retrieves rows of the params tensor. The behavior is similar to using indexing with arrays in numpy
eos_step_embedded = tf.nn.embedding_lookup(embeddings, eos_time_slice)
pad_step_embedded = tf.nn.embedding_lookup(embeddings, pad_time_slice)
def loop_fn_initial():
initial_elements_finished = (0 >= decoder_lengths) # all False at the initial step
#end of sentence
initial_input = eos_step_embedded
#last time steps cell state
initial_cell_state = encoder_final_state
#none
initial_cell_output = None
#none
initial_loop_state = None # we don't need to pass any additional information
return (initial_elements_finished,
initial_input,
initial_cell_state,
initial_cell_output,
initial_loop_state)
def loop_fn_transition(time, previous_output, previous_state, previous_loop_state):
def get_next_input():
output_logits = tf.add(tf.matmul(previous_output, W), b)
#Returns the index with the largest value across axes of a tensor.
prediction = tf.argmax(output_logits, axis=1)
#embed prediction for the next input
next_input = tf.nn.embedding_lookup(embeddings, prediction)
return next_input

elements_finished = (time >= decoder_lengths) # this operation produces boolean tensor of [batch_size]
# defining if corresponding sequence has ended
#Computes the "logical and" of elements across dimensions of a tensor.
finished = tf.reduce_all(elements_finished) # -> boolean scalar
#Return either fn1() or fn2() based on the boolean predicate pred.
input = tf.cond(finished, lambda: pad_step_embedded, get_next_input)
#set previous to current
state = previous_state
output = previous_output
loop_state = None
return (elements_finished,
input,
state,
output,
loop_state)
def loop_fn(time, previous_output, previous_state, previous_loop_state):
if previous_state is None: # time == 0
assert previous_output is None and previous_state is None
return loop_fn_initial()
else:
return loop_fn_transition(time, previous_output, previous_state, previous_loop_state)
decoder_outputs_ta, decoder_final_state, _ = tf.nn.raw_rnn(decoder_cell, loop_fn)
decoder_outputs = decoder_outputs_ta.stack()
decoder_max_steps, decoder_batch_size, decoder_dim = tf.unstack(tf.shape(decoder_outputs))
#flettened output tensor
decoder_outputs_flat = tf.reshape(decoder_outputs, (-1, decoder_dim))
#pass flattened tensor through decoder
decoder_logits_flat = tf.add(tf.matmul(decoder_outputs_flat, W), b)
#prediction vals
decoder_logits = tf.reshape(decoder_logits_flat, (decoder_max_steps, decoder_batch_size, vocab_size))
decoder_prediction = tf.argmax(decoder_logits, 2)

 

5£©Optimizer

stepwise_cross_entropy = tf.nn.softmax_cross_entropy_with_logits(
labels=tf.one_hot(decoder_targets, depth=vocab_size, dtype=tf.float32),
logits=decoder_logits,
) #loss function
loss = tf.reduce_mean(stepwise_cross_entropy)
#train it
train_op = tf.train.AdamOptimizer().minimize(loss)
sess.run(tf.global_variables_initializer())

6£©Training on the toy task

batch_size = 100
batches = helpers.random_sequences(length_from=3, length_to=8,
vocab_lower=2, vocab_upper=10,
batch_size=batch_size)
print('head of the batch:')
for seq in next(batches)[:10]:
print(seq)
def next_feed():
batch = next(batches)
encoder_inputs_, encoder_input_lengths_ = helpers.batch(batch)
decoder_targets_, _ = helpers.batch(
[(sequence) + [EOS] + [PAD] * 2 for sequence in batch]
)
return {
encoder_inputs: encoder_inputs_,
encoder_inputs_length: encoder_input_lengths_,
decoder_targets: decoder_targets_,
}
loss_track = []
max_batches = 3001
batches_in_epoch = 1000
try:
for batch in range(max_batches):
fd = next_feed()
_, l = sess.run([train_op, loss], fd)
loss_track.append(l)
if batch == 0 or batch % batches_in_epoch == 0:
print('batch {}'.format(batch))
print(' minibatch loss: {}'.format(sess.run(loss, fd)))
predict_ = sess.run(decoder_prediction, fd)
for i, (inp, pred) in enumerate(zip(fd[encoder_inputs].T, predict_.T)):
print(' sample {}:'.format(i + 1))
print(' input > {}'.format(inp))
print('predicted > {}'.format(pred))
if i >= 2:
break
print()
except KeyboardInterrupt:
print(' training interrupted')

ÔÚ30000´Îµü´úºó£¬loss Öµ½µµ½ÁË0.0869¡£

´ó¼Ò¿ÉÒÔ³¢ÊÔһϣ¬ÐèÒªÔ´ÂëµÄ»°£¬ÎÒÉÔºó·ÅÔÚ github ÉÏ¡£

   
3051 ´Îä¯ÀÀ       28
Ïà¹ØÎÄÕÂ

»ùÓÚͼ¾í»ýÍøÂçµÄͼÉî¶Èѧϰ
×Ô¶¯¼ÝÊ»ÖеÄ3DÄ¿±ê¼ì²â
¹¤Òµ»úÆ÷ÈË¿ØÖÆÏµÍ³¼Ü¹¹½éÉÜ
ÏîĿʵս£ºÈçºÎ¹¹½¨ÖªÊ¶Í¼Æ×
 
Ïà¹ØÎĵµ

5GÈ˹¤ÖÇÄÜÎïÁªÍøµÄµäÐÍÓ¦ÓÃ
Éî¶ÈѧϰÔÚ×Ô¶¯¼ÝÊ»ÖеÄÓ¦ÓÃ
ͼÉñ¾­ÍøÂçÔÚ½»²æÑ§¿ÆÁìÓòµÄÓ¦ÓÃÑо¿
ÎÞÈË»úϵͳԭÀí
Ïà¹Ø¿Î³Ì

È˹¤ÖÇÄÜ¡¢»úÆ÷ѧϰ&TensorFlow
»úÆ÷ÈËÈí¼þ¿ª·¢¼¼Êõ
È˹¤ÖÇÄÜ£¬»úÆ÷ѧϰºÍÉî¶Èѧϰ
ͼÏñ´¦ÀíËã·¨·½·¨Óëʵ¼ù