±à¼ÍƼö: |
±¾ÎÄÀ´×ÔÓÚ¸öÈ˲©¿Í£¬±¾ÎÄΪÉî¶Èѧϰ¿Î³Ì±Ê¼Ç£¬Í¨¹ýÁ÷³Ìͼ£¬Ïêϸ½éÉÜÁËÉî¶ÈѧϰģÐ͵Ļù±¾½á¹¹£¬Ï£Íû¶ÔÄúµÄѧϰÓÐËù°ïÖú¡£
|
|
Éî¶ÈѧϰµÄ»ù±¾²½Ö裺¶¨ÒåÄ£ÐÍ-->¶¨ÒåËðʧº¯Êý-->ÕÒµ½ÓÅ»¯·½·¨

¿Î³Ì´ó¸Ù

1¡¢ÊìϤ¶¨Òå·ûºÅ£¨ÂÔ¹ý£©

2¡¢RNN

¼òµ¥µØËµ¾ÍÊÇ


RNN¿ÉÒÔ¿´×öÊÇÒ»¸öfunction·´¸´µü´ú¡£
Ϊʲô²»ÓÃfeedFord network£¬ÒòΪÎÒÃÇÊäÈëµÄsequence¿ÉÄÜ»á±È½Ï³¤£¬ÕâÑùµÄ»°feedFord
network¿ÉÄܾͻá²ÎÊýºÜ¶à£¬ÈÝÒ×µ¼Ö¹ýÄâºÏ¡£
RNNµÄÒ»¸öºÃ´¦ÊDzÎÊýÉÙ£¬ÓпÉÄܱȽÏÄÑtrain£¬µ«ÊÇÄãÒ»µ©ÔÚtraining dataÉÏ»ñµÃ±È½ÏºÃµÄ½á¹û£¬ÔÚtesting
dataÉϵÄЧ¹ûͨ³£Ò²²»»á²î¡£

Deep RNNҪעÒâµ½Ò»µãÊÇ f1µÄÊä³öºÍf2µÄÊäÈëµÄά¶È±ØÐëÒ»Ö£¬ÕâÑù²ÅºÃÆ´½Óµ½Ò»Æð¡£

Ë«ÏòRNN¡£¾ÍÊÇÒª¼ÓÈëÒ»¸öf3£¬½«f1µÄÊä³öºÍf2µÄÊä³öÕûºÏÔÚÒ»Æð¡£ÖÁÓÚf1ºÍf2²»±ØÒ»Ñù£¬Äã¿ÉÒÔËæ±ãÉè¼Æ¡£
ÏÂÃæ½éÉÜһЩ¾ßÌåµÄÓ¦ÓÃ

Õâ¸öÊÇÖ±½ÓÓÃsqe2sqeµÄÓïÒôʶ±ð¡£µÚÒ»²ãÊÇbi-rnn£¬µÚ¶þ²ãÊǽ«Èô¸ÉµÚÒ»²ãµÄÊä³ö£¨ºÏÆðÀ´£©×÷ΪÊä³ö£¬Ò²ÊÇbi-rnn£¬ºóÃæ¶¼ÕâÑùÏÂÈ¥¡£ÕâÖֽṻ±Èµ¥´¿µÄÉî¶Èbi-rnn¸üÈÝÒ×ѵÁ·¡£µÚ¶þ²ãblockËäÈ»ÐèÒª´¦ÀíÈô¸ÉµÚÒ»²ãblockµÄÊä³ö£¬µ«ÊÇÕâÖÖ·¨ÓÐÀûÓÚ×ö²¢ÐмÓËÙ¡£RNNºÜÄÑ×ö²¢ÐУ¬ÒòΪÏÂÒ»¸ö½Úµã±ØÐëµÃµÈµÚÒ»¸ö½ÚµãµÄÊä³ö²ÅÄܽøÐУ¬¶øpyramidal
rnnÀï¸ß²ãµÄblock£¬ÐòÁбä¶ÌÁË£¬Ã¿¸öblockÖÐËäÈ»ÔËËãÁ¿½Ï´ó£¬µ«ÊÇ¿ÉÒÔÓò¢ÐÐÔËËã¼ÓËÙ¡£

×î¼òµ¥µÄRNNÔËËãÈçÉÏͼ¡£

C±ä»¯½ÏÂý£¬¿ÉÒÔ¼Çסʱ¼ä±È½Ï¾ÃµÄÐÅÏ¢¡£

Ò²¿ÉÒÔ°ÑCÄÃÏÂÀ´Ò»¿éÔËËã¡£Ò»°ãWÖжÔÓ¦ÓÚCµÄ²ÎÊý¶¼ÊǶԽǾØÕó¡£

È»ºóÈçÏÂÔËËã

z_iÊÇÊäÈëÃÅ£¬¾ö¶¨ÁËzµÄÄÄЩÐÅÏ¢Á÷È룬z_fÊÇÒÅÍüÃÅ£¬¾ö¶¨c_t-1µÄÄÄЩÐÅÏ¢ÒÅÍü¡£

È»ºó¾Í¿ÉÒÔ·´¸´Õâ¸ö²½Öè¡££¬ÒÔÉÏ´Ö¼ýÍ·±íʾ³ËÒÔÒ»¸ö¾ØÕó£¬Ï¸¼ýÍ·±íʾɶҲû×ö¡£

ʹÓÃÀý×Ó£¨ÏÂÃæÊÇһЩÁ¶µ¤Ðĵã¬ÏêϸÁ˽âÐèÒªÈÏÕæÈ¥¶ÁÔÂÛÎÄ£©

¼ò»¯°æµÄÓïÒôʶ±ðÈÎÎñ£¬Ã¿Ò»¶ÎÉùÒôʶ±ð³ÉÒ»¸öÒòËØ¼´¿É£¬¶à·ÖÀàÎÊÌâ¡£
ʹÓõ¥ÏîRNNÓÐÒ»¸öÓÐЧµÄtrickÊÇ Target Delay
ÔÚÔʼÉùÒôÐźţ¨frame£©ÓҶ˲¹Èô¸É¸öÁ㣨ͼÖÐÊÇ3¸ö£©£¬ÔÚʶ±ðʱ±êÇ©ÏòºóÑÓºóÈô¸Éλ¿ªÊ¼×öʶ±ð¡£

ÒÔÉÏÈÎÎñÔÚÈô¸ÉÖÖ·½·¨µÄʵÑéЧ¹û

×ÝÖáÊÇ·ÖÀàÕýÈ·ÂÊ£¬ºáÖáWindow sizeÊÇ£¨Èç¹ûÊÇMLP£©ÊÇÖ¸£ºÈç¹ûinputÊÇÒ»ÅÅframe£¨Ã¿¸öframeµÄÉùÒôÐźſÉÄÜÖ»ÓÐ0.01Ã룩£¬Ã¿´ÎÅжÏʱ½«Ç°ºóÈô¸ÉframeÆ´½ÓÆðÀ´×÷Ϊһ¸ö±È½Ï³¤µÄvectorÈ¥ÅжÏÖмäÒ»¸öframeµÄ±êÇ©ÊÇʲô¡£Í¼ÖÐMLP·½·¨µÄWindow
sizeΪ10ʱЧ¹û×îºÃ¡£
ºáÖátarget delayÊÇ£¨Èç¹ûÊÇRNN»òLSTM£©ÊÇÖ¸labelÍùÓÒshift¼¸¸öframe¡£

ÉÏÃæÕâ¸öͼµÄºáÖáÊÇѵÁ·µÄÂÖÊý£¬´ÓͼÖпÉÒÔ¿´³öLSTMµÄѵÁ·ËٶȱÈRNN¸ü¿ì
ÏÂÃæÕâ¸öͼÊÇLSTMµÄ¸÷Öּܹ¹ÅܵĽá¹û¡£ÂÛÎÄÃû×Ö½ÐLSTM ÂþÓΡ£³¢ÊÔÁ˸÷ÖÖ¸÷ÑùµÄLSTM¼Ü¹¹ºÍ²ÎÊý¡£


ºáÖáÊÇ´íÎóÂÊ£¬×ÝÖáÊÇѵÁ·Ê±¼ä¡£²»Í¬µÄÑÕÉ«´ú±í²»Í¬µÄLSTM¼Ü¹¹£¬ÏàͬµÄÑÕÉ«²»Í¬µÄµã´ú±íͬÑùµÄ¼Ü¹¹£¬²»Í¬µÄ²ÎÊý¡£

ºóÃæÈý¸öMUT1¡¢MUT2¡¢MUT3ʹÓûùÒòÑÝË㷨ȥѰÕÒ×î¼ÑµÄRNN½á¹¹£¬Ê¹ÓÃLSTMºÍGRUµÄ½á¹¹Îª³õʼ»ùÒò£¬Ê¹Óò»Í¬µÄ×éºÏÈ¥ÕÒ¡£

»¹ÓÐÒ»¸ö±È½Ï¾µäµÄRNN½á¹¹£¬ÆÕͨµÄRNNÊÜÊäÈ볤¶ÈµÄÏÞÖÆ¡£ÊäÈë¹ý³¤Ê±£¬ÄÚ´æ»òÏÔ´æ·Å²»Ï¡£¶øÏÂÃæÕâ¸ö½á¹¹Stack
RNNÊäÈë¿ÉÒÔÎÞÏÞ³¤£¬ËüÊǽ«ÊäÈëÿN¸öÈ¡³öÀ´·Åµ½Ò»¸öfunctionÀïÊä³öÐèÒª´æ´¢µÄinformationͬʱÊä³öpush¡¢pop¡¢nothing²Ù×÷¡£¾ßÌåÏê¼ûÂÛÎİɡ£

|