Äú¿ÉÒÔ¾èÖú£¬Ö§³ÖÎÒÃǵĹ«ÒæÊÂÒµ¡£

1Ôª 10Ôª 50Ôª





ÈÏÖ¤Â룺  ÑéÖ¤Âë,¿´²»Çå³þ?Çëµã»÷Ë¢ÐÂÑéÖ¤Âë ±ØÌî



  ÇóÖª ÎÄÕ ÎÄ¿â Lib ÊÓÆµ iPerson ¿Î³Ì ÈÏÖ¤ ×Éѯ ¹¤¾ß ½²×ù Model Center   Code  
»áÔ±   
   
 
     
   
 ¶©ÔÄ
  ¾èÖú
È¥ÄĶùÍøÏûÏ¢¶ÓÁÐÉè¼ÆÓëʵÏÖ
 
  2946  次浏览      27
 2018-12-17
 
±à¼­ÍƼö:

±¾ÎÄÀ´×ÔÓÚinfoq£¬±¾ÎÄ´ÓÈ¥ÄĶùÍøÊ¹ÓÃÏûÏ¢¶ÓÁÐËùÅöµ½µÄ¸÷ÖÖÎÊÌâ³ö·¢Ì½ÌÖÈ¥ÄĶùÍøÏûÏ¢¶ÓÁеÄÉè¼ÆÓëʵÏÖ¡£

±³¾°

2012 Äê£¬Ëæ×Ź«Ë¾ÒµÎñµÄ¿ìËÙÔö³¤£¬¹«Ë¾µ±Ê±µÄµ¥ÌåÓ¦Óüܹ¹ºÜÄÑÂú×ãÒµÎñ¿ìËÙÔö³¤µÄÒªÇó£¬ºÍÆäËûºÜ¶à¹«Ë¾Ò»Ñù£¬È¥ÄĶùÍøÒ²¿ªÊ¼ÁË·þÎñ»¯¸ÄÔ죬°´ÕÕÒµÎñµÈÒªËØ½«Ô­À´ÅÓ´óµÄµ¥ÌåÓ¦Óòð·Ö³É²»Í¬µÄ·þÎñ¡£ÄÇôÔÚ½øÐзþÎñ»¯¸ÄÔì֮ǰÊ×ÏȾÍÊÇÃæÁÙÊÇ·þÎñ»¯»ù´¡ÉèÊ©µÄ¼¼ÊõÑ¡ÐÍ£¬ÆäÖÐ×îÖØÒªµÄ¾ÍÊÇ·þÎñÖ®¼äµÄͨÐÅÖмä¼þ¡£Ò»°ãÀ´½²·þÎñÖ®¼äµÄͨÐÅ¿ÉÒÔ·ÖΪͬ²½·½Ê½ºÍÒì²½·½Ê½¡£Í¬²½µÄ·½Ê½µÄ´ú±í¾ÍÊÇ RPC£¬ÎÒÃÇÑ¡ÔñÁ˵±Ê±»¹ÔÚ»îÔ¾¿ª·¢µÄ Alibaba Dubbo(ÔÚÖ®ºó Dubbo ¹Ù·½Í£Ö¹ÁË¿ª·¢£¬µ«ÊÇ×î½ü Dubbo ÏîÄ¿ÓÖÖØÐÂÆô¶¯ÁË)¡£

Òì²½·½Ê½µÄ´ú±í¾ÍÊÇÏûÏ¢¶ÓÁÐ (Message Queue)£¬MQ ÔÚµ±Ê±Ò²Óкܶ࿪ԴµÄÑ¡Ôñ£ºRabbitMQ, ActiveMQ, Kafka, MetaQ(RocketMQ µÄǰÉí)¡£Ê×ÏÈÒòΪ¼¼ÊõÕ»ÎÒÃÇÅųýÁË erlang ¿ª·¢µÄ RabbitMQ£¬¶ø Kafka ÒÔ¼° Java °æ Kafka µÄ MetaQ ÔÚµ±Ê±»¹²¢²»³ÉÊìºÍÎȶ¨¡£¶ø ActiveMQ ÔÚÈ¥ÄĶùÍøÒѾ­ÓкܶàÓ¦ÓÃÔÚʹÓÃÁË£¬µ«ÊÇʹÓùý³ÌÖв¢²»Ò»·«·ç˳£ºå´»ú£¬ÏûÏ¢¶ªÊ§£¬ÏûÏ¢¶ÂÈûµÈÎÊÌâÂżû²»ÏÊ£¬¶øÇÒ ActiveMQ ·¢Õ¹¶àÄ꣬´úÂëÒ²·Ç³£¸´ÔÓ£¬ÏëÒªÍêÈ«°Ñ¿ØÒ²²»ÈÝÒ×£¬ËùÒÔÎÒÃǾö¶¨×Ô¼ºÔìÒ»¸öÂÖ×Ó¡£

ÎÊÌâ

Èç¹ûÎÒÃÇÒªÔÚ·þÎñ»¯²ð·ÖÖÐʹÓÃÏûÏ¢¶ÓÁУ¬ÄÇôÎÒÃÇÐèÒª½â¾öÄÄЩÎÊÌâÄØ£¿Ê×ÏÈÈ¥ÄĶùÍøÌṩÁËÂÃÓβúÆ·ÔÚÏßÔ¤¶©·þÎñ£¬ÄÇô¾ÍÉæ¼°µçÉ̽»Ò×£¬ÔÚµçÉ̽»Ò×ÖÐÎÒÃÇÈÏΪÊý¾ÝµÄÒ»ÖÂÐÔÊǷdz£¹Ø¼üµÄÒªËØ¡£ÄÇôÎÒÃÇµÄ MQ ±ØÐëÌṩһÖÂÐÔ±£Ö¤¡£

MQ ÌṩһÖÂÐÔ±£Ö¤ÓÖ·ÖΪÁ½¸ö·½Ã棺·¢ÏûϢʱÎÒÃÇÈçºÎÈ·±£ÒµÎñ²Ù×÷ºÍ·¢ÏûÏ¢ÊÇÒ»Öµģ¬Ò²¾ÍÊDz»ÄܳöÏÖÒµÎñ²Ù×÷³É¹¦ÏûϢδ·¢³ö»òÕßÏûÏ¢·¢³öÁ˵«ÊÇÒµÎñ²¢Ã»Óгɹ¦µÄÇé¿ö¡£¾ÙÀýÀ´Ëµ£¬Ö§¸¶·þÎñʹÓÃÏûϢ֪ͨ³öƱ·þÎñ£¬ÄÇô²»ÄܳöÏÖÖ§¸¶³É¹¦£¬µ«ÊÇÏûϢûÓз¢³ö£¬Õâ»áÒýÆðÓû§Í¶Ëߣ»µ«ÊÇÒ²²»ÄܳöÏÖÖ§¸¶Î´³É¹¦£¬µ«ÊÇÏûÏ¢·¢³ö×îºó³öƱÁË£¬Õâ»áµ¼Ö¹«Ë¾Ëðʧ¡£×ܽáһϾÍÊÇ·¢ÏûÏ¢ºÍÒµÎñÐèÒªÓÐÊÂÎñ±£Ö¤¡£Ò»ÖÂÐÔµÄÁíÒ»¶ËÊÇÏû·ÑÕߣ¬±ÈÈçÏû·ÑÕßÁÙʱ³ö´í»òÍøÂç¹ÊÕÏ£¬ÎÒÃÇÈçºÎÈ·±£ÏûÏ¢×îÖÕ±»´¦ÀíÁË¡£ÄÇôÎÒÃÇͨ¹ýÏû·Ñ ACK ºÍÖØÊÔÀ´´ïµ½×îÖÕÒ»ÖÂÐÔ¡£

¶ø·þÎñ¶ËÉè¼Æ£¬ÔÚµ±Ê±ÎÒÃÇ¿¼ÂǵIJ¢²»¶à£¬ÎÒÃÇÔ­¼Æ»®Ö»ÔÚ½»Ò×»·½ÚÖÐʹÓÃ×Ô¼º¿ª·¢µÄ MQ£¬¾­¹ý¶ÔδÀ´Êý¾ÝµÄÔ¤¹ÀÎÒÃÇÑ¡ÔñÊý¾Ý¿â×÷Ϊ MQ Server µÄÏûÏ¢´æ´¢£¬µ«ÊÇËæ×Å MQ ÔÚ¸÷ϵͳÖеĴóÁ¿Ó¦Ó㬾Ͳ»½öÏÞÓÚ½»Ò׳¡¾°ÁË£¬¶øÇÒ´ó¼Ò¶¼ÆÚÍûËùÓг¡¾°ÖÐֻʹÓÃÒ»Ì× API£¬ËùÒÔºóÀ´ÏûÏ¢Á¿Ñ¸ËÙÔö³¤£¬ÆÈʹÎÒÃǶԴ洢ģÐͽøÐÐÁËÖØÐÂÉè¼Æ¡£ÔÙ¼ÓÉÏÂÃÓβúÆ·Ô¤¶¨µÄÌØÕ÷£¬´ó²¿·ÖÔ¤¶¨¶¼ÊÇδÀ´Ä³¸öʱ¼äµãµÄ£¬Õâ¸öʱ¼ä¿É³¤¿É¶Ì£¬¶ÌµÄ»°¿ÉÄÜÊǼ¸¸öСʱ£¬³¤µÄ»°¿ÉÄÜÊǰëÄêÒÔÉÏ£¬ÄÇôÎÒÃǶÔÑÓʱÏûÏ¢µÄÐèÇóÒ²ºÜÇ¿ÁÒ¡£ÄÇôÕâÖÖÑÓʱʱ¼ä²»¹Ì¶¨µÄ·½Ê½Ò²¶Ô·þÎñ¶ËÉè¼ÆÌá³öÁËÌôÕ½¡£

½ÓÏÂÀ´±¾ÎÄ»á´Ó¿Í»§¶ËÒ»ÖÂÐÔÉè¼ÆºÍ·þÎñ¶Ë´æ´¢Ä£ÐÍÁ½·½Ãæ½øÐÐÌÖÂÛ¡£

¿Í»§¶ËÒ»ÖÂÐÔ

Ìáµ½Ò»ÖÂÐÔ£¬´ó¼Ò¿Ï¶¨¾ÍÏëµ½ÊÂÎñ£¬¶øÒ»Ìáµ½ÊÂÎñ£¬¿Ï¶¨¾ÍÏëµ½¹ØÏµÐÍÊý¾Ý¿â£¬ÄÇôÎÒÃÇÊDz»ÊÇ¿ÉÒÔ½èÖú¹ØÏµÐÍ DB Àï¾Ã¾­¿¼ÑéµÄÊÂÎñÀ´ÊµÏÖÕâ¸öÒ»ÖÂÐÔÄØ¡£ÎÒÃÇÒÔ MySQL ΪÀý£¬¶ÔÓÚ MySQL ÖÐͬһ¸öʵÀýÀïÃæµÄ db£¬Èç¹û¹²ÏíÏàͬµÄ Connection µÄ»°ÊÇ¿ÉÒÔÔÚͬһ¸öÊÂÎñÀïµÄ¡£ÒÔÏÂͼΪÀý£¬ÎÒÃÇÓÐÒ»¸ö MySQL ʵÀý¼àÌýÔÚ 3306 ¶Ë¿ÚÉÏ£¬È»ºó¸ÃʵÀýÉÏÓÐ A,B Á½¸ö DB£¬ÄÇôÏÂÃæµÄα´úÂëÊÇ¿ÉÒÔÅÜÔÚͬһ¸öÊÂÎñÀïµÄ£º

begin transaction
insert into A.tbl1(name, age) values('admin', 18);
insert into B.tbl2(num) values(20);
end transaction

ÓÐÁËÕâ²ã±£Ö¤£¬ÎÒÃǾͿÉÒÔ͸Ã÷µÄʵÏÖÒµÎñ²Ù×÷ºÍÏûÏ¢·¢ËÍÔÚͬһ¸öÊÂÎñÀïÁË£¬Ê×ÏÈÎÒÃÇÔÚ¹«Ë¾ËùÓÐ MySQL ʵÀýÀï³õʼ»¯³öÒ»¸ö message db£¬Õâ¸ö¿ÉÒԷŵ½×Ô¶¯»¯Á÷³ÌÖУ¬¶ÔÓ¦ÓÃ͸Ã÷¡£È»ºóÎÒÃÇÖ»Òª½«·¢ÏûÏ¢ÓëÒµÎñ²Ù×÷·Åµ½Í¬Ò»¸ö DB ÊÂÎñÀï¼´¿É¡£

ÎÒÃÇÀ´¿´Ò»¸öʵ¼ÊµÄ³¡¾°£ºÔÚÖ§¸¶³¡¾°ÖУ¬Ö§¸¶³É¹¦ºóÎÒÃÇÐèÒª²åÈëÒ»ÌõÖ§¸¶Á÷Ë®£¬²¢ÇÒ·¢ËÍÒ»ÌõÖ§¸¶Íê³ÉµÄÏûϢ֪ͨÆäËûϵͳ¡£ÄÇôÕâÀï²åÈëÖ§¸¶Á÷Ë®ºÍ·¢ËÍÏûÏ¢¾ÍÐèÒªÊÇÒ»Öµģ¬ÈκÎÒ»²½Ã»Óгɹ¦×îºó¶¼»áµ¼ÖÂÎÊÌâ¡£ÄÇô¾ÍÓÐÏÂÃæµÄ´úÂë:

@Transactional
public void pay(Order order){
PayTransaction t = buildPayTransaction(order);
payDao.append(t);
producer.sendMessage(buildMessage(t));
}

ÉÏÃæµÄ´úÂë¿ÉÒÔÓÃÏÂÃæµÄα´úÂë½âÊÍ£º

@Transactional
public void pay(Order order){
PayTransaction t = buildPayTransaction(order);
payDao.append(t);
//producer.sendMessage(buildMessage(t));
final Message message = buildMessage(t);
messageDao.insert(message);
// ÔÚÊÂÎñÌá½»ºóÖ´ÐÐ
triggerAfterTransactionCommit(()->{
messageClient.send(message);
messageDao.delete(message);
});
}

ʵ¼ÊÉÏÔÚ producer.sendMessage Ö´ÐеÄʱºò£¬ÏûÏ¢²¢Ã»ÓÐͨ¹ýÍøÂç·¢ËͳöÈ¥£¬¶ø½ö½öÊÇÍùÒµÎñ DB ͬһ¸öʵÀýÉϵÄÏûÏ¢¿â²åÈëÁËÒ»Ìõ¼Ç¼£¬È»ºó×¢²áÊÂÎñµÄ»Øµ÷£¬ÔÚÕâ¸öÊÂÎñÕæÕýÌá½»ºóÏûÏ¢²Å´ÓÍøÂç·¢ËͳöÈ¥£¬Õâ¸öʱºòÈç¹û·¢Ë͵½ server ³É¹¦µÄ»°ÏûÏ¢»á±»Á¢¼´É¾³ýµô¡£¶øÈç¹ûÏûÏ¢·¢ËÍʧ°ÜÔòÏûÏ¢¾ÍÁôÔÚÏûÏ¢¿âÀÕâ¸öʱºòÎÒÃÇ»áÓÐÒ»¸ö²¹³¥ÈÎÎñ»á½«ÕâЩÏûÏ¢´ÓÏûÏ¢¿âÀïÀ̳öÈ»ºóÖØÐ·¢ËÍ£¬Ö±µ½·¢Ëͳɹ¦¡£Õû¸öÁ÷³Ì¾ÍÈçÏÂͼËùʾ£º

1.begin tx ¿ªÆô±¾µØÊÂÎñ

2.do work Ö´ÐÐÒµÎñ²Ù×÷

3.insert message ÏòͬʵÀýÏûÏ¢¿â²åÈëÏûÏ¢

4.end tx ÊÂÎñÌá½»

5.send message ÍøÂçÏò server ·¢ËÍÏûÏ¢

6.reponse server »ØÓ¦ÏûÏ¢

7.delete message Èç¹û server »Ø¸´³É¹¦Ôòɾ³ýÏûÏ¢

8.scan messages ²¹³¥ÈÎÎñɨÃèδ·¢ËÍÏûÏ¢

9.send message ²¹³¥ÈÎÎñ²¹³¥ÏûÏ¢

10.delete messages ²¹³¥ÈÎÎñɾ³ý²¹³¥³É¹¦µÄÏûÏ¢

·þÎñ¶Ë´æ´¢Ä£ÐÍ

·ÖÎöÁ˿ͻ§¶ËΪÁËÒ»ÖÂÐÔËù×÷µÄÉè¼Æºó£¬ÎÒÃÇÔÙÀ´¿´¿´·þÎñ¶ËµÄ´æ´¢Éè¼Æ¡£ÎÒ»á´ÓÁ½¸ö·½ÃæÀ´½éÉÜ£ºÀàËÆ Kafka Ö®ÀàµÄ»ùÓÚ partition ´æ´¢Ä£ÐÍÓÐʲôÎÊÌ⣬ÒÔ¼°ÎÒÃÇÊÇÈçºÎ½â¾öµÄ¡£

»ùÓÚ partition ´æ´¢Ä£Ð͵ÄÎÊÌâ

ÎÒÃǶ¼ÖªµÀ Kafka ºÍ RocketMQ ¶¼ÊÇ»ùÓÚ partition µÄ´æ´¢Ä£ÐÍ£¬Ò²¾ÍÊÇÿ¸ö subject ·ÖΪһ¸ö»ò¶à¸ö partition£¬¶ø Server ÊÕµ½ÏûÏ¢ºó½«Æä·Ö·¢µ½Ä³¸ö partition ÉÏ£¬¶ø Consumer Ïû·ÑµÄʱºòÊÇÓë partition ¶ÔÓ¦µÄ¡£±ÈÈ磬ÎÒÃÇij¸ö subject a ·ÖÅäÁË 3 ¸ö partition(p1, p2, p3)£¬ÓÐ 3 ¸öÏû·ÑÕß (c1, c2, c3£©Ïû·Ñ¸ÃÏûÏ¢£¬Ôò»á½¨Á¢ c1 - p1, c2 - p2, c3 - p3 ÕâÑùµÄÏû·Ñ¹ØÏµ¡£

ÄÇôÈç¹ûÎÒÃÇµÄ consumer ¸öÊý±È partition ¸öÊý¶àÄØ£¿ÔòÓÐµÄ consumer »áÊÇ¿ÕÏеġ£

¶øÈç¹û partition ¸öÊý±È consumer ¸öÊý¶àÄØ£¿Ôò¿ÉÄÜ´æÔÚÓÐµÄ consumer Ïû·ÑµÄ partition ¸öÊý»á±ÈÆäËûµÄ consumer ¶àµÄÇé¿ö¡£

ÄÇôºÏÀíµÄ·ÖÅä²ßÂÔÖ»ÓÐÊÇ partition ¸öÊýÓë consumer ¸öÊý³É±¶Êý¹ØÏµ¡£

ÒÔÉ϶¼ÊÇ»ùÓÚ partition µÄ MQ Ëù´øÀ´µÄ¸ºÔؾùºâÎÊÌâ¡£ÒòΪÕâÖÖ¾²Ì¬µÄ°ó¶¨µÄ¹ØÏµ£¬»¹»áµ¼Ö Consumer À©ÈÝËõÈÝÂé·³¡£Ò²¾ÍÊÇʹÓà Kafka »òÕß RocketMQ ÕâÖÖ»ùÓÚ partition µÄÏûÏ¢¶ÓÁÐʱ£¬Èç¹ûÓöµ½´¦ÀíËٶȸú²»ÉÏʱ£¬¹â¼òµ¥µÄÔö¼Ó Consumer ²¢²»ÄÜÂíÉÏÌá¸ß´¦ÀíÄÜÁ¦£¬ÐèÒª¶ÔÓ¦µÄÔö¼Ó partition ¸öÊý£¬¶øÌرðÔÚ Kafka Àï partition ÊÇÒ»¸ö±È½ÏÖØµÄ×ÊÔ´£¬Ôö¼ÓÌ«¶à parition »¹ÐèÒª¿¼ÂÇÕû¸ö¼¯ÈºµÄ´¦ÀíÄÜÁ¦£»µ±¸ß·åÆÚ¹ýÁËÖ®ºó£¬Èç¹ûÏëËõÈÝ Consumer Ò²±È½ÏÂé·³£¬ÒòΪ partition Ö»ÄÜÔö¼Ó£¬²»ÄܼõÉÙ¡£

¸úÀ©ÈÝÏà¹ØµÄÁíÍâÒ»¸öÎÊÌâÊÇ£¬ÒѾ­¶Ñ»ýµÄÏûÏ¢ÊDz»ÄÜ¿ìËÙÏû·ÑµÄ¡£±ÈÈ翪ʼµÄʱºòÎÒÃÇ·ÖÅäÁË 2 ¸ö partition£¬ÓÉ 2 ¸ö Consumer À´Ïû·Ñ£¬µ«ÊÇͻȻ·¢ËÍ·½´óÁ¿·¢ËÍÏûÏ¢ (Õâ¸öÔÚÈÕ³£ÔËάÖо­³£Óöµ½)£¬µ¼ÖÂÏûÏ¢¿ìËٵĶѻý£¬Õâ¸öʱºòÎÒÃÇÈçºÎÄÜ¿ìËÙÀ©ÈÝÏû·ÑÕâЩÏûÏ¢ÄØ£¿ÆäʵÔö¼Ó partition ºÍ Consumer ¶¼ÊÇûÓÐÓõģ¬Ôö¼ÓµÄ Consumer °®ÄªÄÜÖú, ÒòΪ¶Ñ»ýµÄÄÇ 3 ¸ö partition Ö»ÄÜÓÉ 2 ¸ö Consumer À´Ïû·Ñ£¬Õâ¸öʱºòÄãÖ»ÄÜ×ÝÏòÀ©Õ¹£¬¶ø²»ÄܺáÏòÀ©Õ¹£¬¶øÎÒÃǶ¼ÖªµÀ×ÝÏòÀ©Õ¹ºÜ¶àʱºòÊDz»ÏÖʵµÄ£¬»òÕßÖ´ÐбȽÏÖØµÄÔÙ¾ùºâ²Ù×÷¡£

È¥ÄĶùÏûÏ¢¶ÓÁд洢ģÐÍ

ÉÏÃæÒѾ­½éÉÜÁË»ùÓÚ partition µÄ´æ´¢Ä£ÐÍ´æÔÚµÄÎÊÌ⣬ÄÇôÕâЩÎÊÌâ¶ÔÓÚÎÒÃÇÊÇÎÊÌâÂ𣿻òÕßÎÒÃǵij¡¾°ÊÇ·ñÄܿ˷þÕâЩÎÊÌâÄØ£¿

ÏÖÔÚÈ¥ÄĶùÍøµÄϵͳ¼Ü¹¹»ù±¾É϶¼ÊÇÏûÏ¢Çý¶¯µÄ£¬Ò²¾ÍÊǾø´ó¶àÊýÒµÎñÁ÷³Ì¶¼ÊÇ¿¿ÏûÏ¢À´Çý¶¯£¬ÄÇôÕâÑùµÄ¼Ü¹¹ÓÐÊ²Ã´ÌØÕ÷ÄØ£º

1.ÏûÏ¢Ö÷ÌâÌØ±ð¶à ÏÖÔÚÉú²úÉÏÒÑÓÐ 4W+ ÏûÏ¢Ö÷Ìâ¡£ÕâÊÇÒµÎñÖÐʹÓõÄÏûÏ¢ÓëÊý¾ÝÁ÷´¦ÀíÖÐʹÓõÄ×î´óµÄ²»Í¬£¬Êý¾ÝÁ÷ÖÐÒ»°ãÏûÏ¢Ö÷ÌâÉÙ£¬µ«ÊÇÿ¸öÏûÏ¢Ö÷ÌâµÄÍÌÍÂÁ¿¶¼¼«´ó¡£¶øÒµÎñÖеÄÏûÏ¢ÊÇÖ÷Ì⼫¶à£¬µ«ÊÇÓкܶàÖ÷ÌâËûµÄÁ¿ÊǼ«Ð¡µÄ¡£

2.ÏûÏ¢Ïû·ÑµÄÉȳö´ó Ò²¾ÍÊÇÒ»¸öÏûÏ¢Ö÷ÌâÓм¸Ê®¸öÉõÖÁÉϰٸö²»Í¬µÄÓ¦Óö©ÔÄÊǷdz£³£¼ûµÄ¡£ÒÔÈ¥ÄĶù¾Æµê¶©µ¥×´Ì¬±ä¸üµÄÏûϢΪÀý£¬Ä¿Ç°Óн«½ü 70 ¶à¸ö²»Í¬µÄϵͳÀ´¶©ÔÄÕâ¸öÏûÏ¢¡£

½áºÏÇ°Ãæ¶Ô»ùÓÚ partition µÄ´æ´¢Ä£Ð͵ÄÌÖÂÛ£¬ÎÒÃǾõµÃÕâÖִ洢ģÐͲ»Ì«ÈÝÒ×·ûºÏÎÒÃǵÄÐèÇó¡£

ËäÈ»ÎÒÃDz¢²»Ïë²ÉÓûùÓÚ partition µÄ´æ´¢Ä£ÐÍ£¬µ«ÊÇ Kafka ºÍ RocketMQ ÀïºÜ¶àÉè¼ÆÎÒÃÇ»¹ÊÇ¿ÉÒÔѧϰµÄ£º

1.˳Ðò append Îļþ£¬ÌṩºÜºÃµÄÐÔÄÜ

2.˳ÐòÏû·ÑÎļþ£¬Ê¹Óà offset ±íʾÏû·Ñ½ø¶È£¬²»ÓøøÃ¿¸öÏûÏ¢¼Ç¼Ïû·Ñ״̬£¬³É±¾¼«µÍ

3.½«ËùÓÐ subject µÄÏûÏ¢ºÏ²¢ÔÚÒ»Æð£¬¼õÉÙ parition ÊýÁ¿£¬µ¥Ò»¼¯Èº¿ÉÒÔÖ§³Å¸ü¶àµÄ subject(RocketMQ)

ÔÚÉè¼Æ QMQ µÄ´æ´¢Ä£ÐÍʱ£¬¾õµÃÕ⼸µãÊǷdz£ÖØÒªµÄ¡£ÄÇÈçºÎÔÚ²»Ê¹ÓûùÓÚ partition µÄÇé¿öÏ£¬ÓÖÄܵõ½ÕâÐ©ÌØÐÔÄØ£¿ÕýËùνÓÐǰ±²´óʦ˵£º¼ÆËã»úÖÐËùÓÐÎÊÌâ¶¼¿ÉÒÔͨ¹ýÌí¼ÓÒ»¸öÖмä²ãÀ´½â¾ö£¬Ò»¸öÖмä²ã½â¾ö²»ÁËÄǾÍÌí¼ÓÁ½¸ö¡£

ÎÒÃÇͨ¹ýÌí¼ÓÒ»²ãÀ­È¡µÄ log(pull log) À´¶¯Ì¬Ó³Éä consumer Óë partition µÄÂß¼­¹ØÏµ£¬ÕâÑù²»½ö½â¾öÁË consumer µÄ¶¯Ì¬À©ÈÝËõÈÝÎÊÌ⣬»¹¿ÉÒÔ¼ÌÐøÊ¹ÓÃÒ»¸ö offset ±íʾÏû·Ñ½ø¶È¡£¶ø pull log Óë consumer Ò»Ò»¶ÔÓ¦¡£

ÏÂͼÊÇ QMQ µÄ´æ´¢Ä£ÐÍ

ÏȽâÊÍÒ»ÏÂÉÏͼÖеÄÊý×ÖµÄÒâÒå¡£ÉÏͼÖз½¿òÉÏ·½µÄÊý×Ö£¬±íʾ¸Ã·½¿òÔÚ×Ô¼º log ÖÐµÄÆ«ÒÆ£¬¶ø·½¿òÄÚµÄÊý×ÖÊǸÃÏîµÄÄÚÈÝ¡£±ÈÈç message log ·½¿òÉÏ·½µÄÊý×Ö:3,6,9 ±íʾÕ⼸ÌõÏûÏ¢ÔÚ message log ÖÐµÄÆ«ÒÆ¡£¶ø consume log Öз½¿òÄÚµÄÊý×Ö 3,6,9,20 Õý¶ÔÓ¦×Å messzge log µÄÆ«ÒÆ£¬±íʾÕ⼸¸öλÖÃÉϵÄÏûÏ¢¶¼ÊÇ topic1 µÄÏûÏ¢£¬consume log ·½¿òÉÏ·½µÄ 1,2,3,4 ±íʾÕ⼸¸ö·½¿òÔÚ consume log ÖеÄÂß¼­Æ«ÒÆ¡£ÏÂÃæµÄ pull log ·½¿òÄÚµÄÄÚÈݶÔÓ¦×Å consume log µÄÂß¼­Æ«ÒÆ£¬¶ø pull log ·½¿òÍâµÄÊý×Ö±íʾ pull log µÄÂß¼­Æ«ÒÆ¡£

ÕâÑù´æ´¢ÖÐÓÐÈýÖÖÖØÒªµÄ log:

1.message log ËùÓÐ subject µÄÏûÏ¢½øÈë¸Ã log£¬ÏûÏ¢µÄÖ÷´æ´¢

2.consume log consume log ´æ´¢µÄÊÇ message log µÄË÷ÒýÐÅÏ¢

3.pull log ÿ¸ö consumer À­È¡ÏûÏ¢µÄʱºò»á²úÉú pull log£¬pull log ¼Ç¼µÄÊÇÀ­È¡µÄÏûÏ¢ÔÚ consume log ÖÐµÄ sequence

ÄÇôÏû·ÑÕ߾ͿÉÒÔʹÓà pull log É쵀 sequence À´±íʾÏû·Ñ½ø¶È£¬ÕâÑùÒ»À´ÎÒÃǾͽâñîÁË consumer Óë partition Ö®¼äµÄñîºÏ¹ØÏµ£¬Á½Õß¿ÉÒÔÈÎÒâµÄÀ©Õ¹¡£

ÑÓ³ÙÏûÏ¢¶ÓÁд洢ģÐÍ

³ýÁ˶ÔʵʱÏûÏ¢µÄÖ§³Ö£¬QMQ »¹Ö§³ÖÁËÈÎÒâʱ¼äµÄÑÓʱÏûÏ¢, ÔÚ¿ªÔ´°æ±¾µÄ RocektMQ ÀïÌṩÁ˶àÖ̶ֹ¨ÑÓ³Ù level µÄÑÓʱÏûÏ¢Ö§³Ö£¬Ò²¾ÍÊÇÄã¿ÉÒÔ·¢Ëͼ¸¸ö¹Ì¶¨µÄÑÓʱʱ¼äµÄÑÓʱÏûÏ¢£¬±ÈÈçÑÓʱ 10s, 30s¡­£¬µ«ÊÇ»ùÓÚÎÒÃÇÏÖÓеÄÒµÎñÌØÕ÷£¬ÎÒÃǾõµÃÕâÖÖ²»Í¬ÑÓʱ level µÄÑÓʱÏûÏ¢²¢²»ÄÜÂú×ãÎÒÃǵÄÐèÒª£¬ÎÒÃǸü¶àµÄÊÇÐèÒªÈÎÒâʱ¼äÑÓʱ¡£ÔÚ OTA ³¡¾°ÖУ¬¿ÍÈ˾­³£ÊÇÔ¤¶©Î´À´Ä³¸öʱ¿ÌµÄ¾Æµê»òÕß»úƱ£¬Õâ¸öʱ¼äÊDz»¹Ì¶¨µÄ£¬ÎÒÃÇÎÞ·¨Ê¹Óü¸¸ö¹Ì¶¨µÄÑÓʱ level À´ÊµÏÖÕâ¸ö³¡¾°¡£

ÎÒÃǵÄÑÓʱ / ¶¨Ê±ÏûϢʹÓõÄÊÇÁ½²ã hash wheel timer À´ÊµÏֵġ£µÚÒ»²ãλÓÚ´ÅÅÌÉÏ£¬Ã¿¸öСʱΪһ¸ö¿Ì¶È£¬Ã¿¸ö¿Ì¶È»áÉú³ÉÒ»¸öÈÕÖ¾Îļþ£¬¸ù¾ÝÒµÎñÌØÕ÷£¬ÎÒÃǾõµÃÖ§³ÖÁ½ÄêÄÚÈÎÒâʱ¼äÑÓʱ¾Í¹»ÁË£¬ÄÇô×î¶à»áÉú³É 2 * 366 * 24 = 17568 ¸öÎļþ¡£µÚ¶þ²ãÔÚÄÚ´æÖУ¬µ±ÏûÏ¢µÄͶµÝʱ¼ä¼´½«µ½À´µÄʱºò£¬»á½«Õâ¸öСʱµÄÏûÏ¢Ë÷Òý (Æ«ÒÆÁ¿£¬Í¶µÝʱ¼äµÈ) ´Ó´ÅÅÌÎļþ¼ÓÔØµ½ÄÚ´æÖÐµÄ hash wheel timer ÉÏ¡£

ÔÚÑÓʱ / ¶¨Ê±ÏûÏ¢ÀïÒ²´æÔÚÈýÖÖ log:

1.message log ºÍʵʱÏûÏ¢ÀïµÄ message log ÀàËÆ£¬ÊÕµ½ÏûÏ¢ºó append µ½¸Ã log£¬append ³É¹¦ºó¾ÍÁ¢¼´·µ»Ø

2.schedule log °´ÕÕͶµÝʱ¼ä×éÖ¯£¬Ã¿¸öСʱһ¸ö¡£¸Ã log ÊǻطŠmessage log ºó¸ù¾ÝÑÓʱʱ¼ä·ÅÖöÔÓ¦µÄ log ÉÏ£¬ÕâÊÇÉÏÃæÃèÊöµÄÁ½²ã hash wheel timer µÄµÚÒ»²ã£¬Î»ÓÚ´ÅÅÌÉÏ

3.dispatch log ÑÓʱ / ¶¨Ê±ÏûϢͶµÝºóдÈ룬Ö÷ÒªÓÃÓÚÔÚÓ¦ÓÃÖØÆôºóÄܹ»È·¶¨ÄÄЩÏûÏ¢ÒѾ­Í¶µÝ

×ܽá

ÏûÏ¢¶ÓÁÐÊǹ¹½¨Î¢·þÎñ¼Ü¹¹ºÜ¹Ø¼üµÄ»ù´¡ÉèÊ©£¬±¾Îĸù¾ÝÎÒÃÇ×Ô¼ºµÄʵ¼ÊʹÓó¡¾°£¬²¢ÇÒ¶ÔĿǰÊÐÃæÉÏһЩ»îÔ¾µÄ¿ªÔ´²úÆ·½øÐжԱȣ¬Ìá³öÈ¥ÄĶùÍøµÄÏûÏ¢¶ÓÁеÄÉè¼ÆÓëʵÏÖ¡£±¾ÎÄÖ»ÊÇ´Óºê¹Û¼Ü¹¹ÉÏ×ö³öһЩ̽ÌÖ£¬¾ßÌåµÄʵÏÖϸ½ÚÒ²ÓкܶàÓÐȤµÄµØ·½£¬Ä¿Ç°È¥ÄĶùÍøµÄÏûÏ¢¶ÓÁÐ QMQ Ò²ÒѾ­ÔÚGitHubÉÏ¿ªÔ´£¬»¶Ó­´ó¼ÒÊÔÓ㬻¶Ó­ PR£¬Ð»Ð»¡£

 

   
2946 ´Îä¯ÀÀ       27
Ïà¹ØÎÄÕÂ

ÆóÒµ¼Ü¹¹¡¢TOGAFÓëArchiMate¸ÅÀÀ
¼Ü¹¹Ê¦Ö®Â·-ÈçºÎ×öºÃÒµÎñ½¨Ä££¿
´óÐÍÍøÕ¾µçÉÌÍøÕ¾¼Ü¹¹°¸ÀýºÍ¼¼Êõ¼Ü¹¹µÄʾÀý
ÍêÕûµÄArchimateÊÓµãÖ¸ÄÏ£¨°üÀ¨Ê¾Àý£©
Ïà¹ØÎĵµ

Êý¾ÝÖÐ̨¼¼Êõ¼Ü¹¹·½·¨ÂÛÓëʵ¼ù
ÊÊÓÃArchiMate¡¢EA ºÍ iSpace½øÐÐÆóÒµ¼Ü¹¹½¨Ä£
ZachmanÆóÒµ¼Ü¹¹¿ò¼Ü¼ò½é
ÆóÒµ¼Ü¹¹ÈÃSOAÂ䵨
Ïà¹Ø¿Î³Ì

ÔÆÆ½Ì¨Óë΢·þÎñ¼Ü¹¹Éè¼Æ
ÖÐ̨սÂÔ¡¢ÖÐ̨½¨ÉèÓëÊý×ÖÉÌÒµ
ÒÚ¼¶Óû§¸ß²¢·¢¡¢¸ß¿ÉÓÃϵͳ¼Ü¹¹
¸ß¿ÉÓ÷ֲ¼Ê½¼Ü¹¹Éè¼ÆÓëʵ¼ù