Äú¿ÉÒÔ¾èÖú£¬Ö§³ÖÎÒÃǵĹ«ÒæÊÂÒµ¡£

1Ôª 10Ôª 50Ôª





ÈÏÖ¤Â룺  ÑéÖ¤Âë,¿´²»Çå³þ?Çëµã»÷Ë¢ÐÂÑéÖ¤Âë ±ØÌî



  ÇóÖª ÎÄÕ ÎÄ¿â Lib ÊÓÆµ iPerson ¿Î³Ì ÈÏÖ¤ ×Éѯ ¹¤¾ß ½²×ù Model Center   Code  
»áÔ±   
   
 
     
   
 ¶©ÔÄ
  ¾èÖú
TiDB-DM ¼Ü¹¹Éè¼ÆÓëʵÏÖÔ­Àí
 
×÷Õߣº ÕÅѧ³Ì
  1841  次浏览      29
2019-12-23
 
±à¼­ÍƼö:
±¾ÎĽéÉÜÈçºÎ´Ó MySQL/MariaDB Ç¨ÒÆµ½ TiDB£¬ÒÔ¼°½« TiDB ×÷Ϊ MySQL/MariaDB µÄ´Ó¿âµÄʵÏÖÔ­Àí£¬Ï£Íû¶ÔÄúµÄѧϰÓÐËù°ïÖú¡£
±¾ÎÄÀ´×ÔÓÚ¼òÊ飬ÓÉ»ðÁú¹ûÈí¼þAlice±à¼­¡¢ÍƼö¡£

¼ò½é

TiDB-DM£¨Data Migration£©ÊÇÓÃÓÚ½«Êý¾Ý´Ó MySQL/MariaDB Ç¨ÒÆµ½ TiDB µÄ¹¤¾ß¡£¸Ã¹¤¾ß¼ÈÖ§³ÖÒÔÈ«Á¿±¸·ÝÎļþµÄ·½Ê½½« MySQL/MariaDB µÄÊý¾Ýµ¼Èëµ½ TiDB£¬Ò²Ö§³Öͨ¹ý½âÎöÖ´ÐÐ MySQL/MariaDB binlog µÄ·½Ê½½«Êý¾ÝÔöÁ¿Í¬²½µ½ TiDB¡£ÌØ±ðµØ£¬¶ÔÓÚÓжà¸ö MySQL/MariaDB ʵÀýµÄ·Ö¿â·Ö±íÐèÒªºÏ²¢ºóͬ²½µ½Í¬Ò»¸ö TiDB ¼¯ÈºµÄ³¡¾°£¬DM ÌṩÁËÁ¼ºÃµÄÖ§³Ö¡£

DM ÊǼ¯ÈºÄ£Ê½µÄ£¬ÆäÖ÷ÒªÓÉ DM-master¡¢DM-worker Óë DM-ctl Èý¸ö×é¼þ×é³É£¬Äܹ»ÒÔ¶à¶Ô¶àµÄ·½Ê½½«¶à¸öÉÏÓÎ MySQL ʵÀýµÄÊý¾Ýͬ²½µ½¶à¸öÏÂÓÎ TiDB ¼¯Èº£¬Æä¼Ü¹¹Í¼ÈçÏ£º

DM-master£º¹ÜÀíÕû¸ö DM ¼¯Èº£¬Î¬»¤¼¯ÈºµÄÍØÆËÐÅÏ¢£¬¼à¿Ø¸÷¸ö DM-worker ʵÀýµÄÔËÐÐ״̬£»½øÐÐÊý¾Ýͬ²½ÈÎÎñµÄ²ð½âÓë·Ö·¢£¬¼à¿ØÊý¾Ýͬ²½ÈÎÎñµÄÖ´ÐÐ״̬£»ÔÚ½øÐкϿâºÏ±íµÄÔöÁ¿Êý¾Ýͬ²½Ê±£¬Ð­µ÷¸÷ DM-worker ÉÏ DDL µÄÖ´ÐлòÌø¹ý£»ÌṩÊý¾Ýͬ²½ÈÎÎñ¹ÜÀíµÄͳһÈë¿Ú¡£

DM-worker£ºÓëÉÏÓÎ MySQL ʵÀýÒ»Ò»¶ÔÓ¦£¬Ö´ÐоßÌåµÄÈ«Á¿¡¢ÔöÁ¿Êý¾Ýͬ²½ÈÎÎñ£»½«ÉÏÓÎ MySQL µÄ binlog À­È¡µ½±¾µØ²¢³Ö¾Ã»¯±£´æ£»¸ù¾Ý¶¨ÒåµÄÊý¾Ýͬ²½ÈÎÎñ£¬½«ÉÏÓÎ MySQL Êý¾ÝÈ«Á¿µ¼³ö³É SQL Îļþºóµ¼Èëµ½ÏÂÓÎ TiDB£¬»ò½âÎö±¾µØ³Ö¾Ã»¯µÄ binlog ºóÔöÁ¿Í¬²½µ½ÏÂÓÎ TiDB£»±àÅÅ DM-master ²ð½âºóµÄÊý¾Ýͬ²½×ÓÈÎÎñ£¬¼à¿Ø×ÓÈÎÎñµÄÔËÐÐ״̬¡£

DM-ctl£ºÃüÁîÐн»»¥¹¤¾ß£¬Í¨¹ýÁ¬½Óµ½ DM-master ºó£¬Ö´ÐÐ DM ¼¯ÈºµÄ¹ÜÀíÓëÊý¾Ýͬ²½ÈÎÎñµÄ¹ÜÀí¡£

ʵÏÖÔ­Àí

Êý¾ÝÇ¨ÒÆÁ÷³Ì

µ¥¸ö DM ¼¯Èº¿ÉÒÔͬʱÔËÐжà¸öÊý¾Ýͬ²½ÈÎÎñ£»¶ÔÓÚÿһ¸öͬ²½ÈÎÎñ£¬¿ÉÒÔ²ð½âΪ¶à¸ö×ÓÈÎÎñͬʱÓɶà¸ö DM-worker ½Úµã³Ðµ££¬ÆäÖÐÿ¸ö DM-worker ½Úµã¸ºÔðͬ²½À´×Ô¶ÔÓ¦µÄÉÏÓÎ MySQL ʵÀýµÄÊý¾Ý¡£¶ÔÓÚµ¥¸ö DM-worker ½ÚµãÉϵĵ¥¸öÊý¾Ýͬ²½×ÓÈÎÎñ£¬ÆäÊý¾ÝÇ¨ÒÆÁ÷³ÌÈçÏ£¬ÆäÖÐÉϲ¿µÄÊý¾ÝÁ÷ÏòΪȫÁ¿Êý¾ÝÇ¨ÒÆ¡¢Ï²¿µÄÊý¾ÝÁ÷ÏòΪÔöÁ¿Êý¾Ýͬ²½£º

ÔÚÿ¸ö DM-worker ½ÚµãÄÚ²¿£¬¶ÔÓÚÌØ¶¨µÄÊý¾Ýͬ²½×ÓÈÎÎñ£¬Ö÷ÒªÓÉ dumper¡¢loader¡¢relay Óë syncer£¨binlog replication£©µÈÊý¾Ýͬ²½´¦Àíµ¥ÔªÖ´ÐоßÌåµÄÊý¾Ýͬ²½²Ù×÷¡£

¶ÔÓÚÈ«Á¿Êý¾ÝÇ¨ÒÆ£¬DM Ê×ÏÈʹÓà dumper µ¥Ôª´ÓÉÏÓÎ MySQL Öн«±í½á¹¹ÓëÊý¾Ýµ¼³ö³É SQL Îļþ£»È»ºóʹÓà loader µ¥Ôª¶ÁÈ¡ÕâЩ SQL Îļþ²¢Í¬²½µ½ÏÂÓÎ TiDB¡£

¶ÔÓÚÔöÁ¿Êý¾Ýͬ²½£¬Ê×ÏÈʹÓà relay µ¥Ôª×÷Ϊ slave Á¬½Óµ½ÉÏÓÎ MySQL ²¢À­È¡ binlog Êý¾Ýºó×÷Ϊ relay log ³Ö¾Ã»¯´æ´¢ÔÚ±¾µØ£¬È»ºóʹÓà syncer µ¥Ôª¶ÁÈ¡ÕâЩ relay log ²¢½âÎö¹¹Ôì³É SQL Óï¾äºóͬ²½µ½ÏÂÓÎ TiDB¡£Õâ¸öÔöÁ¿Í¬²½µÄ¹ý³ÌÓë MySQL µÄÖ÷´Ó¸´ÖÆÀàËÆ£¬Ö÷񻂿±ðÔÚÓÚÔÚ DM ÖУ¬±¾µØ³Ö¾Ã»¯µÄ relay log ¿ÉÒÔͬʱ¹©¶à¸ö²»Í¬×ÓÈÎÎñµÄ syncer µ¥ÔªËù¹²Ó㬱ÜÃâÁ˶à¸öÈÎÎñÐèÒªÖØ¸´´ÓÉÏÓÎ MySQL À­È¡ binlog µÄÎÊÌâ¡£

Êý¾ÝÇ¨ÒÆ²¢·¢Ä£ÐÍ

Ϊ¼Ó¿ìÊý¾Ýµ¼ÈëËÙ¶È£¬ÔÚ DM Öв»ÂÛÊÇÈ«Á¿Êý¾ÝÇ¨ÒÆ£¬»¹ÊÇÔöÁ¿Êý¾Ýͬ²½£¬¶¼ÔÚÆäÖв¿·Ö½×¶ÎʹÓÃÁ˲¢·¢´¦Àí¡£

¶ÔÓÚÈ«Á¿Êý¾ÝÇ¨ÒÆ£¬ÔÚµ¼³ö½×¶Î£¬dumper µ¥Ôªµ÷Óà mydumper µ¼³ö¹¤¾ßÖ´ÐÐʵ¼ÊµÄÊý¾Ýµ¼³ö²Ù×÷£¬¶ÔÓ¦µÄ²¢·¢Ä£ÐÍ¿ÉÒÔÖ±½Ó²Î¿¼ mydumper µÄÔ´Âë¡£ÔÚʹÓà loader µ¥ÔªÖ´Ðеĵ¼Èë½×¶Î£¬¶ÔÓ¦µÄ²¢·¢Ä£ÐͽṹÈçÏ£º

ʹÓà mydumper Ö´Ðе¼³öʱ£¬¿ÉÒÔͨ¹ý --chunk-filesize µÈ²ÎÊý½«µ¥¸ö±í²ð·Ö³É¶à¸ö SQL Îļþ£¬ÕâЩ SQL Îļþ¶ÔÓ¦µÄ¶¼ÊÇÉÏÓÎ MySQL ijһ¸öʱ¿ÌµÄ¾²Ì¬¿ìÕÕÊý¾Ý£¬ÇÒ¸÷ SQL Îļþ¼äµÄÊý¾Ý²»´æÔÚ¹ØÁª¡£Òò´Ë£¬ÔÚʹÓà loader µ¥ÔªÖ´Ðе¼Èëʱ£¬¿ÉÒÔÖ±½ÓÔÚÒ»¸ö loader µ¥ÔªÄÚÆô¶¯¶à¸ö worker ¹¤×÷Э³Ì£¬Óɸ÷ worker Э³Ì²¢·¢¡¢¶ÀÁ¢µØÃ¿´Î¶Áȡһ¸ö´ýµ¼ÈëµÄ SQL Îļþ½øÐе¼Èë¡£¼´ loader µ¼Èë½×¶Î£¬ÊÇÒÔ SQL Îļþ¼¶±ðÁ£¶È²¢·¢½øÐеġ£ÔÚ DM µÄÈÎÎñÅäÖÃÖУ¬¶ÔÓÚ loader µ¥Ôª£¬ÆäÖÐµÄ pool-size ²ÎÊý¼´ÓÃÓÚ¿ØÖÆ´Ë´¦ worker Э³ÌÊýÁ¿¡£

¶ÔÓÚÔöÁ¿Êý¾Ýͬ²½£¬ÔÚ´ÓÉÏÓÎÀ­È¡ binlog ²¢³Ö¾Ã»¯µ½±¾µØµÄ½×¶Î£¬ÓÉÓÚÉÏÓÎ MySQL ÉÏ binlog µÄ²úÉúÓë·¢ËÍÊÇÒÔ stream ÐÎʽ½øÐеģ¬Òò´ËÕⲿ·ÖÖ»ÄÜ´®Ðд¦Àí¡£ÔÚʹÓà syncer µ¥ÔªÖ´Ðеĵ¼Èë½×¶Î£¬ÔÚÒ»¶¨µÄÏÞÖÆÌõ¼þÏ£¬¿ÉÒÔÖ´Ðв¢·¢µ¼È룬¶ÔÓ¦µÄÄ£ÐͽṹÈçÏ£º

µ± syncer ¶ÁÈ¡Óë½âÎö±¾µØ relay log ʱ£¬Óë´ÓÉÏÓÎÀ­È¡ binlog ÀàËÆ£¬ÊÇÒÔ stream ÐÎʽ½øÐеģ¬Òò´ËÒ²Ö»ÄÜ´®Ðд¦Àí¡£µ± syncer ½âÎö³ö¸÷ binlog event ²¢¹¹Ôì³É´ýͬ²½µÄ job ºó£¬Ôò¿ÉÒÔ¸ù¾Ý¶ÔÓ¦ÐÐÊý¾ÝµÄÖ÷¼ü¡¢Ë÷ÒýµÈÐÅÏ¢¾­¹ý hash ¼ÆËãºó·Ö·¢µ½¶à¸ö²»Í¬µÄ´ýͬ²½ job channel ÖУ»ÔÚ channel µÄÁíÒ»¶Ë£¬Óë¸÷¸ö channel ¶ÔÓ¦µÄ worker Э³Ì²¢·¢µØ´Ó channel ÖÐÈ¡³ö job ºóͬ²½µ½ÏÂÓ뵀 TiDB¡£¼´ syncer µ¼Èë½×¶Î£¬ÊÇÒÔ binlog event ¼¶±ðÁ£¶È²¢·¢½øÐеġ£ÔÚ DM µÄÈÎÎñÅäÖÃÖУ¬¶ÔÓÚ syncer µ¥Ôª£¬ÆäÖÐµÄ worker-count ²ÎÊý¼´ÓÃÓÚ¿ØÖÆ´Ë´¦ worker Э³ÌÊýÁ¿¡£

µ« syncer ²¢·¢Í¬²½µ½ÏÂÓÎ TiDB ʱ£¬´æÔÚһЩÏÞÖÆ£¬Ö÷Òª°üÀ¨£º

¶ÔÓÚ DDL£¬ÓÉÓÚ»á±ä¸üÏÂÓεıí½á¹¹£¬Òò´Ë±ØÐëÈ·±£Ôھɱí½á¹¹¶ÔÓ¦µÄ DML ¶¼Í¬²½Íê³Éºó£¬²ÅÄܽøÐÐͬ²½¡£ÔÚ DM ÖУ¬µ±½âÎö binlog event µÃµ½ DDL ºó£¬»áÏòÿһ¸ö job channel ·¢ËÍÒ»¸öÌØÊâµÄ flush job£»µ±¸÷ worker Э³ÌÓöµ½ flush job ʱ£¬»áÁ¢¿ÌÏòÏÂÓÎ TiDB ͬ²½Ö®Ç°ÒѾ­È¡³öµÄËùÓÐ job£»µÈ¸÷ job channel ÖÐµÄ job ¶¼Í¬²½µ½ÏÂÓÎ TiDB ºó£¬¿ªÊ¼Í¬²½ DDL£»µÈ´ý DDL ͬ²½Íê³Éºó£¬¼ÌÐøÍ¬²½ºóÐøµÄ DML¡£¼´ DDL ²»ÄÜÓë DML ²¢·¢Í¬²½£¬ÇÒ DDL ֮ǰÓëÖ®ºóµÄ DML Ò²²»Äܲ¢·¢Í¬²½¡£sharding ³¡¾°Ï DDL µÄͬ²½´¦Àí¼ûºóÎÄ¡£

¶ÔÓÚ DML£¬¶àÌõ DML ¿ÉÄÜ»áÐÞ¸ÄͬһÐеÄÊý¾Ý£¬ÉõÖÁÊÇÖ÷¼ü¡£Èç¹û²¢·¢µØÍ¬²½ÕâЩ DML£¬Ôò¿ÉÄÜÔì³Éͬ²½ºóÊý¾ÝµÄ²»Ò»Ö¡£DM ÖжÔÓÚ DML Ö®¼äµÄ³åÍ»¼ì²âÓë´¦Àí£¬Óë TiDB-Binlog ÖеĴ¦ÀíÀàËÆ£¬¾ßÌåÔ­Àí¿ÉÒÔÔĶÁ¡¶TiDB EcoSystem Tools Ô­Àí½â¶Á£¨Ò»£©TiDB-Binlog ¼Ü¹¹ÑݽøÓëʵÏÖÔ­Àí¡·ÖйØÓÚ Drainer ÄÚ SQL Ö®¼ä³åÍ»¼ì²âµÄÌÖÂÛ¡£

ºÏ¿âºÏ±íÊý¾Ýͬ²½

ÔÚʹÓà MySQL Ö§³Å´óÁ¿Êý¾Ýʱ£¬¾­³£»áÑ¡ÔñʹÓ÷ֿâ·Ö±íµÄ·½°¸¡£µ«µ±½«Êý¾Ýͬ²½µ½ TiDB ºó£¬Í¨³£Ï£ÍûÂß¼­ÉϽøÐкϿâºÏ±í¡£DM Ϊ֧³ÖºÏ¿âºÏ±íµÄÊý¾Ýͬ²½£¬Ö÷ҪʵÏÖÁËÒÔϵÄһЩ¹¦ÄÜ¡£

table router

Ϊ˵Ã÷ DM ÖÐ table router£¨±íÃû·ÓÉ£©¹¦ÄÜ£¬ÏÈ¿´ÈçÏÂͼËùʾµÄÒ»¸öÀý×Ó£º

ÔÚÕâ¸öÀý×ÓÖУ¬ÉÏÓÎÓÐ 2 ¸ö MySQL ʵÀý£¬Ã¿¸öʵÀýÓÐ 2 ¸öÂß¼­¿â£¬Ã¿¸ö¿âÓÐ 2 ¸ö±í£¬×ܹ² 8 ¸ö±í¡£µ±Í¬²½µ½ÏÂÓÎ TiDB ºó£¬Ï£ÍûËùÓеÄÕâ 8 ¸ö±í×îÖÕ¶¼ºÏ²¢Í¬²½µ½Í¬Ò»¸ö±íÖС£

µ«ÎªÁËÄܽ« 8 ¸öÀ´×Ô²»Í¬ÊµÀý¡¢²»Í¬¿âÇÒÓв»Í¬ÃûµÄ±íͬ²½µ½Í¬Ò»¸ö±íÖУ¬Ê×ÏÈÒª´¦ÀíµÄ£¬¾ÍÊÇÒªÄܸù¾ÝijЩ¶¨ÒåºÃµÄ¹æÔò£¬½«À´×Ô²»Í¬±íµÄÊý¾Ý¶¼Â·Óɵ½ÏÂÓεÄͬһ¸ö±íÖС£ÔÚ DM ÖУ¬ÕâÀà¹æÔò½Ð×ö router-rules¡£¶ÔÓÚÉÏÃæµÄʾÀý£¬Æä¹æÔòÈçÏ£º

name-of-router-rule:
schema-pattern: "schema_*"
table-pattern: "table_*"
target-schema: "schema"
target-table: "table"

name-of-router-rule£º¹æÔòÃû£¬Óû§Ö¸¶¨¡£µ±Óжà¸öÉÏÓÎʵÀýÐèҪʹÓÃÏàͬµÄ¹æÔòʱ£¬¿ÉÒÔÖ»¶¨ÒåÒ»Ìõ¹æÔò£¬¶à¸ö²»Í¬µÄʵÀýͨ¹ý¹æÔòÃû½øÐÐÒýÓá£

schema-pattern£ºÓÃÓÚÆ¥ÅäÉÏÓο⣨schema£©ÃûµÄģʽ£¬Ö§³ÖÔÚβ²¿Ê¹ÓÃͨÅä·û£¨*£©¡£ÕâÀïʹÓà schema_* ¼´¿ÉÆ¥Å䵽ʾÀýÖеÄÁ½¸ö¿âÃû¡£

table-pattern£ºÓÃÓÚÆ¥ÅäÉÏÓαíÃûµÄģʽ£¬Óë schema-pattern ÀàËÆ¡£ÕâÀïʹÓà table_* ¼´¿ÉÆ¥Å䵽ʾÀýÖеÄÁ½¸ö±íÃû¡£

target-schema£ºÄ¿±ê¿âÃû¡£¶ÔÓÚ¿âÃû¡¢±íÃûÆ¥ÅäµÄÊý¾Ý£¬½«±»Â·Óɵ½Õâ¸ö¿âÖС£

target-table£ºÄ¿±ê±íÃû¡£¶ÔÓÚ¿âÃû¡¢±íÃûÆ¥ÅäµÄÊý¾Ý£¬½«±»Â·Óɵ½ target-schema ¿âϵÄÕâ¸ö±íÖС£

ÔÚ DM ÄÚ²¿ÊµÏÖÉÏ£¬Ê×Ïȸù¾Ý schema-pattern / table-pattern ¹¹Ôì¶ÔÓ¦µÄ trie ½á¹¹£¬²¢½«¹æÔò´æ´¢ÔÚ trie ½ÚµãÖУ»µ±ÓÐ SQL ÐèҪͬ²½µ½ÏÂÓÎʱ£¬Í¨¹ýʹÓÃÉÏÓοâÃû¡¢±íÃû²éѯ trie ¼´¿ÉµÃµ½¶ÔÓ¦µÄ¹æÔò£¬²¢¸ù¾Ý¹æÔòÌæ»»Ô­ SQL ÖеĿâÃû¡¢±íÃû£»Í¨¹ýÏòÏÂÓÎ TiDB Ö´ÐÐÌæ»»ºóµÄ SQL ¼´Íê³ÉÁ˸ù¾Ý±íÃûµÄ·ÓÉͬ²½¡£ÓÐ¹Ø router-rules ¹æÔòµÄ¾ßÌåʵÏÖ£¬¿ÉÒÔÔĶÁ TiDB-Tools Ï嵀 table-router pkg Ô´´úÂë¡£

column mapping

ÓÐÁË table router ¹¦ÄÜ£¬ÒѾ­¿ÉÒÔÍê³É»ù±¾µÄºÏ¿âºÏ±íÊý¾Ýͬ²½ÁË¡£µ«ÔÚÊý¾Ý¿âÖУ¬ÎÒÃǾ­³£»áʹÓÃ×ÔÔöÀàÐ͵ÄÁÐ×÷ΪÖ÷¼ü¡£Èç¹û¶à¸öÉÏÓηֱíµÄÖ÷¼ü¸÷×Ô¶ÀÁ¢µØ×ÔÔö£¬½«ËüÃǺϲ¢Í¬²½µ½ÏÂÓκ󣬾ͺܿÉÄÜ»á³öÏÖÖ÷¼ü³åÍ»£¬Ôì³ÉÊý¾ÝµÄ²»Ò»Ö¡£ÎÒÃǿɿ´Ò»¸öÈçϵÄÀý×Ó£º

ÔÚÕâ¸öÀý×ÓÖУ¬ÉÏÓÎ 4 ¸öÐèÒªºÏ²¢Í¬²½µ½ÏÂÓεıíÖУ¬¶¼´æÔÚ id ÁÐֵΪ 1 µÄ¼Ç¼¡£¼ÙÉèÕâ¸ö id ÁÐÊDZíµÄÖ÷¼ü¡£ÔÚͬ²½µ½ÏÂÓεĹý³ÌÖУ¬ÓÉÓÚÏà¹Ø¸üвÙ×÷ÊÇÒÔ id ÁÐ×÷ΪÌõ¼þÀ´È·¶¨ÐèÒª¸üеļǼ£¬Òò´Ë»áÔì³Éºóͬ²½µÄÊý¾Ý¸²¸ÇÇ°ÃæÒѾ­Í¬²½¹ýµÄÊý¾Ý£¬µ¼Ö²¿·ÖÊý¾ÝµÄ¶ªÊ§¡£

ÔÚ DM ÖУ¬ÎÒÃÇͨ¹ý column mapping ¹¦ÄÜÔÚÊý¾Ýͬ²½µÄ¹ý³ÌÖÐÒÀ¾ÝÖ¸¶¨¹æÔò¶ÔÏà¹ØÁеÄÊý¾Ý½øÐÐת»»¸ÄдÀ´±ÜÃâÊý¾Ý³åÍ»Ó붪ʧ¡£¶ÔÓÚÉÏÃæµÄʾÀý£¬ÆäÖÐ MySQL ʵÀý 1 µÄ column mapping ¹æÔòÈçÏ£º

mapping-rule-of-instance-1:
schema-pattern: "schema_*"
table-pattern: "table_*"
expression: "partition id"
source-column: "id"
target-column: "id"
arguments: ["1", "schema_", "table_"]

mapping-rule-of-instance-1£º¹æÔòÃû£¬Óû§Ö¸¶¨¡£ÓÉÓÚ²»Í¬µÄÉÏÓÎ MySQL ʵÀýÐèҪת»»µÃµ½²»Í¬µÄÖµ£¬Òò´Ëͨ³£Ã¿¸ö MySQL ʵÀýʹÓÃÒ»ÌõרÓеĹæÔò¡£

schema-pattern / table-pattern£ºÉÏÓοâÃû¡¢±íÃûÆ¥Åäģʽ£¬Óë router-rules ÖеĶÔÓ¦ÅäÖÃÏîÒ»Ö¡£

expression£º½øÐÐÊý¾Ýת»»µÄ±í´ïʽÃû¡£Ä¿Ç°³£Óõıí´ïʽ¼´Îª "partition id"£¬Óйظñí´ïʽµÄ¾ßÌå˵Ã÷¼ûÏÂÎÄ¡£

source-column£º×ª»»±í´ïʽµÄÊäÈëÊý¾Ý¶ÔÓ¦µÄÀ´Ô´ÁÐÃû£¬"id" ±íʾÕâ¸ö±í´ïʽ½«×÷ÓÃÓÚ±íÖÐÃûΪ id µÄÁС£ÔÝʱֻ֧³Ö¶Ôµ¥¸öÀ´Ô´ÁнøÐÐÊý¾Ýת»»¡£

target-column£º×ª»»±í´ïʽµÄÊä³öÊý¾Ý¶ÔÓ¦µÄÄ¿±êÁÐÃû£¬Óë source-column ÀàËÆ¡£ÔÝʱֻ֧³Ö¶Ôµ¥¸öÄ¿±êÁнøÐÐÊý¾Ýת»»£¬ÇÒ¶ÔÓ¦µÄÄ¿±êÁбØÐëÒѾ­´æÔÚ¡£

arguments£º×ª»»±í´ïʽËùÒÀÀµµÄ²ÎÊý¡£²ÎÊý¸öÊýÓ뺬ÒåÒÀ¾ßÌå±í´ïʽ¶ø¶¨¡£

partition id ÊÇĿǰÖ÷ÒªÊÜÖ§³ÖµÄת»»±í´ïʽ£¬Æäͨ¹ýΪ bigint ÀàÐ͵ÄÖµÔö¼Ó¶þ½øÖÆÇ°×ºÀ´½â¾öÀ´×Ô²»Í¬±íµÄÊý¾ÝºÏ²¢Í¬²½ºó¿ÉÄܲúÉú³åÍ»µÄÎÊÌâ¡£partition id µÄ arguments °üÀ¨ 3 ¸ö²ÎÊý£¬·Ö±ðΪ£º

MySQL ʵÀý ID£º±êʶÊý¾ÝµÄÀ´Ô´ MySQL ʵÀý£¬Óû§×ÔÓÉÖ¸¶¨¡£Èç "1" ±íʾƥÅä¸Ã¹æÔòµÄÊý¾ÝÀ´×ÔÓÚ MySQL ʵÀý 1£¬ÇÒÕâ¸ö±êʶ½«±»×ª»»³ÉÊýÖµºóÒÔ¶þ½øÖƵÄÐÎʽ×÷Ϊǰ׺µÄÒ»²¿·ÖÌí¼Óµ½×ª»»ºóµÄÖµÖС£

¿âÃûǰ׺£º±êʶÊý¾ÝµÄÀ´Ô´Âß¼­¿â¡£Èç "schema_" Ó¦ÓÃÓÚ schema_2 Âß¼­¿âʱ£¬±íʾȥ³ýǰ׺ºóʣϵIJ¿·Ö£¨Êý×Ö 2£©½«ÒÔ¶þ½øÖƵÄÐÎʽ×÷Ϊǰ׺µÄÒ»²¿·ÖÌí¼Óµ½×ª»»ºóµÄÖµÖС£

±íÃûǰ׺£º±êʶÊý¾ÝµÄÀ´Ô´±í¡£Èç "table_" Ó¦ÓÃÓÚ table_3 ±íʱ£¬±íʾȥ³ýǰ׺ºóʣϵIJ¿·Ö£¨Êý×Ö 3£©½«ÒÔ¶þ½øÖƵÄÐÎʽ×÷Ϊǰ׺µÄÒ»²¿·ÖÌí¼Óµ½×ª»»ºóµÄÖµÖС£

¸÷²¿·ÖÔÚ¾­¹ýת»»ºóµÄÊýÖµÖеĶþ½øÖÆ·Ö²¼ÈçÏÂͼËùʾ£¨¸÷²¿·ÖĬÈÏËùÕ¼ÓÃµÄ bits λÊýÈçͼËùʾ£©£º

¼ÙÈçת»»Ç°µÄԭʼÊý¾ÝΪ 123£¬ÇÒÓÐÈçÉ쵀 arguments ²ÎÊýÉèÖã¬Ôòת»»ºóµÄֵΪ£º

1<<(64-1-4) | 2<<(64-1-4-7) | 3<<(64-1-4-7-8) | 123

ÁíÍ⣬arguments ÖÐµÄ 3 ¸ö²ÎÊý¾ù¿ÉÉèÖÃΪ¿Õ×Ö·û´®£¨""£©£¬¼´±íʾ¸Ã²¿·Ö²»±»Ìí¼Óµ½×ª»»ºóµÄÖµÖУ¬ÇÒ²»Õ¼ÓöîÍâµÄ bits¡£±ÈÈ罫ÆäÉèÖÃΪ["1", "", "table_"]£¬Ôòת»»ºóµÄֵΪ£º

1 << (64-1-4) | 3<< (64-1-4-8) | 123

ÓÐ¹Ø column mapping ¹¦ÄܵľßÌåʵÏÖ£¬¿ÉÒÔÔĶÁ TiDB-Tools Ï嵀 column-mapping pkg Ô´´úÂë¡£

sharding DDL

ÓÐÁË table router ºÍ column mapping ¹¦ÄÜ£¬DML µÄºÏ¿âºÏ±íÊý¾Ýͬ²½ÒѾ­¿ÉÒÔÕý³£½øÐÐÁË¡£µ«Èç¹ûÔÚÔöÁ¿Êý¾Ýͬ²½µÄ¹ý³ÌÖУ¬ÉÏÓδýºÏ²¢µÄ·Ö±íÉÏÖ´ÐÐÁË DDL ²Ù×÷£¬Ôò¿ÉÄܳöÏÖÎÊÌâ¡£ÎÒÃÇÏÈÀ´¿´Ò»¸ö¼ò»¯ºóµÄÔÚ·Ö±íÉÏÖ´ÐÐ DDL µÄÀý×Ó¡£

ÔÚÉÏͼµÄÀý×ÓÖУ¬·Ö±íµÄºÏ¿âºÏ±í¼ò»¯³ÉÁËÉÏÓÎÖ»ÓÐÁ½¸ö MySQL ʵÀý£¬Ã¿¸öʵÀýÄÚÖ»ÓÐÒ»¸ö±í¡£¼ÙÉèÔÚ¿ªÊ¼Êý¾Ýͬ²½Ê±£¬½«Á½¸ö·Ö±íµÄ±í½á¹¹ schema µÄ°æ±¾¼ÇΪ schema V1£¬½« DDL Ö´ÐÐÍê³ÉºóµÄ±í½á¹¹ schema µÄ°æ±¾¼ÇΪ schema V2¡£

ÏÖÔÚ£¬¼ÙÉèÊý¾Ýͬ²½¹ý³ÌÖУ¬´ÓÁ½¸öÉÏÓηֱíÊÕµ½µÄ binlog Êý¾ÝÓÐÈçϵÄʱÐò£º

¿ªÊ¼Í¬²½Ê±£¬´ÓÁ½¸ö·Ö±íÊÕµ½µÄ¶¼ÊÇ schema V1 µÄ DML¡£

ÔÚ t1 ʱ¿Ì£¬ÊÕµ½ÊµÀý 1 ÉÏ·Ö±íµÄ DDL¡£

´Ó t2 ʱ¿Ì¿ªÊ¼£¬´ÓʵÀý 1 ÊÕµ½µÄÊÇ schema V2 µÄ DML£»µ«´ÓʵÀý 2 ÊÕµ½µÄÈÔÊÇ schema V1 µÄ DML¡£

ÔÚ t3 ʱ¿Ì£¬ÊÕµ½ÊµÀý 2 ÉÏ·Ö±íµÄ DDL¡£

´Ó t4 ʱ¿Ì¿ªÊ¼£¬´ÓʵÀý 2 ÊÕµ½µÄÒ²ÊÇ schema V2 µÄ DML¡£

¼ÙÉèÔÚÊý¾Ýͬ²½¹ý³ÌÖУ¬²»¶Ô·Ö±íµÄ DDL ½øÐд¦Àí¡£µ±½«ÊµÀý 1 µÄ DDL ͬ²½µ½ÏÂÓκó£¬ÏÂÓεıí½á¹¹»á±ä¸ü³ÉΪ schema V2¡£µ«¶ÔÓÚʵÀý 2£¬ÔÚ t2 ʱ¿Ìµ½ t3 ʱ¿ÌÕâ¶Îʱ¼äÄÚÊÕµ½µÄÈÔÈ»ÊÇ schema V1 µÄ DML¡£µ±³¢ÊÔ°ÑÕâЩÓë schema V1 ¶ÔÓ¦µÄ DML ͬ²½µ½ÏÂÓÎʱ£¬¾Í»áÓÉÓÚ DML Óë±í½á¹¹µÄ²»Ò»Ö¶ø·¢Éú´íÎó£¬Ôì³ÉÊý¾ÝÎÞ·¨Õýȷͬ²½¡£

¼ÌÐøÊ¹ÓÃÉÏÃæµÄÀý×Ó£¬À´¿´¿´ÎÒÃÇÔÚ DM ÖÐÊÇÈçºÎ´¦ÀíºÏ¿âºÏ±í¹ý³ÌÖÐµÄ DDL ͬ²½µÄ¡£

ÔÚÕâ¸öÀý×ÓÖУ¬DM-worker-1 ÓÃÓÚͬ²½À´×Ô MySQL ʵÀý 1 µÄÊý¾Ý£¬DM-worker-2 ÓÃÓÚͬ²½À´×Ô MySQL ʵÀý 2 µÄÊý¾Ý£¬DM-master ÓÃÓÚЭµ÷¶à¸ö DM-worker ¼äµÄ DDL ͬ²½¡£´Ó DM-worker-1 ÊÕµ½ DDL ¿ªÊ¼£¬¼ò»¯ºóµÄ DDL ͬ²½Á÷³ÌΪ£º

1.DM-worker-1 ÔÚ t1 ʱ¿ÌÊÕµ½À´×Ô MySQL ʵÀý 1 µÄ DDL£¬×ÔÉíÔÝÍ£¸Ã DDL ¶ÔÓ¦ÈÎÎñµÄ DDL ¼° DML Êý¾Ýͬ²½£¬²¢½« DDL Ïà¹ØÐÅÏ¢·¢Ë͸ø DM-master¡£

2.DM-master ¸ù¾Ý DDL ÐÅÏ¢ÅжÏÐèҪЭµ÷¸Ã DDL µÄͬ²½£¬Îª¸Ã DDL ´´½¨Ò»¸öËø£¬²¢½« DDL ËøÐÅÏ¢·¢»Ø¸ø DM-worker-1£¬Í¬Ê±½« DM-worker-1 ±ê¼ÇΪÕâ¸öËøµÄ owner¡£

3.DM-worker-2 ¼ÌÐø½øÐÐ DML µÄͬ²½£¬Ö±µ½ÔÚ t3 ʱ¿ÌÊÕµ½À´×Ô MySQL ʵÀý 2 µÄ DDL£¬×ÔÉíÔÝÍ£¸Ã DDL ¶ÔÓ¦ÈÎÎñµÄÊý¾Ýͬ²½£¬²¢½« DDL Ïà¹ØÐÅÏ¢·¢Ë͸ø DM-master¡£

4.DM-master ¸ù¾Ý DDL ÐÅÏ¢Åжϸà DDL ¶ÔÓ¦µÄËøÐÅÏ¢ÒѾ­´æÔÚ£¬Ö±½Ó½«¶ÔÓ¦ËøÐÅÏ¢·¢»Ø¸ø DM-worker-2¡£

5.DM-master ¸ù¾ÝÆô¶¯ÈÎÎñʱµÄÅäÖÃÐÅÏ¢¡¢ÉÏÓÎ MySQL ʵÀý·Ö±íÐÅÏ¢¡¢²¿ÊðÍØÆËÐÅÏ¢µÈ£¬ÅжϵÃÖªÒѾ­ÊÕµ½ÁËÐèÒªºÏ±íµÄËùÓÐÉÏÓηֱíµÄ¸Ã DDL£¬ÇëÇó DDL ËøµÄ owner£¨DM-worker-1£©ÏòÏÂÓÎͬ²½Ö´Ðиà DDL¡£

6.DM-worker-1 ¸ù¾Ý step 2 ʱÊÕµ½µÄ DDL ËøÐÅÏ¢ÑéÖ¤ DDL Ö´ÐÐÇëÇó£»ÏòÏÂÓÎÖ´ÐÐ DDL£¬²¢½«Ö´Ðнá¹û·´À¡¸ø DM-master£»ÈôÖ´ÐÐ DDL ³É¹¦£¬Ôò×ÔÉí¿ªÊ¼¼ÌÐøÍ¬²½ºóÐøµÄ£¨´Ó t2 ʱ¿Ì¶ÔÓ¦µÄ binlog ¿ªÊ¼µÄ£©DML¡£

7.DM-master ÊÕµ½À´×Ô owner Ö´ÐÐ DDL ³É¹¦µÄÏìÓ¦£¬ÇëÇóÔڵȴý¸Ã DDL ËøµÄËùÓÐÆäËû DM-worker£¨DM-worker-2£©ºöÂԸà DDL£¬Ö±½Ó¼ÌÐøÍ¬²½ºóÐøµÄ£¨´Ó t4 ʱ¿Ì¶ÔÓ¦µÄ binlog ¿ªÊ¼µÄ£©DML¡£

¸ù¾ÝÉÏÃæ DM ´¦Àí¶à¸ö DM-worker ¼äµÄ DDL ͬ²½µÄÁ÷³Ì£¬¹éÄÉһϠDM ÄÚ´¦Àí¶à¸ö DM-worker ¼ä sharding DDL ͬ²½µÄÌØµã£º

1.¸ù¾ÝÈÎÎñÅäÖÃÓë DM ¼¯Èº²¿ÊðÍØÆËÐÅÏ¢£¬ÔÚ DM-master ÄÚ½¨Á¢Ò»¸öÐèҪЭµ÷ DDL ͬ²½µÄÂß¼­ sharding group£¬group ÖеijÉԱΪ´¦Àí¸ÃÈÎÎñ²ð½âºó¸÷×ÓÈÎÎñµÄ DM-worker¡£

2.¸÷ DM-worker ÔÚ´Ó binlog event ÖлñÈ¡µ½ DDL ºó£¬»á½« DDL ÐÅÏ¢·¢Ë͸ø DM-master¡£

3.DM-master ¸ù¾ÝÀ´×Ô DM-worker µÄ DDL ÐÅÏ¢¼° sharding group ÐÅÏ¢´´½¨/¸üРDDL Ëø¡£

4.Èç¹û sharding group µÄËùÓгÉÔ±¶¼ÊÕµ½ÁËijһÌõ DDL£¬Ôò±íÃ÷ÉÏÓηֱíÔڸà DDL Ö´ÐÐǰµÄ DML ¶¼ÒѾ­Í¬²½Íê³É£¬¿ÉÒÔÖ´ÐÐ DDL£¬²¢¼ÌÐøºóÐøµÄ DML ͬ²½¡£

5.ÉÏÓηֱíµÄ DDL ÔÚ¾­¹ý table router ת»»ºó£¬¶ÔÓ¦ÐèÒªÔÚÏÂÓÎÖ´ÐÐµÄ DDL Ó¦¸ÃÒ»Ö£¬Òò´Ë½öÐè DDL ËøµÄ owner Ö´ÐÐÒ»´Î¼´¿É£¬ÆäËû DM-worker ¿ÉÖ±½ÓºöÂÔ¶ÔÓ¦µÄ DDL¡£

´Ó DM ´¦Àí DM-worker ¼ä sharding DDL ͬ²½µÄÌØµã£¬¿ÉÒÔ¿´³ö¸Ã¹¦ÄÜ´æÔÚÒÔÏÂһЩÏÞÖÆ£º

1.ÉÏÓÎµÄ·Ö±í±ØÐëÒÔÏàͬµÄ˳ÐòÖ´ÐУ¨table router ת»»ºóÏàͬµÄ£©DDL£¬±ÈÈç±í 1 ÏÈÔö¼ÓÁÐ a ºóÔÙÔö¼ÓÁÐ b£¬¶ø±í 2 ÏÈÔö¼ÓÁÐ b ºóÔÙÔö¼ÓÁÐ a£¬ÕâÖÖ²»Í¬Ë³ÐòµÄ DDL Ö´Ðз½Ê½ÊDz»Ö§³ÖµÄ¡£

2.Ò»¸öÂß¼­ sharding group ÄÚµÄËùÓÐ DM-worker ¶ÔÓ¦µÄÉÏÓÎ·Ö±í£¬¶¼Ó¦¸ÃÖ´ÐжÔÓ¦µÄ DDL£¬±ÈÈçÆäÖÐÓÐ DM-worker-2 ¶ÔÓ¦µÄÉÏÓηֱíδִÐÐ DDL£¬ÔòÆäËûÒÑÖ´ÐÐ DDL µÄ DM-worker ¶¼»áÔÝͣͬ²½ÈÎÎñ£¬µÈ´ý DM-worker-2 ÊÕµ½¶ÔÓ¦ÉÏÓ뵀 DDL¡£

3.ÓÉÓÚÒѾ­ÊÕµ½µÄ DDL µÄ DM-worker »áÔÝÍ£ÈÎÎñÒԵȴýÆäËû DM-worker ÊÕµ½¶ÔÓ¦µÄ DDL£¬Òò´ËÊý¾Ýͬ²½ÑÓ³Ù»áÔö¼Ó¡£

4.ÔöÁ¿Í¬²½¿ªÊ¼Ê±£¬ÐèÒªºÏ²¢µÄËùÓÐÉÏÓηֱí½á¹¹±ØÐëÒ»Ö£¬²ÅÄÜÈ·±£À´×Ô²»Í¬·Ö±íµÄ DML ¿ÉÒÔͬ²½µ½Ò»¸öÈ·¶¨±í½á¹¹µÄÏÂÓΣ¬Ò²²ÅÄÜÈ·±£ºóÐø¸÷·Ö±íµÄ DDL Äܹ»ÕýÈ·Æ¥ÅäÓëͬ²½¡£

ÔÚÉÏÃæµÄʾÀýÖУ¬Ã¿¸ö DM-worker ¶ÔÓ¦µÄÉÏÓÎ MySQL ʵÀýÖÐÖ»ÓÐÒ»¸öÐèÒª½øÐкϲ¢µÄ·Ö±í¡£µ«ÔÚʵ¼Ê³¡¾°Ï£¬Ò»¸ö MySQL ʵÀý¿ÉÄÜÓжà¸ö·Ö¿âÄڵĶà¸ö·Ö±íÐèÒª½øÐкϲ¢£¬±ÈÈçÇ°Ãæ½éÉÜ table router Óë column mapping ¹¦ÄÜʱµÄÀý×Ó¡£µ±Ò»¸ö MySQL ʵÀýÖÐÓжà¸ö·Ö±íÐèÒªºÏ²¢Ê±£¬sharding DDL µÄЭµ÷ͬ²½¹ý³ÌÔö¼ÓÁ˸ü¶àµÄ¸´ÔÓÐÔ¡£

¼ÙÉèͬһ¸ö MySQL ʵÀýÖÐÓÐ table_1 ºÍ table_2 Á½¸ö·Ö±íÐèÒª½øÐкϲ¢£¬ÈçÏÂͼ£º

ÓÉÓÚÊý¾ÝÀ´×Ôͬһ¸ö MySQL ʵÀý£¬Òò´ËËùÓÐÊý¾Ý¶¼ÊÇ´Óͬһ¸ö binlog Á÷ÖлñµÃ¡£ÔÚÕâ¸öÀý×ÓÖУ¬Ê±ÐòÈçÏ£º

1.¿ªÊ¼Í¬²½Ê±£¬Á½¸ö·Ö±íÊÕµ½µÄÊý¾Ý¶¼ÊÇ schema V1 µÄ DML¡£

2.ÔÚ t1 ʱ¿Ì£¬ÊÕµ½ÁË table_1 µÄ DDL¡£

3.´Ó t2 ʱ¿Ìµ½ t3 ʱ¿Ì£¬ÊÕµ½µÄÊý¾Ýͬʱ°üº¬ table_1 schema V2 µÄ DML ¼° table_2 schema V1 µÄ DML¡£

4.ÔÚ t3 ʱ¿Ì£¬ÊÕµ½ÁË table_2 µÄ DDL¡£

5.´Ó t4 ʱ¿Ì¿ªÊ¼£¬Á½¸ö·Ö±íÊÕµ½µÄÊý¾Ý¶¼ÊÇ schema V2 µÄ DML¡£

¼ÙÉèÔÚÊý¾Ýͬ²½¹ý³ÌÖв»¶Ô DDL ½øÐÐÌØÊâ´¦Àí£¬µ± table_1 µÄ DDL ͬ²½µ½ÏÂÓΡ¢±ä¸üÏÂÓαí½á¹¹ºó£¬table_2 schema V1 µÄ DML ½«ÎÞ·¨Õý³£Í¬²½¡£Òò´Ë£¬ÔÚµ¥¸ö DM-worker ÄÚ²¿£¬ÎÒÃÇÒ²¹¹ÔìÁËÓë DM-master ÄÚÀàËÆµÄÂß¼­ sharding group£¬µ« group µÄ³ÉÔ±ÊÇͬһ¸öÉÏÓÎ MySQL ʵÀýµÄ²»Í¬·Ö±í¡£

µ« DM-worker ÄÚЭµ÷´¦Àí sharding group µÄͬ²½²»ÄÜÍêÈ«Óë DM-master ´¦ÀíʱһÖ£¬Ö÷ÒªÔ­Òò°üÀ¨£º

µ±ÊÕµ½ table_1 µÄ DDL ʱ£¬Í¬²½²»ÄÜÔÝÍ££¬ÐèÒª¼ÌÐø½âÎö binlog ²ÅÄÜ»ñµÃºóÐø table_2µÄ DDL£¬¼´ÐèÒª´Ó t2 ʱ¿Ì¼ÌÐøÏòǰ½âÎöÖ±µ½ t3 ʱ¿Ì¡£

ÔÚ¼ÌÐø½âÎö t2 ʱ¿Ìµ½ t3 ʱ¿ÌµÄ binlog µÄ¹ý³ÌÖУ¬table_1 µÄ schema V2 µÄ DML ²»ÄÜÏòÏÂÓÎͬ²½£»µ«ÔÚ sharding DDL ͬ²½²¢Ö´Ðгɹ¦ºó£¬ÕâЩ DML ÐèҪͬ²½µ½ÏÂÓΡ£

ÔÚ DM ÖУ¬¼ò»¯ºóµÄ DM-worker ÄÚ sharding DDL ͬ²½Á÷³ÌΪ£º

1.ÔÚ t1 ʱ¿ÌÊÕµ½ table_1 µÄ DDL£¬¼Ç¼ DDL ÐÅÏ¢¼°´ËʱµÄ binlog λÖõãÐÅÏ¢¡£

2.¼ÌÐøÏòǰ½âÎö t2 ʱ¿Ìµ½ t3 ʱ¿ÌµÄ binlog¡£

3.¶ÔÓÚÊôÓÚ table_1 µÄ schema V2 DML£¬ºöÂÔ£»¶ÔÓÚÊôÓÚ table_2 µÄ schema V1 DML£¬Õý³£Í¬²½µ½ÏÂÓΡ£

4.ÔÚ t3 ʱ¿ÌÊÕµ½ table_2 µÄ DDL£¬¼Ç¼ DDL ÐÅÏ¢¼°´ËʱµÄ binlog λÖõãÐÅÏ¢¡£

5.¸ù¾Ýͬ²½ÈÎÎñÅäÖÃÐÅÏ¢¡¢ÉÏÓοâ±íÐÅÏ¢µÈ£¬Åжϸà MySQL ʵÀýÉÏËùÓзֱíµÄ DDL ¶¼ÒѾ­ÊÕµ½£»½« DDL ͬ²½µ½ÏÂÓÎÖ´ÐС¢±ä¸üÏÂÓαí½á¹¹¡£

6.ÉèÖÃÐ嵀 binlog Á÷µÄ½âÎöÆðʼλÖõãΪ step 1 ʱ±£´æµÄλÖõ㡣

7.ÖØÐ¿ªÊ¼½âÎö´Ó t2 ʱ¿Ìµ½ t3 ʱ¿ÌµÄ binlog¡£

8.¶ÔÓÚÊôÓÚ table_1 µÄ schema V2 DML£¬Õý³£Í¬²½µ½ÏÂÓΣ»¶ÔÓÚÊôÓÚ table_2 µÄ shema V1 DML£¬ºöÂÔ¡£

9.½âÎöµ½´ï step 4 ʱ±£´æµÄ binlog λÖõ㣬¿ÉµÃÖªÔÚ step 3 ʱ±»ºöÂÔµÄËùÓÐ DML ¶¼ÒѾ­ÖØÐÂͬ²½µ½ÏÂÓΡ£

10.¼ÌÐø´Ó t4 ʱ¿Ì¶ÔÓ¦µÄ binlog λÖõãÕý³£Í¬²½¡£

´ÓÉÏÃæµÄ·ÖÎö¿ÉÒÔÖªµÀ£¬DM ÔÚ´¦Àí sharding DDL ͬ²½Ê±£¬Ö÷Ҫͨ¹ýÁ½¼¶ sharding group À´½øÐÐЭµ÷¿ØÖÆ£¬¼ò»¯µÄÁ÷³ÌΪ£º

1.¸÷ DM-worker ¶ÀÁ¢µØÐ­µ÷¶ÔÓ¦ÉÏÓÎ MySQL ʵÀýÄÚ¶à¸ö·Ö±í×é³ÉµÄ sharding group µÄ DDL ͬ²½¡£

2.µ± DM-worker ÄÚËùÓзֱíµÄ DDL ¶¼ÊÕµ½Ê±£¬Ïò DM-master ·¢ËÍ DDL Ïà¹ØÐÅÏ¢¡£

3.DM-master ¸ù¾Ý DM-worker ·¢À´µÄ DDL ÐÅÏ¢£¬Ð­µ÷Óɸ÷ DM-worker ×é³ÉµÄ sharing group µÄ DDL ͬ²½¡£

4.µ± DM-master ÊÕµ½ËùÓÐ DM-worker µÄ DDL ÐÅϢʱ£¬ÇëÇó DDL lock µÄ owner£¨Ä³¸ö DM-worker£©Ö´ÐÐ DDL¡£

5.owner Ö´ÐÐ DDL£¬²¢½«½á¹û·´À¡¸ø DM-master£»×ÔÉí¿ªÊ¼ÖØÐÂͬ²½ÔÚÄÚ²¿Ð­µ÷ DDL ͬ²½¹ý³ÌÖб»ºöÂ﵀ DML¡£

6.µ± DM-master ·¢ÏÖ owner Ö´ÐÐ DDL ³É¹¦ºó£¬ÇëÇóÆäËûËùÓÐ DM-worker ¿ªÊ¼¼ÌÐøÍ¬²½¡£

7.ÆäËûËùÓÐ DM-worker ¸÷×Ô¿ªÊ¼ÖØÐÂͬ²½ÔÚÄÚ²¿Ð­µ÷ DDL ͬ²½¹ý³ÌÖб»ºöÂ﵀ DML¡£

8.ËùÓÐ DM-worker ÔÚÖØÐÂͬ²½Íê³É±»ºöÂ﵀ DML ºó£¬¼ÌÐøÕý³£Í¬²½¡£

Êý¾Ýͬ²½¹ýÂË

ÔÚ½øÐÐÊý¾Ýͬ²½µÄ¹ý³ÌÖУ¬ÓÐʱ¿ÉÄܲ¢²»ÐèÒª½«ÉÏÓÎËùÓеÄÊý¾Ý¶¼Í¬²½µ½ÏÂÓΣ¬Õâʱһ°ãÆÚÍûÄÜÔÚͬ²½¹ý³ÌÖиù¾ÝijЩ¹æÔò£¬¹ýÂ˵ô²¿·Ö²»ÆÚÍûͬ²½µÄÊý¾Ý¡£ÔÚ DM ÖУ¬Ö§³Ö 2 ÖÖ²»Í¬¼¶±ðµÄͬ²½¹ýÂË·½Ê½¡£

¿â±íºÚ°×Ãûµ¥

DM ÔÚ dumper¡¢loader¡¢syncer Èý¸ö´¦Àíµ¥ÔªÖж¼Ö§³ÖÅäÖùæÔòֻͬ²½/²»Í¬²½²¿·Ö¿â»ò±í¡£

¶ÔÓÚ dumper µ¥Ôª£¬Æäʵ¼Êµ÷Óà mydumper À´ dump ÉÏÓÎ MySQL µÄÊý¾Ý¡£±ÈÈçÖ»ÆÚÍûµ¼³ö test ¿âÖÐµÄ t1¡¢t2 Á½¸ö±íµÄÊý¾Ý£¬Ôò¿ÉÒÔΪ dumper µ¥ÔªÅäÖÃÈçϹæÔò£º

name-of-dump-rule:
extra-args: "-B test -T t1,t2"

name-of-dump-rule£º¹æÔòÃû£¬Óû§Ö¸¶¨¡£µ±Óжà¸öÉÏÓÎʵÀýÐèҪʹÓÃÏàͬµÄ¹æÔòʱ£¬¿ÉÒÔÖ»¶¨ÒåÒ»Ìõ¹æÔò£¬¶à¸ö²»Í¬µÄʵÀýͨ¹ý¹æÔòÃû½øÐÐÒýÓá£

extra-args£ºdumper µ¥Ôª¶îÍâ²ÎÊý¡£³ý dumper µ¥ÔªÖÐÃ÷È·¶¨ÒåµÄÅäÖÃÏîÍâµÄÆäËûËùÓÐ mydumper ÅäÖÃÏͨ¹ý´Ë²ÎÊý´«È룬¸ñʽÓëʹÓà mydumper ʱһÖ¡£

ÓÐ¹Ø mydumper ¶Ô¿â±íºÚ°×Ãûµ¥µÄÖ§³Ö£¬¿É²é¿´ mydumper µÄ²ÎÊý¼° mydumper µÄÔ´Âë¡£

¶ÔÓÚ loader ºÍ syncer µ¥Ôª£¬Æä¶ÔÓ¦µÄ¿â±íºÚ°×Ãûµ¥¹æÔòΪ black-white-list¡£¼ÙÉèÖ»ÆÚÍûͬ²½ test ¿âÖÐµÄ t1¡¢t2 Á½¸ö±íµÄÊý¾Ý£¬Ôò¿ÉÅäÖÃÈçϹæÔò£º

name-of-bwl-rule:
do-tables:
- db-name: "test"
tbl-name: "t1"
- db-name: "test"
tbl-name: "t2"

ʾÀýÖÐֻʹÓÃÁ˸ùæÔòµÄ²¿·ÖÅäÖÃÏÍêÕûµÄÅäÖÃÏî¼°¸÷ÅäÖÃÏîµÄº¬Ò壬¿ÉÔĶÁ¸Ã¹¦ÄܶÔÓ¦µÄÓû§Îĵµ¡£DM ÖиùæÔòÓë MySQL µÄÖ÷´Óͬ²½¹ýÂ˹æÔòÀàËÆ£¬Òò´ËÒ²¿É²Î¿¼ Evaluation of Database-Level Replication and Binary Logging Options Óë Evaluation of Table-Level Replication Options¡£

¶ÔÓÚ loader µ¥Ôª£¬ÔÚ½âÎö SQL ÎļþÃû»ñµÃ¿âÃû±íÃûºó£¬»áÓëÅäÖõĺڰ×Ãûµ¥¹æÔò½øÐÐÆ¥Å䣬Èç¹ûÆ¥Åä½á¹ûΪ²»ÐèҪͬ²½£¬Ôò»áºöÂÔ¶ÔÓ¦µÄÕû¸ö SQL Îļþ¡£¶ÔÓÚ syncer µ¥Ôª£¬ÔÚ½âÎö binlog »ñµÃ¿âÃû±íÃûºó£¬»áÓëÅäÖõĺڰ×Ãûµ¥¹æÔò½øÐÐÆ¥Å䣬Èç¹ûÆ¥Åä½á¹ûΪ²»ÐèҪͬ²½£¬Ôò»áºöÂÔ¶ÔÓ¦µÄ£¨²¿·Ö£©binlog event Êý¾Ý¡£

binlog event ¹ýÂË

ÔÚ½øÐÐÔöÁ¿Êý¾Ýͬ²½Ê±£¬ÓÐʱ»áÆÚÍû¹ýÂ˵ôÄ³Ð©ÌØ¶¨ÀàÐ굀 binlog event£¬Á½¸öµäÐ͵ij¡¾°°üÀ¨£º

ÉÏÓÎÖ´ÐÐ TRUNCATE TABLE ʱ²»Ï£ÍûÇå¿ÕÏÂÓαíÖеÄÊý¾Ý¡£

ÉÏÓηֱíÉÏÖ´ÐÐ DROP TABLE ʱ²»Ï£Íû DROP ÏÂÓκϲ¢ºóµÄ±í¡£

ÔÚ DM ÖÐÖ§³Ö¸ù¾Ý binlog event µÄÀàÐͽøÐйýÂË£¬¶ÔÓÚÐèÒª¹ýÂË TRUNCATE TABLE Óë DROP TABLE µÄ³¡¾°£¬¿ÉÅäÖùæÔòÈçÏ£º

name-of-filter-rule:
schema-pattern: "test_*"
table-pattern: "t_*"
events: ["truncate table", "drop table"]
action: Ignore

¹æÔòµÄÆ¥ÅäģʽÓë table router¡¢column mapping ÀàËÆ£¬¾ßÌåµÄÅäÖÃÏî¿ÉÔĶÁ¸Ã¹¦ÄܶÔÓ¦µÄÓû§Îĵµ¡£

ÔÚʵÏÖÉÏ£¬µ±½âÎö binlog event »ñµÃ¿âÃû¡¢±íÃû¼° binlog event ÀàÐͺó£¬ÓëÅäÖõĹæÔò½øÐÐÆ¥Å䣬²¢ÔÚÆ¥ÅäºóÒÀ¾Ý action ÅäÖÃÏîÀ´¾ö¶¨ÊÇ·ñÐèÒª½øÐйýÂË¡£ÓÐ¹Ø binlog event ¹ýÂ˹¦ÄܵľßÌåʵÏÖ£¬¿ÉÒÔÔĶÁ TiDB-Tools Ï嵀 binlog-filter pkg Ô´´úÂë¡£

   
1841 ´Îä¯ÀÀ       29
Ïà¹ØÎÄÕÂ

»ùÓÚEAµÄÊý¾Ý¿â½¨Ä£
Êý¾ÝÁ÷½¨Ä££¨EAÖ¸ÄÏ£©
¡°Êý¾Ýºþ¡±£º¸ÅÄî¡¢ÌØÕ÷¡¢¼Ü¹¹Óë°¸Àý
ÔÚÏßÉ̳ÇÊý¾Ý¿âϵͳÉè¼Æ ˼·+Ч¹û
 
Ïà¹ØÎĵµ

GreenplumÊý¾Ý¿â»ù´¡Åàѵ
MySQL5.1ÐÔÄÜÓÅ»¯·½°¸
ijµçÉÌÊý¾ÝÖÐ̨¼Ü¹¹Êµ¼ù
MySQL¸ßÀ©Õ¹¼Ü¹¹Éè¼Æ
Ïà¹Ø¿Î³Ì

Êý¾ÝÖÎÀí¡¢Êý¾Ý¼Ü¹¹¼°Êý¾Ý±ê×¼
MongoDBʵս¿Î³Ì
²¢·¢¡¢´óÈÝÁ¿¡¢¸ßÐÔÄÜÊý¾Ý¿âÉè¼ÆÓëÓÅ»¯
PostgreSQLÊý¾Ý¿âʵսÅàѵ