±à¼ÍƼö: |
±¾ÎĽéÉÜÈçºÎ´Ó
MySQL/MariaDB Ç¨ÒÆµ½ TiDB£¬ÒÔ¼°½« TiDB ×÷Ϊ MySQL/MariaDB
µÄ´Ó¿âµÄʵÏÖÔÀí£¬Ï£Íû¶ÔÄúµÄѧϰÓÐËù°ïÖú¡£
±¾ÎÄÀ´×ÔÓÚ¼òÊ飬ÓÉ»ðÁú¹ûÈí¼þAlice±à¼¡¢ÍƼö¡£
|
|
¼ò½é
TiDB-DM£¨Data Migration£©ÊÇÓÃÓÚ½«Êý¾Ý´Ó MySQL/MariaDB
Ç¨ÒÆµ½ TiDB µÄ¹¤¾ß¡£¸Ã¹¤¾ß¼ÈÖ§³ÖÒÔÈ«Á¿±¸·ÝÎļþµÄ·½Ê½½« MySQL/MariaDB µÄÊý¾Ýµ¼Èëµ½
TiDB£¬Ò²Ö§³Öͨ¹ý½âÎöÖ´ÐÐ MySQL/MariaDB binlog µÄ·½Ê½½«Êý¾ÝÔöÁ¿Í¬²½µ½ TiDB¡£ÌØ±ðµØ£¬¶ÔÓÚÓжà¸ö
MySQL/MariaDB ʵÀýµÄ·Ö¿â·Ö±íÐèÒªºÏ²¢ºóͬ²½µ½Í¬Ò»¸ö TiDB ¼¯ÈºµÄ³¡¾°£¬DM ÌṩÁËÁ¼ºÃµÄÖ§³Ö¡£
DM ÊǼ¯ÈºÄ£Ê½µÄ£¬ÆäÖ÷ÒªÓÉ DM-master¡¢DM-worker Óë DM-ctl Èý¸ö×é¼þ×é³É£¬Äܹ»ÒÔ¶à¶Ô¶àµÄ·½Ê½½«¶à¸öÉÏÓÎ
MySQL ʵÀýµÄÊý¾Ýͬ²½µ½¶à¸öÏÂÓÎ TiDB ¼¯Èº£¬Æä¼Ü¹¹Í¼ÈçÏ£º
DM-master£º¹ÜÀíÕû¸ö DM ¼¯Èº£¬Î¬»¤¼¯ÈºµÄÍØÆËÐÅÏ¢£¬¼à¿Ø¸÷¸ö DM-worker ʵÀýµÄÔËÐÐ״̬£»½øÐÐÊý¾Ýͬ²½ÈÎÎñµÄ²ð½âÓë·Ö·¢£¬¼à¿ØÊý¾Ýͬ²½ÈÎÎñµÄÖ´ÐÐ״̬£»ÔÚ½øÐкϿâºÏ±íµÄÔöÁ¿Êý¾Ýͬ²½Ê±£¬Ðµ÷¸÷
DM-worker ÉÏ DDL µÄÖ´ÐлòÌø¹ý£»ÌṩÊý¾Ýͬ²½ÈÎÎñ¹ÜÀíµÄͳһÈë¿Ú¡£
DM-worker£ºÓëÉÏÓÎ MySQL ʵÀýÒ»Ò»¶ÔÓ¦£¬Ö´ÐоßÌåµÄÈ«Á¿¡¢ÔöÁ¿Êý¾Ýͬ²½ÈÎÎñ£»½«ÉÏÓÎ MySQL
µÄ binlog ÀÈ¡µ½±¾µØ²¢³Ö¾Ã»¯±£´æ£»¸ù¾Ý¶¨ÒåµÄÊý¾Ýͬ²½ÈÎÎñ£¬½«ÉÏÓÎ MySQL Êý¾ÝÈ«Á¿µ¼³ö³É
SQL Îļþºóµ¼Èëµ½ÏÂÓÎ TiDB£¬»ò½âÎö±¾µØ³Ö¾Ã»¯µÄ binlog ºóÔöÁ¿Í¬²½µ½ÏÂÓÎ TiDB£»±àÅÅ
DM-master ²ð½âºóµÄÊý¾Ýͬ²½×ÓÈÎÎñ£¬¼à¿Ø×ÓÈÎÎñµÄÔËÐÐ״̬¡£
DM-ctl£ºÃüÁîÐн»»¥¹¤¾ß£¬Í¨¹ýÁ¬½Óµ½ DM-master ºó£¬Ö´ÐÐ DM ¼¯ÈºµÄ¹ÜÀíÓëÊý¾Ýͬ²½ÈÎÎñµÄ¹ÜÀí¡£
ʵÏÖÔÀí
Êý¾ÝÇ¨ÒÆÁ÷³Ì
µ¥¸ö DM ¼¯Èº¿ÉÒÔͬʱÔËÐжà¸öÊý¾Ýͬ²½ÈÎÎñ£»¶ÔÓÚÿһ¸öͬ²½ÈÎÎñ£¬¿ÉÒÔ²ð½âΪ¶à¸ö×ÓÈÎÎñͬʱÓɶà¸ö DM-worker
½Úµã³Ðµ££¬ÆäÖÐÿ¸ö DM-worker ½Úµã¸ºÔðͬ²½À´×Ô¶ÔÓ¦µÄÉÏÓÎ MySQL ʵÀýµÄÊý¾Ý¡£¶ÔÓÚµ¥¸ö
DM-worker ½ÚµãÉϵĵ¥¸öÊý¾Ýͬ²½×ÓÈÎÎñ£¬ÆäÊý¾ÝÇ¨ÒÆÁ÷³ÌÈçÏ£¬ÆäÖÐÉϲ¿µÄÊý¾ÝÁ÷ÏòΪȫÁ¿Êý¾ÝÇ¨ÒÆ¡¢Ï²¿µÄÊý¾ÝÁ÷ÏòΪÔöÁ¿Êý¾Ýͬ²½£º
ÔÚÿ¸ö DM-worker ½ÚµãÄÚ²¿£¬¶ÔÓÚÌØ¶¨µÄÊý¾Ýͬ²½×ÓÈÎÎñ£¬Ö÷ÒªÓÉ dumper¡¢loader¡¢relay
Óë syncer£¨binlog replication£©µÈÊý¾Ýͬ²½´¦Àíµ¥ÔªÖ´ÐоßÌåµÄÊý¾Ýͬ²½²Ù×÷¡£
¶ÔÓÚÈ«Á¿Êý¾ÝÇ¨ÒÆ£¬DM Ê×ÏÈʹÓà dumper µ¥Ôª´ÓÉÏÓÎ MySQL Öн«±í½á¹¹ÓëÊý¾Ýµ¼³ö³É SQL
Îļþ£»È»ºóʹÓà loader µ¥Ôª¶ÁÈ¡ÕâЩ SQL Îļþ²¢Í¬²½µ½ÏÂÓÎ TiDB¡£
¶ÔÓÚÔöÁ¿Êý¾Ýͬ²½£¬Ê×ÏÈʹÓà relay µ¥Ôª×÷Ϊ slave Á¬½Óµ½ÉÏÓÎ MySQL ²¢ÀÈ¡ binlog
Êý¾Ýºó×÷Ϊ relay log ³Ö¾Ã»¯´æ´¢ÔÚ±¾µØ£¬È»ºóʹÓà syncer µ¥Ôª¶ÁÈ¡ÕâЩ relay
log ²¢½âÎö¹¹Ôì³É SQL Óï¾äºóͬ²½µ½ÏÂÓÎ TiDB¡£Õâ¸öÔöÁ¿Í¬²½µÄ¹ý³ÌÓë MySQL µÄÖ÷´Ó¸´ÖÆÀàËÆ£¬Ö÷񻂿±ðÔÚÓÚÔÚ
DM ÖУ¬±¾µØ³Ö¾Ã»¯µÄ relay log ¿ÉÒÔͬʱ¹©¶à¸ö²»Í¬×ÓÈÎÎñµÄ syncer µ¥ÔªËù¹²Ó㬱ÜÃâÁ˶à¸öÈÎÎñÐèÒªÖØ¸´´ÓÉÏÓÎ
MySQL ÀÈ¡ binlog µÄÎÊÌâ¡£
Êý¾ÝÇ¨ÒÆ²¢·¢Ä£ÐÍ
Ϊ¼Ó¿ìÊý¾Ýµ¼ÈëËÙ¶È£¬ÔÚ DM Öв»ÂÛÊÇÈ«Á¿Êý¾ÝÇ¨ÒÆ£¬»¹ÊÇÔöÁ¿Êý¾Ýͬ²½£¬¶¼ÔÚÆäÖв¿·Ö½×¶ÎʹÓÃÁ˲¢·¢´¦Àí¡£
¶ÔÓÚÈ«Á¿Êý¾ÝÇ¨ÒÆ£¬ÔÚµ¼³ö½×¶Î£¬dumper µ¥Ôªµ÷Óà mydumper µ¼³ö¹¤¾ßÖ´ÐÐʵ¼ÊµÄÊý¾Ýµ¼³ö²Ù×÷£¬¶ÔÓ¦µÄ²¢·¢Ä£ÐÍ¿ÉÒÔÖ±½Ó²Î¿¼
mydumper µÄÔ´Âë¡£ÔÚʹÓà loader µ¥ÔªÖ´Ðеĵ¼Èë½×¶Î£¬¶ÔÓ¦µÄ²¢·¢Ä£ÐͽṹÈçÏ£º
ʹÓà mydumper Ö´Ðе¼³öʱ£¬¿ÉÒÔͨ¹ý --chunk-filesize µÈ²ÎÊý½«µ¥¸ö±í²ð·Ö³É¶à¸ö
SQL Îļþ£¬ÕâЩ SQL Îļþ¶ÔÓ¦µÄ¶¼ÊÇÉÏÓÎ MySQL ijһ¸öʱ¿ÌµÄ¾²Ì¬¿ìÕÕÊý¾Ý£¬ÇÒ¸÷ SQL
Îļþ¼äµÄÊý¾Ý²»´æÔÚ¹ØÁª¡£Òò´Ë£¬ÔÚʹÓà loader µ¥ÔªÖ´Ðе¼Èëʱ£¬¿ÉÒÔÖ±½ÓÔÚÒ»¸ö loader µ¥ÔªÄÚÆô¶¯¶à¸ö
worker ¹¤×÷г̣¬Óɸ÷ worker г̲¢·¢¡¢¶ÀÁ¢µØÃ¿´Î¶Áȡһ¸ö´ýµ¼ÈëµÄ SQL Îļþ½øÐе¼Èë¡£¼´
loader µ¼Èë½×¶Î£¬ÊÇÒÔ SQL Îļþ¼¶±ðÁ£¶È²¢·¢½øÐеġ£ÔÚ DM µÄÈÎÎñÅäÖÃÖУ¬¶ÔÓÚ loader
µ¥Ôª£¬ÆäÖÐµÄ pool-size ²ÎÊý¼´ÓÃÓÚ¿ØÖÆ´Ë´¦ worker гÌÊýÁ¿¡£
¶ÔÓÚÔöÁ¿Êý¾Ýͬ²½£¬ÔÚ´ÓÉÏÓÎÀÈ¡ binlog ²¢³Ö¾Ã»¯µ½±¾µØµÄ½×¶Î£¬ÓÉÓÚÉÏÓÎ MySQL ÉÏ binlog
µÄ²úÉúÓë·¢ËÍÊÇÒÔ stream ÐÎʽ½øÐеģ¬Òò´ËÕⲿ·ÖÖ»ÄÜ´®Ðд¦Àí¡£ÔÚʹÓà syncer µ¥ÔªÖ´Ðеĵ¼Èë½×¶Î£¬ÔÚÒ»¶¨µÄÏÞÖÆÌõ¼þÏ£¬¿ÉÒÔÖ´Ðв¢·¢µ¼È룬¶ÔÓ¦µÄÄ£ÐͽṹÈçÏ£º
µ± syncer ¶ÁÈ¡Óë½âÎö±¾µØ relay log ʱ£¬Óë´ÓÉÏÓÎÀÈ¡ binlog ÀàËÆ£¬ÊÇÒÔ stream
ÐÎʽ½øÐеģ¬Òò´ËÒ²Ö»ÄÜ´®Ðд¦Àí¡£µ± syncer ½âÎö³ö¸÷ binlog event ²¢¹¹Ôì³É´ýͬ²½µÄ
job ºó£¬Ôò¿ÉÒÔ¸ù¾Ý¶ÔÓ¦ÐÐÊý¾ÝµÄÖ÷¼ü¡¢Ë÷ÒýµÈÐÅÏ¢¾¹ý hash ¼ÆËãºó·Ö·¢µ½¶à¸ö²»Í¬µÄ´ýͬ²½ job
channel ÖУ»ÔÚ channel µÄÁíÒ»¶Ë£¬Óë¸÷¸ö channel ¶ÔÓ¦µÄ worker г̲¢·¢µØ´Ó
channel ÖÐÈ¡³ö job ºóͬ²½µ½ÏÂÓ뵀 TiDB¡£¼´ syncer µ¼Èë½×¶Î£¬ÊÇÒÔ binlog
event ¼¶±ðÁ£¶È²¢·¢½øÐеġ£ÔÚ DM µÄÈÎÎñÅäÖÃÖУ¬¶ÔÓÚ syncer µ¥Ôª£¬ÆäÖÐµÄ worker-count
²ÎÊý¼´ÓÃÓÚ¿ØÖÆ´Ë´¦ worker гÌÊýÁ¿¡£
µ« syncer ²¢·¢Í¬²½µ½ÏÂÓÎ TiDB ʱ£¬´æÔÚһЩÏÞÖÆ£¬Ö÷Òª°üÀ¨£º
¶ÔÓÚ DDL£¬ÓÉÓÚ»á±ä¸üÏÂÓεıí½á¹¹£¬Òò´Ë±ØÐëÈ·±£Ôھɱí½á¹¹¶ÔÓ¦µÄ DML ¶¼Í¬²½Íê³Éºó£¬²ÅÄܽøÐÐͬ²½¡£ÔÚ
DM ÖУ¬µ±½âÎö binlog event µÃµ½ DDL ºó£¬»áÏòÿһ¸ö job channel ·¢ËÍÒ»¸öÌØÊâµÄ
flush job£»µ±¸÷ worker гÌÓöµ½ flush job ʱ£¬»áÁ¢¿ÌÏòÏÂÓÎ TiDB ͬ²½Ö®Ç°ÒѾȡ³öµÄËùÓÐ
job£»µÈ¸÷ job channel ÖÐµÄ job ¶¼Í¬²½µ½ÏÂÓÎ TiDB ºó£¬¿ªÊ¼Í¬²½ DDL£»µÈ´ý
DDL ͬ²½Íê³Éºó£¬¼ÌÐøÍ¬²½ºóÐøµÄ DML¡£¼´ DDL ²»ÄÜÓë DML ²¢·¢Í¬²½£¬ÇÒ DDL ֮ǰÓëÖ®ºóµÄ
DML Ò²²»Äܲ¢·¢Í¬²½¡£sharding ³¡¾°Ï DDL µÄͬ²½´¦Àí¼ûºóÎÄ¡£
¶ÔÓÚ DML£¬¶àÌõ DML ¿ÉÄÜ»áÐÞ¸ÄͬһÐеÄÊý¾Ý£¬ÉõÖÁÊÇÖ÷¼ü¡£Èç¹û²¢·¢µØÍ¬²½ÕâЩ DML£¬Ôò¿ÉÄÜÔì³Éͬ²½ºóÊý¾ÝµÄ²»Ò»Ö¡£DM
ÖжÔÓÚ DML Ö®¼äµÄ³åÍ»¼ì²âÓë´¦Àí£¬Óë TiDB-Binlog ÖеĴ¦ÀíÀàËÆ£¬¾ßÌåÔÀí¿ÉÒÔÔĶÁ¡¶TiDB
EcoSystem Tools ÔÀí½â¶Á£¨Ò»£©TiDB-Binlog ¼Ü¹¹ÑݽøÓëʵÏÖÔÀí¡·ÖйØÓÚ Drainer
ÄÚ SQL Ö®¼ä³åÍ»¼ì²âµÄÌÖÂÛ¡£
ºÏ¿âºÏ±íÊý¾Ýͬ²½
ÔÚʹÓà MySQL Ö§³Å´óÁ¿Êý¾Ýʱ£¬¾³£»áÑ¡ÔñʹÓ÷ֿâ·Ö±íµÄ·½°¸¡£µ«µ±½«Êý¾Ýͬ²½µ½ TiDB ºó£¬Í¨³£Ï£ÍûÂß¼ÉϽøÐкϿâºÏ±í¡£DM
Ϊ֧³ÖºÏ¿âºÏ±íµÄÊý¾Ýͬ²½£¬Ö÷ҪʵÏÖÁËÒÔϵÄһЩ¹¦ÄÜ¡£
table router
Ϊ˵Ã÷ DM ÖÐ table router£¨±íÃû·ÓÉ£©¹¦ÄÜ£¬ÏÈ¿´ÈçÏÂͼËùʾµÄÒ»¸öÀý×Ó£º
ÔÚÕâ¸öÀý×ÓÖУ¬ÉÏÓÎÓÐ 2 ¸ö MySQL ʵÀý£¬Ã¿¸öʵÀýÓÐ 2 ¸öÂß¼¿â£¬Ã¿¸ö¿âÓÐ 2 ¸ö±í£¬×ܹ²
8 ¸ö±í¡£µ±Í¬²½µ½ÏÂÓÎ TiDB ºó£¬Ï£ÍûËùÓеÄÕâ 8 ¸ö±í×îÖÕ¶¼ºÏ²¢Í¬²½µ½Í¬Ò»¸ö±íÖС£
µ«ÎªÁËÄܽ« 8 ¸öÀ´×Ô²»Í¬ÊµÀý¡¢²»Í¬¿âÇÒÓв»Í¬ÃûµÄ±íͬ²½µ½Í¬Ò»¸ö±íÖУ¬Ê×ÏÈÒª´¦ÀíµÄ£¬¾ÍÊÇÒªÄܸù¾ÝijЩ¶¨ÒåºÃµÄ¹æÔò£¬½«À´×Ô²»Í¬±íµÄÊý¾Ý¶¼Â·Óɵ½ÏÂÓεÄͬһ¸ö±íÖС£ÔÚ
DM ÖУ¬ÕâÀà¹æÔò½Ð×ö router-rules¡£¶ÔÓÚÉÏÃæµÄʾÀý£¬Æä¹æÔòÈçÏ£º
name-of-router-rule:
schema-pattern: "schema_*"
table-pattern: "table_*"
target-schema: "schema"
target-table: "table" |
name-of-router-rule£º¹æÔòÃû£¬Óû§Ö¸¶¨¡£µ±Óжà¸öÉÏÓÎʵÀýÐèҪʹÓÃÏàͬµÄ¹æÔòʱ£¬¿ÉÒÔÖ»¶¨ÒåÒ»Ìõ¹æÔò£¬¶à¸ö²»Í¬µÄʵÀýͨ¹ý¹æÔòÃû½øÐÐÒýÓá£
schema-pattern£ºÓÃÓÚÆ¥ÅäÉÏÓο⣨schema£©ÃûµÄģʽ£¬Ö§³ÖÔÚβ²¿Ê¹ÓÃͨÅä·û£¨*£©¡£ÕâÀïʹÓÃ
schema_* ¼´¿ÉÆ¥Å䵽ʾÀýÖеÄÁ½¸ö¿âÃû¡£
table-pattern£ºÓÃÓÚÆ¥ÅäÉÏÓαíÃûµÄģʽ£¬Óë schema-pattern ÀàËÆ¡£ÕâÀïʹÓÃ
table_* ¼´¿ÉÆ¥Å䵽ʾÀýÖеÄÁ½¸ö±íÃû¡£
target-schema£ºÄ¿±ê¿âÃû¡£¶ÔÓÚ¿âÃû¡¢±íÃûÆ¥ÅäµÄÊý¾Ý£¬½«±»Â·Óɵ½Õâ¸ö¿âÖС£
target-table£ºÄ¿±ê±íÃû¡£¶ÔÓÚ¿âÃû¡¢±íÃûÆ¥ÅäµÄÊý¾Ý£¬½«±»Â·Óɵ½ target-schema
¿âϵÄÕâ¸ö±íÖС£
ÔÚ DM ÄÚ²¿ÊµÏÖÉÏ£¬Ê×Ïȸù¾Ý schema-pattern / table-pattern ¹¹Ôì¶ÔÓ¦µÄ
trie ½á¹¹£¬²¢½«¹æÔò´æ´¢ÔÚ trie ½ÚµãÖУ»µ±ÓÐ SQL ÐèҪͬ²½µ½ÏÂÓÎʱ£¬Í¨¹ýʹÓÃÉÏÓοâÃû¡¢±íÃû²éѯ
trie ¼´¿ÉµÃµ½¶ÔÓ¦µÄ¹æÔò£¬²¢¸ù¾Ý¹æÔòÌæ»»Ô SQL ÖеĿâÃû¡¢±íÃû£»Í¨¹ýÏòÏÂÓÎ TiDB Ö´ÐÐÌæ»»ºóµÄ
SQL ¼´Íê³ÉÁ˸ù¾Ý±íÃûµÄ·ÓÉͬ²½¡£ÓÐ¹Ø router-rules ¹æÔòµÄ¾ßÌåʵÏÖ£¬¿ÉÒÔÔĶÁ TiDB-Tools
Ï嵀 table-router pkg Ô´´úÂë¡£
column mapping
ÓÐÁË table router ¹¦ÄÜ£¬ÒѾ¿ÉÒÔÍê³É»ù±¾µÄºÏ¿âºÏ±íÊý¾Ýͬ²½ÁË¡£µ«ÔÚÊý¾Ý¿âÖУ¬ÎÒÃǾ³£»áʹÓÃ×ÔÔöÀàÐ͵ÄÁÐ×÷ΪÖ÷¼ü¡£Èç¹û¶à¸öÉÏÓηֱíµÄÖ÷¼ü¸÷×Ô¶ÀÁ¢µØ×ÔÔö£¬½«ËüÃǺϲ¢Í¬²½µ½ÏÂÓκ󣬾ͺܿÉÄÜ»á³öÏÖÖ÷¼ü³åÍ»£¬Ôì³ÉÊý¾ÝµÄ²»Ò»Ö¡£ÎÒÃǿɿ´Ò»¸öÈçϵÄÀý×Ó£º
ÔÚÕâ¸öÀý×ÓÖУ¬ÉÏÓÎ 4 ¸öÐèÒªºÏ²¢Í¬²½µ½ÏÂÓεıíÖУ¬¶¼´æÔÚ id ÁÐֵΪ 1 µÄ¼Ç¼¡£¼ÙÉèÕâ¸ö id
ÁÐÊDZíµÄÖ÷¼ü¡£ÔÚͬ²½µ½ÏÂÓεĹý³ÌÖУ¬ÓÉÓÚÏà¹Ø¸üвÙ×÷ÊÇÒÔ id ÁÐ×÷ΪÌõ¼þÀ´È·¶¨ÐèÒª¸üеļǼ£¬Òò´Ë»áÔì³Éºóͬ²½µÄÊý¾Ý¸²¸ÇÇ°ÃæÒѾͬ²½¹ýµÄÊý¾Ý£¬µ¼Ö²¿·ÖÊý¾ÝµÄ¶ªÊ§¡£
ÔÚ DM ÖУ¬ÎÒÃÇͨ¹ý column mapping ¹¦ÄÜÔÚÊý¾Ýͬ²½µÄ¹ý³ÌÖÐÒÀ¾ÝÖ¸¶¨¹æÔò¶ÔÏà¹ØÁеÄÊý¾Ý½øÐÐת»»¸ÄдÀ´±ÜÃâÊý¾Ý³åÍ»Ó붪ʧ¡£¶ÔÓÚÉÏÃæµÄʾÀý£¬ÆäÖÐ
MySQL ʵÀý 1 µÄ column mapping ¹æÔòÈçÏ£º
mapping-rule-of-instance-1:
schema-pattern: "schema_*"
table-pattern: "table_*"
expression: "partition id"
source-column: "id"
target-column: "id"
arguments: ["1", "schema_",
"table_"] |
mapping-rule-of-instance-1£º¹æÔòÃû£¬Óû§Ö¸¶¨¡£ÓÉÓÚ²»Í¬µÄÉÏÓÎ MySQL
ʵÀýÐèҪת»»µÃµ½²»Í¬µÄÖµ£¬Òò´Ëͨ³£Ã¿¸ö MySQL ʵÀýʹÓÃÒ»ÌõרÓеĹæÔò¡£
schema-pattern / table-pattern£ºÉÏÓοâÃû¡¢±íÃûÆ¥Åäģʽ£¬Óë router-rules
ÖеĶÔÓ¦ÅäÖÃÏîÒ»Ö¡£
expression£º½øÐÐÊý¾Ýת»»µÄ±í´ïʽÃû¡£Ä¿Ç°³£Óõıí´ïʽ¼´Îª "partition
id"£¬Óйظñí´ïʽµÄ¾ßÌå˵Ã÷¼ûÏÂÎÄ¡£
source-column£º×ª»»±í´ïʽµÄÊäÈëÊý¾Ý¶ÔÓ¦µÄÀ´Ô´ÁÐÃû£¬"id" ±íʾÕâ¸ö±í´ïʽ½«×÷ÓÃÓÚ±íÖÐÃûΪ
id µÄÁС£ÔÝʱֻ֧³Ö¶Ôµ¥¸öÀ´Ô´ÁнøÐÐÊý¾Ýת»»¡£
target-column£º×ª»»±í´ïʽµÄÊä³öÊý¾Ý¶ÔÓ¦µÄÄ¿±êÁÐÃû£¬Óë source-column ÀàËÆ¡£ÔÝʱֻ֧³Ö¶Ôµ¥¸öÄ¿±êÁнøÐÐÊý¾Ýת»»£¬ÇÒ¶ÔÓ¦µÄÄ¿±êÁбØÐëÒѾ´æÔÚ¡£
arguments£º×ª»»±í´ïʽËùÒÀÀµµÄ²ÎÊý¡£²ÎÊý¸öÊýÓ뺬ÒåÒÀ¾ßÌå±í´ïʽ¶ø¶¨¡£
partition id ÊÇĿǰÖ÷ÒªÊÜÖ§³ÖµÄת»»±í´ïʽ£¬Æäͨ¹ýΪ bigint ÀàÐ͵ÄÖµÔö¼Ó¶þ½øÖÆÇ°×ºÀ´½â¾öÀ´×Ô²»Í¬±íµÄÊý¾ÝºÏ²¢Í¬²½ºó¿ÉÄܲúÉú³åÍ»µÄÎÊÌâ¡£partition
id µÄ arguments °üÀ¨ 3 ¸ö²ÎÊý£¬·Ö±ðΪ£º
MySQL ʵÀý ID£º±êʶÊý¾ÝµÄÀ´Ô´ MySQL ʵÀý£¬Óû§×ÔÓÉÖ¸¶¨¡£Èç "1"
±íʾƥÅä¸Ã¹æÔòµÄÊý¾ÝÀ´×ÔÓÚ MySQL ʵÀý 1£¬ÇÒÕâ¸ö±êʶ½«±»×ª»»³ÉÊýÖµºóÒÔ¶þ½øÖƵÄÐÎʽ×÷Ϊǰ׺µÄÒ»²¿·ÖÌí¼Óµ½×ª»»ºóµÄÖµÖС£
¿âÃûǰ׺£º±êʶÊý¾ÝµÄÀ´Ô´Âß¼¿â¡£Èç "schema_" Ó¦ÓÃÓÚ schema_2
Âß¼¿âʱ£¬±íʾȥ³ýǰ׺ºóʣϵIJ¿·Ö£¨Êý×Ö 2£©½«ÒÔ¶þ½øÖƵÄÐÎʽ×÷Ϊǰ׺µÄÒ»²¿·ÖÌí¼Óµ½×ª»»ºóµÄÖµÖС£
±íÃûǰ׺£º±êʶÊý¾ÝµÄÀ´Ô´±í¡£Èç "table_" Ó¦ÓÃÓÚ table_3 ±íʱ£¬±íʾȥ³ýǰ׺ºóʣϵIJ¿·Ö£¨Êý×Ö
3£©½«ÒÔ¶þ½øÖƵÄÐÎʽ×÷Ϊǰ׺µÄÒ»²¿·ÖÌí¼Óµ½×ª»»ºóµÄÖµÖС£
¸÷²¿·ÖÔÚ¾¹ýת»»ºóµÄÊýÖµÖеĶþ½øÖÆ·Ö²¼ÈçÏÂͼËùʾ£¨¸÷²¿·ÖĬÈÏËùÕ¼ÓÃµÄ bits λÊýÈçͼËùʾ£©£º
¼ÙÈçת»»Ç°µÄÔʼÊý¾ÝΪ 123£¬ÇÒÓÐÈçÉ쵀 arguments ²ÎÊýÉèÖã¬Ôòת»»ºóµÄֵΪ£º
1<<(64-1-4)
| 2<<(64-1-4-7) | 3<<(64-1-4-7-8)
| 123 |
ÁíÍ⣬arguments ÖÐµÄ 3 ¸ö²ÎÊý¾ù¿ÉÉèÖÃΪ¿Õ×Ö·û´®£¨""£©£¬¼´±íʾ¸Ã²¿·Ö²»±»Ìí¼Óµ½×ª»»ºóµÄÖµÖУ¬ÇÒ²»Õ¼ÓöîÍâµÄ
bits¡£±ÈÈ罫ÆäÉèÖÃΪ["1", "", "table_"]£¬Ôòת»»ºóµÄֵΪ£º
1 << (64-1-4)
| 3<< (64-1-4-8) | 123 |
ÓÐ¹Ø column mapping ¹¦ÄܵľßÌåʵÏÖ£¬¿ÉÒÔÔĶÁ TiDB-Tools Ï嵀 column-mapping
pkg Ô´´úÂë¡£
sharding DDL
ÓÐÁË table router ºÍ column mapping ¹¦ÄÜ£¬DML µÄºÏ¿âºÏ±íÊý¾Ýͬ²½ÒѾ¿ÉÒÔÕý³£½øÐÐÁË¡£µ«Èç¹ûÔÚÔöÁ¿Êý¾Ýͬ²½µÄ¹ý³ÌÖУ¬ÉÏÓδýºÏ²¢µÄ·Ö±íÉÏÖ´ÐÐÁË
DDL ²Ù×÷£¬Ôò¿ÉÄܳöÏÖÎÊÌâ¡£ÎÒÃÇÏÈÀ´¿´Ò»¸ö¼ò»¯ºóµÄÔÚ·Ö±íÉÏÖ´ÐÐ DDL µÄÀý×Ó¡£
ÔÚÉÏͼµÄÀý×ÓÖУ¬·Ö±íµÄºÏ¿âºÏ±í¼ò»¯³ÉÁËÉÏÓÎÖ»ÓÐÁ½¸ö MySQL ʵÀý£¬Ã¿¸öʵÀýÄÚÖ»ÓÐÒ»¸ö±í¡£¼ÙÉèÔÚ¿ªÊ¼Êý¾Ýͬ²½Ê±£¬½«Á½¸ö·Ö±íµÄ±í½á¹¹
schema µÄ°æ±¾¼ÇΪ schema V1£¬½« DDL Ö´ÐÐÍê³ÉºóµÄ±í½á¹¹ schema µÄ°æ±¾¼ÇΪ
schema V2¡£
ÏÖÔÚ£¬¼ÙÉèÊý¾Ýͬ²½¹ý³ÌÖУ¬´ÓÁ½¸öÉÏÓηֱíÊÕµ½µÄ binlog Êý¾ÝÓÐÈçϵÄʱÐò£º
¿ªÊ¼Í¬²½Ê±£¬´ÓÁ½¸ö·Ö±íÊÕµ½µÄ¶¼ÊÇ schema V1 µÄ DML¡£
ÔÚ t1 ʱ¿Ì£¬ÊÕµ½ÊµÀý 1 ÉÏ·Ö±íµÄ DDL¡£
´Ó t2 ʱ¿Ì¿ªÊ¼£¬´ÓʵÀý 1 ÊÕµ½µÄÊÇ schema V2 µÄ DML£»µ«´ÓʵÀý 2 ÊÕµ½µÄÈÔÊÇ
schema V1 µÄ DML¡£
ÔÚ t3 ʱ¿Ì£¬ÊÕµ½ÊµÀý 2 ÉÏ·Ö±íµÄ DDL¡£
´Ó t4 ʱ¿Ì¿ªÊ¼£¬´ÓʵÀý 2 ÊÕµ½µÄÒ²ÊÇ schema V2 µÄ DML¡£
¼ÙÉèÔÚÊý¾Ýͬ²½¹ý³ÌÖУ¬²»¶Ô·Ö±íµÄ DDL ½øÐд¦Àí¡£µ±½«ÊµÀý 1 µÄ DDL ͬ²½µ½ÏÂÓκó£¬ÏÂÓεıí½á¹¹»á±ä¸ü³ÉΪ
schema V2¡£µ«¶ÔÓÚʵÀý 2£¬ÔÚ t2 ʱ¿Ìµ½ t3 ʱ¿ÌÕâ¶Îʱ¼äÄÚÊÕµ½µÄÈÔÈ»ÊÇ schema
V1 µÄ DML¡£µ±³¢ÊÔ°ÑÕâЩÓë schema V1 ¶ÔÓ¦µÄ DML ͬ²½µ½ÏÂÓÎʱ£¬¾Í»áÓÉÓÚ DML
Óë±í½á¹¹µÄ²»Ò»Ö¶ø·¢Éú´íÎó£¬Ôì³ÉÊý¾ÝÎÞ·¨Õýȷͬ²½¡£
¼ÌÐøÊ¹ÓÃÉÏÃæµÄÀý×Ó£¬À´¿´¿´ÎÒÃÇÔÚ DM ÖÐÊÇÈçºÎ´¦ÀíºÏ¿âºÏ±í¹ý³ÌÖÐµÄ DDL ͬ²½µÄ¡£
ÔÚÕâ¸öÀý×ÓÖУ¬DM-worker-1 ÓÃÓÚͬ²½À´×Ô MySQL ʵÀý 1 µÄÊý¾Ý£¬DM-worker-2
ÓÃÓÚͬ²½À´×Ô MySQL ʵÀý 2 µÄÊý¾Ý£¬DM-master ÓÃÓÚе÷¶à¸ö DM-worker ¼äµÄ
DDL ͬ²½¡£´Ó DM-worker-1 ÊÕµ½ DDL ¿ªÊ¼£¬¼ò»¯ºóµÄ DDL ͬ²½Á÷³ÌΪ£º
1.DM-worker-1 ÔÚ t1 ʱ¿ÌÊÕµ½À´×Ô MySQL ʵÀý
1 µÄ DDL£¬×ÔÉíÔÝÍ£¸Ã DDL ¶ÔÓ¦ÈÎÎñµÄ DDL ¼° DML Êý¾Ýͬ²½£¬²¢½« DDL Ïà¹ØÐÅÏ¢·¢Ë͸ø
DM-master¡£
2.DM-master ¸ù¾Ý DDL ÐÅÏ¢ÅжÏÐèҪе÷¸Ã DDL
µÄͬ²½£¬Îª¸Ã DDL ´´½¨Ò»¸öËø£¬²¢½« DDL ËøÐÅÏ¢·¢»Ø¸ø DM-worker-1£¬Í¬Ê±½« DM-worker-1
±ê¼ÇΪÕâ¸öËøµÄ owner¡£
3.DM-worker-2 ¼ÌÐø½øÐÐ DML µÄͬ²½£¬Ö±µ½ÔÚ t3
ʱ¿ÌÊÕµ½À´×Ô MySQL ʵÀý 2 µÄ DDL£¬×ÔÉíÔÝÍ£¸Ã DDL ¶ÔÓ¦ÈÎÎñµÄÊý¾Ýͬ²½£¬²¢½« DDL
Ïà¹ØÐÅÏ¢·¢Ë͸ø DM-master¡£
4.DM-master ¸ù¾Ý DDL ÐÅÏ¢Åжϸà DDL ¶ÔÓ¦µÄËøÐÅÏ¢ÒѾ´æÔÚ£¬Ö±½Ó½«¶ÔÓ¦ËøÐÅÏ¢·¢»Ø¸ø
DM-worker-2¡£
5.DM-master ¸ù¾ÝÆô¶¯ÈÎÎñʱµÄÅäÖÃÐÅÏ¢¡¢ÉÏÓÎ MySQL
ʵÀý·Ö±íÐÅÏ¢¡¢²¿ÊðÍØÆËÐÅÏ¢µÈ£¬ÅжϵÃÖªÒѾÊÕµ½ÁËÐèÒªºÏ±íµÄËùÓÐÉÏÓηֱíµÄ¸Ã DDL£¬ÇëÇó DDL ËøµÄ
owner£¨DM-worker-1£©ÏòÏÂÓÎͬ²½Ö´Ðиà DDL¡£
6.DM-worker-1 ¸ù¾Ý step 2 ʱÊÕµ½µÄ DDL
ËøÐÅÏ¢ÑéÖ¤ DDL Ö´ÐÐÇëÇó£»ÏòÏÂÓÎÖ´ÐÐ DDL£¬²¢½«Ö´Ðнá¹û·´À¡¸ø DM-master£»ÈôÖ´ÐÐ DDL
³É¹¦£¬Ôò×ÔÉí¿ªÊ¼¼ÌÐøÍ¬²½ºóÐøµÄ£¨´Ó t2 ʱ¿Ì¶ÔÓ¦µÄ binlog ¿ªÊ¼µÄ£©DML¡£
7.DM-master ÊÕµ½À´×Ô owner Ö´ÐÐ DDL ³É¹¦µÄÏìÓ¦£¬ÇëÇóÔڵȴý¸Ã
DDL ËøµÄËùÓÐÆäËû DM-worker£¨DM-worker-2£©ºöÂԸà DDL£¬Ö±½Ó¼ÌÐøÍ¬²½ºóÐøµÄ£¨´Ó
t4 ʱ¿Ì¶ÔÓ¦µÄ binlog ¿ªÊ¼µÄ£©DML¡£
¸ù¾ÝÉÏÃæ DM ´¦Àí¶à¸ö DM-worker ¼äµÄ DDL ͬ²½µÄÁ÷³Ì£¬¹éÄÉһϠDM ÄÚ´¦Àí¶à¸ö
DM-worker ¼ä sharding DDL ͬ²½µÄÌØµã£º
1.¸ù¾ÝÈÎÎñÅäÖÃÓë DM ¼¯Èº²¿ÊðÍØÆËÐÅÏ¢£¬ÔÚ DM-master
ÄÚ½¨Á¢Ò»¸öÐèҪе÷ DDL ͬ²½µÄÂß¼ sharding group£¬group ÖеijÉԱΪ´¦Àí¸ÃÈÎÎñ²ð½âºó¸÷×ÓÈÎÎñµÄ
DM-worker¡£
2.¸÷ DM-worker ÔÚ´Ó binlog event ÖлñÈ¡µ½
DDL ºó£¬»á½« DDL ÐÅÏ¢·¢Ë͸ø DM-master¡£
3.DM-master ¸ù¾ÝÀ´×Ô DM-worker µÄ DDL
ÐÅÏ¢¼° sharding group ÐÅÏ¢´´½¨/¸üРDDL Ëø¡£
4.Èç¹û sharding group µÄËùÓгÉÔ±¶¼ÊÕµ½ÁËijһÌõ
DDL£¬Ôò±íÃ÷ÉÏÓηֱíÔڸà DDL Ö´ÐÐǰµÄ DML ¶¼ÒѾͬ²½Íê³É£¬¿ÉÒÔÖ´ÐÐ DDL£¬²¢¼ÌÐøºóÐøµÄ
DML ͬ²½¡£
5.ÉÏÓηֱíµÄ DDL ÔÚ¾¹ý table router ת»»ºó£¬¶ÔÓ¦ÐèÒªÔÚÏÂÓÎÖ´ÐеÄ
DDL Ó¦¸ÃÒ»Ö£¬Òò´Ë½öÐè DDL ËøµÄ owner Ö´ÐÐÒ»´Î¼´¿É£¬ÆäËû DM-worker ¿ÉÖ±½ÓºöÂÔ¶ÔÓ¦µÄ
DDL¡£
´Ó DM ´¦Àí DM-worker ¼ä sharding DDL ͬ²½µÄÌØµã£¬¿ÉÒÔ¿´³ö¸Ã¹¦ÄÜ´æÔÚÒÔÏÂһЩÏÞÖÆ£º
1.ÉÏÓÎµÄ·Ö±í±ØÐëÒÔÏàͬµÄ˳ÐòÖ´ÐУ¨table router ת»»ºóÏàͬµÄ£©DDL£¬±ÈÈç±í
1 ÏÈÔö¼ÓÁÐ a ºóÔÙÔö¼ÓÁÐ b£¬¶ø±í 2 ÏÈÔö¼ÓÁÐ b ºóÔÙÔö¼ÓÁÐ a£¬ÕâÖÖ²»Í¬Ë³ÐòµÄ DDL Ö´Ðз½Ê½ÊDz»Ö§³ÖµÄ¡£
2.Ò»¸öÂß¼ sharding group ÄÚµÄËùÓÐ DM-worker
¶ÔÓ¦µÄÉÏÓÎ·Ö±í£¬¶¼Ó¦¸ÃÖ´ÐжÔÓ¦µÄ DDL£¬±ÈÈçÆäÖÐÓÐ DM-worker-2 ¶ÔÓ¦µÄÉÏÓηֱíδִÐÐ
DDL£¬ÔòÆäËûÒÑÖ´ÐÐ DDL µÄ DM-worker ¶¼»áÔÝͣͬ²½ÈÎÎñ£¬µÈ´ý DM-worker-2
ÊÕµ½¶ÔÓ¦ÉÏÓ뵀 DDL¡£
3.ÓÉÓÚÒѾÊÕµ½µÄ DDL µÄ DM-worker »áÔÝÍ£ÈÎÎñÒԵȴýÆäËû
DM-worker ÊÕµ½¶ÔÓ¦µÄ DDL£¬Òò´ËÊý¾Ýͬ²½ÑÓ³Ù»áÔö¼Ó¡£
4.ÔöÁ¿Í¬²½¿ªÊ¼Ê±£¬ÐèÒªºÏ²¢µÄËùÓÐÉÏÓηֱí½á¹¹±ØÐëÒ»Ö£¬²ÅÄÜÈ·±£À´×Ô²»Í¬·Ö±íµÄ
DML ¿ÉÒÔͬ²½µ½Ò»¸öÈ·¶¨±í½á¹¹µÄÏÂÓΣ¬Ò²²ÅÄÜÈ·±£ºóÐø¸÷·Ö±íµÄ DDL Äܹ»ÕýÈ·Æ¥ÅäÓëͬ²½¡£
ÔÚÉÏÃæµÄʾÀýÖУ¬Ã¿¸ö DM-worker ¶ÔÓ¦µÄÉÏÓÎ MySQL ʵÀýÖÐÖ»ÓÐÒ»¸öÐèÒª½øÐкϲ¢µÄ·Ö±í¡£µ«ÔÚʵ¼Ê³¡¾°Ï£¬Ò»¸ö
MySQL ʵÀý¿ÉÄÜÓжà¸ö·Ö¿âÄڵĶà¸ö·Ö±íÐèÒª½øÐкϲ¢£¬±ÈÈçÇ°Ãæ½éÉÜ table router Óë
column mapping ¹¦ÄÜʱµÄÀý×Ó¡£µ±Ò»¸ö MySQL ʵÀýÖÐÓжà¸ö·Ö±íÐèÒªºÏ²¢Ê±£¬sharding
DDL µÄе÷ͬ²½¹ý³ÌÔö¼ÓÁ˸ü¶àµÄ¸´ÔÓÐÔ¡£
¼ÙÉèͬһ¸ö MySQL ʵÀýÖÐÓÐ table_1 ºÍ table_2 Á½¸ö·Ö±íÐèÒª½øÐкϲ¢£¬ÈçÏÂͼ£º
ÓÉÓÚÊý¾ÝÀ´×Ôͬһ¸ö MySQL ʵÀý£¬Òò´ËËùÓÐÊý¾Ý¶¼ÊÇ´Óͬһ¸ö binlog Á÷ÖлñµÃ¡£ÔÚÕâ¸öÀý×ÓÖУ¬Ê±ÐòÈçÏ£º
1.¿ªÊ¼Í¬²½Ê±£¬Á½¸ö·Ö±íÊÕµ½µÄÊý¾Ý¶¼ÊÇ schema V1 µÄ DML¡£
2.ÔÚ t1 ʱ¿Ì£¬ÊÕµ½ÁË table_1 µÄ DDL¡£
3.´Ó t2 ʱ¿Ìµ½ t3 ʱ¿Ì£¬ÊÕµ½µÄÊý¾Ýͬʱ°üº¬ table_1
schema V2 µÄ DML ¼° table_2 schema V1 µÄ DML¡£
4.ÔÚ t3 ʱ¿Ì£¬ÊÕµ½ÁË table_2 µÄ DDL¡£
5.´Ó t4 ʱ¿Ì¿ªÊ¼£¬Á½¸ö·Ö±íÊÕµ½µÄÊý¾Ý¶¼ÊÇ schema V2
µÄ DML¡£
¼ÙÉèÔÚÊý¾Ýͬ²½¹ý³ÌÖв»¶Ô DDL ½øÐÐÌØÊâ´¦Àí£¬µ± table_1 µÄ DDL ͬ²½µ½ÏÂÓΡ¢±ä¸üÏÂÓαí½á¹¹ºó£¬table_2
schema V1 µÄ DML ½«ÎÞ·¨Õý³£Í¬²½¡£Òò´Ë£¬ÔÚµ¥¸ö DM-worker ÄÚ²¿£¬ÎÒÃÇÒ²¹¹ÔìÁËÓë
DM-master ÄÚÀàËÆµÄÂß¼ sharding group£¬µ« group µÄ³ÉÔ±ÊÇͬһ¸öÉÏÓÎ
MySQL ʵÀýµÄ²»Í¬·Ö±í¡£
µ« DM-worker ÄÚе÷´¦Àí sharding group µÄͬ²½²»ÄÜÍêÈ«Óë DM-master
´¦ÀíʱһÖ£¬Ö÷ÒªÔÒò°üÀ¨£º
µ±ÊÕµ½ table_1 µÄ DDL ʱ£¬Í¬²½²»ÄÜÔÝÍ££¬ÐèÒª¼ÌÐø½âÎö binlog ²ÅÄÜ»ñµÃºóÐø table_2µÄ
DDL£¬¼´ÐèÒª´Ó t2 ʱ¿Ì¼ÌÐøÏòǰ½âÎöÖ±µ½ t3 ʱ¿Ì¡£
ÔÚ¼ÌÐø½âÎö t2 ʱ¿Ìµ½ t3 ʱ¿ÌµÄ binlog µÄ¹ý³ÌÖУ¬table_1 µÄ schema V2
µÄ DML ²»ÄÜÏòÏÂÓÎͬ²½£»µ«ÔÚ sharding DDL ͬ²½²¢Ö´Ðгɹ¦ºó£¬ÕâЩ DML ÐèҪͬ²½µ½ÏÂÓΡ£
ÔÚ DM ÖУ¬¼ò»¯ºóµÄ DM-worker ÄÚ sharding DDL ͬ²½Á÷³ÌΪ£º
1.ÔÚ t1 ʱ¿ÌÊÕµ½ table_1 µÄ DDL£¬¼Ç¼ DDL
ÐÅÏ¢¼°´ËʱµÄ binlog λÖõãÐÅÏ¢¡£
2.¼ÌÐøÏòǰ½âÎö t2 ʱ¿Ìµ½ t3 ʱ¿ÌµÄ binlog¡£
3.¶ÔÓÚÊôÓÚ table_1 µÄ schema V2 DML£¬ºöÂÔ£»¶ÔÓÚÊôÓÚ
table_2 µÄ schema V1 DML£¬Õý³£Í¬²½µ½ÏÂÓΡ£
4.ÔÚ t3 ʱ¿ÌÊÕµ½ table_2 µÄ DDL£¬¼Ç¼ DDL
ÐÅÏ¢¼°´ËʱµÄ binlog λÖõãÐÅÏ¢¡£
5.¸ù¾Ýͬ²½ÈÎÎñÅäÖÃÐÅÏ¢¡¢ÉÏÓοâ±íÐÅÏ¢µÈ£¬Åжϸà MySQL ʵÀýÉÏËùÓзֱíµÄ
DDL ¶¼ÒѾÊÕµ½£»½« DDL ͬ²½µ½ÏÂÓÎÖ´ÐС¢±ä¸üÏÂÓαí½á¹¹¡£
6.ÉèÖÃÐ嵀 binlog Á÷µÄ½âÎöÆðʼλÖõãΪ step 1 ʱ±£´æµÄλÖõ㡣
7.ÖØÐ¿ªÊ¼½âÎö´Ó t2 ʱ¿Ìµ½ t3 ʱ¿ÌµÄ binlog¡£
8.¶ÔÓÚÊôÓÚ table_1 µÄ schema V2 DML£¬Õý³£Í¬²½µ½ÏÂÓΣ»¶ÔÓÚÊôÓÚ
table_2 µÄ shema V1 DML£¬ºöÂÔ¡£
9.½âÎöµ½´ï step 4 ʱ±£´æµÄ binlog λÖõ㣬¿ÉµÃÖªÔÚ
step 3 ʱ±»ºöÂÔµÄËùÓÐ DML ¶¼ÒÑ¾ÖØÐÂͬ²½µ½ÏÂÓΡ£
10.¼ÌÐø´Ó t4 ʱ¿Ì¶ÔÓ¦µÄ binlog λÖõãÕý³£Í¬²½¡£
´ÓÉÏÃæµÄ·ÖÎö¿ÉÒÔÖªµÀ£¬DM ÔÚ´¦Àí sharding DDL ͬ²½Ê±£¬Ö÷Ҫͨ¹ýÁ½¼¶ sharding
group À´½øÐÐе÷¿ØÖÆ£¬¼ò»¯µÄÁ÷³ÌΪ£º
1.¸÷ DM-worker ¶ÀÁ¢µØÐµ÷¶ÔÓ¦ÉÏÓÎ MySQL ʵÀýÄÚ¶à¸ö·Ö±í×é³ÉµÄ
sharding group µÄ DDL ͬ²½¡£
2.µ± DM-worker ÄÚËùÓзֱíµÄ DDL ¶¼ÊÕµ½Ê±£¬Ïò DM-master
·¢ËÍ DDL Ïà¹ØÐÅÏ¢¡£
3.DM-master ¸ù¾Ý DM-worker ·¢À´µÄ DDL
ÐÅÏ¢£¬Ðµ÷Óɸ÷ DM-worker ×é³ÉµÄ sharing group µÄ DDL ͬ²½¡£
4.µ± DM-master ÊÕµ½ËùÓÐ DM-worker µÄ DDL
ÐÅϢʱ£¬ÇëÇó DDL lock µÄ owner£¨Ä³¸ö DM-worker£©Ö´ÐÐ DDL¡£
5.owner Ö´ÐÐ DDL£¬²¢½«½á¹û·´À¡¸ø DM-master£»×ÔÉí¿ªÊ¼ÖØÐÂͬ²½ÔÚÄÚ²¿Ðµ÷
DDL ͬ²½¹ý³ÌÖб»ºöÂ﵀ DML¡£
6.µ± DM-master ·¢ÏÖ owner Ö´ÐÐ DDL ³É¹¦ºó£¬ÇëÇóÆäËûËùÓÐ
DM-worker ¿ªÊ¼¼ÌÐøÍ¬²½¡£
7.ÆäËûËùÓÐ DM-worker ¸÷×Ô¿ªÊ¼ÖØÐÂͬ²½ÔÚÄÚ²¿Ðµ÷ DDL
ͬ²½¹ý³ÌÖб»ºöÂ﵀ DML¡£
8.ËùÓÐ DM-worker ÔÚÖØÐÂͬ²½Íê³É±»ºöÂ﵀ DML ºó£¬¼ÌÐøÕý³£Í¬²½¡£
Êý¾Ýͬ²½¹ýÂË
ÔÚ½øÐÐÊý¾Ýͬ²½µÄ¹ý³ÌÖУ¬ÓÐʱ¿ÉÄܲ¢²»ÐèÒª½«ÉÏÓÎËùÓеÄÊý¾Ý¶¼Í¬²½µ½ÏÂÓΣ¬Õâʱһ°ãÆÚÍûÄÜÔÚͬ²½¹ý³ÌÖиù¾ÝijЩ¹æÔò£¬¹ýÂ˵ô²¿·Ö²»ÆÚÍûͬ²½µÄÊý¾Ý¡£ÔÚ
DM ÖУ¬Ö§³Ö 2 ÖÖ²»Í¬¼¶±ðµÄͬ²½¹ýÂË·½Ê½¡£
¿â±íºÚ°×Ãûµ¥
DM ÔÚ dumper¡¢loader¡¢syncer Èý¸ö´¦Àíµ¥ÔªÖж¼Ö§³ÖÅäÖùæÔòֻͬ²½/²»Í¬²½²¿·Ö¿â»ò±í¡£
¶ÔÓÚ dumper µ¥Ôª£¬Æäʵ¼Êµ÷Óà mydumper À´ dump ÉÏÓÎ MySQL µÄÊý¾Ý¡£±ÈÈçÖ»ÆÚÍûµ¼³ö
test ¿âÖÐµÄ t1¡¢t2 Á½¸ö±íµÄÊý¾Ý£¬Ôò¿ÉÒÔΪ dumper µ¥ÔªÅäÖÃÈçϹæÔò£º
name-of-dump-rule:
extra-args: "-B test -T t1,t2" |
name-of-dump-rule£º¹æÔòÃû£¬Óû§Ö¸¶¨¡£µ±Óжà¸öÉÏÓÎʵÀýÐèҪʹÓÃÏàͬµÄ¹æÔòʱ£¬¿ÉÒÔÖ»¶¨ÒåÒ»Ìõ¹æÔò£¬¶à¸ö²»Í¬µÄʵÀýͨ¹ý¹æÔòÃû½øÐÐÒýÓá£
extra-args£ºdumper µ¥Ôª¶îÍâ²ÎÊý¡£³ý dumper µ¥ÔªÖÐÃ÷È·¶¨ÒåµÄÅäÖÃÏîÍâµÄÆäËûËùÓÐ
mydumper ÅäÖÃÏͨ¹ý´Ë²ÎÊý´«È룬¸ñʽÓëʹÓà mydumper ʱһÖ¡£
ÓÐ¹Ø mydumper ¶Ô¿â±íºÚ°×Ãûµ¥µÄÖ§³Ö£¬¿É²é¿´ mydumper µÄ²ÎÊý¼° mydumper
µÄÔ´Âë¡£
¶ÔÓÚ loader ºÍ syncer µ¥Ôª£¬Æä¶ÔÓ¦µÄ¿â±íºÚ°×Ãûµ¥¹æÔòΪ black-white-list¡£¼ÙÉèÖ»ÆÚÍûͬ²½
test ¿âÖÐµÄ t1¡¢t2 Á½¸ö±íµÄÊý¾Ý£¬Ôò¿ÉÅäÖÃÈçϹæÔò£º
name-of-bwl-rule:
do-tables:
- db-name: "test"
tbl-name: "t1"
- db-name: "test"
tbl-name: "t2" |
ʾÀýÖÐֻʹÓÃÁ˸ùæÔòµÄ²¿·ÖÅäÖÃÏÍêÕûµÄÅäÖÃÏî¼°¸÷ÅäÖÃÏîµÄº¬Ò壬¿ÉÔĶÁ¸Ã¹¦ÄܶÔÓ¦µÄÓû§Îĵµ¡£DM ÖиùæÔòÓë
MySQL µÄÖ÷´Óͬ²½¹ýÂ˹æÔòÀàËÆ£¬Òò´ËÒ²¿É²Î¿¼ Evaluation of Database-Level
Replication and Binary Logging Options Óë Evaluation
of Table-Level Replication Options¡£
¶ÔÓÚ loader µ¥Ôª£¬ÔÚ½âÎö SQL ÎļþÃû»ñµÃ¿âÃû±íÃûºó£¬»áÓëÅäÖõĺڰ×Ãûµ¥¹æÔò½øÐÐÆ¥Å䣬Èç¹ûÆ¥Åä½á¹ûΪ²»ÐèҪͬ²½£¬Ôò»áºöÂÔ¶ÔÓ¦µÄÕû¸ö
SQL Îļþ¡£¶ÔÓÚ syncer µ¥Ôª£¬ÔÚ½âÎö binlog »ñµÃ¿âÃû±íÃûºó£¬»áÓëÅäÖõĺڰ×Ãûµ¥¹æÔò½øÐÐÆ¥Å䣬Èç¹ûÆ¥Åä½á¹ûΪ²»ÐèҪͬ²½£¬Ôò»áºöÂÔ¶ÔÓ¦µÄ£¨²¿·Ö£©binlog
event Êý¾Ý¡£
binlog event ¹ýÂË
ÔÚ½øÐÐÔöÁ¿Êý¾Ýͬ²½Ê±£¬ÓÐʱ»áÆÚÍû¹ýÂ˵ôÄ³Ð©ÌØ¶¨ÀàÐ굀 binlog event£¬Á½¸öµäÐ͵ij¡¾°°üÀ¨£º
ÉÏÓÎÖ´ÐÐ TRUNCATE TABLE ʱ²»Ï£ÍûÇå¿ÕÏÂÓαíÖеÄÊý¾Ý¡£
ÉÏÓηֱíÉÏÖ´ÐÐ DROP TABLE ʱ²»Ï£Íû DROP ÏÂÓκϲ¢ºóµÄ±í¡£
ÔÚ DM ÖÐÖ§³Ö¸ù¾Ý binlog event µÄÀàÐͽøÐйýÂË£¬¶ÔÓÚÐèÒª¹ýÂË TRUNCATE TABLE
Óë DROP TABLE µÄ³¡¾°£¬¿ÉÅäÖùæÔòÈçÏ£º
name-of-filter-rule:
schema-pattern: "test_*"
table-pattern: "t_*"
events: ["truncate table", "drop
table"]
action: Ignore |
¹æÔòµÄÆ¥ÅäģʽÓë table router¡¢column mapping ÀàËÆ£¬¾ßÌåµÄÅäÖÃÏî¿ÉÔĶÁ¸Ã¹¦ÄܶÔÓ¦µÄÓû§Îĵµ¡£
ÔÚʵÏÖÉÏ£¬µ±½âÎö binlog event »ñµÃ¿âÃû¡¢±íÃû¼° binlog event ÀàÐͺó£¬ÓëÅäÖõĹæÔò½øÐÐÆ¥Å䣬²¢ÔÚÆ¥ÅäºóÒÀ¾Ý
action ÅäÖÃÏîÀ´¾ö¶¨ÊÇ·ñÐèÒª½øÐйýÂË¡£ÓÐ¹Ø binlog event ¹ýÂ˹¦ÄܵľßÌåʵÏÖ£¬¿ÉÒÔÔĶÁ
TiDB-Tools Ï嵀 binlog-filter pkg Ô´´úÂë¡£
|