Äú¿ÉÒÔ¾èÖú£¬Ö§³ÖÎÒÃǵĹ«ÒæÊÂÒµ¡£

1Ôª 10Ôª 50Ôª





ÈÏÖ¤Â룺  ÑéÖ¤Âë,¿´²»Çå³þ?Çëµã»÷Ë¢ÐÂÑéÖ¤Âë ±ØÌî



  ÇóÖª ÎÄÕ ÎÄ¿â Lib ÊÓÆµ iPerson ¿Î³Ì ÈÏÖ¤ ×Éѯ ¹¤¾ß ½²×ù Modeler   Code  
»áÔ±   
 
   
 
 
     
   
 ¶©ÔÄ
  ¾èÖú
mysqlµÄ°²×°ÒÔ¼°ºÍmapreduceµÄ½»»¥
 
À´Ô´£ºOPEN¾­Ñé¿â »ðÁú¹ûÈí¼þ  ·¢²¼ÓÚ 2015-1-19
  3917  次浏览      31
 

MySQL°²×°

MySQLÏÂÔØµØÖ·£ºhttp://www.mysql.com/downloads/

1.1 Windowsƽ̨

1£©×¼±¸Èí¼þ

MySQL°æ±¾£ºmysql-5.5.21-win32.msi

2£©°²×°»·¾³£º

²Ù×÷ϵͳ£ºWindows 7Æì½¢°æ

3£©¿ªÊ¼°²×°

µÚÒ»²½£ºË«»÷"msi"°²×°Îļþ£¬³öÏÖÈçͼ1.1-1½çÃæ——"MySQL°²×°Ïòµ¼"£¬°´"Next"¼ÌÐø¡£

ͼ1.1-1 MySQL°²×°Ïòµ¼

µÚ¶þ²½£ºÔÚ"I accept …."Ç°Ãæ¹´ÉÏ£¬Í¬ÒâЭÒ飬°´"Next"°´Å¥¼ÌÐø¡£

ͼ1.1-2 Èí¼þЭÒé

µÚÈý²½£ºÑ¡Ôñ°²×°ÀàÐÍ£¬ÓÐ"Typical£¨Ä¬ÈÏ£©"¡¢"Custom£¨¶¨Öư²×°£©"¡¢"Complete£¨ÍêÈ«£©"Èý¸öÑ¡Ïî¡£

µäÐͰ²×°£º°²×°Ö»°²×°MySQL·þÎñÆ÷¡¢mysqlÃüÁîÐпͻ§¶ËºÍÃüÁîÐÐʵÓóÌÐò¡£ÃüÁîÐпͻ§¶ËºÍʵÓóÌÐò°üÀ¨mysqldump¡¢myisamchkºÍÆäËü¼¸¸ö¹¤¾ßÀ´°ïÖúÄã¹ÜÀíMySQL·þÎñÆ÷¡£

¶¨Öư²×°£º°²×°ÔÊÐíÄãÍêÈ«¿ØÖÆÄãÏëÒª°²×°µÄÈí¼þ°üºÍ°²×°Â·¾¶¡£

ÍêÈ«°²×°£º°²×°½«°²×°Èí¼þ°üÄÚ°üº¬µÄËùÓÐ×é¼þ¡£ÍêÈ«°²×°Èí¼þ°ü°üÀ¨µÄ×é¼þ°üÀ¨Ç¶Èëʽ·þÎñÆ÷¿â¡¢»ù×¼Ì×¼þ¡¢Ö§³Ö½Å±¾ºÍÎĵµ¡£

ÎÒÃÇÑ¡Ôñ"Custom"£¬Óиü¶àµÄÑ¡ÏҲ·½±ãÊìϤ°²×°¹ý³Ì¡£

ͼ1.1-3 °²×°ÀàÐÍ

µÚËIJ½£ºÑ¡Ôñ×é¼þ¼°¸ü¸ÄÎļþ¼ÐλÖá£

ͼ1.1-4 ×Ô¶¨Òå½çÃæ

ËùÓпÉÓÃ×é¼þÁÐÈë¶¨ÖÆ°²×°¶Ô»°¿ò×ó²àµÄÊ÷×´ÊÓͼÄÚ¡£Î´°²×°µÄ×é¼þÓúìÉ«X Í¼±ê±íʾ£»ÒѾ­°²×°µÄ×é¼þÓлÒɫͼ±ê¡£ÒªÏë¸ü¸Ä×é¼þ£¬µã»÷¸Ã×é¼þµÄͼ±ê²¢´ÓÏÂÀ­ÁбíÖÐÑ¡ÔñеÄÑ¡Ïî¡£×é¼þÎÒÑ¡ÔñÁËĬÈϰ²×°£¬Î»ÖÃÎÒ»á¸ü¸Äһϣ¬µã»÷Browse¡£

ͼ1.1-5 ·¾¶Ñ¡Ôñ

°´"OK"°´Å¥·µ»Ø£¬²¢°´"Next"°´Å¥¼ÌÐø¡£

±¸×¢£º°²×°mysqlµÄ·¾¶ÖУ¬²»Äܺ¬ÓÐÖÐÎÄ¡£

µÚÎå²½£ºÈ·ÈÏÒ»ÏÂÏÈǰµÄÉèÖã¬Èç¹ûÓÐÎ󣬰´"Back"·µ»ØÖØ×ö¡£°´"Install"¿ªÊ¼°²×°¡£

ͼ1.1-6 ×¼±¸°²×°

µÚÁù²½£ºÕýÔÚ°²×°ÖУ¬ÇëÉÔºò……

ͼ1.1-7 ÕýÔÚ°²×°

µÚÆß²½£ºµ¯³öÒ»¸öÒ³ÃæÀ´£¬ÊǹØÓÚ½éÉÜMySQLÆóÒµ°æµÄÐÅÏ¢£¬Ã»ÓÐʲô¿É²Ù×÷µÄ£¬°´"Next"°´Å¥¼ÌÐø¡£

ͼ1.1-8 MySQLÆóÒµ°æ½éÉÜ

È»ºóµ¯³öÒ»¸öÀàËÆ½çÃæ£¬½Ó×Ű´"Next"°´Å¥¼ÌÐø¡£

µÚ°Ë²½£ºÄǸö´ø¸´Ñ¡¿òµÄÊÇ"MySQL·þÎñÆ÷ʵÀýÅäÖÃÏòµ¼"£¬±£³ÖĬÈϹ´Ñ¡£¬°´"Finish"°´Å¥¡£ÖÁ´Ë£¬°²×°¹ý³ÌÒѾ­½áÊøÁË£¬µ«ÊÇ»¹ÐèÒªÅäÖÃһϡ£

ͼ1.1-9 °²×°½áÊø

µÚ¾Å²½£ºMySQLÅäÖÃÏòµ¼Æô¶¯½çÃæ£¬°´"Next"°´Å¥¼ÌÐø¡£

ͼ1.1-10 ÅäÖÃÏòµ¼

MySQL Configuration Wizard£¨ÅäÖÃÏòµ¼£©¿ÉÒÔ°ïÖú×Ô¶¯ÅäÖÃWindowsÖеķþÎñÆ÷¡£MySQL Configuration Wizard£¨ÅäÖÃÏòµ¼£©ÎÊÄãһϵÁÐÎÊÌ⣬Ȼºó½«»Ø´ð·Åµ½Ä£°åÖÐÉú³ÉÒ»¸ömy.iniÎļþ£¬¸ÃÎļþÓëÄãµÄ°²×°Ò»Ö¡£Ä¿Ç°Ö»ÊÊÓÃÓÚWindowsÓû§¡£

Ò»°ãÇé¿öµ±MySQL°²×°°ïÖúÍ˳öʱ£¬´ÓMySQL°²×°°ïÖúÆô¶¯MySQL Configuration Wizard£¨ÅäÖÃÏòµ¼£©¡£»¹¿ÉÒÔµã»÷WindowsÆô¶¯²Ëµ¥ÖÐMySQL·þÎñÆ÷ʵÀýÅäÖÃÏòµ¼ÌõÄ¿ÖеÄMySQL²¿·ÖÀ´Æô¶¯MySQL Configuration Wizard£¨ÅäÖÃÏòµ¼£©¡£²¢ÇÒ£¬»¹¿ÉÒÔ½øÈëMySQL°²×°binĿ¼ֱ½ÓÆô¶¯MySQLInstanceConfig.exeÎļþ¡£

;µÚÊ®²½£ºÑ¡ÔñÅäÖÃÀàÐÍ£¬¿ÉÒÔÑ¡Ôñ Á½ÖÖÅäÖÃÀàÐÍ£ºDetailed Configuration£¨ÏêϸÅäÖ㩺ÍStandard Configuration£¨±ê×¼ÅäÖã©¡£Standard Configuration£¨±ê×¼ÅäÖã©Ñ¡ÏîÊʺÏÏëÒª¿ìËÙÆô¶¯MySQL¶ø²»±Ø¿¼ÂÇ·þÎñÆ÷ÅäÖõÄÐÂÓû§¡£ÏêϸÅäÖÃÑ¡ÏîÊʺÏÏëÒª¸ü¼ÓϸÁ£¶È¿ØÖÆ·þÎñÆ÷ÅäÖÃµÄ ¸ß¼¶Óû§¡£ÎÒÃÇÑ¡Ôñ"Detailed Configuration"£¬·½±ãÊìϤÅäÖùý³Ì¡£

ͼ1.1-11 ÅäÖÃÀàÐÍ

±¸×¢£º

Èç¹ûÄãÊÇMySQLµÄÐÂÊÖ£¬ÐèÒªÅäÖÃΪµ¥Óû§¿ª·¢»úµÄ·þÎñÆ÷£¬Standard Configuration£¨±ê×¼ÅäÖã©Ó¦µ±ÊʺÏÄãµÄÐèÇó¡£Ñ¡ÔñStandard Configuration£¨±ê×¼ÅäÖã©Ñ¡ÏÔò MySQL Configuration Wizard£¨ÅäÖÃÏòµ¼£©×Ô¶¯ÉèÖÃËùÓÐÅäÖÃÑ¡Ïµ«²»°üÀ¨·þÎñÑ¡ÏîºÍ°²È«Ñ¡Ïî¡£

Standard Configuration£¨±ê×¼ÅäÖã©ÉèÖÃÑ¡Ïî¿ÉÄÜÓë°²×°MySQLµÄϵͳ²»¼æÈÝ¡£Èç¹ûϵͳÉÏÒѾ­°²×°ÁËMySQLºÍÄãÏëÒªÅäÖõݲװ£¬½¨ÒéÑ¡ÔñÏêϸÅäÖá£

µÚʮһ²½£ºÑ¡Ôñ·þÎñÆ÷ÀàÐÍ£¬¿ÉÒÔÑ¡Ôñ3ÖÖ·þÎñÆ÷ÀàÐÍ£¬Ñ¡ÔñÄÄÖÖ·þÎñÆ÷½«Ó°Ïìµ½MySQL Configuration Wizard£¨ÅäÖÃÏòµ¼£©¶ÔÄÚ´æ¡¢Ó²Å̺͹ý³Ì»òʹÓõľö²ß¡£

Developer Machine£¨¿ª·¢»úÆ÷£©£º¸ÃÑ¡Ïî´ú±íµäÐ͸öÈËÓÃ×ÀÃæ¹¤×÷Õ¾¡£¼Ù¶¨»úÆ÷ÉÏÔËÐÐ×Ŷà¸ö×ÀÃæÓ¦ÓóÌÐò¡£½«MySQL·þÎñÆ÷ÅäÖóÉʹÓÃ×îÉÙµÄϵͳ×ÊÔ´¡£

Server Machine£¨·þÎñÆ÷£©£º¸ÃÑ¡Ïî´ú±í·þÎñÆ÷£¬MySQL·þÎñÆ÷¿ÉÒÔͬÆäËüÓ¦ÓóÌÐòÒ»ÆðÔËÐУ¬ÀýÈçFTP¡¢emailºÍweb·þÎñÆ÷¡£MySQL·þÎñÆ÷ÅäÖóÉʹÓÃÊʵ±±ÈÀýµÄϵͳ×ÊÔ´¡£

Dedicated MySQL Server Machine£¨×¨ÓÃMySQL·þÎñÆ÷£©£º¸ÃÑ¡Ïî´ú±íÖ»ÔËÐÐMySQL·þÎñµÄ·þÎñÆ÷¡£¼Ù¶¨ÔËÐÐûÓÐÔËÐÐÆäËüÓ¦ÓóÌÐò¡£MySQL·þÎñÆ÷ÅäÖóÉʹÓÃËùÓпÉÓÃϵͳ×ÊÔ´¡£

´ó¼Ò¸ù¾Ý×Ô¼ºµÄÀàÐÍÑ¡ÔñÁË£¬Ò»°ãÑ¡"Server Machine"£¬²»»áÌ«ÉÙ£¬Ò²²»»áÕ¼Âú¡£ÎÒÃÇÑ¡Ôñ"Server Machine"£¬°´"Next"°´Å¥¼ÌÐø¡£

ͼ1.1-12 ·þÎñÆ÷ÀàÐÍ

µÚÊ®¶þ²½£ºÑ¡ÔñÊý¾Ý¿âÓÃ;£¬Í¨¹ý Database Usage£¨Êý¾Ý¿âʹÓã©¶Ô»°¿ò£¬Äã¿ÉÒÔÖ¸³ö´´½¨MySQL±íʱʹÓÃµÄ±í´¦ÀíÆ÷¡£Í¨¹ý¸ÃÑ¡ÏÄã¿ÉÒÔÑ¡ÔñÊÇ·ñʹÓÃInnoDB´¢´æÒýÇæ£¬ÒÔ¼°InnoDB Õ¼Óöà´ó±ÈÀýµÄ·þÎñÆ÷×ÊÔ´¡£"Multifunctional Database£¨Í¨Óö๦ÄÜÐÍ£¬ºÃ£©"¡¢"Transactional Database Only£¨·þÎñÆ÷ÀàÐÍ£¬×¨×¢ÓÚÊÂÎñ´¦Àí£¬Ò»°ã£©"¡¢"Non-Transactional Database Only£¨·ÇÊÂÎñ´¦ÀíÐÍ£¬½Ï¼òµ¥£¬Ö÷Òª×öһЩ¼à¿Ø¡¢¼ÇÊýÓ㬶ÔMyISAM Êý¾ÝÀàÐ͵ÄÖ§³Ö½öÏÞÓÚnon-transactional£©"£¬Ëæ×Ô¼ºµÄÓÃ;¶øÑ¡ÔñÁË£¬Ò»°ãÑ¡ÔñµÚÒ»Öֶ๦Äܵġ£ÎÒÃÇÑ¡Ôñ"Multifunctional Database"£¬°´"Next"°´Å¥¼ÌÐø¡£

ͼ1.1-13 Êý¾Ý¿âÓÃ;

µÚÊ®Èý²½£º¶ÔInnoDB Tablespace ½øÐÐÅäÖ㬾ÍÊÇΪInnoDB Êý¾Ý¿âÎļþÑ¡ÔñÒ»¸ö´æ´¢¿Õ¼ä£¬Èç¹ûÐÞ¸ÄÁË£¬Òª¼ÇסλÖã¬ÖØ×°µÄʱºòҪѡÔñÒ»ÑùµÄµØ·½£¬·ñÔò¿ÉÄÜ»áÔì³ÉÊý¾Ý¿âË𻵣¬µ±È»£¬¶ÔÊý¾Ý¿â×ö¸ö±¸·Ý¾ÍûÎÊÌâÁË£¬ÕâÀï²»ÏêÊö¡£ÕâÀïûÓÐÐ޸ģ¬Ê¹ÓÃÓÃĬÈÏλÖ㬰´"Next"°´Å¥¼ÌÐø¡£

ͼ1.1-14 ÅäÖÃInnoDB±í¿Õ¼ä

µÚÊ®ËIJ½£ºÑ¡ÔñMySQLÔÊÐíµÄ ×î´óÁ¬½ÓÊý£¬ÏÞÖÆËù´´½¨µÄÓëMySQL·þÎñÆ÷Ö®¼äµÄ²¢ÐÐÁ¬½ÓÊýÁ¿ºÜÖØÒª£¬ÒÔ±ã·ÀÖ¹·þÎñÆ÷ºÄ¾¡×ÊÔ´¡£ÔÚConcurrent Connections£¨²¢ÐÐÁ¬½Ó£©¶Ô»°¿òÖУ¬¿ÉÒÔÑ¡Ôñ·þÎñÆ÷µÄʹÓ÷½·¨£¬²¢¸ù¾ÝÇé¿öÏÞÖÆ²¢ÐÐÁ¬½ÓµÄÊýÁ¿¡£»¹¿ÉÒÔÊÖ¶¯ÉèÖò¢ÐÐÁ¬½ÓµÄÏÞÖÆ¡£µÚÒ»ÖÖÊÇ×î´ó 20¸öÁ¬½Ó²¢·¢Êý£¬µÚ¶þÖÖÊÇ×î´ó500¸ö²¢·¢Á¬½ÓÊý£¬×îºóÒ»ÖÖÊÇ×Ô¶¨Òå¡£ÎÒÃÇÑ¡Ôñ"Online Transaction PRocessing(OLTP)"£¬°´"Next"°´Å¥¼ÌÐø¡£

ͼ1.1-15 ×î´óÁ¬½ÓÊý

µÚÊ®Îå²½£º½øÐÐÍøÂçÅäÖã¬ÔÚ Networking Options£¨ÍøÂçÑ¡Ï¶Ô»°¿òÖпÉÒÔÆôÓûò½ûÓÃTCP/IPÍøÂ磬²¢ÅäÖÃÓÃÀ´Á¬½ÓMySQL·þÎñÆ÷µÄ¶Ë¿ÚºÅ¡£Ä¬ÈÏÇé¿öÆôÓÃTCP/IPÍøÂç¡£ÒªÏë½ûÓà TCP/IPÍøÂ磬ȡÏûÑ¡ÔñEnable TCP/IP NetworkingÑ¡ÏîÅԱߵļì²é¿ò¡£Ä¬ÈÏʹÓÃ3306¶Ë¿Ú¡£ÒªÏë¸ü·ÃÎÊMySQLʹÓõĶ˿ڣ¬´ÓÏÂÀ­¿òÑ¡ÔñÒ»¸öж˿ںŻòÖ±½ÓÏòÏÂÀ­¿òÊäÈëÐÂµÄ¶Ë¿Ú ºÅ¡£Èç¹ûÄãÑ¡ÔñµÄ¶Ë¿ÚºÅÒѾ­±»Õ¼Ó㬽«ÌáʾȷÈÏÑ¡ÔñµÄ¶Ë¿ÚºÅ¡£ÎÒÃDZ£³ÖĬÈÏÑ¡Ï°´"Next"°´Å¥¼ÌÐø¡£

ͼ1.1-16 ÍøÂçÅäÖÃ

µÚÊ®Áù²½£º¶ÔÊý¾Ý¿âÓïÑÔ±àÂë½øÐÐÉèÖ㬷dz£ÖØÒª£¬ÒòΪHadoopÀïĬÈϱàÂëΪUTF-8£¬ËùÒÔΪÁ˱ÜÃâ³öÏÖÂÒÂ룬ÎÒÃÇÕâÀïÑ¡Ôñ"UTF-8"×÷ΪMySQLÊý¾Ý¿âµÄÓïÑÔ±àÂë¡£

ͼ1.1-17 Êý¾Ý¿â±àÂë

µÚÊ®Æß²½£ºÊÇ·ñÒª°ÑMySQLÉèÖóÉWindowsµÄ·þÎñ£¬Ò»°ãÑ¡ÔñÉè³É·þÎñ£¬ÕâÑùÒÔºó¾Í¿ÉÒÔͨ¹ý·þÎñÖÐÆô¶¯ºÍ¹Ø±ÕmysqlÊý¾Ý¿âÁË¡£ÍƼö£ºÏÂÃæµÄ¸´Ñ¡¿òÒ²¹´Ñ¡ÉÏ£¬ÕâÑù£¬ÔÚcmdģʽÏ£¬²»±Ø·Çµ½mysqlµÄbinĿ¼ÏÂÖ´ÐÐÃüÁî¡£ÎÒÃÇÈ«²¿´òÉÏÁ˹´£¬Service Name ²»±ä¡£°´"Next"°´Å¥¼ÌÐø¡£

ͼ1.1-18 ·þÎñÑ¡Ïî

µÚÊ®°Ë²½£ºÉèÖÃMySQLµÄ³¬¼¶Óû§ÃÜÂ룬Õâ¸ö³¬¼¶Óû§·Ç³£ÖØÒª£¬¶ÔMySQLÓµÓÐÈ«²¿µÄȨÏÞ£¬ÇëÉèÖúò¢Àμdz¬¼¶Óû§µÄÃÜÂ룬ÏÂÃæÓиö¸´Ñ¡¿òÊÇÑ¡ÔñÊÇ·ñÔÊÐíÔ¶³Ì»úÆ÷ÓÃrootÓû§Á¬½Óµ½ÄãµÄMySQL·þÎñÆ÷ÉÏÃæ£¬Èç¹ûÓÐÕâ¸öÐèÇó£¬Ò²Çë¹´Ñ¡¡£ÎÒÃÇÕâÀïµÄrootÓû§ÃÜÂëÉèÖÃΪ"hadoop"£¬²¢¹´ÉÏ"ÔÊÐíÔ¶³ÌÁ¬½Ó"¸´Ñ¡¿ò£¬°´"Next"°´Å¥¼ÌÐø¡£

ͼ1.1-19 °²È«Ñ¡Ïî

±¸×¢£º

"Enable root access from remote machines£¨ÊÇ·ñÔÊÐíroot Óû§ÔÚÆäËüµÄ»úÆ÷Éϵǽ£¬Èç¹ûÒª°²È«£¬¾Í²»Òª¹´ÉÏ£¬Èç¹ûÒª·½±ã£¬¾Í¹´ÉÏËü£©"¡£

"Create An Anonymous Account£¨Ð½¨Ò»¸öÄäÃûÓû§£¬ÄäÃûÓû§¿ÉÒÔÁ¬½ÓÊý¾Ý¿â£¬²»ÄܲÙ×÷Êý¾Ý£¬°üÀ¨²éѯ£©"£¬Ò»°ã¾Í²»Óù´ÁË¡£

µÚÊ®¾Å²½£ºÈ·ÈÏÉèÖÃÎÞÎó£¬Èç¹ûÓÐÎ󣬰´"Back"·µ»Ø¼ì²é¡£Èç¹ûûÓУ¬°´"Execute"ʹÉèÖÃÉúЧ¡£

ͼ1.1-20 È·ÈÏÅäÖÃ

µÚ¶þÊ®²½£ºÉèÖÃÍê±Ï£¬°´"Finish"°´Å¥½áÊøMySQLµÄ°²×°ÓëÅäÖá£

ͼ1.1-21 ÅäÖÃÍê³É

±¸×¢£ºÕâÀïÓÐÒ»¸ö±È½Ï³£¼ûµÄ´íÎ󣬾ÍÊDz» ÄÜ"Start service"£¬Ò»°ã³öÏÖÔÚÒÔǰÓа²×°MySQLµÄ·þÎñÆ÷ÉÏ£¬½â¾öµÄ°ì·¨£¬Ïȱ£Ö¤ÒÔǰ°²×°µÄMySQL ·þÎñÆ÷³¹µ×Ð¶ÔØµôÁË£»²»Ðеϰ£¬¼ì²éÊÇ·ñ°´ÉÏÃæÒ»²½Ëù˵£¬Ö®Ç°µÄÃÜÂëÊÇ·ñÓÐÐ޸ģ¬ÕÕÉÏÃæµÄ²Ù×÷£»Èç¹ûÒÀÈ»²»ÐУ¬½«MySQL °²×°Ä¿Â¼ÏµÄdata Îļþ¼Ð±¸·Ý£¬È»ºóɾ³ý£¬ÔÚ°²×°Íê³Éºó£¬½«°²×°Éú³ÉµÄ data Îļþ¼Ðɾ³ý£¬±¸·ÝµÄdata Îļþ¼ÐÒÆ»ØÀ´£¬ÔÙÖØÆôMySQL ·þÎñ¾Í¿ÉÒÔÁË£¬ÕâÖÖÇé¿öÏ£¬¿ÉÄÜÐèÒª½«Êý¾Ý¿â¼ì²éһϣ¬È»ºóÐÞ¸´Ò»´Î£¬·ÀÖ¹Êý¾Ý³ö´í¡£

4£©ÑéÖ¤³É¹¦

µÚÒ»ÖÖ£º´ò¿ªÈÎÎñ¹ÜÀíÆ÷ ¿´µ½MySQL·þÎñÊÇ·ñÒѾ­Æô¶¯¡£

ͼ1.1-22 ÈÎÎñ¹ÜÀíÆ÷

µÚ¶þÖÖ£º"¿ªÊ¼àÆô¶¯cmd࿪´òcmdģʽ"£¬ÊäÈë"mysql –u root –p"Á¬½ÓÊý¾Ý¿â¡£

ͼ1.1-23 Á¬½ÓÊý¾Ý¿â

1.2 Linuxƽ̨

1£©×¼±¸Èí¼þ

MySQLÊý¾Ý¿â£ºMySQL-server-5.5.21-1.linux2.6.i386.rpm

MySQL¿Í»§¶Ë£ºMySQL-client-5.5.21-1.linux2.6.i386.rpm

2£©°²×°»·¾³£º

²Ù×÷ϵͳ£ºCentOS6.0 Linux

3£©¼ì²é°²×°

ÔÚ°²×°MySQL֮ǰ£¬Ïȼì²éCentOSϵͳÖÐÊÇ·ñÒѾ­°²×°ÁËÒ»¸öMySQL£¬Èç¹ûÒѾ­°²×°ÏÈÐ¶ÔØ£¬²»È»»áµ¼Ö°²×°ÐµÄMySQLʧ°Ü¡£

ÓÃÏÂÃæÃüÁî²é¿´ÏµÍ³Ö®Ç°ÊÇ·ñÒѰ²×°MySQL¡£

rpm -qa | grep mysql

²é¿´½á¹ûÈçÏ£º

´ÓÉÏͼµÃÖª£¬CentOS6.0ϵͳ×Ô´øÁËÒ»¸öMySQL£¬ÎÒÃÇÐèҪɾ³ýÕâ¸öÀϰ汾£¬ÓÃrootÓû§Ö´ÐÐÏÂÃæÓï¾ä¡£

rpm -e --nodeps mysql-libs-5.1.47-4.el6.i686

ÉÏͼÖУ¬ÎÒÃÇÏÈÇл»µ½"root"Óû§Ï£¬È»ºóÖ´ÐÐɾ³ýÓï¾ä£¬É¾³ýÖ®ºó£¬ÎÒÃÇÔٴβ鿴£¬·¢ÏÖÒѾ­³É¹¦É¾³ýÁËCentOS6.0×Ô´øµÄ¾ÉMySQL°æ±¾¡£

ÔÚɾ³ýMySQLµÄrpmºó£¬»¹Òª½øÐÐһЩɨβ²Ù×÷£¬ÍøÉÏÓÐÁ½ÖÖ²Ù×÷¡££¨±¸×¢£ºÎÒÔÚÕâÀïÁ½ÖÖ¶¼Ã»ÓÐÓõ½£¬·¢ÏÖϵͳÖв¢Ã»ÓÐÆäËû²ÐÓàµÄMySQLÐÅÏ¢¡££©

µÚÒ»ÖÖÉÆºó´¦Àí£ºÊ¹ÓÃÏÂÃæÃüÁî½øÐд¦Àí¡£

rm -rf /var/lib/mysql*

rm -rf /usr/share/mysql*

ÁíÒ»ÖÖÉÆºó´¦Àí£ºÐ¶ÔØºó/var/lib/mysqlÖеÄ/etc/my.cnf»áÖØÃüÃûΪmy.cnf.rpmsave£¬/var/log/mysqld.log »áÖØÃüÃûΪ/var/log/mysqld.log.rpmsave£¬Èç¹ûÈ·¶¨Ã»Óúó¾ÍÊÖ¹¤É¾³ý¡£

4£©¿ªÊ¼°²×°

µÚÒ»²½£ºÉÏ´«ËùÐèÈí¼þ¡£Í¨¹ý"FlashFXP"Èí¼þʹÓÃ"vsftpd"ÉÏ´«Óõ½µÄÁ½¸öÈí¼þµ½"/home/hadoop"Ŀ¼Ï¡£

µÚ¶þ²½£º°²×°MySQL·þÎñ¶Ë¡£ÓÃ"root"Óû§ÔËÐÐÈçÏÂÃüÁî½øÐа²×°£º£¨±¸×¢£ºÒÔϲ½Öè¶¼ÊÇÓÃ"root"Óû§Ö´ÐС££©

rpm -ivh MySQL-server-5.5.21-1.linux2.6.i386.rpm

ͨ¹ýSecureCRT²é¿´ÈçÏ£º

Èç³öÏÖÈçÉÏÐÅÏ¢£¬·þÎñ¶Ë°²×°Íê±Ï¡£

µÚÈý²½£º¼ì²âMySQL 3306¶Ë¿ÚÊÇ·ñ°²´ò¿ª¡£²âÊÔÊÇ·ñ³É¹¦¿ÉÔËÐÐnetstat¿´MySQL¶Ë¿ÚÊÇ·ñ´ò¿ª£¬Èç´ò¿ª±íʾ·þÎñÒѾ­Æô¶¯£¬°²×°³É¹¦¡£MySQLĬÈϵĶ˿ÚÊÇ3306¡£

netstat -nat

´ÓÉÏͼÖз¢ÏÖ²¢Ã»ÓÐÓë"3306"ÓйصÄÐÅÏ¢£¬ËµÃ÷"MySQL·þÎñÆ÷"ûÓÐÆô¶¯¡£Í¨¹ýÏÂÃæÃüÁîÆô¶¯MySQL¡£

service mysql start

´ÓÉÏͼÖÐÒѾ­·¢ÏÖÎÒÃǵÄMySQL·þÎñÆ÷ÒѾ­ÆðÀ´ÁË¡£

µÚËIJ½£º°²×°MySQL¿Í»§¶Ë¡£ÓÃÏÂÃæÃüÁî½øÐа²×°£º

rpm -ivh MySQL-client-5.5.21-1.linux2.6.i386.rpm

Ö´ÐÐÃüÁîÏÔʾÈçÏ£º

´ÓÉÏͼÖÐÏÔʾMySQL¿Í»§¶ËÒѾ­°²×°Íê±Ï¡£

µÚÎå²½£ºMySQLµÄ¼¸¸öÖØÒªÄ¿Â¼¡£MySQL°²×°Íê³Éºó²»ÏñSQL ServerĬÈϰ²×°ÔÚÒ»¸öĿ¼£¬ËüµÄÊý¾Ý¿âÎļþ¡¢ÅäÖÃÎļþºÍÃüÁîÎļþ·Ö±ðÔÚ²»Í¬µÄĿ¼£¬Á˽âÕâЩĿ¼·Ç³£ÖØÒª£¬ÓÈÆä¶ÔÓÚLinuxµÄ³õѧÕߣ¬ÒòΪ Linux±¾ÉíµÄĿ¼½á¹¹¾Í±È½Ï¸´ÔÓ£¬Èç¹û¸ã²»Çå³þMySQLµÄ°²×°Ä¿Â¼ÄǾÍÎÞ´Ó̸ÆðÉîÈëѧϰ¡£

ÏÂÃæ¾Í½éÉÜÒ»ÏÂÕ⼸¸öĿ¼¡£

a¡¢Êý¾Ý¿âĿ¼

/var/lib/mysql/

b¡¢ÅäÖÃÎļþ

/usr/share/mysql£¨mysql.serverÃüÁî¼°ÅäÖÃÎļþ£©

c¡¢Ïà¹ØÃüÁî

/usr/bin(mysqladmin mysqldumpµÈÃüÁî)

d¡¢Æô¶¯½Å±¾

/etc/rc.d/init.d/£¨Æô¶¯½Å±¾ÎļþmysqlµÄĿ¼£©

È磺/etc/rc.d/init.d/mysql start/restart/stop/status

ÏÂÃæ¾Í·Ö±ðչʾÉÏÃæµÄ¼¸¸öĿ¼ÄÚÈÝ£º

1.Êý¾Ý¿âĿ¼

2.ÅäÖÃÎļþ

3.Ïà¹ØÃüÁî

4.Æô¶¯½Å±¾

µÚÁù²½£º¸ü¸ÄMySQLĿ¼¡£ÓÉÓÚMySQLÊý¾Ý¿âĿ¼ռÓôÅÅ̱Ƚϴ󣬶øMySQLĬÈϵÄÊý¾ÝÎļþ´æ´¢Ä¿Â¼Îª/"var/lib/mysql"£¬ËùÒÔÎÒÃÇÒª°ÑÄ¿Â¼ÒÆµ½"/"¸ùĿ¼ÏµÄ"mysql_data"Ŀ¼ÖС£

ÐèÒªÒÔϼ¸¸ö²½Ö裺

"/"¸ùĿ¼Ï½¨Á¢"mysql_data"Ŀ¼

cd /

mkdir mysql_data

°ÑMySQL·þÎñ½ø³ÌÍ£µô

¿ÉÒÔÓÃÁ½ÖÖ·½·¨:

service mysql stop

»òÕß

mysqladmin -u root -p shutdown

´ÓÉÏͼÖÐÎÒÃǵÃÖª"MySQL·þÎñ½ø³Ì"ÒѾ­Í£µô¡£

±¸×¢£ºMySQLĬÈÏÓû§ÃûΪ"root"£¬´Ë´¦µÄ"root"ÓëLinuxµÄ×î¸ßȨÏÞÓû§"root"²»ÊÇÒ»»á¶ù£¬¶øÇÒĬÈϵÄÓû§"root"µÄÃÜÂëΪ¿Õ£¬ËùÒÔÉÏͼÖÐÈÃÊäÈëÃÜÂ룬ֱ½Óµã»÷»Ø³µ¼´¿É¡£

°Ñ"/var/lib/mysql"Õû¸öÄ¿Â¼ÒÆµ½"/mysql_data"

mv /var/lib/mysql /mysql_data

ÕâÑù¾Í°ÑMySQLµÄÊý¾ÝÎļþÒÆ¶¯µ½ÁË"/mysql_data/mysql"Ï¡£

ÕÒµ½my.cnfÅäÖÃÎļþ

Èç¹û"/etc/"Ŀ¼ÏÂûÓÐmy.cnfÅäÖÃÎļþ£¬Çëµ½"/usr/share/mysql/"ÏÂÕÒµ½*.cnfÎļþ£¬¿½±´ÆäÖÐÒ»¸öºÏÊʵÄÅäÖÃÎļþµ½"/etc/"²¢¸ÄÃûΪ"my.cnf"ÖС£ÃüÁîÈçÏ£º

cp /usr/share/mysql/my-medium.cnf     /etc/my.cnf

ÉÏͼÖУ¬Ï²鿴"/etc/"ÏÂÃæÊÇ·ñÓÐ"my.cnf"Îļþ£¬·¢ÏÖûÓУ¬È»ºóͨ¹ýÉÏÃæµÄÃüÁî½øÐп½±´£¬¿½±´ÍêÖ®ºó£¬½øÐв鿴£¬·¢ÏÖ¿½±´³É¹¦¡£

±¸×¢£º"/usr/share/mysql/"ÏÂÓкü¸¸ö½áβΪcnfµÄÎļþ£¬ËüÃǵÄ×÷Ó÷ֱðÊÇ¡£

a¡¢my-small.cnf£ºÊÇΪÁËСÐÍÊý¾Ý¿â¶øÉè¼ÆµÄ¡£²»Ó¦¸Ã°ÑÕâ¸öÄ£ÐÍÓÃÓÚº¬ÓÐһЩ³£ÓÃÏîÄ¿µÄÊý¾Ý¿â¡£

b¡¢my-medium.cnf£ºÊÇΪÖеȹæÄ£µÄÊý¾Ý¿â¶øÉè¼ÆµÄ¡£Èç¹ûÄãÕýÔÚÆóÒµÖÐʹÓÃRHEL,¿ÉÄÜ»á±ÈÕâ¸ö²Ù×÷ϵͳµÄ×îСRAMÐèÇó(256MB)Ã÷ÏÔ¶àµÃ¶àµÄÎïÀíÄÚ´æ¡£Óɴ˿ɼû£¬Èç¹ûÓÐÄÇô¶àRAMÄÚ´æ¿ÉÒÔʹÓã¬×ÔÈ»¿ÉÒÔÔÚͬһ̨»úÆ÷ÉÏÔËÐÐÆäËü·þÎñ¡£

c¡¢my-large.cnf£ºÊÇΪרÓÃÓÚÒ»¸öSQLÊý¾Ý¿âµÄ¼ÆËã»ú¶øÉè¼ÆµÄ¡£ÓÉÓÚËü¿ÉÒÔΪ¸ÃÊý¾Ý¿âʹÓöà´ï512MBµÄÄڴ棬ËùÒÔÔÚÕâÖÖÀàÐ͵ÄϵͳÉϽ«ÐèÒªÖÁÉÙ1GBµÄRAM,ÒÔ±ãËüÄܹ»Í¬Ê±´¦Àí²Ù×÷ϵͳÓëÊý¾Ý¿âÓ¦ÓóÌÐò¡£

d¡¢my-huge.cnf£ºÊÇΪÆóÒµÖеÄÊý¾Ý¿â¶øÉè¼ÆµÄ¡£ÕâÑùµÄÊý¾Ý¿âÒªÇóרÓ÷þÎñÆ÷ºÍ1GB»ò1GBÒÔÉϵÄRAM¡£

ÕâЩѡÔñ¸ß¶ÈÒÀÀµÓÚÄÚ´æµÄÊýÁ¿¡¢¼ÆËã»úµÄÔËËãËÙ¶È¡¢Êý¾Ý¿âµÄϸ½Ú´óС¡¢·ÃÎÊÊý¾Ý¿âµÄÓû§ÊýÁ¿ÒÔ¼°ÔÚÊý¾Ý¿âÖÐ×°Èë²¢·ÃÎÊÊý¾ÝµÄÓû§ÊýÁ¿¡£Ëæ×ÅÊý¾Ý¿âºÍÓû§µÄ²»¶ÏÔö¼Ó£¬Êý¾Ý¿âµÄÐÔÄÜ¿ÉÄܻᷢÉú±ä»¯¡£

±¸×¢£ºÕâÀïÎÒÃǸù¾Ýʵ¼ÊÇé¿ö£¬Ñ¡ÔñÁË"my-medium.cnf"½øÐÐÅäÖá£

±à¼­MySQLµÄÅäÖÃÎļþ"/etc/my.cnf"

Ϊ±£Ö¤MySQLÄܹ»Õý³£¹¤×÷£¬ÐèÒªÖ¸Ã÷"mysql.sock"ÎļþµÄ²úÉúλÖã¬ÒÔ¼°Ä¬ÈϱàÂëÐÞ¸ÄΪUTF-8¡£ÓÃÏÂÃæÃüÁ

vim /etc /my.cnf

ÐèÒªÐ޸ĺÍÌí¼ÓµÄÄÚÈÝÈçÏ£º

¡¾client¡¿

socket = /mysql_data/mysql/mysql.sock

default-character-set=utf8

¡¾mysqld¡¿

socket = /mysql_data/mysql/mysql.sock

datadir =/mysql_data/mysql

character-set-server=utf8

lower_case_table_names=1£¨×¢ÒâlinuxÏÂmysql°²×°ÍêºóÊÇĬÈÏ£ºÇø·Ö±íÃûµÄ´óСд£¬²»Çø·ÖÁÐÃûµÄ´óСд£»lower_case_table_names = 00£ºÇø·Ö´óСд£¬1£º²»Çø·Ö´óСд£©

±¸×¢£º¡¾client¡¿ºÍ¡¾mysqld¡¿ÉèÖõıàÂëʱǰµØÃû³Æ²»Ò»Ñù¡£

ÐÞ¸ÄMySQLÆô¶¯½Å±¾"/etc/rc.d/init.d/mysql"

×îºó£¬ÐèÒªÐÞ¸ÄMySQLÆô¶¯½Å±¾/etc/rc.d/init.d/mysql£¬ÐÞ¸Ädatadir=/mysql_data/mysql¡£

vim /etc/rc.d/init.d/mysql

ÖØÐÂÆô¶¯MySQL·þÎñ

service mysql start

Õý×¼±¸¸ßÐËʱ£¬·¢ÏÖMySQLÆô¶¯²»ÁËÁË£¬ÍøÉÏËÑÁËÒ»ÏÂÎ磬¸÷ÖÖ¶¼Ã»Óнâ¾ö¡£ºóÀ´ÔÚһƪÎÄÕ²ŵÃÖªÓÖÊÇ"SELinux"Èǵûö¡£½â¾ö°ì·¨ÈçÏ£º

´ò¿ª/etc/selinux/config£¬°ÑSELINUX=enforcing¸ÄΪSELINUX=disabledºó´æÅÌÍ˳öÖØÆô»úÆ÷ÊÔÊÔ£¬±ØÐëÒªÖØÆô£¬ºÜ¹Ø¼ü¡£

»úÆ÷ÖØÆôÖ®ºó£¬ÔÚ°Ñ"mysql·þÎñ"Æô¶¯¡£

µÚÆß²½£ºÐ޸ĵǼÃÜÂë¡£

MySQLĬÈÏûÓÐÃÜÂ룬°²×°Íê±ÏÔö¼ÓÃÜÂëµÄÖØÒªÐÔÊDz»ÑÔ¶øÓ÷µÄ¡£

ÐÞ¸Äǰ£¬Ö±½ÓµÇ¼

ÔÚûÓÐÌí¼ÓÃÜÂëǰ£¬Ö±½ÓÊäÈë"mysql"¾ÍÄܵǼµ½MySQLÊý¾Ý¿âÀï¡£

Ð޸ĵǼÃÜÂë

Óõ½µÄÃüÁîÈçÏ£ºmysqladmin -u root password 'new-password'

¸ñʽ£ºmysqladmin -uÓû§Ãû -p¾ÉÃÜÂë password ÐÂÃÜÂë

ÎÒÃÇÕâÀïÉèÖÃMySQLÊý¾Ý¿â"root"Óû§µÄÃÜÂëΪ"hadoop"¡£Ö´ÐеÄÃüÁîÈçÏ£ºmysqladmin –u root password hadoop

²âÊÔÊÇ·ñÐ޸ijɹ¦

£¨1£©²»ÓÃÃÜÂëµÇ¼

´ËʱÏÔʾ´íÎó£¬ËµÃ÷ÃÜÂëÒѾ­Ð޸ġ£

£¨2£©ÓÃÐ޸ĺóµÄÃÜÂëµÇ¼

´ÓÉÏͼÖеÃÖª£¬ÎÒÃÇÒѾ­³É¹¦ÐÞ¸ÄÁËÃÜÂ룬²¢ÇÒÓÃеÄÃÜÂëµÇ¼ÁËMySQL·þÎñÆ÷¡£

µÚ°Ë²½£ºÅäÖ÷À»ðǽ

µÚÒ»ÖÖ£ºÐ޸ķÀ»ðǽÅäÖÃÎļþ"/etc/sysconfig/iptables"£¬Ìí¼ÓÈçÏÂÄÚÈÝ£º

-A INPUT -m state --state NEW -m tcp -p tcp --sport 3306 -j ACCEPT

-A OUTPUT -m state --state NEW -m tcp -p tcp --dport 3306 -j ACCEPT

È»ºóÖ´ÐÐÏÂÃæÃüÁʹ·À»ðǽÁ¢¼´ÉúЧ¡£

service iptables restart

µÚ¶þÖÖ£º¹Ø±Õ·À»ðǽ

ͨ¹ýÏÂÃæÁ½¸öÃüÁîʹ·À»ðǽ¹Ø±Õ£¬²¢ÇÒÓÀÔ¶²»Æð×÷Óá£

service iptables stop

chkconfig iptables off

ÎÒÃÇÔÚÕâÀïΪÁË·½±ã£¬²ÉÓõڶþÖÖ·½·¨£¬Ö´ÐÐЧ¹ûÈçÏ¡£

µÚ¾Å²½£ºÑéÖ¤MySQLÊý¾Ý¿â±àÂëÊÇ·ñΪUTF-8¡£

Á¬½ÓÉÏÊý¾Ý¿âÖ®ºó£¬ÊäÈëÃüÁ"SHOW VARIABLES LIKE '%char%';"¼´¿É²é¿´µ½ÏÖÔÚÄãµÄÊý¾Ý¿âËùʹÓõÄ×Ö·û¼¯ÁË¡£

µÚÊ®²½£ºÉ¾³ý¿ÕÓû§£¬ÔöÇ¿°²È«¡£

ĿǰΪֹÎÒÃǶ¼ÊÇÒÔ"root"µÄÉí·Ý½øÐе쬵«Êǵ±ÎÒÃÇÇл»ÖÁÆÕͨÓû§µÇ¼MySQLʱ£¬Ö±½ÓÊäÈë"mysql"¾Í½øÈ¥ÁË£¬ÎÒÃǸղŲ»ÊÇÉèÖÃÃÜÂëÁËÂð£¿Ôõô¾ÍʧЧÁËÄØ£¿ËµÃ÷ÓпÕÓû§´æÔÚ¡£ÏÈÓÃÃüÁî"exit"Í˳ö£¬ÔÚ°´ÕÕÏÂÃæÃüÁî½øÐÐÐÞÕý¡£

½â¾ö²½ÖèÈçÏ£º

ÒÔMySQLÓû§"root"ÓÃÃÜÂëÐÎʽµÇ¼¡£

mysql -u root -p

ɾ³ý¿ÕÓû§£¬Ç¿ÁÒ½¨Òé¡£

mysql>delete from mysql.user where user='';

Ë¢ÐÂȨÏÞ±í£¬ÒÔ±ã¿ÉÒÔʹ¸ü¸ÄÁ¢¼´ÉúЧ¡£

mysql>flush privileges;

ÊäÈë"exit"£¬Í˳öMySQL¡£

mysql>exit

ÔÙÖØÐÂÒÔ"mysql"µÇ¼²âÊÔ

mysql

·¢ÏÖÒÔ"mysql"µÇ¼ÒѾ­Ê§Ð§£¬±ØÐëÒÔ"mysql –u root -p"²ÅÄܵǼ¡£

ÏÂÃæÊÇÖ´ÐÐЧ¹û½ØÍ¼£º

2¡¢MapReduceÓëMySQL½»»¥

MapReduce¼¼ÊõÍÆ³öºó£¬ÔøÔâµ½¹ØÏµÊý¾Ý¿âÑо¿ÕßµÄÌôÌÞºÍÅúÆÀ£¬ÈÏΪMapReduce²»¾ß±¸ÓÐÀàËÆÓÚ¹ØÏµÊý¾Ý¿âÖеĽṹ»¯Êý¾Ý´æ´¢ºÍ´¦ÀíÄÜÁ¦¡£Îª´Ë£¬GoogleºÍMapReduceÉçÇø½øÐÐÁ˺ܶàŬÁ¦¡£Ò»·½Ã棬ËûÃÇÉè¼ÆÁËÀàËÆÓÚ¹ØÏµÊý¾ÝÖнṹ»¯Êý¾Ý±íµÄ¼¼Êõ£¨GoogleµÄBigTable£¬HadoopµÄHBase£©ÌṩһЩ´ÖÁ£¶ÈµÄ½á¹¹»¯Êý¾Ý´æ´¢ºÍ´¦ÀíÄÜÁ¦£»ÁíÒ»·½Ã棬ΪÁËÔöÇ¿Óë¹ØÏµÊý¾Ý¿âµÄ¼¯³ÉÄÜÁ¦£¬Hadoop MapReduceÌṩÁËÏàÓ¦µÄ·ÃÎʹØÏµÊý¾Ý¿â¿âµÄ±à³Ì½Ó¿Ú¡£

MapReduceÓëMySQL½»»¥µÄÕûÌå¼Ü¹¹ÈçÏÂͼËùʾ¡£

ͼ2-1Õû¸ö»·¾³µÄ¼Ü¹¹

¾ßÌåµ½MapReduce¿ò¼Ü¶Á/дÊý¾Ý¿â£¬ÓÐ2¸öÖ÷ÒªµÄ³ÌÐò·Ö±ðÊÇDBInputFormatºÍDBOutputFormat£¬DBInputFormat ¶ÔÓ¦µÄÊÇSQLÓï¾äselect£¬¶øDBOutputFormat ¶ÔÓ¦µÄÊÇInster/update£¬Ê¹Óà DBInputFormatºÍDBOutputFormaʱºòÐèҪʵÏÖInputFormatÕâ¸ö³éÏóÀ࣬Õâ¸ö³éÏóÀຬÓÐgetSplits()ºÍ createRecordReader()³éÏó·½·¨£¬ÔÚDBInputFormatÀàÖÐÓÉ protected String getCountQuery() ·½·¨´«Èë½á¹û¼¯µÄ¸öÊý£¬getSplits()·½·¨ÔÙÈ·¶¨ÊäÈëµÄÇзÖÔ­Ôò£¬ÀûÓÃSQLÖÐµÄ LIMIT ºÍ OFFSET ½øÐÐÇзֻñµÃÊý¾Ý¼¯µÄ·¶Î§ £¬Çë²Î¿¼DBInputFormatÔ´ÂëÖÐpublic InputSplit[] getSplits(JobConf job, int chunks) throws IOExceptionµÄ·½·¨£¬ÔÚDBInputFormatÔ´ÂëÖÐcreateRecordReader()Ôò¿ÉÒÔ°´Ò»¶¨¸ñʽ¶ÁÈ¡ÏàÓ¦Êý¾Ý¡£

1£©½¨Á¢¹ØÏµÊý¾Ý¿âÁ¬½Ó

DBConfiguration£ºÌṩÊý¾Ý¿âÅäÖúʹ´½¨Á¬½ÓµÄ½Ó¿Ú¡£

DBConfigurationÀàÖÐÌṩÁËÒ»¸ö¾²Ì¬·½·¨´´½¨Êý¾Ý¿âÁ¬½Ó£º

public static void configureDB(Job job,String driverClass,StringdbUrl,String userName,String Password)

ÆäÖУ¬jobΪµ±Ç°×¼±¸Ö´ÐеÄ×÷Òµ£¬driverClasssΪÊý¾Ý¿â³§ÉÌÌṩµÄ·ÃÎÊÆäÊý¾Ý¿âµÄÇý¶¯³ÌÐò£¬dbUrlΪÔËÐÐÊý¾Ý¿âµÄÖ÷»úµÄµØÖ·£¬userNameºÍpassword·Ö±ðΪÊý¾Ý¿âÌṩ·ÃÎʵØÓû§ÃûºÍÏàÓ¦µÄ·ÃÎÊÃÜÂë¡£

2£©ÏàÓ¦µÄ´Ó¹ØÏµÊý¾Ý¿â²éѯºÍ¶ÁÈ¡Êý¾ÝµÄ½Ó¿Ú

DBInputFormat£ºÌṩ´ÓÊý¾Ý¿â¶ÁÈ¡Êý¾ÝµÄ¸ñʽ¡£

  • DBRecordReader£ºÌṩ¶ÁÈ¡Êý¾Ý¼Ç¼µÄ½Ó¿Ú¡£

3£©ÏàÓ¦µÄÏò¹ØÏµÊý¾Ý¿âÖ±½ÓÊä³ö½á¹ûµÄ±à³Ì½Ó¿Ú

  • DBOutputFormat£ºÌṩÏòÊý¾Ý¿âÊä³öÊý¾ÝµÄ¸ñʽ¡£
  • DBRecordWrite£ºÌṩÊý¾Ý¿âдÈëÊý¾Ý¼Ç¼µÄ½Ó¿Ú¡£

Êý¾Ý¿âÁ¬½ÓÍê³Éºó£¬¼´¿ÉÍê³É´ÓMapReduce³ÌÐòÏò¹ØÏµÊý¾Ý¿âдÈëÊý¾ÝµÄ²Ù×÷¡£ÎªÁ˸æÖªÊý¾Ý¿â½«Ð´ÈëÄĸö±íÖеÄÄÄЩ×ֶΣ¬DBOutputFormatÖÐÌṩÁËÒ»¸ö¾²Ì¬·½·¨À´Ö¸¶¨ÐèҪдÈëµÄÊý¾Ý±íºÍ×ֶΣº

public static void setOutput(Job job,String tableName,String ... fieldName)

ÆäÖУ¬tableNameÖ¸¶¨¼´½«Ð´ÈëµÄÊý¾Ý±í£¬ºóÐø²ÎÊý½«Ö¸¶¨ÄÄЩ×Ö¶ÎÊý¾Ý½«Ð´Èë¸Ã±í¡£

2.1 ´ÓÊý¾Ý¿âÖÐÊäÈëÊý¾Ý

ËäÈ»HadoopÔÊÐí´ÓÊý¾Ý¿âÖÐÖ±½Ó¶ÁÈ¡Êý¾Ý¼Ç¼×÷ΪMapReduceµÄÊäÈ룬µ«´¦ÀíЧÂʽϵͣ¬¶øÇÒ´óÁ¿Æµ·±µØ´ÓMapReduce³ÌÐòÖвéѯºÍ¶ÁÈ¡¹ØÏµÊý¾Ý¿â¿ÉÄÜ»á´ó´óÔö¼ÓÊý¾Ý¿âµÄ·ÃÎʸºÔØ£¬Òò´ËDBInputFormat½öÊʺ϶ÁȡСÁ¿Êý¾Ý¼Ç¼µÄ¼ÆËãºÍÓ¦Ó㬲»ÊʺÏÊý¾Ý²Ö¿âÁª»úÊý¾Ý·ÖÎö´óÁ¿Êý¾ÝµÄ¶ÁÈ¡´¦Àí¡£

¶ÁÈ¡´óÁ¿Êý¾Ý¼Ç¼һ¸ö¸üºÃµÄ½â¾ö°ì·¨ÊÇ£ºÓÃÊý¾Ý¿âÖеÄDump¹¤¾ß½«´óÁ¿´ý·ÖÎöÊý¾ÝÊä³öΪÎı¾Êý¾ÝÎļþ£¬²¢ÉÏÔØµ½HDFSÖнøÐд¦Àí¡£

1£©Ê×ÏÈ´´½¨Òª¶ÁÈëµÄÊý¾Ý

  • Windows»·¾³

Ê×ÏÈ´´½¨Êý¾Ý¿â"school"£¬Ê¹ÓÃÏÂÃæÃüÁî½øÐУº

create database school;

È»ºóͨ¹ýÒÔϼ¸¾ä»°£¬°ÑÎÒÃÇÊÂÏÈ×¼±¸ºÃµÄsqlÓï¾ä£¨student.sqlÊÂÏȷŵ½ÁËDÅÌĿ¼£©µ¼Èëµ½¸Õ´´½¨µÄ"school"Êý¾Ý¿âÖС£Óõ½µÄÃüÁîÈçÏ£º

use school;

source d:\student.sql

"student.sql"ÖеÄÄÚÈÝÈçÏÂËùʾ£º

DROP TABLE IF EXISTS `school`.`student`;

CREATE TABLE `school`.`student` (

`id` int(11) NOT NULL default '0',

`name` varchar(20) default NULL,

`sex` varchar(10) default NULL,

`age` int(10) default NULL,

PRIMARY KEY (`id`)

) ENGINE=InnoDB DEFAULT CHARSET=utf8;

INSERT INTO `student` VALUES ('201201', 'ÕÅÈý', 'ÄÐ', '21');

INSERT INTO `student` VALUES ('201202', 'ÀîËÄ', 'ÄÐ', '22');

INSERT INTO `student` VALUES ('201203', 'ÍõÎå', 'Å®', '20');

INSERT INTO `student` VALUES ('201204', 'ÕÔÁù', 'ÄÐ', '21');

INSERT INTO `student` VALUES ('201205', 'Сºì', 'Å®', '19');

INSERT INTO `student` VALUES ('201206', 'СÃ÷', 'ÄÐ', '22');

Ö´Ðнá¹ûÈçÏÂËùʾ£º

²éѯ¸Õ²Å´´½¨µÄÊý¾Ý¿â±í"student"µÄÄÚÈÝ¡£

½á¹û·¢ÏÖÏÔʾÊÇÂÒÂ룬¼ÇµÃÎÒµ±Ê±ÊÇÉèÖõÄUTF-8£¬Ôõô¾Í³öÏÖÂÒÂëÁËÄØ£¿ÆäʵÎÒÃÇʹÓõIJÙ×÷ϵͳµÄϵͳΪÖÐÎÄ£¬ÇÒËüµÄĬÈϱàÂëÊÇgbk£¬¶øMySQLµÄ±àÂëÓÐÁ½ÖÖ£¬ËüÃÇ·Ö±ðÊÇ£º

¡¾client¡¿£º¿Í»§¶ËµÄ×Ö·û¼¯¡£¿Í»§¶ËĬÈÏ×Ö·û¼¯¡£µ±¿Í»§¶ËÏò·þÎñÆ÷·¢ËÍÇëÇóʱ£¬ÇëÇóÒÔ¸Ã×Ö·û¼¯½øÐбàÂë¡£

¡¾mysqld¡¿£º·þÎñÆ÷×Ö·û¼¯£¬Ä¬ÈÏÇé¿öÏÂËù²ÉÓõġ£

ÕÒµ½°²×°MySQLĿ¼£¬±ÈÈçÎÒÃǵݲװĿ¼Ϊ£º

E:\HadoopWorkPlat\MySQL Server 5.5

´ÓÖÐÕÒµ½"my.ini"ÅäÖÃÎļþ£¬×îÖÕ·¢ÏÖmy.iniÀïµÄ2¸öcharacter_set°Ñclient¸Ä³Égbk£¬°Ñserver¸Ä³Éutf8¾Í¿ÉÒÔÁË¡£

¡¾client¡¿¶Ë£º

[client]

port=3306

[mysql]

default-character-set=gbk

¡¾mysqld¡¿¶Ë£º

[mysqld]

# The default character set that will be used when a new schema or table is

# created and no character set is defined

character-set-server=utf8

°´ÕÕÉÏÃæÐÞ¸ÄÍêÖ®ºó£¬ÖØÆôMySQL·þÎñ¡£

´ËʱÔÚWindowsÏÂÃæµÄÊý¾Ý¿â±íÒѾ­×¼±¸Íê³ÉÁË¡£

  • Linux»·¾³

Ê×ÏÈͨ¹ý"FlashFXP"°ÑÎÒÃǸղŵÄ"student.sql"ÉÏ´«µ½"/home/hadoop"Ŀ¼ÏÂÃæ£¬È»ºó°´ÕÕÉÏÃæµÄÓï¾ä´´½¨"school"Êý¾Ý¿â¡£

²é¿´ÎÒÃÇÉÏ´«µÄ"student.sql"ÄÚÈÝ£º

´´½¨"school"Êý¾Ý¿â£¬²¢µ¼Èë"student.sql"Óï¾ä¡£

ÏÔʾÊý¾Ý¿â"school"Öеıí"student"ÐÅÏ¢¡£

ÏÔʾ±í"student"ÖеÄÄÚÈÝ¡£

µ½´ËΪֹÔÚ"Windows"ºÍ"Linux"Á½ÖÖ»·¾³ÏÂÃæ¶¼´´½¨Á˱í"student"±í£¬²¢³õʼ»¯ÁËÖµ¡£ÏÂÃæ¾Í¿ªÊ¼Í¨¹ýMapReduce¶ÁÈ¡MySQL¿âÖбí"student"µÄÐÅÏ¢¡£

2£©Ê¹MySQLÄÜÔ¶³ÌÁ¬½Ó

MySQLĬÈÏÊÇÔÊÐí±ðµÄ»úÆ÷½øÐÐÔ¶³Ì·ÃÎʵأ¬ÎªÁËʹHadoop¼¯ÈºÄÜ·ÃÎÊMySQLÊý¾Ý¿â£¬ËùÒÔ½øÐÐÏÂÃæ²Ù×÷¡£

  • ÓÃMySQLÓû§"root"µÇ¼¡£

mysql -u root -p

  • ʹÓÃÏÂÃæÓï¾ä½øÐÐÊÚȨ£¬¸³ÓèÈκÎÖ÷»ú·ÃÎÊÊý¾ÝµÄȨÏÞ¡£

GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'hadoop' WITH GRANT OPTION;

  • ˢУ¬Ê¹Ö®Á¢¼´ÉúЧ¡£

FLUSH PRIVILEGES;

Ö´Ðнá¹ûÈçÏÂͼ¡£

WindowsÏÂÃæ£º

LinuxÏÂÃæ£º

µ½Ä¿Ç°ÎªÖ¹£¬Èç¹ûÁ¬½ÓWin7ÉÏÃæµÄMySQLÊý¾Ý¿â»¹²»ÐУ¬´ó¼Ò»¹Ó¦¸Ã¼ÇµÃÇ°ÃæÔÚLinuxÏÂÃæ¹ØµôÁË·À»ðǽ£¬µ«ÊÇÎÒÃÇÔÚWin7϶ԷÀ»ðǽ²¢Ã»ÓÐ×öÈκδ¦Àí£¬Èç¹û²»¶Ô·À»ðǽ×ö´¦Àí£¬¼´Ê¹Ö´ÐÐÁËÉÏÃæµÄÔ¶³ÌÊÚȨ£¬ÈÔÈ»²»ÄÜÁ¬½Ó¡£ÏÂÃæÊÇÉèÖÃWin7ÉÏÃæµÄ·À»ðǽ£¬Ê¹Ô¶³Ì»úÆ÷ÄÜͨ¹ý3306¶Ë¿Ú·ÃÎÊMySQLÊý¾Ý¿â¡£

½â¾ö·½°¸£ºÖ»ÒªÔÚ'ÈëÕ¾¹æÔò'ÉϽ¨Á¢Ò»¸ö3306¶Ë¿Ú¼´¿É¡£

Ö´ÐÐ˳Ðò£º¿ØÖÆÃæ°åà¹ÜÀí¹¤¾ßà¸ß¼¶°²È«µÄWindows·À»ðǽàÈëÕ¾¹æÔò

È»ºóн¨¹æÔòàÑ¡Ôñ'¶Ë¿Ú'àÔÚ'ÌØ¶¨±¾µØ¶Ë¿Ú'ÉÏÊäÈëÒ»¸ö'3306'àÑ¡Ôñ'ÔÊÐíÁ¬½Ó'=>Ñ¡Ôñ'Óò'¡¢'רÓÃ'¡¢'¹«ÓÃ'=>¸ø¸öÃû³Æ£¬È磺MySqlInput

3£©¶ÔJDBCµÄJar°ü´¦Àí

ÒòΪ³ÌÐòËäÈ»ÓÃEclipse±àÒëÔËÐе«×îÖÕÒªÌá½»µ½Hadoop¼¯ÈºÉÏ£¬ËùÒÔJDBCµÄjar±ØÐë·Åµ½Hadoop¼¯ÈºÖС£ÓÐÁ½ÖÖ·½Ê½£º

£¨1£©ÔÚÿ¸ö½ÚµãϵÄ${HADOOP_HOME}/libÏÂÌí¼Ó¸Ã°ü£¬ÖØÆô¼¯Èº£¬Ò»°ãÊDZȽÏԭʼµÄ·½·¨¡£

ÎÒÃǵÄHadoop°²×°°üÔÚ"/usr/hadoop"£¬ËùÒÔ°ÑJar·Åµ½"/usr/hadoop/lib"ÏÂÃæ£¬È»ºóÖØÆô£¬¼ÇµÃÊÇHadoop¼¯ÈºÖÐËùÓеĽڵ㶼Ҫ·Å£¬ÒòΪִÐзֲ¼Ê½ÊdzÌÐòÊÇÔÚÿ¸ö½Úµã±¾µØ»úÆ÷ÉϽøÐС£

£¨2£©ÔÚHadoop¼¯ÈºµÄ·Ö²¼Ê½ÎļþϵͳÖд´½¨"/lib"Îļþ¼Ð£¬²¢°ÑÎÒÃǵĵÄJDBCµÄjar°üÉÏ´«ÉÏÈ¥£¬È»ºóÔÚÖ÷³ÌÐòÌí¼ÓÈçÏÂÓï¾ä£¬¾ÍÄܱ£Ö¤Hadoop¼¯ÈºÖÐËùÓеĽڵ㶼ÄÜʹÓÃÕâ¸öjar°ü¡£ÒòΪÕâ¸öjar°ü·ÅÔÚÁËHDFSÉÏ£¬¶ø²»ÊDZ¾µØÏµÍ³£¬Õâ¸öÒªÀí½âÇå³þ¡£

DistributedCache.addFileToClassPath(new Path("/lib/mysql-connector-java-5.1.18-bin.jar"), conf);

ÎÒÃÇÓõÄJDBCµÄjarÈçÏÂËùʾ£º

mysql-connector-java-5.1.18-bin.jar

ͨ¹ýEclipseÏÂÃæµÄDFS Locations½øÐд´½¨"/lib"Îļþ¼Ð£¬²¢ÉÏ´«JDBCµÄjar°ü¡£Ö´Ðнá¹ûÈçÏ£º

±¸×¢£ºÎÒÃÇÕâÀï²ÉÓÃÁ˵ڶþÖÖ·½Ê½¡£

4£©Ô´³ÌÐò´úÂëÈçÏÂËùʾ

package com.hebut.mr;

import java.io.IOException;

import java.io.DataInput;

import java.io.DataOutput;

import java.sql.Connection;

import java.sql.DriverManager;

import java.sql.PreparedStatement;

import java.sql.ResultSet;

import java.sql.SQLException;

import org.apache.hadoop.filecache.DistributedCache;

import org.apache.hadoop.fs.Path;

import org.apache.hadoop.io.LongWritable;

import org.apache.hadoop.io.Text;

import org.apache.hadoop.io.Writable;

import org.apache.hadoop.mapred.JobClient;

import org.apache.hadoop.mapred.JobConf;

import org.apache.hadoop.mapred.MapReduceBase;

import org.apache.hadoop.mapred.Mapper;

import org.apache.hadoop.mapred.OutputCollector;

import org.apache.hadoop.mapred.FileOutputFormat;

import org.apache.hadoop.mapred.Reporter;

import org.apache.hadoop.mapred.lib.IdentityReducer;

import org.apache.hadoop.mapred.lib.db.DBWritable;

import org.apache.hadoop.mapred.lib.db.DBInputFormat;

import org.apache.hadoop.mapred.lib.db.DBConfiguration;

public class ReadDB {

public static class Mapextends MapReduceBaseimplements

Mapper<LongWritable, StudentRecord, LongWritable, Text> {

//ʵÏÖmapº¯Êý

public void map(LongWritable key, StudentRecord value,

OutputCollector<LongWritable, Text> collector, Reporter reporter)

throws IOException {

collector.collect(new LongWritable(value.id),

new Text(value.toString()));

}

}

public static class StudentRecordimplements Writable, DBWritable {

public int id;

public Stringname;

public Stringsex;

public int age;

@Override

public void readFields(DataInput in)throws IOException {

this.id = in.readInt();

this.name = Text.readString(in);

this.sex = Text.readString(in);

this.age = in.readInt();

}

@Override

public void write(DataOutput out)throws IOException {

out.writeInt(this.id);

Text.writeString(out,this.name);

Text.writeString(out,this.sex);

out.writeInt(this.age);

}

@Override

public void readFields(ResultSet result)throws SQLException {

this.id = result.getInt(1);

this.name = result.getString(2);

this.sex = result.getString(3);

this.age = result.getInt(4);

}

@Override

public void write(PreparedStatement stmt)throws SQLException {

stmt.setInt(1, this.id);

stmt.setString(2, this.name);

stmt.setString(3, this.sex);

stmt.setInt(4, this.age);

}

@Override

public String toString() {

return new String("ѧºÅ£º" + this.id +"_ÐÕÃû£º" +this.name

+ "_ÐÔ±ð£º"+this.sex +"_ÄêÁ䣺" +this.age);

}

}

public static void main(String[] args)throws Exception {

JobConf conf = new JobConf(ReadDB.class);

//Õâ¾ä»°ºÜ¹Ø¼ü

conf.set("mapred.job.tracker","192.168.1.2:9001");

// ·Ç³£ÖØÒª£¬ÖµµÃ¹Ø×¢

DistributedCache.addFileToClassPath(new Path(

"/lib/mysql-connector-java-5.1.18-bin.jar"), conf);

//ÉèÖÃÊäÈëÀàÐÍ

conf.setInputFormat(DBInputFormat.class);

//ÉèÖÃÊä³öÀàÐÍ

conf.setOutputKeyClass(LongWritable.class);

conf.setOutputValueClass(Text.class);

//ÉèÖÃMapºÍReduceÀà

conf.setMapperClass(Map.class);

conf.setReducerClass(IdentityReducer.class);

//ÉèÖÃÊä³öĿ¼

FileOutputFormat.setOutputPath(conf,new Path("rdb_out"));

//½¨Á¢Êý¾Ý¿âÁ¬½Ó

DBConfiguration.configureDB(conf,"com.mysql.jdbc.Driver",

"jdbc:mysql://192.168.1.24:3306/school","root", "hadoop");

//¶ÁÈ¡"student"±íÖеÄÊý¾Ý

String[] fields = { "id", "name","sex", "age" };

DBInputFormat.setInput(conf, StudentRecord.class,"student", null,"id", fields);

JobClient.runJob(conf);

}

}

±¸×¢£ºÓÉÓÚHadoop1.0.0еÄAPI¶Ô¹ØÏµÐÍÊý¾Ý¿âÔݲ»Ö§³Ö£¬Ö»ÄÜÓþɵÄAPI½øÐУ¬ËùÒÔÏÂÃæµÄ"ÏòÊý¾Ý¿âÖÐÊä³öÊý¾Ý"Ò²ÊÇÈç´Ë¡£

5£©ÔËÐнá¹ûÈçÏÂËùʾ

¾­¹ýÉÏÃæµÄÉèÖúó£¬ÒѾ­Í¨¹ýÁ¬½ÓWin7ºÍLinuxÉϵÄMySQLÊý¾Ý¿â£¬Ö´Ðнá¹û¶¼Ò»Ñù¡£Î¨¶À±äµÃ¾ÍÊÇ´úÂëÖÐ"DBConfiguration.configureDB"ÖÐMySQLÊý¾Ý¿âËùÔÚ»úÆ÷µÄIPµØÖ·¡£

2.2 ÏòÊý¾Ý¿âÖÐÊä³öÊý¾Ý

»ùÓÚÊý¾Ý²Ö¿âµÄÊý¾Ý·ÖÎöºÍÍÚ¾òÊä³ö½á¹ûµÄÊý¾ÝÁ¿Ò»°ã²»»áÌ«´ó£¬Òò¶ø¿ÉÄÜÊʺÏÓÚÖ±½ÓÏòÊý¾Ý¿âдÈë¡£ÎÒÃÇÕâÀï³¢ÊÔÓë"WordCount"³ÌÐòÏà½áºÏ£¬°Ñµ¥´Êͳ¼ÆµÄ½á¹û´æÈëµ½¹ØÏµÐÍÊý¾Ý¿âÖС£

1£©´´½¨Ð´ÈëµÄÊý¾Ý¿â±í

ÎÒÃÇ»¹Ê¹ÓøղŴ´½¨µÄÊý¾Ý¿â"school"£¬Ö»ÊÇÔÚÀïÌí¼ÓÒ»¸öеıí"wordcount"£¬»¹ÊÇʹÓÃÏÂÃæÓï¾äÖ´ÐУº

use school;

source sql½Å±¾È«Â·¾¶

ÏÂÃæÊÇÒª´´½¨µÄ"wordcount"±íµÄsql½Å±¾¡£

DROP TABLE IF EXISTS `school`.`wordcount`;

CREATE TABLE `school`.`wordcount` (

`id` int(11) NOT NULL auto_increment,

`word` varchar(20) default NULL,

`number` int(11) default NULL,

PRIMARY KEY (`id`)

) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Ö´ÐÐЧ¹ûÈçÏÂËùʾ£º

  • Windows»·¾³

  • Linux»·¾³

2£©³ÌÐòÔ´´úÂëÈçÏÂËùʾ

package com.hebut.mr;

import java.io.IOException;

import java.io.DataInput;

import java.io.DataOutput;

import java.sql.PreparedStatement;

import java.sql.ResultSet;

import java.sql.SQLException;

import java.util.Iterator;

import java.util.StringTokenizer;

import org.apache.hadoop.filecache.DistributedCache;

import org.apache.hadoop.fs.Path;

import org.apache.hadoop.io.IntWritable;

import org.apache.hadoop.io.Text;

import org.apache.hadoop.io.Writable;

import org.apache.hadoop.mapred.FileInputFormat;

import org.apache.hadoop.mapred.JobClient;

import org.apache.hadoop.mapred.JobConf;

import org.apache.hadoop.mapred.MapReduceBase;

import org.apache.hadoop.mapred.Mapper;

import org.apache.hadoop.mapred.OutputCollector;

import org.apache.hadoop.mapred.Reducer;

import org.apache.hadoop.mapred.Reporter;

import org.apache.hadoop.mapred.TextInputFormat;

import org.apache.hadoop.mapred.lib.db.DBOutputFormat;

import org.apache.hadoop.mapred.lib.db.DBWritable;

import org.apache.hadoop.mapred.lib.db.DBConfiguration;

public class WriteDB {

// Map´¦Àí¹ý³Ì

public static class Mapextends MapReduceBaseimplements

Mapper<Object, Text, Text, IntWritable> {

private final static IntWritableone = new IntWritable(1);

private Textword = new Text();

@Override

public void map(Object key, Text value,

OutputCollector<Text, IntWritable> output, Reporter reporter)

throws IOException {

String line = value.toString();

StringTokenizer tokenizer = new StringTokenizer(line);

while (tokenizer.hasMoreTokens()) {

word.set(tokenizer.nextToken());

output.collect(word,one);

}

}

}

// Combine´¦Àí¹ý³Ì

public static class Combineextends MapReduceBaseimplements

Reducer<Text, IntWritable, Text, IntWritable> {

@Override

public void reduce(Text key, Iterator<IntWritable> values,

OutputCollector<Text, IntWritable> output, Reporter reporter)

throws IOException {

int sum = 0;

while (values.hasNext()) {

sum += values.next().get();

}

output.collect(key, new IntWritable(sum));

}

}

// Reduce´¦Àí¹ý³Ì

public static class Reduceextends MapReduceBaseimplements

Reducer<Text, IntWritable, WordRecord, Text> {

@Override

public void reduce(Text key, Iterator<IntWritable> values,

OutputCollector<WordRecord, Text> collector, Reporter reporter)

throws IOException {

int sum = 0;

while (values.hasNext()) {

sum += values.next().get();

}

WordRecord wordcount = new WordRecord();

wordcount.word = key.toString();

wordcount.number = sum;

collector.collect(wordcount, new Text());

}

}

public static class WordRecordimplements Writable, DBWritable {

public Stringword;

public int number;

@Override

public void readFields(DataInput in)throws IOException {

this.word = Text.readString(in);

this.number = in.readInt();

}

@Override

public void write(DataOutput out)throws IOException {

Text.writeString(out,this.word);

out.writeInt(this.number);

}

@Override

public void readFields(ResultSet result)throws SQLException {

this.word = result.getString(1);

this.number = result.getInt(2);

}

@Override

public void write(PreparedStatement stmt)throws SQLException {

stmt.setString(1, this.word);

stmt.setInt(2, this.number);

}

}

public static void main(String[] args)throws Exception {

JobConf conf = new JobConf(WriteDB.class);

//Õâ¾ä»°ºÜ¹Ø¼ü

conf.set("mapred.job.tracker","192.168.1.2:9001");

DistributedCache.addFileToClassPath(new Path(

"/lib/mysql-connector-java-5.1.18-bin.jar"), conf);

//ÉèÖÃÊäÈëÊä³öÀàÐÍ

conf.setInputFormat(TextInputFormat.class);

conf.setOutputFormat(DBOutputFormat.class);

// ²»¼ÓÕâÁ½¾ä£¬Í¨²»¹ý£¬µ«ÊÇÍøÉϸøµÄÀý×ÓûÓÐÕâÁ½¾ä¡£

conf.setOutputKeyClass(Text.class);

conf.setOutputValueClass(IntWritable.class);

//ÉèÖÃMapºÍReduceÀà

conf.setMapperClass(Map.class);

conf.setCombinerClass(Combine.class);

conf.setReducerClass(Reduce.class);

//ÉèÖÃÊäÈçĿ¼

FileInputFormat.setInputPaths(conf,new Path("wdb_in"));

//½¨Á¢Êý¾Ý¿âÁ¬½Ó

DBConfiguration.configureDB(conf,"com.mysql.jdbc.Driver",

"jdbc:mysql://192.168.1.24:3306/school","root", "hadoop");

//дÈë"wordcount"±íÖеÄÊý¾Ý

String[] fields = { "word", "number" };

DBOutputFormat.setOutput(conf,"wordcount", fields);

JobClient.runJob(conf);

}

}

3£©ÔËÐнá¹ûÈçÏÂËùʾ

  • Windows»·¾³

²âÊÔÊý¾Ý£º

£¨1£©file1.txt

hello word

hello hadoop

£¨2£©file2.txt

ϺƤ hadoop

ϺƤ word

Èí¼þ Èí¼þ

ÔËÐнá¹û£º

ÎÒÃÇ·¢ÏÖÉÏͼÖгöÏÖÁË"?"£¬ºóÀ´²éÕÒÔ­À´ÊÇÒòΪÎҵIJâÊÔÊý¾ÝʱÔÚWindowsÓüÇʱ¾Ð´µÄÈ»ºó±£´æÎª"UTF-8"£¬ÔÚ±£´æÊ±ÎªÁËÇø·Ö±àÂ룬×Ô¶¯ÔÚÇ°Ãæ¼ÓÁËÒ»¸ö"BOM"£¬µ«ÊDz»»áÏÔʾÈκνá¹û¡£È»¶øÎÒÃǵĴúÂë°ÑËüʶ±ðΪ"?"½øÐд¦Àí¡£Õâ¾Í³öÏÖÁËÉÏÃæµÄ½á¹û£¬Èç¹ûÎÒÃÇÔÚÿ¸öÒª´¦ÀíµÄÎļþÇ°ÃæµÄµÚÒ»ÐмÓÒ»¸ö¿Õ¸ñ£¬½á¹û¾Í³ÉÈçÏÂÏÔʾ£º

½Ó×ÅÓÖ×öÁËÒ»¸ö²âÊÔ£¬ÔÚLinuxÉÏÃæÓÃÏÂÃæÃüÁî´´½¨ÁËÒ»¸öÎļþ£¬²¢Ð´ÉÏÖÐÎÄÄÚÈÝ¡£½á¹ûÏÔʾ²¢Ã»ÓгöÏÖ"?"£¬¶øÇÒÍøÉÏ˵²»Í¬µÄ¼Çʱ¾Èí¼þ£¨EmEditor¡¢UE£©±£´æÎª"UTF-8"¾ÍûÓÐÕâ¸öÎÊÌâ¡£¾­¹ýÐÞ¸ÄÖ®ºóµÄMapÀ࣬¾ÍÄܹ»Õý³£Ê¶±ðÁË¡£

// Map´¦Àí¹ý³Ì

public static class Mapextends MapReduceBaseimplements

Mapper<Object, Text, Text, IntWritable> {

private final static IntWritableone = new IntWritable(1);

private Textword = new Text();

@Override

public void map(Object key, Text value,

OutputCollector<Text, IntWritable> output, Reporter reporter)

throws IOException {

String line = value.toString();

//´¦Àí¼Çʱ¾UTF-8µÄBOMÎÊÌâ

if (line.getBytes().length > 0) {

if ((int) line.charAt(0) == 65279) {

line = line.substring(1);

}

}

StringTokenizer tokenizer = new StringTokenizer(line);

while (tokenizer.hasMoreTokens()) {

word.set(tokenizer.nextToken());

output.collect(word,one);

}

}

}

´¦ÀíÖ®ºóµÄ½á¹û£º

´ÓÉÏͼÖеÃÖª£¬ÎÒÃǵÄÎÊÌâÒѾ­½â¾öÁË£¬Òò´Ë£¬Ôڱ༭¡¢¸ü¸ÄÈκÎÎı¾Îļþʱ£¬ÇëÎñ±ØÊ¹Óò»»áÂÒ¼ÓBOMµÄ±à¼­Æ÷¡£Linuxϵı༭Æ÷Ó¦¸Ã¶¼Ã»ÓÐÕâ¸ö ÎÊÌâ¡£WindowsÏ£¬ÇëÎðʹÓüÇʱ¾µÈ±à¼­Æ÷¡£ÍƼöµÄ±à¼­Æ÷ÊÇ£º Editplus 2.12°æ±¾ÒÔÉÏ£» EmEditor£» UltraEdit£¨ÐèҪȡÏû'Ìí¼ÓBOM'µÄÏà¹ØÑ¡Ï£» Dreamweaver£¨ÐèҪȡÏû'Ìí¼ÓBOM'µÄÏà¹ØÑ¡Ï µÈ¡£

¶ÔÓÚÒѾ­Ìí¼ÓÁËBOMµÄÎļþ£¬ÒªÈ¡ÏûµÄ»°£¬¿ÉÒÔÓÃÒÔÉϱ༭Æ÷Áí´æÒ»´Î¡££¨EditplusÐèÒªÏÈÁí´æÎªgb£¬ÔÙÁí´æÎªUTF-8¡££© DW½â¾ö°ì·¨ÈçÏ£º ÓÃDW´ò¿ªÖ¸¶¨Îļþ£¬°´Ctrl+Jà±êÌâ/±àÂëà±àÂëÑ¡Ôñ"UTF-8"£¬È¥µô"°üÀ¨UnicodeÇ©Ãû(BOM)"¹´Ñ¡à±£´æ/Áí´æÎª£¬¼´¿É¡£

¹úÍâÓÐÒ»¸öÅ£ÈËÒѾ­°ÑÕâ¸öÎÊÌâ½â¾öÁË£¬Ê¹ÓÃ"UnicodeInputStream"¡¢"UnicodeReader"¡£

µØÖ·£ºhttp://koti.mbnet.fi/akini/java/unicodereader/

ʾÀý£ºJava¶Á´øÓÐBOMµÄUTF-8ÎļþÂÒÂëÔ­Òò¼°½â¾ö·½·¨

´úÂ룺http://download.csdn.net/detail/xia520pi/4146123

  • Linux»·¾³

²âÊÔÊý¾Ý£º

£¨1£©file1.txt

MapReduce is simple

£¨2£©file2.txt

MapReduce is powerful is simple

£¨3£©file2.txt

Hello MapReduce bye MapReduce

ÔËÐнá¹û£º

µ½Ä¿Ç°ÎªÖ¹£¬MapReduceÓë¹ØÏµÐÍÊý¾Ý¿â½»»¥ÒѾ­½áÊø£¬´Ó½á¹ûÖеÃÖª£¬Ä¿Ç°Ð°æµÄAPI»¹²»ÄܺܺõÄÖ§³Ö¹ØÏµÐÍÊý¾Ý¿âµÄ²Ù×÷£¬ÉÏÃæÁ½¸öÀý×Ó¶¼ÊÇʹÓõľɰæµÄAPI¡£

   
3917 ´Îä¯ÀÀ       31
Ïà¹ØÎÄÕÂ

»ùÓÚEAµÄÊý¾Ý¿â½¨Ä£
Êý¾ÝÁ÷½¨Ä££¨EAÖ¸ÄÏ£©
¡°Êý¾Ýºþ¡±£º¸ÅÄî¡¢ÌØÕ÷¡¢¼Ü¹¹Óë°¸Àý
ÔÚÏßÉ̳ÇÊý¾Ý¿âϵͳÉè¼Æ ˼·+Ч¹û
 
Ïà¹ØÎĵµ

GreenplumÊý¾Ý¿â»ù´¡Åàѵ
MySQL5.1ÐÔÄÜÓÅ»¯·½°¸
ijµçÉÌÊý¾ÝÖÐ̨¼Ü¹¹Êµ¼ù
MySQL¸ßÀ©Õ¹¼Ü¹¹Éè¼Æ
Ïà¹Ø¿Î³Ì

Êý¾ÝÖÎÀí¡¢Êý¾Ý¼Ü¹¹¼°Êý¾Ý±ê×¼
MongoDBʵս¿Î³Ì
²¢·¢¡¢´óÈÝÁ¿¡¢¸ßÐÔÄÜÊý¾Ý¿âÉè¼ÆÓëÓÅ»¯
PostgreSQLÊý¾Ý¿âʵսÅàѵ
×îл¼Æ»®
DeepSeek´óÄ£ÐÍÓ¦Óÿª·¢ 6-12[ÏÃÃÅ]
È˹¤ÖÇÄÜ.»úÆ÷ѧϰTensorFlow 6-22[Ö±²¥]
»ùÓÚ UML ºÍEA½øÐзÖÎöÉè¼Æ 6-30[±±¾©]
ǶÈëʽÈí¼þ¼Ü¹¹-¸ß¼¶Êµ¼ù 7-9[±±¾©]
Óû§ÌåÑé¡¢Ò×ÓÃÐÔ²âÊÔÓëÆÀ¹À 7-25[Î÷°²]
ͼÊý¾Ý¿âÓë֪ʶͼÆ× 8-23[±±¾©]

MySQLË÷Òý±³ºóµÄÊý¾Ý½á¹¹
MySQLÐÔÄܵ÷ÓÅÓë¼Ü¹¹Éè¼Æ
SQL ServerÊý¾Ý¿â±¸·ÝÓë»Ö¸´
ÈÃÊý¾Ý¿â·ÉÆðÀ´ 10´óDB2ÓÅ»¯
oracleµÄÁÙʱ±í¿Õ¼äдÂú´ÅÅÌ
Êý¾Ý¿âµÄ¿çƽ̨Éè¼Æ


²¢·¢¡¢´óÈÝÁ¿¡¢¸ßÐÔÄÜÊý¾Ý¿â
¸ß¼¶Êý¾Ý¿â¼Ü¹¹Éè¼ÆÊ¦
HadoopÔ­ÀíÓëʵ¼ù
Oracle Êý¾Ý²Ö¿â
Êý¾Ý²Ö¿âºÍÊý¾ÝÍÚ¾ò
OracleÊý¾Ý¿â¿ª·¢Óë¹ÜÀí


GE Çø¿éÁ´¼¼ÊõÓëʵÏÖÅàѵ
º½Ìì¿Æ¹¤Ä³×Ó¹«Ë¾ Nodejs¸ß¼¶Ó¦Óÿª·¢
ÖÐÊ¢Òæ»ª ׿Խ¹ÜÀíÕß±ØÐë¾ß±¸µÄÎåÏîÄÜÁ¦
ijÐÅÏ¢¼¼Êõ¹«Ë¾ PythonÅàѵ
ij²©²ÊITϵͳ³§ÉÌ Ò×ÓÃÐÔ²âÊÔÓëÆÀ¹À
ÖйúÓÊ´¢ÒøÐÐ ²âÊÔ³ÉÊì¶ÈÄ£Ðͼ¯³É(TMMI)
ÖÐÎïÔº ²úÆ·¾­ÀíÓë²úÆ·¹ÜÀí