ÕªÒª£ºÍ¨¹ý±¾ÏµÁÐÎÄÕ£¬Äãѧ»áÁËÔÚVPCÖÐʹÓö¨ÖÆDNS·þÎñÆ÷À´·¢²¼Ò»¸öÓµÓж¨ÖÆË½ÓÐÓòÃûµÄEMR¼¯Èº¡£Ö»ÒªÄãÄܹ»½âÎö¼¯ÈºÖеÄËùÓнڵ㣬Äã¿ÉÒÔʹÓÃÈκη½·¨À´·¢²¼Ò»¸öEMR¼¯Èº¡£
ÔÚµÚÒ»²¿·Ö£¬ÎÒÃÇѧϰÁË Amazon EMRÈçºÎʹÓà Amazon VPC
,DNSÖ÷»úÃû³ÆºÍDHCPÉèÖÃÀ´Âú×ãHadoopÐèÇó¡£¶øÔÚʵ¼ÊÓ¦ÓÃÖУ¬ÔÚ DHCPÑ¡ÏÖн«ÓòÃûÐ޸ijɶ¨ÖÆ»¥ÁªÍøÓòÃûµÄ²Ù×÷·Ç³£ÆÕ±é£¬±¾Æª²©ÎĽ«½éÉÜÈç³É¹¦µØ¸øEMRÅäÖò»Í¬µÄDNS·þÎñ£¬±ÈÈçBIND¡¢DnsmasqÒÔ¼°
Amazon Route 544¡£Í¨³£Çé¿öÏ£¬ÎÒÃÇÐèÒªÖ´ÐÐÏÂÊö¶à¸ö²Ù×÷£º
1. ΪVPC½øÐÐBIND·þÎñÆ÷ÅäÖá£
2. ÐÞ¸ÄÒÑÓÐVPCÀ´Ê¹ÓÃÒ»¸ö¶¨ÖÆÓòÃûºÍDNS·þÎñÆ÷¡£
3. Ϊ¶¨ÖÆ»¯ÓòÃû°²×°ºÍÅäÖÃDnsmasqÀ´½âÎöÓòÃû²éѯ¡£
4. ÅäÖÃRoute 53À´ÍйÜ˽ÓÐÓò¡£
ÈçÉÏÆª²©ÎÄËùÊö£¬ÎÒÃǽ«·¢²¼Ò»¸ö¼òµ¥µÄ²âÊÔ¼¯ÈºÀ´±£Ö¤Ò»ÇпÉÒÔÕý³£µÄÆô¶¯ºÍÔËÐС£
DNSÏà¹Ø
ΪÁË×Ô¶¯µØÉú³ÉDNSÖ÷»úÃû³Æ¼Ç¼£¬ÎÒÃÇÐèÒªÒ»¸öDHCP·þÎñÆ÷¡£È»¶ø£¬ÔÚVPCÖв¢²»´æÔÚÕâ¸öDHCPÑ¡ÏÒò´ËÓû§±ØÐëΪÿ¸ö·¢²¼¼¯ÈºµÄ×ÓÍøÊÖ¶¯µØÁгöËùÓÐIPµØÖ·¼Ç¼¡£´ËÍ⣬ΪÁËÂú×ãHadoop
DNSµÄ²éÕÒÐèÇó£¬Õý·´Ïò²éÕÒ¶¼ÐèÒª±»³É¹¦µØÉèÖá£Õâ¾ÍÒâζ×ÅÓû§ÐèҪΪhostname A¼Ç¼ÉèÖÃÒ»¸öÕýÏòÓò£¬Í¬Ê±Ò²ÒªÎªIP
PTR¼Ç¼ÉèÖÃÒ»¸ö·´Ïò²éÕÒÓò¡£ÔÚ±¾ÎÄÖУ¬ÎÒÃǽ«Ê¹ÓÃÒ»¸öСµÄ×ÓÍø£¨11Ìõ¿ÉÓÃÖ÷»ú¼Ç¼£©£¬Êµ¼Ê²Ù×÷ÖУ¬×ÓÍøºÍ¼¯Èº·â×°´óС¸ù¾Ýʵ¼ÊÇé¿öÖÆ¶¨¡£»ùÓÚÕâЩÐèÇó£¬Èç¹ûVPC
DNS·þÎñ²»ÄܽâÎöDHCPÑ¡Ï£¨³ý·ÇĬÈÏregion.compute.internalÓòÃûºÍRoute
53ÖÐÍйܵÄÓòÃû£¬ÆäËûµÄ²»Äܱ»½âÎö£©ÖеÄÓòÃû£¬ÄÇô±ØÐëÒª´æÔÚ¹¤¾ß¶ÔÆä½âÎö¡£ÏÂÃæÑÔ¹éÕý´«¡£
½¨Á¢Ò»¸öBIND·þÎñÆ÷ÅäÖÃ
Èç¹ûÒѾӵÓÐÒ»¸öDNS·þÎñÆ÷£¬¼òµ¥µÄ±ØÒªÖ÷»úºÍIPµØÖ·¼Ç¼ÅäÖÃÒѲ»¿ÉÐС£ÕâÒ»½ÚÎÒÃǽ«Õ¹Ê¾ÈçºÎÅäÖÃÒ»¸öBIND·þÎñÆ÷¡£»ùÓÚBIND·þÎñÆ÷ÉèÖõĸ´ÔÓÐÔ£¬ÕâÀï²»»áÒ»²½Ò»²½µÄ½øÐа²×°¡£ÕâÀï¼ÙÉèÒѾ´æÔÚÒ»¸öÏÖÓеÄBIND·þÎñÆ÷£¬ËüÒѾ±»ÅäÖúÍÔËÐÐһЩÃû³Æ½âÎö·þÎñ¡£
Èç¹ûÄãÔÚBINDµÄÉèÖÃÉÏ»¹ÊǸöÐÂÊÖ£¬ÐèҪעÒâµÄÊÇËü¶ÔÓòÎļþ´íÎó·Ç³£Ãô¸Ð¡£µ±È»£¬Èç¹ûÄãÕæµÄÒ»µã»ù´¡¶¼Ã»Óеϰ£¬
Setting up a DNS Server in Amazon EC2 Ò»ÎÄ¿ÉÒÔ°ïÖúµ½Ä㣬»òÕßÕâ¸öͨ¹ýUbuntuÒ»²½Ò»²½ÉèÖÃÒ»¸ö˽ÓеÄ
DNS·þÎñÆ÷ ¡£·ñÔò£¬Äã¿ÉÒÔÖ±½ÓÌøµ½ÏÂÃæµÄѧϰÉèÖÃDnsmasqÒ»½Ú¡£Í¬Ñù£¬¾¡¹ÜչʾÓÃÀýʹÓõÄÊÇBIND·þÎñÆ÷£¬Óû§Ò²¿ÉÒÔʹÓÃÈÎºÎÆäËûDNS·þÎñÆ÷¡£
Õâ¸öʾÀýBIND·þÎñÆ÷±»ÅäÖÃÓÃÀ´¼àÌý²éѯ£¬²»¹ÜÊDZ¾µØ»Øµ÷µØÖ·£¬Òà»òÊÇ˽Óл¥ÁªÍøIPµØÖ·£º
listen-on port 53 { 127.0.0.1; 10.20.30.5; }; |
ÔÚVPCÖУ¬ËùÓÐIPµØÖ·¶¼ÔÊÐí×öµÝ¹é²éѯÒÔ·¢ÏÖδ֪Ö÷»ú¡£ÐèҪעÒâµÄÊÇ£¬È·±£ÕâЩÉèÖÃÒÑ×ã¹»ÑϸñÀ´×èÖ¹DNSÖØ·Å¹¥»÷£º
recursion yes; allow-recursive { 10.20.30.0/24; }; allow-query { 10.20.30.0/24; }; |
ÕâÀïÐèÒªÅäÖÃÁ½¸öÓòÎļþ£º¡°hadoop.local¡±ºÍ¡°30.20.10.in-addr.arpa¡±¡£
// // named.conf // // Provided by Red Hat bind package to configure the ISC BIND named(8) DNS // server as a caching-only nameserver (as a localhost DNS resolver only). // // See /usr/share/doc/bind*/sample/ for example named configuration files. // options { listen-on port 53 { 127.0.0.1; 10.20.30.5; }; // listen-on-v6 port 53 { ::1; }; directory "/var/named"; dump-file "/var/named/data/cache_dump.db"; statistics-file "/var/named/data/named_stats.txt"; memstatistics-file "/var/named/data/named_mem_stats.txt"; allow-query { 10.20.30.0/24; }; recursion yes; allow-recursion { 10.20.30.0/24; }; dnssec-enable no; dnssec-validation no; // dnssec-lookaside auto;
/* Path to ISC DLV key */
bindkeys-file "/etc/named.iscdlv.key";
managed-keys-directory "/var/named/dynamic";
};
logging {
channel default_debug {
file "data/named.run";
severity dynamic;
};
};
zone "." IN {
type hint;
file "named.ca";
};
zone "hadoop.local" IN {
type master;
file "hadoop.zone";
};
zone "30.20.10.in-addr.arpa" IN {
type master;
file "reverse.zone";
};
include "/etc/named.rfc1912.zones";
include "/etc/named.root.key"; |
ÕýÏòÓòhadoop.local£¬°üº¬ÁËËùÓлص½10.20.30.5 BIND·þÎñÆ÷NS¼Ç¼£»ËùÓпÉÄܵÄÖ÷»úÃû³Æ¶¼±»¶¨ÒåÈçÏÂÎÄA¼Ç¼£º
$TTL 5M @ IN SOA ns.hadoop.local. root.hadoop.local. ( 0 ; serial 1D ; refresh 1H ; retry 1W ; expire 3H ) ; minimum IN NS ns.hadoop.local. ;host records
ns IN A 10.20.30.5
ip-10-20-30-6 IN A 10.20.30.6
ip-10-20-30-7 IN A 10.20.30.7
ip-10-20-30-8 IN A 10.20.30.8
ip-10-20-30-9 IN A 10.20.30.9
ip-10-20-30-10 IN A 10.20.30.10
ip-10-20-30-11 IN A 10.20.30.11
ip-10-20-30-12 IN A 10.20.30.12
ip-10-20-30-13 IN A 10.20.30.13
ip-10-20-30-14 IN A 10.20.30.14
;Alias for nameserver
ip-10-20-30-5 IN CNAME ns |
ÉÏÊöÇé¿öÒ²ÊÊÓÃÓÚ·´ÏòÓò30.20.10.in-addr.arpa¡£ËùÓÐ×ÓÍøÖпÉÄܵÄIPµØÖ·±»Ó³É仨 hadoop.localÓòϵÄÖ÷»úA¼Ç¼¡£
$TTL 5M @ IN SOA ns.hadoop.local. root.hadoop.local. ( 0 ; serial 1D ; refresh 1H ; retry 1W ; expire 3H ) ; minimum IN NS ns.hadoop.local. ;ptr records 5 IN PTR ip-10-20-30-5.hadoop.local. 6 IN PTR ip-10-20-30-6.hadoop.local. 7 IN PTR ip-10-20-30-7.hadoop.local. 8 IN PTR ip-10-20-30-8.hadoop.local. 9 IN PTR ip-10-20-30-9.hadoop.local. 10 IN PTR ip-10-20-30-10.hadoop.local. 11 IN PTR ip-10-20-30-11.hadoop.local. 12 IN PTR ip-10-20-30-12.hadoop.local. 13 IN PTR ip-10-20-30-13.hadoop.local. 14 IN PTR ip-10-20-30-14.hadoop.local. |
ÐÞ¸ÄVPCÒÔʹÓÃÒ»¸ö¶¨ÖÆ»¯ÓòÃûºÍDNS·þÎñÆ÷
ÉÏÆª²©ÎÄչʾÁËÈçºÎ½¨Á¢ºÍʹÓÃÒ»¸öVPC£¬ÏÂÒ»¸ö²½ÖèÔòÊÇͨ¹ýDHCPÑ¡ÏÐÞ¸ÄÓòÃûºÍDNS·þÎñÆ÷£¬²¢½¨Á¢Ò»¸ö½âÎöhadoop.localÓòµÄDNS·þÎñÆ÷¡£DHCPÑ¡Ï½«Ê¹ÓÃÒ»¸öλÓÚ×ÓÍøÄÚÊ׸ö¿ÉÓÃIPµØÖ·µÄ¶¨ÖÆ»¯DNS·þÎñÆ÷¡£×¢Ò⣺AWS»á±£´æÈκÎ×ÓÍøµÄǰËĸöºÍ×îºóÒ»¸öµØÖ·£¬Òò´ËÄãµÄ¿ÉÓõØÖ·Ó¦¸Ã´Ó.5¿ªÊ¼¡£
Ê×ÏÈ·¢²¼Ò»¸öAmazon EC2ʵÀý£¬²¢ÎªÆäÅäÖÃÒ»¸ö10.20.30.5µÄ˽ÓÐIPµØÖ·¡£ÇëÈ·±£ÒѾΪSSH·ÃÎÊÌîÈëÒ»¸ö¼üÖµ¶Ô¡£ÔÚÕâ¸öÀý×ÓÖУ¬ÎÒÃǽ«Õ¹Ê¾Ê¹ÓöíÀÕ¸ÔÖÝ×îеÄLinux
HVM EBS-backed AMI¡£Äã¿ÉÒÔͨ¹ýÒÔÏ´úÂëΪregion»ñµÃAMI ID£º
aws ec2 run-instances --image-id ami-dfc39aef --count 1 --key-name
--instance-type t2.micro --subnet-id subnet-907af9f5 --private-ip-address 10.20.30.5 --associate-public-ip-address |
ͨ¹ýÏÂÊö´úÂ뽨Á¢Ò»¸öÓòÃûΪ¡°hadoop.local¡±µÄDHCPÑ¡Ï£¬²¢ÇÒ½«DNS·þÎñÆ÷ÉèÖÃΪ¡°10.20.30.5,
AmazonProvidedDNS'¡±£º
$ aws ec2 create-dhcp-options --dhcp-configurations Key=domain-name,Values=hadoop.local Key=domain-name-servers,Values=10.20.30.5,AmazonProvidedDNS |
{ "DhcpOptions": { "DhcpConfigurations": [ { "Values": [ { "Value": "hadoop.local" } ], "Key": "domain-name" }, { "Values": [ { "Value": "10.20.30.5" }, { "Value": "AmazonProvidedDNS" } ], "Key": "domain-name-servers" } ], "DhcpOptionsId": "dopt-793fdf1c" } } |
ÏÖÔÚ£¬ÎªVPCÉèÖÃDHCPÑ¡ÏºÍVPC-idÖµ£¬Ö÷»úÃû³ÆÔÚ£º
{ "DhcpOptions": { "DhcpConfigurations": [ { "Values": [ { "Value": "hadoop.local" } ], "Key": "domain-name" }, { "Values": [ { "Value": "10.20.30.5" }, { "Value": "AmazonProvidedDNS" } ], "Key": "domain-name-servers" } ], "DhcpOptionsId": "dopt-793fdf1c" } } |
ÐÞ¸ÄʵÀýËùÔڵݲȫ×éÒÔÔÊÐí´ÓVPC CIDR ÖÐʵÀýÁ÷ÈëµÄDNS²éѯ¡£Èç¹ûÄãûÓÐΪrun-instancesÃüÁîÌṩһ¸ö×ÓÍø×éID£¬ÄÇôËü½«ÊÇVPCÖеÄĬÈϰ²È«×飺
aws ec2 authorize-security-group-ingress --group-id sg-08bf9b6d --protocol udp --port 53 --cidr 10.20.30.0/24 |
°²×°ºÍÅäÖÃDnsmasqÀ´½âÎöÓòÃû²éѯ
Èç¹ûÄãÔÚVPCÖÐûÓÐÔËÐÐÒ»¸öDNS·þÎñÆ÷£¬²¢ÇÒÏëÔÚÎÞ¸½¼Ó´ëÊ©µÄÇé¿öÏÂÔËÐÐÒ»¸öEMR¼¯Èº£¬Ò»¸öÀàËÆDnsmasqÕâÑùµÄ×é¼þ·Ç³£Êʺϡ£ÔÚ×ÊÔ´ÓÐÏÞµÄÇé¿öÏ£¬Dnsmasq¿ÉÒÔÌṩÃû³Æ»º´æ²¢×ª·¢·þÎñ¡£ÔÚ2.6.7°æ±¾ÖУ¬DnsmasqÒýÈëÁËsynth-domain¹¦ÄÜ£¬Ëü¿ÉÒÔΪÄãÑ¡ÔñµÄÈκÎIP/hostname·¶Î§Ìṩ¶¨ÖƵÄA/PTR¡£
--synth-domain=<domain>,<address range>[,<prefix>] |
ͨ¹ýÕâ¸öÌØÐÔ£¬Äã²»ÔÙÐèÒªµ¥¶ÀµØÖ¸¶¨£¬»òÕßÏÞÖÆµ½Ò»¸öÒѾ¶¨ÒåÁËÓò¼Ç¼µÄµ¥Ò»×ÓÍø£¬ËüÔÊÐíÄãÏìÓ¦ÈκοÉÄܵÄIPµØÖ·»òÕßhostname²éѯ¡£amazon-linux
yum repoÖ»ÌṩÁË2.4.8°æ±¾£¬Òò´ËΪÁËʹÓÃÕâ¸ö¹¦ÄÜ£¬Äã±ØÐëÔÊÐíÍⲿ°æ±¾¿â»òÕß´ÓÔ´´úÂë±àÒë¡£
ʹÓÃSSHÈ¥Á¬½Óн¨Á¢µÄ·þÎñÆ÷£¬²¢Í¨¹ýÒÔÏÂÃüÁîÀ´½¨Á¢¿ª·¢¹¤¾ß£¨°üÀ¨Ò»¸ö±àÒëÆ÷£©£º
[ec2-user ~]$ sudo yum ¨Cy groupinstall "Development Tools" |
½«ÏÂÊöÃüÁî¸´ÖÆºÍÕ³Ìùµ½ÖÕ¶ËÒÔÏÂÔØÒ»¸ö×îеÄDnsmasqÔ´´úÂ룬untar¡¢build²¢link»ØÆÚÍûϵͳµÄsbin·¾¶¡£
wget <a href="http://www.thekelleys.org.uk/dnsmasq/dnsmasq-2.72.tar.gz">
http://www.thekelleys.org.uk/dnsmasq/dnsmasq-2.72.tar.gz</a>gunzip dnsmasq-*.gz tar -xf dnsmasq-*.tar cd dnsmasq-2.72 sudo make install sudo ln -s /usr/local/sbin/dnsmasq /usr/sbin/dnsmasq |
ʹÓÃÃüÁîÐÐÔÚ¼àÌýÆ÷¶Ë¿ÚÉÏ¿ªÆôÒ»¸öDNS·þÎñÆ÷£¬Õâ¸öÃüÁîÌṩÁËÕû¸öVPC CIDR·¶Î§µÄ½âÎö¡£ÄãµÄVPC
CIDR·¶Î§ÊÇ10.20.30.0/24£¨Õâ¸öÔÚµÚһƪ²©ÎÄÖлñµÃ£©¡£·þÎñÆ÷²ÎÊý»áת·¢ÄÇЩ²»¿ÉÒÔ±»Ä¬ÈÏAWS
DNS·þÎñÆ÷½âÎöµÄ²éѯ£¬Ïê¼û DHCP Options SetsÒ»ÎÄ¡£
sudo dnsmasq --interface=eth0 --listen-address=127.0.0.1 --synth-domain=hadoop.local,10.20.30.0/24,ip- --server=10.20.30.2 |
ÔÚVPCÖÐÒ»¸ö¶ÀÁ¢µÄʵÀýÉϲâÊÔµ½Õâ¸ö·þÎñÆ÷ÉϵIJéѯ½«µÃµ½ÕýÈ·µÄ·µ»Ø¡£Ê¹ÓÃip-hostnameÖµ¶ÔÐ鹹ʵÀý·¢²¼Ò»¸önslookup²éѯ¡£
ÏÖÔÚ£¬¶ÔÕâ¸öµØÖ··¢²¼ÏàͬµÄ²éÕÒ£º
[ec2-user@ip-10-20-30-6 ~]$ nslookup ip-10-20-30-40.hadoop.local 10.20.30.5 Server: 10.20.30.5 Address: 10.20.30.5#53
Name: ip-10-20-30-40.hadoop.local
Address: 10.20.30.40 |
[ec2-user@ip-10-20-30-6 ~]$ nslookup 10.20.30.40 10.20.30.5 Server: 10.20.30.5 Address: 10.20.30.5#53
40.30.20.10.in-addr.arpa name = ip-10-20-30-40.hadoop.local |
Á½¸ö¶¼¿ÉÒÔÕý³£½âÎö£»ÏÖÔÚÐèÒª²âÊÔµÄÖ»ÓÐÒ»¸öÊÂÇéÁË¡£±£Ö¤Äã¿ÉÒÔΪs3.amazonaws.com½âÎöÒ»¸öIPµØÖ·£»¶ÔÓÚ½âÎö¹«¹²DNSĿ¼£¬ÕâÊÇÒ»¸ö²»´íµÄ²âÊÔ£¬ÌرðÊÇ·þÎñÆ÷¶Ëµã¡£
[ec2-user@ip-10-20-30-6 ~]$ nslookup s3.amazonaws.com 10.20.30.5 Server: 10.20.30.5 Address: 10.20.30.5#53 Non-authoritative answer: s3.amazonaws.com canonical name = s3.a-geo.amazonaws.com. s3.a-geo.amazonaws.com canonical name = s3-2.amazonaws.com. Name: s3-2.amazonaws.com Address: 54.231.244.4 |
µ±ÏÂÒ»ÇоÍÐ÷£¬¿ªÆôÁíÒ»¸öwordcount×÷Òµ²âÊÔ¡£
aws emr create-cluster --bootstrap-actions Path= s3://support.elasticmapreduce/bootstrap-actions/other/sethostforvpc.sh --steps Type=STREAMING,Name='Streaming Program',ActionOnFailure=CONTINUE,Args=[-files,
s3://elasticmapreduce/samples/wordcount/wordSplitter.py,-mapper,wordSplitter.py,-reducer,aggregate,
-input,s3://elasticmapreduce/samples/wordcount/input,-output,
s3:///wordcount/output/] --ami-version 3.6.0 --instance-groups InstanceGroupType=MASTER,
InstanceCount=1,InstanceType=m3.xlarge InstanceGroupType=CORE,
InstanceCount=2,InstanceType=m3.xlarge --auto-terminate |
Èç¹ûÒ»ÇÐÉèÖÃÕýÈ·£¬Äã¾Í¿ÉÒÔÔÚÊý·ÖÖÓÄڳɹ¦µÄÓµÓÐÒ»¸ö¼¯Èº¡£Èç¹ûÄãµÄÖÕÖ¹ÔÒòÊÇ¡°Steps completed¡±£¬Ò»Çж¼»áÈçÔ¤ÆÚ°ãµÄ¹¤×÷¡£
aws emr describe-cluster --cluster-id --query Cluster.Status.StateChangeReason.Message "Steps completed" |
Èç¹ûÄã¼Æ»®Ê¹ÓÃRoute 53À´½âÎöÖ÷»ú¼Ç¼£¬ÔÚÕâÀïÄã¿ÉÒԳɹ¦µÄÖÕÖ¹Ò»¸öDNSmasq£»Äã²»ÔÙÐèÒªÈÎºÎÆäËû²Ù×÷¡£
aws ec2 terminate-instances --instance-ids i-22d6792f |
ÅäÖÃRoute 53µ½ÄãµÄ˽ÓÐÓò
±¾ÏµÁеÄ×îºóÒ»¸ö½Ì³ÌÊÇʹÓÃRoute 53˽ÓÐDNSÓòÀ´Ö§³ÅÕý·½ÏòÓò¼Ç¼¡£Èç֮ǰËùÊö£¬ÔÚÒÑ´æDNS·þÎñÆ÷ÖлᷢÉúÀàËÆµÄÎÊÌ⣻ÔÚδʹÓÃDHCP·þÎñÆ÷Çé¿öÏ£¬ÎÒÃÇÎÞ·¨Ìî³ähost/address¼Ç¼¡£ÎªÁËÈÃʾÀý¸ü¼Ó¼òµ¥£¬ÎÒÃÇʹÓÃÁËÒ»¸öСµÄ×ÓÍø£¬²¢¶¨ÒåÈκοÉÄܵĵØÖ·¡£Í¬Ê±£¬ÎÒÃÇʹÓÃÁËʾÀýÖнéÉܵÄBINDÓò£¬²¢ÈÃÊäÈëÖ±½ÓͨÍùRoute
53¡£
Route 53ÐèÇó¿ªÆôDns Hostnames²ÎÊý£¬Èç¹ûÔÚ֮ǰûÓпªÆô£¬Äã¿ÉÒÔͨ¹ýÏÂÊöÃüÁîÍê³É£º
aws ec2 modify-vpc-attribute --vpc-id vpc-055ef660 --enable-dns-hostnames |
½¨Á¢Á½¸öÓò£¬·Ö±ðÓµÓÐÕý·´Ïò²éÕÒ£¬Í¬Ê±È·±£ÎªÃ¿¸öÇëÇóÖ§³Öunique-caller-reference
Öµ¡£
aws route53 create-hosted-zone --name hadoop.local --vpc VPCRegion=us-west-2,
VPCId=vpc-055ef660 --caller-reference Support01 |
¼ÇÏ·µ»ØµÄÖ÷»úÓòID¡£
"Id": "/hostedzone/Z34WMQP8SCULM7 " |
ͨ¹ýÒÔÏ´úÂëÁ¬½ÓRoute 53˽ÓÐDNSÓòºÍVPC¡£
aws route53 associate-vpc-with-hosted-zone
--hosted-zone-id /hostedzone/Z34WMQP8SCULM7 --vpc VPCRegion=us-west-2,VPCId=vpc-055ef660 |
ÔÚ·´ÏòÓòÖÐÖ´ÐÐÏàͬµÄ²Ù×÷¡£
aws route53 associate-vpc-with-hosted-zone
--hosted-zone-id /hostedzone/Z1KTAQQMOLRO1Z --vpc VPCRegion=us-west-2,VPCId=vpc-055ef660 |
ÏÖÔÚ£¬´ÓÉÏÊöʾÀýBIND·þÎñÆ÷Öе¼ÈëÓò¼Ç¼¡£ÔÚRoute 53¿ØÖÆÌ¨£¬Ñ¡Ôñhadoop.localÓò£¬Ñ¡Ôñ
Go to Record Sets¡¢Import Zone File£¬ËæºóÕ³ÌùÓòÎļþÄÚÈݵ½Zone File¡£Ñ¡ÔñÆÁĻϷ½µÄImport¡£ÔÚ30.20.10.in-addr.arpaÓòÖÐÖ´ÐÐÏàͬµÄ²Ù×÷¡£
½¨Á¢Ò»¸öеÄDHCPÑ¡Ï£¬²¢½«DNS·þÎñÆ÷hadoop.localÖ¸ÏòAmazonProvidedDNS£¬ËæºóÒÆ³ýDnsmasqʾÀýʹÓõÄDNS·þÎñÆ÷¡£
aws ec2 create-dhcp-options --dhcp-configurations Key=domain-name,Values=hadoop.local Key=domain-name-servers,Values=AmazonProvidedDNS |
¼ÇÏÂÐÂDHCPÑ¡ÏID¡£
{ "DhcpOptions": { "DhcpConfigurations": [ { "Values": [ { "Value": "hadoop.local" } ], "Key": "domain-name" }, { "Values": [ { "Value": "AmazonProvidedDNS" } ], "Key": "domain-name-servers" } ], "DhcpOptionsId": "dopt-936484f6" } } |
Á¬½ÓÐÂÑ¡ÏºÍVPC¡£
aws ec2 associate-dhcp-options --dhcp-options-id dopt-936484f6 --vpc-id vpc-055ef660 |
Ëæºó·¢²¼×îÖÕ¼¯Èº£¬²¢ÑéÖ¤Ëù×öµÄ¹¤×÷¡£
È·±£ÄãÒѾʹÓÃÄã×Ô¼ºµÄÖµÌæ»»ÁËʾÀýÖÐʹÓõÄÖµ£¬²¢ÇÒʹÓÃÁËÄã×Ô¼ºµÄÊä³öµØÖ·£¬·ñÔò²Ù×÷½«Ê§°Ü¡£
aws emr create-cluster --bootstrap-actions Path= s3://support.elasticmapreduce/bootstrap-actions/other/sethostforvpc.sh --steps Type=STREAMING,Name='Streaming Program',ActionOnFailure=CONTINUE,Args=[-files,
s3://elasticmapreduce/samples/wordcount/wordSplitter.py,-mapper,wordSplitter.py,
-reducer,aggregate,-input,s3://elasticmapreduce/samples/wordcount/input,-output,
s3:///wordcount/output/] --ami-version 3.6.0 --instance-groups InstanceGroupType=MASTER,
InstanceCount=1,InstanceType=m3.xlarge InstanceGroupType=CORE,InstanceCount=2,
InstanceType=m3.xlarge --auto-terminate |
µÈ´ý¼¸·ÖÖÓ£¬²¢¼ì²é×îÖÕ״̬µÄ±ä»¯ÔÒò£º
aws emr describe-cluster --cluster-id j-2TEFHMDR3LXWD --query Cluster.Status.StateChangeReason.Message "Steps completed" |
×¢Ò⣺Èç¹ûÄãÆÚÍû±ÜÃâËùÓÐDNS¹ÜÀí²Ù×÷£¬Äã¿ÉÒÔÀûÓà bootstrap actions½«¼¯ÈºÖ÷»úÃû³Æ¸Ä»ØÄ¬ÈÏÖµ£¬²¢ÔÊÐíAWS·þÎñ¹ÜÀíËùÓÐÊÂÇé¡£½«ÏÂÃæµÄ¶¨ÖÆbootstrap
action¼ÓÈ뼯Ⱥ£º
s3://support.elasticmapreduce/bootstrap-actions/other/sethostforvpc.sh |
|