KubernetesÊÇÔÉúµÄÈÝÆ÷±àÅŹÜÀíϵͳ£¬¶ÔÓÚ¸ºÔؾùºâ¡¢·þÎñ·¢ÏÖ¡¢¸ß¿ÉÓᢹö¶¯Éý¼¶¡¢×Ô¶¯ÉìËõµÈÈÝÆ÷ÔÆÆ½Ì¨µÄ¹¦ÄÜÒªÇóÓÐÔÉúÖ§³Ö¡£½ñÌìÎÒ·ÖÏíÒ»ÏÂÎÒÃÇÔÚKubernetes¼¯ÈºÖÐÈÕÖ¾¹ÜÀíµÄʵ¼ù·½°¸¡£ÔÚÕâ¸ö·½°¸ÖУ¬³ýÁËDockerºÍKubernetes£¬Ö÷Òª»¹Éæ¼°µÄ¼¼Êõ°üÀ¨£ºFluentd¡¢Elasticsearch¡¢KibanaºÍSwift¡£

Fig00-KubernetesÈÕ־ϵͳÖÐÉæ¼°µÄ¼¼Êõ
ÆÀ¹ÀÈÝÆ÷ÔÆÆ½Ì¨ÈÕ־ϵͳµÄ±ê×¼£º
Ò×À©Õ¹£ºÄܹ»Ö§³Å¼¯Èº¹æÄ£µÄÔö³¤
¿ªÏúµÍ£º¾¡Á¿Õ¼ÓýÏÉÙµÄϵͳ×ÊÔ´
ÈëÇÖС£º¾¡Á¿²»ÐèÒª¸Ä¶¯Ó¦ÓÃÈÝÆ÷ºÍÔÆÆ½Ì¨ÏµÍ³
´ó¼¯ÖУº½«ËùÓзֲ¼ÔÚ¸÷¸öÖ÷»ú½ÚµãÉϵÄÈÕÖ¾¼¯ÖÐÔÚÒ»Æð·ÖÎöºÍ²éѯ
Òײ¿Ê𣺷½±ã×Ô¶¯»¯²¿Êðµ½·Ö²¼Ê½¼¯ÈºÖÐ
Ò×¶¨ÖÆ£º·½±ã´¦Àí²»Í¬ÈÕÖ¾¸ñʽ£¬·½±ã¶Ô½Ó²»Í¬µÄ´æ´¢·½Ê½
ʵЧÐÔ£ºÈÕÖ¾ÔÚ²úÉúÖ®ºóÐèÒªÄÜÔÚ¶Ìʱ¼äÄÚ¼´¿ÉÒÔ½øÐв鿴·ÖÎö
ÉçÇø»îÔ¾£º·½±ãδÀ´µÄά»¤ºÍ¸üУ¬·½±ã¹¦ÄÜÀ©Õ¹
FluentdµÄ½éÉÜ
FluentdÊÇÒ»¸öʵʱÈÕÖ¾ÊÕ¼¯ÏµÍ³£¬Ëü°ÑJSON×÷ΪÈÕÖ¾µÄÖм䴦Àí¸ñʽ£¬Í¨¹ýÁé»îµÄ²å¼þ»úÖÆ£¬¿ÉÒÔÖ§³Ö·á¸»¶àÑùµÄÈÕÖ¾ÊäÈëÓ¦Óá¢Êä³öÓ¦Óá¢ÒÔ¼°¶àÖÖÈÕÖ¾½âÎö¡¢»º´æ¡¢¹ýÂ˺͸ñʽ»¯Êä³ö»úÖÆ¡£
FluentdµÄ¼Ü¹¹
Fluentd½«JSON×÷ΪÊý¾Ý´¦ÀíµÄÖмä¸ñʽ£¬Í¨¹ý²å¼þʽµÄ¼Ü¹¹¿ÉÀ©Õ¹µØÖ§³Ö²»Í¬ÖÖÓ¦Óûòϵͳ×÷ΪÈÕÖ¾Ô´ºÍÈÕÖ¾Êä³ö¶Ë¡£¼ÙÉèÓÐMÖÖÊäÈëÔ´Wordpress¡¢MySQL¡¢Tomcat¡¡£»NÖÖÊä³ö¶ËMySQL¡¢MongoDB¡¢ElasticSearch¡¡ÄÇô´¦ÀíÈÕÖ¾µÄ´úÂëÄ£¿éÓÉMxN¼õÉÙΪM+N¡£ÎÒÃÇÔÚKubernetes¼¯ÈºÖÐÊÕ¼¯ÈÕÖ¾Ö÷ÒªÓõ½https://hub.docker.com/r/fabric8/fluentd-kubernetes/Õâ¸ö¾µÏñºÍhttps://github.com/fabric8io/docker-fluentd-kubernetesÕâ¸öPlugins¡£

Fig01-Fluend¼Ü¹¹

Fig02-Fluentd¹¦ÄÜ
FluentdµÄÌØµã£º
½«JSON×÷ΪͳһµÄÖмä²ãÈÕÖ¾¸ñʽ×öÈÕÖ¾´¦Àí
»ùÓÚRubyµÄÈÕÖ¾ÊÕ¼¯¹¤¾ß£º½Ï»ùÓÚJRubyµÄLogstashµÄFootprintС
¼æÈݵÄÊäÈëÔ´Êä³ö¶Ë»ù±¾ºÍLogstashÏ൱
ÐÔÄÜÏà¹ØµÄ²¿·ÖΪC´úÂ룺ËٶȽϿì
Ö§³Ö²å¼þÀ©Õ¹£ºInput¡¢Parser¡¢Filter¡¢Output¡¢Formatter
and Buffer
FluentdÔÚKubernetes¼¯ÈºÖеIJ¿Êð¼Ü¹¹
ÿ¸öNode½ÚµãÉ϶¼ÒªÓÐFluentd-ElasticsearchÕâ¸öPod£¬ÓÐÁ½ÖÖ·½Ê½Ö§³Ö£º1.
·ÅÔÚ/etc/kubernetes/manifestÏ£¬Óýű¾×Ô¶¯Æô¶¯£»2. ÓÃDaemonSetÆô¶¯¡£ÕâÁ½ÖÖģʽ¶¼ÊÇΪÁ˱£Ö¤ÔÚÿһ¸öKubernetes¼¯Èº½ÚµãÉ϶¼ÓÐÒ»¸öFluentdµÄפÁôPodÔËÐÐÀ´ÊÕ¼¯ÈÕÖ¾¡£KubernetesÖÐDaemonSetÕâ¸öAPI¶ÔÏó¾ÍÊÇΪÁËפÁôPod¶øÉè¼ÆµÄ¡£

Fig03-FluentdÔÚKubernetes¼¯ÈºÖеIJ¿Êð¼Ü¹¹
ÔÚKubenetesÖеIJ¿Êð°¸ÀýYAML
ÔÚKubernetes¼¯ÈºÖв¿ÊðFluentdʱ£¬¿ÉÒÔ²ÉÓÃÀàËÆÏÂÃæµÄYAMLÎļþ£¬½«Ê¹ÓÃDocker¾µÏñFluentd-ElasticsearchµÄPod²¿Êðµ½Ã¿Ò»¸öKubernetes½ÚµãÉÏ¡£

Fig04-FluentdÔÚKubernetes¼¯ÈºÖеIJ¿ÊðYAML
Fluentd podµÄÔËÐÐʱ״̬£º

Fig05-FluentdÔÚKubernetes¼¯ÈºÖеÄÔËÐÐ״̬
Ñ¡ÓÃFluentdµÄÀíÓÉ£º
¿ªÏúµÍ£ººËÐÄ´úÂëΪC£¬²å¼þ´úÂëΪRuby£¬²»ÐèÒª´ò°üJDK
ÈëÇÖС£ºÓÃPod²¿Ê𣬲»¸ÉÈÅÓ¦ÓÃÈÝÆ÷ºÍÖ÷»ú·þÎñ
Òײ¿Êð£ºÊ¹ÓÃÈÝÆ÷¾µÏñ×÷Ϊµ¥ÈÝÆ÷Pod²¿Êð
Ò×¶¨ÖÆ£ººÜ·½±ãÔö¼ÓºÍ¸ü¸ÄÊʺÏ×Ô¼ºÓ¦ÓõIJå¼þ
ElasticSearch
ElasticsearchÊÇÒ»¸öʵʱµÄ·Ö²¼Ê½ËÑË÷ºÍ·ÖÎöÒýÇæ¡£Ëü¿ÉÒÔÓÃÓÚÎĵµ´æ´¢£¬È«ÎÄËÑË÷£¬½á¹¹»¯ËÑË÷ÒÔ¼°ÊµÊ±·ÖÎö£¬Óë³£¼ûµÄ»¥ÁªÍøÓ¦ÓÃ×îµäÐ͵ÄÓ¦Óó¡¾°ÊÇÈÕÖ¾·ÖÎö²éѯºÍÈ«ÎÄËÑË÷¡£
ElasticSearchµÄ¼Ü¹¹
ÔÚElasticSearchÖÐÓÐÈýÀà½Úµã£ºµÚÒ»ÀàÊÇData Node£¬ÓÃÀ´´æ´¢Êý¾Ý£¬ElasticSearchÖжÔÓÚÒ»·ÝÊý¾Ý¿ÉÒÔÓжà¸ö¸±±¾£¬ÒÔÌṩÊý¾Ý¸ß¿ÉÓÃÄÜÁ¦£»µÚ¶þÀàÊÇClient
Node£¬»òÕ߽вéѯ½Úµã£¬Ìṩ¶Ô²éѯµÄ¸ºÔؾùºâ£»µÚÈýÀàÊÇMaster Eligible node£¬»òÕß½ÐË÷Òý½Úµã£¬ÕâЩ½Úµã¿ÉÒÔ±»Ñ¡¾ÙΪMaster
Node£¬¶øMaster Node»á¿ØÖÆÕû¸öElasticSearch¼¯ÈºµÄ״̬¡£

Fig06-ElasticsearchµÄ¼Ü¹¹
ElasticSearchÔÚKubernetesÖеIJ¿Êð¼Ü¹¹
ÎÒÃÇÔÚKubernetesÖУ¬ÈýÀà½Úµã¶¼ÊÇÒ»¸ö°üº¬Í¬Ò»¸ö¾µÏñPod
elasticsearch-cloud-kubernetes£¬Çø±ðÖ»ÊÇÆô¶¯Ê±µÄ»·¾³role²»Ò»Ñù¡£²éѯºÍË÷Òý½ÚµãÐèÒªÌṩ¶ÔÍâµÄWeb·þÎñ£¬ÐèÒª·¢²¼ÎªÒ»¸öKubernetes
Service¡£Êý¾Ý½ÚµãÐèÒª°ó¶¨Ò»¸ö³Ö¾Ã»¯´æ´¢£¬ÎÒÃÇÓÃKubernetes PV´´½¨³ö´æ´¢¾í£¬Êý¾Ý½ÚµãÉÏÃæÍ¨¹ýKubernetes
PVC°ó¶¨µ½ÏàÓ¦µÄÊý¾Ý¾í¡£PVµÄʵ¼Ê´æ´¢¿ÉÒÔÊÇNFS¡¢GlusterFSµÈ¹²Ïí´æ´¢¡£

Fig07-ElasticsearchµÄ¼Ü¹¹
ElasticSearchÔÚKubernetesÖеIJ¿Êð¼Ü¹¹
ÎÒÃÇÔÚKubernetesÖУ¬ÈýÀà½Úµã¶¼ÊÇÒ»¸ö°üº¬Í¬Ò»¸ö¾µÏñPod
elasticsearch-cloud-kubernetes£¬Çø±ðÖ»ÊÇÆô¶¯Ê±µÄ»·¾³role²»Ò»Ñù¡£²éѯºÍË÷Òý½ÚµãÐèÒªÌṩ¶ÔÍâµÄWeb·þÎñ£¬ÐèÒª·¢²¼ÎªÒ»¸öKubernetes
Service¡£Êý¾Ý½ÚµãÐèÒª°ó¶¨Ò»¸ö³Ö¾Ã»¯´æ´¢£¬ÎÒÃÇÓÃKubernetes PV´´½¨³ö´æ´¢¾í£¬Êý¾Ý½ÚµãÉÏÃæÍ¨¹ýKubernetes
PVC°ó¶¨µ½ÏàÓ¦µÄÊý¾Ý¾í¡£PVµÄʵ¼Ê´æ´¢¿ÉÒÔÊÇNFS¡¢GlusterFSµÈ¹²Ïí´æ´¢¡£

Fig08-ElasticSearchÓ봫ͳÊý¾Ý¿âµÄ¶Ô±È
ElasticSearchÔÚKubernetes¼¯ÈºÖеIJ¿Êð
ÔÚKubernetes¼¯ÈºÖв¿ÊðElasticSearch£¬ÎÒÃǻᲿÊðÀàËÆÍ¼ÖеÄ3Öֽڵ㣺es-dataÀàÊÇÓÃÀ´´æ·ÅË÷ÒýÊý¾ÝµÄ£»es-masterÀàÊÇÓÃÀ´ÌṩË÷Òýд·þÎñµÄ£»es-clientÊÇÓÃÀ´ÌṩË÷Òý²éѯ·þÎñµÄ¡£

Fig09-ElasticSearchÔÚKubernetes¼¯ÈºÖеIJ¿Êð
ElasticSearchÊý¾ÝÔÚKubernetes¼¯ÈºÖеij־û¯´æ´¢
ÔÚ²¿Êðes-data½ÚµãµÄʱºò£¬ËûÃǵÄÊý¾Ý¾íÎÒÃÇÊÇÒÔ¹²Ïí´æ´¢¾í¹ÒÔØµÄ£¬ÕâÀï²ÉÓÃPVC/PVµÄģʽ¹ÒÔØÒ»¸öNFSµÄPVÌṩÊý¾Ý¾í£¬ÈçÏÂͼËùʾ¡£

Fig10-ElasticSearchÊý¾ÝÔÚKubernetes¼¯ÈºÖеij־û¯´æ´¢
ÈÕÖ¾µÄ±¸·ÝºÍ»Ö¸´
ElasticSearchÔÊÐí¶ÔÓÚµ¥¸öË÷Òý»òÕû¸ö¼¯Èº×ö±¸·ÝºÍ»Ö¸´¡£±¸·Ý»Ö¸´ËùÖ§³ÖµÄÄ¿±ê´æ´¢²Ö¿âÀàÐͰüÀ¨£º
S3²Ö¿â£º½«AWS S3×÷Ϊ±¸·Ý²Ö¿â
°²×°ÃüÁ
sudo bin/elasticsearch-plugin install repository-s3 |
´´½¨²Ö¿âʾÀý£º
PUT _snapshot/my-s3-repository-1
{
"type": "s3",
"settings": {
"bucket": "s3_repository_1",
"region": "us-west"
}
} |
Azure²Ö¿â£º½«Azure×÷Ϊ±¸·Ý²Ö¿â
°²×°ÃüÁ
sudo bin/elasticsearch-plugin
install repository-azure |
´´½¨²Ö¿âʾÀý£º
PUT _snapshot/my-azure-repository-1
{
"type": "azure",
"settings": { "container":
"backup-container", "base_path":
"backups", "chunk_size":
"32m", "compress": true
}
} |
HDFS²Ö¿â£º½«HDFS×÷Ϊ±¸·Ý²Ö¿â
°²×°ÃüÁ
sudo bin/elasticsearch-plugin
install repository-hdfs |
´´½¨²Ö¿âʾÀý£º
PUT _snapshot/my-hdfs-repository-1
{
"type": "hdfs",
"settings": {
"uri": "hdfs://namenode:8020/",
"path": "elasticsearch/respositories/my_hdfs_repository",
"conf.dfs.client.read.shortcircuit":
"true"
}
} |
GCS²Ö¿â£º½«Google Cloud Storage×÷Ϊ±¸·Ý²Ö¿â
°²×°ÃüÁ
sudo bin/elasticsearch-plugin
install repository-gcs |
´´½¨²Ö¿âʾÀý£º
PUT _snapshot/my-gcs-repository-1
{
"type": "gcs",
"settings": {
"bucket": "my_bucket",
"service_account": "_default_"
}
} |
×÷Ϊ˽ÓÐÔÆ²¿ÊðµÄ»·¾³£¬¶àÊý»ùÓÚOpenStackµÄIaaS²ã£¬¿ÉÒÔ²ÉÓÃOpenStackµÄ¶ÔÏó´æ´¢Swift×÷Ϊ±¸·Ý¡£
Swift²Ö¿â£º½«OpenStack Swift×÷Ϊ±¸·Ý²Ö¿â
°²×°ÃüÁ
sudo bin/elasticsearch-plugin
install org.
wikimedia.elasticsearch.swift/swift-repository-plugin/2.1.1 |
´´½¨²Ö¿âʾÀý£º
sudo bin/elasticsearch-plugin
install org.
wikimedia.elasticsearch.swift/swift-repository-plugin/2.1.1
´´½¨²Ö¿âʾÀý£º
PUT _snapshot/my-gcs-repository-1
{
"type": "swift",
"settings": { "swift_url":
"http://localhost:8080/auth/v1.0/",
"swift_container": "my-container",
"swift_username": "myuser",
"swift_password": "mypass!"
}
} |
Ñ¡ÓÃElasticSearchµÄÀíÓÉ£º
Ò×À©Õ¹£º¿ÉÒÔËæ×ÅÔö¼Ó½ÚµãˮƽÀ©Õ¹´æ´¢ÈÝÁ¿ºÍË÷ÒýÄÜÁ¦
´ó¼¯ÖУº½«ËùÓÐPodºÍÈÝÆ÷µÄÊý¾Ý¼¯ÖÐÔÚÒ»Æð·½±ã²éѯºÍ·ÖÎö
Òײ¿Êð£ºÊ¹ÓÃÈÝÆ÷¾µÏñ×÷Ϊµ¥ÈÝÆ÷Pod²¿Êð
Ò×¶¨ÖÆ£ººÜ·½±ãÔö¼ÓºÍ¸ü¸ÄÊʺÏ×Ô¼ºÓ¦ÓõIJå¼þ
ʵЧÐÔ£ºÈÕÖ¾ÔÚ²úÉúÖ®ºóÐèÒªÄÜÔÚ¶Ìʱ¼äÄÚ¼´¿ÉÒÔ½øÐв鿴·ÖÎö
ÉçÇø»îÔ¾£ºElasticSearchÉçÇøÔ½À´Ô½»îÔ¾£¬´óÓиϳ¬SolrÖ®ÊÆ
Kibana
KibanaÔÚKubernetesÖеIJ¿Êð
Kibana¸úElasticSearchµÄ¼¯³ÉÏà¶ÔÀ´Ëµ±È½ÏÖ±¹Û£¬ÀûÓÃhttps://hub.docker.com/r/fabric8/kibana4/¾µÏñ£¬ÉèÖúÃELASTICSEARCH_URL²ÎÊý¾Í¿ÉÒÔ£¬KibanaÊÇÒ»¸ö²¿ÊðÔÚKubernetes¼¯ÈºÖеÄWebǰ¶Ë·þÎñ£¬¶øËüÒýÓÃELASTICSEARCH_URLÕâ¸ö»·¾³±äÁ¿×÷Ϊ×ÊԴʹÓá£

ÕûÌåÈÕÖ¾¹ÜÀíϵͳµÄ¼Ü¹¹
ÔÚKubernetes¼¯ÈºÖеÄÿ¸ö½ÚµãÉÏÔËÐÐÒ»¸öFluentdµÄÈÝÆ÷£¬ÊÕ¼¯ÈÝÆ÷µÄÈÕÖ¾·¢Ë͸øµ½ElasticSearch¼¯ÈºÖС£ElasticSearch¼¯Èº»á±£´æÒ»ÖܵÄÈÕÖ¾×÷ΪÈÈÊý¾ÝÒÔ¹©ÊµÊ±·ÖÎöºÍ²éѯ£¬Óû§¿ÉÒÔͨ¹ýKibana²é¿´ÈÎÒâNode¡¢Namespace¡¢Service¡¢PodºÍÈÝÆ÷µÄÊý¾Ý¡£¶ÔÓÚ³¬¹ýÒ»ÖܵÄÈÕÖ¾Êý¾Ý£¬ElasticSearch»á×Ô¶¯±¸·Ýµ½Swift¶ÔÏó´æ´¢µÄÏàÓ¦BucketÖС£

Fig12-ÕûÌåKubernetesÈÕÖ¾¹ÜÀíϵͳµÄ¼Ü¹¹
Q&A
Q£ºÇëÎÊKubernetesËÞÖ÷»úµÄÈÕÖ¾ÊÇÈçºÎÊÕ¼¯µÄ£¿
A£º¸úÊÕ¼¯ÈÝÆ÷µÄÈÕÖ¾ÊÇÀàËÆµÄ£¬ÊÂʵÉÏÈÝÆ÷µÄÈÕÖ¾Ò²ÊÇ´ÓÖ÷»úµÄÈÕ־Ŀ¼ÊÕ¼¯¹ýÀ´µÄ¡£
Q£ºÈç¹û°ÑÒÆ¶¯É豸µÄÕû»úÈÕÖ¾ÊäÈëϵͳ£¬ÓÈÆäÊÇÒÆ¶¯É豸ÐèҪעÒâÄÄЩÎÊÌ⣿²úÉúÈÕ־ĿǰÏëµ½ÓÐÁ½ÖÖ·½°¸£º£¨1£©Ê¹ÓÃAPPת·¢¸øFluentd£¨2£©Ê¹ÓÃSyslog£¬¸öÈ˸оõµÚ¶þ¸ö¸üºÏÊÊ£¬»òÕß»¹ÓÐÆäËû¸üºÃµÄ·½°¸Ã´£¿
A£º±§Ç¸£¬ÎÒÃDZȽϹØ×¢µÄÊÇÔÆÆ½Ì¨·þÎñÆ÷¶ËµÄÈÕÖ¾£¬Òƶ¯É豸µÄÈÕ־ûÓÐÑо¿¹ý¡£Èç¹ûÊÇÒÆ¶¯É豸ÈÕ־ͨ¹ý·þÎñÆ÷µÄAPIÉÏ´«µ½·þÎñÆ÷ÁË£¬ÄÇô¾ÍÊÇÒ»ÑùµÄ´¦Àí¡£Ò»°ãÎÒÃǵÄÀí½â£¬Òƶ¯É豸µÄÈÕÖ¾ÊÇͨ¹ýÓ¦ÓÃ×Ô¼ºµÄһЩÈÕÖ¾³ÌÐò£¬¶¨ÆÚѹËõ·¢Ë͵½·þÎñÆ÷»òÕßµÚÈý·½µÄÈÕÖ¾ÊÖ»úƽ̨£¬È»ºóÔÚ·þÎñÆ÷¶Ë£¬µ±×÷ÆÕͨµÄ·þÎñÆ÷Ó¦ÓÃÈÕÖ¾À´´¦Àí£¬Ö»²»¹ýÒª´òÉÏÒÆ¶¯É豸ºÍÓû§µÄÏà¹ØTag¡£
Q£ºElasticSearchÊÇ¿ÉÒÔÉèÖñ¸·ÝÖÜÆÚµÄʱ¼ä°É£¿Èç¹ûÎÒÏë±£ÁôÒ»¸öÔµÄÈÕÖ¾À´½øÐвéѯ£¿
A£º¿ÉÒÔÉèÖᣵ«ÊÇҪͨ¹ý×Ô¼ºµÄ½Å±¾»òÕßcrontabÈÎÎñÀ´ÊµÏÖ¡£ESĿǰÖ÷ÒªÌṩµÄÊÇͨ¹ýREST
API´´½¨¡¢É¾³ý±¸·ÝºÍͨ¹ý±£ÁôµÄ±¸·Ý»Ö¸´Ä³¸ö¼¯Èº¡£
Q£ºFluentd¿É·ñÉèÖÃÊÕ¼¯ÈÝÆ÷Ó¦ÓÃÖ¸¶¨Ä¿Â¼ÈÕÖ¾£¨·Ç±ê×¼Êä³öĿ¼£©£¬ÔõôÉèÖã¿
A£ºÈÝÆ÷Ó¦ÓÃĿ¼ÔÚÈÝÆ÷ÄÚ£¬FluentdÊÇÊÕ¼¯²»µ½µÄ£¬³ý·ÇÄãµÄÊä³öĿ¼ҲÊÇÍⲿ¹ÒÔØµÄµÄ¹²ÏíĿ¼¡£Èç¹ûÊÇÒ»¸öµ¥´¿Docker
Engine¹ÜÀíµÄ½Úµã£¬¿ÉÒÔͨ¹ý¨Cvolumn-from·ÃÎÊÁíÒ»¸öÈÝÆ÷´æ´¢µÄÊý¾Ý£¬µ±È»ÕâÒ²ÊÇÔÚÄǸöÈÝÆ÷-vÉùÃ÷µÄvolumn¶ø²»ÊÇÈÎÒâĿ¼£»ÕâÖÖ·½Ê½¶ÔKubernetes¼¯ÈºÃ»Ê²Ã´°ïÖú¡£
Q£ºElasticSearchÈÕÖ¾µÄ±£Áô²ßÂÔ£¬ ÔõôÉèÖÃÄØ£¬Êǵ÷APIɾ³ý»¹ÊÇElasticSearch×Ô´øÄØ£¿
A£ºÎÒÃÇÏÖÔÚÓõķ½Ê½ÊÇÓÃʱ¼ä×öË÷Òý£¬È»ºó½Å±¾¶¨Ê±É¾³ý¡£
Q£ºÊý¾Ý½ÚµãPVC/PV ¹ÒÔØµÄÎļþϵͳÊÇÄÇÖÖÄØ£¿Êµ¼ÊʹÓÃÉÏÓöµ½Ê²Ã´ÎÊÌâûÓУ¿
A£ºÎÒÃÇÓùýµÄÖ÷ÒªÊÇNFSºÍGlusterFS¡£×î³õʵÏÖµÄPV±È½ÏÈõ£¬PVC²»ÄÜͨ¹ýLabelÓëPVÆ¥Å䣬ֻÄÜͨ¹ýsizeºÍ·ÃÎÊÀàÐÍÆ¥Å䣬ÎÞ·¨×¼È·Ñ¡ÔñPV´æ´¢¡£ÏÖÔÚ×îÐÂKubernetesÖ§³ÖPVCµÄselectorÖ§³ÖÑ¡Ôñ´øÓÐÌØ¶¨LabelµÄPVÁË¡£
Q£ºÇëÎÊKubernetesËÞÖ÷»úµÄÈÕÖ¾ÊÇÈçºÎÊÕ¼¯µÄ£¿
A£ºÓÃÏàÓ¦µÄ²»Í¬µÄFluentd²å¼þ£¬ÀàËÆµÄmountµ½ÏàÓ¦µÄÖ÷»úÈÕ־Ŀ¼¼´¿É¡£ÏÖÔÚÈÝÆ÷µÄÈÕÖ¾Ò²ÊÇͨ¹ýÖ÷»úÊÕ¼¯µÄ¡£
Q£ºÈÕÖ¾°üÀ¨ÈÝÆ÷µÄ²¶»ñµÄ±ê×¼Êä³öÈÕÖ¾ºÍÓ¦Óôòµ½ÈÕÖ¾ÎļþÖеÄÈÕÖ¾¡£¶ÔÓÚÕâÁ½Àೡ¾°£¬ÈçºÎÓÃFluentdʵÏÖÐÂÆô¶¯ÈÝÆ÷µÄÈÕÖ¾×Ô¶¯·¢ÏÖºÍÊÕ¼¯£¿
A£º¶ÔÓÚ´òµ½ÈÕÖ¾ÎļþÖеÄÈÕÖ¾£¬ÔÔòÉϽ¨ÒéÈÕ־Ŀ¼ÊÇÖ÷»ú°ó¶¨ÉϵĻòÊǹ²ÏíĿ¼¡£ÈÕÖ¾µÄ×Ô¶¯·¢ÏÖºÍÊÕ¼¯ÐèҪͨ¹ýfluentdµÄ²å¼þ£¬½«Ö¸¶¨µÄĿ¼µÄÎļþ¹ýÂ˳öÀ´£¬ÀýÈç±ê×¼Êä³öÈÕÖ¾¿Ï¶¨ÔÚÖ÷»úµÄ/var/lib/docker/containers/container-id/Ï¡£ÎÒÃǼ¯³ÉµÄFluentd¾µÏñ£¬ÒѾ´ò°üÅäÖúÃÁËÏàÓ¦µÄµÄ²å¼þhttps://github.com/fabric8io/fluent-plugin-kubernetes_metadata_filter£¬¿ÉÒԲο¼¡£
Q£ºÎÒÃÇĿǰҲʹÓÃÁËFluentdÊÕ¼¯ÈÝÆ÷ÈÕÖ¾£¬ÊÕ¼¯ÈÝÆ÷дµ½logÎļþÖеÄÈÕÖ¾±ÈÊÕ¼¯´Ó±ê×¼Êä³öµÄÈÕÖ¾ÒªÂý£¬ÇëÎÊÄãÃÇÓÐʲôµ÷Óŵķ½·¨Âð£¿
A£ºÎļþÈÕÖ¾±È±ê×¼Êä³öÂýÊÇÕý³£µÄ£¬µ÷ÓÅFluentdµÄÐÔÄÜ¿ÉÄÜÒª¸ù¾ÝFluentdµÄ˵Ã÷Öð½¥»ýÀÛ¾ÑéÏȶ¨Î»ÄÄÀïÂý£¬ÔÙÊÔÑé¼Ó¿ì·½·¨£¬¸ú¸÷ÖÖϵͳµÄÐÔÄܵ÷ÓÅÊÇͬÑùµÄ˼·¡£ÏÂÃæÕâ¸öÁ´½ÓÓÐЩµ÷ÓŽ¨Òé¡£http://docs.fluentd.org/articles/performance-tuning
Q£ºÇëÎÊÈçºÎ´¦Àí¼¯Èº×ÔÎÒ»Ö¸´»úÖÆ£¬±ÈÈçelasticsearch-master¡¢elasticsearch-client
¹ÒÁË£¿
A£ºÎÒÃÇÔÚKubernetes¼¯ÈºÖУ¬elasticsearch-masterºÍelasticsearch-client¶¼ÊÇÒÔRelication
Controller»òReplication Set·½Ê½Æô¶¯µÄ£¬ÏµÍ³»á×Ô¶¯±£Ö¤·þÎñµÄ¸ß¿ÉÓá£ÆäËû¼¯ÈºÒ²ÊÇÀàËÆµÄ»úÖÆ£¬ºÍÒ»°ãWebÓ¦Óõĸ߿ÉÓÃÊÇÒ»Ñù£¬ÒªÓлúÖÆ±£Ö¤ÖØÆô·þÎñ£¬ÒªÓлúÖÆ×ö·þÎñ·¢Ïֺ͸ºÔؾùºâ£¬ÔÚK8s¼¯ÈºÊÇ¿¿Relication
ControllerºÍKube-proxy¡£
Q£ºÇëÎÊ£¬ÔÚKubernetes¼¯ÈºÖеÄÿ¸ö½ÚµãÉÏÔËÐÐÒ»¸öFluentdµÄÈÝÆ÷£¬Õâ¸ö½ÚµãÊÇÈÝÆ÷»¹ÊDz¿ÊðDockerµÄ½Úµã£¿
A£ºÕâ¸öÊÇÖ÷»ú½Úµã£¨¿ÉÄÜÊÇÎïÀí»ú»òÐéÄâ»ú£©£¬¾ÍÊÇKubernetesµÄNode£¬²¿ÊðDockerµÄ½Úµã¡£ÍƼöµÄ¹Ù·½·½·¨£¬ÊÇͨ¹ýKubernetesµÄDaemonSet×ö²¿Êð¡£µ±È»Ò²¿ÉÒÔ×Ô¼ºÔÚÿ¸ö½ÚµãÉÏά³Ö×Ô¶¯µÄÆô¶¯½Å±¾£¬ÔËÐÐһЩÿ¸ö½Úµã¶¼ÒªÆô¶¯µÄPod·þÎñ¡£
Q£ºÇëÎÊ£¬KubernetesµÄmasterÊǵ¥µãµÄ£¬ÄãÃÇÊÇ·ñÓÐÓÅ»¯¹ý£¬ÏÖÔÚ1.3ÁË£¬ÄãÃÇÆ½Ì¨Éý¼¶ÊÇ·ñÊÇÈȲ¿Êð£¿
A£ºÎÒÃÇÊÇÓÃpodmaster×öÖÁÉÙ3½Úµãmaster¸ß¿ÉÓò¿Êð£¬api-serverÊǶà»îµÄ£¬controllermanager
ºÍscheduleÊÇ1»î2±¸¡£Æ½Ì¨Éý¼¶ÏÖÔÚÊÇÊÖ¶¯µÄ£¬²»»áÓ°ÏìÔËÐÐÖеķþÎñ¡£µ«ÊÇÏÖÔÚÆ½Ì¨Éý¼¶ÐèÒª¹¤³ÌʦСÐÄÒíÒíµÄÊÖ¶¯²Ù×÷£¬»¹²»ÊÇ×Ô¶¯»¯µÄ¡£
Q£ºÇëÎÊFluentdºÍFlume³ýÁË¿ª·¢ÓïÑÔ²»Ò»Ñù£¬ÓÐʲô±¾ÖÊÉϵÄÇø±ð£¿ÒÔ¼°ESÈÕË÷ÒýµÄ·ÖƬÊýÁ¿½¨ÒéÊÇ£¿
A£ºFluentdºÍFlumeµÄÉè¼ÆÀíÄîÊÇÀàËÆµÄ£¬Ò»¸öÓÃCRuby£¬Ò»¸öÓÃJRuby£¬ÓëFluentdºÍLogstashµÄÇé¿öÀàËÆ£¬FluentdµÄ¾µÏñ»áСһЩ£¬ÔËÐÐʱÄÚ´æÏûºÄ»áСһЩ£¬¶øFlumeµÄ¾µÏñÒòΪҪ´ò°üJDK²î²»¶àÒª¼¸°ÙÕס£ÔÚÎ§ÈÆÈÝÆ÷µÄLinux»·¾³ÖУ¬JavaµÄ¿çƽ̨ÐÔ±¾Éí´øÀ´²»ÁËÌØ±ð´óÓÅÊÆ£¬·´¶øFluentd¾µÏñСµÄÓÅÊÆ¸ü¼ÓÃ÷ÏÔ¡£ÁíÍâÒ»¸ö¶ÔÎÒÃǵÄϵͳʵ¼ùÒâÒå±È½Ï´ó£¬¾ÍÊÇfluentd¸úKubernetes¼¯ÈºµÄ·½°¸×öµÄ¼òµ¥Ã÷ÁË¡£ESĬÈϵÄshardsÊýÊÇ5£¬Ò»°ãµÄ¼¯ÈºÇé¿öÒª¸ù¾ÝʹÓÃÇé¿ö²âһϣ¬¶ÔÓÚ²»Í¬µÄindex£¬Õâ¸öshardsÊý¿ÉÒÔÊDz»Ò»ÑùµÄ¡£ShardsµÄ¸öÊý¾¡Á¿²»Éè1°É£¬Éè1µÄ»°£¬½«À´ÏëÒªÔö¼Ó·ÖƬʱ£¬Òƶ¯µÄÊý¾ÝÁ¿Ì«´óÁË¡£ÔÚûÓкܳä·ÖµÄÉú²ú²âÊÔ¾Ñé֮ǰ£¬ÉèÖÃ2µ½5±È½ÏºÃ¡£
|