ÆßÅ£ÔÆµÄ
AI ²¿ÃÅÊôÓÚÈÝÆ÷ÔÆ²¿ÃŵĿͻ§£¬Õë¶ÔÓÚ AI ѵÁ·ÕâÑùÒ»¸öÌØÊâµÄѵÁ·³¡¾°£¬¾ßÌåÂäʵµ½ k8s µÄʵ¼ùÉϾßÌåʵʩ¹¤×÷ÔõÑù×öµÄ£¬´ø¸øÆßÅ£ÔõÑùµÄºÃ´¦£¬ÒÔ¼°´ÓÖÐÅöµ½Ò»Ð©Ê²Ã´ÑùµÄÎÊÌâ½ÁúÎĶԴË×öÁË·ÖÏí¡£
1¡¢AI ѵÁ·µÄÒµÎñÇé¿ö
ÆßÅ£±¾ÉíÓÐÒ»¸öÉî¶ÈѧϰµÄƽ̨£¬ÕâÊÇÒ»¸ö¶Ëµ½¶ËµÄÉî¶Èѧϰƽ̨£¬°üÀ¨´Ó¶ÔÔʼ¸»Ã½ÌåÊý¾ÝµÄ´ò±ê£¬µ½ÖƳÉÒ»¸ö¿ÉÒÔ±»ÑµÁ·ÈÎÎñ¶ÁÈ¡µÄÑù±¾¼¯£¬µ½ÑµÁ·ÈÎÎñµÄ´¥·¢ÒÔ¼°ÑµÁ·³É¹ûµÄ´æ´¢£¬°üÀ¨¶ÔÓÚ×îºóѵÁ·³öÀ´Ä£ÐÍµÄÆÀ¹À£¬ÆÀ¹ÀÍê³ÉÒÔºó×îºó½«Õâ¸öÄ£ÐÍ´ò°ü³ÉÄãµÄÏßÉÏÒµÎñ£¬Í¨¹ý
API ÐÎʽ¶ÔÍâÌṩ·þÎñÒ»ÕûÌ×Á÷³ÌµÄƽ̨¡£


AI ѵÁ·ÊÇÕâ¸öƽ̨ÖеÄÒ»¸ö²¿·Ö¡£AIѵÁ·µü´úÊÇÔõÑùµÄÒ»¸öÊÂÇ飿
AI ѵÁ·µü´ú·ÖÁ½¸ö½×¶Î£º
µÚÒ»£¬Ñù±¾¼¯µÄÉú³É£¬ÈÎÎñÊäÈëÊÇÁ½¸ö£ºÒ»ÊÇÀ´×ÔÓÚÆßÅ£¶ÔÏó´æ´¢µÄÔʼÊý¾Ý£¬Ö÷ÒªÊÇһЩͼƬ¡¢ÒôÊÓÆµÁ÷¸»Ã½ÌåÊý¾Ý£»¶þÊÇ
ava ƽ̨±¾ÉíÓÐÒ»¸ö´ò±êϵͳ£¬¿ÉÒÔ¶ÔÔʼÊý¾Ý½øÐбêÇ©£¬Í¨¹ýÑù±¾Éú³ÉÆ÷Éú³ÉÑù±¾¼¯£¬´æ´¢µ½ÈÝÆ÷ÔÆÆ½Ì¨µÄ´æ´¢µ±ÖУ¬ÕâÊÇÒ»¸ö·Ö²¼Ê½µÄÍøÂç´æ´¢¡£
µÚ¶þ£¬Ò»µ©ÄãµÄÑù±¾¼¯Éú³ÉÍê³ÉÒÔºó»á×Ô¶¯´¥·¢»òÕßÈ˹¤´¥·¢Ò»¸öѵÁ·ÈÎÎñ½øÐÐÒ»¸öѵÁ·£¬¶ÁÈ¡Õû¸öƽ̨ÓÉËã·¨¹¤³ÌʦÊÂǰ׼±¸µÄË㷨ģÐÍ¡¢ÑµÁ·²ÎÊýµ½ÄãµÄѵÁ·ÈÎÎñµ±ÖнøÐÐ
¡£
ѵÁ·£¬×îºó½«ÄãµÄѵÁ·ÈÎÎñÊä³öµ½´æ´¢£¬×îºóÉÏ´«µ½¶ÔÏó´æ´¢µ±ÖÐÈ¥µÄÕû¸ö¹ý³Ì¡£
2¡¢Kubernetes µÄÓÅÊÆ
ÎÒÃÇÕâ±ßÓöµ½µÄÍ´µãÊÇʲô£¿
µÚÒ»£¬Ê¹Óà Kubernetes ×öƽ̨֮ǰ£¬ÑµÁ·Á÷³ÌÉÏÐèÒªËã·¨¹¤³Ìʦͨ¹ý½Å±¾¡¢¿ØÖÆÑµÁ·ÈÎÎñµÄ´¥·¢ÒÔ¼°ÑµÁ·ÈÎÎñÒª´æ´¢µ½Ê²Ã´µØ·½£¬Í¬Ê±ÑµÁ·ÈÎÎñ¿ÉÄÜÒòΪһЩӲ¼þ´íÎóµ¼ÖÂʧ°Ü£¬Ê§°ÜÐèÒªÈ˹¤½éÈë¡£
µÚ¶þ£¬×ÊÔ´¹æ»®·½Ã棬GPU ¼¯ÈºÊǺܶàÈ˹²ÏíµÄ£¬ÆäÖÐ GPU ×ÊÔ´ÐèÒªÈËΪе÷£¬ºÄ·ÑµôºÜ¶à¾«Á¦¡£
µÚÈý£¬ÑµÁ·ÈÎÎñÍê³ÉÒÔºó²¢Ã»ÓаÑÕ¼ÓÃµÄ GPU Êͷŵô£¬Ôì³ÉÒ»¶¨µÄ×ÊÔ´ÀË·Ñ¡£
µÚËÄ£¬´æ´¢£¬ÑµÁ·ÈÎÎñµÄ´æ´¢ÍùÍùÊǷdz£´óµÄÑù±¾¼¯£¬ÐèÒªÈÝÁ¿·Ç³£´óµÄÍøÂç´æ´¢Ö§³Å£¬ÔÚ´Ë֮ǰÎÒÊÇÓõÄÊÇ
NFS£¬·þÎñ¿ÉÓÃÐÔûÓа취´ïµ½ÐèÇó£¬Ë®Æ½À©Õ¹ÒÔ¼°ÐÔÄÜҲûÓа취Âú×ãѵÁ·ÈÎÎñµÄÒªÇó¡£
k8s Ö÷ÒªÓÐÁ½¸öÓÅÊÆ£º
µÚÒ»£¬k8s Ö§³Ö GPU µ÷¶ÈµÄ£¬ÎÒÃÇ»ý¼«½«Õû¸öʵ¼ù¹ý³Ìµ±ÖÐÈ¡µÃµÄ³É¹û»ØÀ¡µ½ÉçÇø£»
µÚ¶þ£¬k8s Ö§³Ö¶àÖÖ Workload µÄµ÷¶È·½Ê½£¬ÊÊÓ¦²»Í¬µÄÒµÎñ³¡¾°£¬JOB ÓëѵÁ·ÈÎÎñÁ½ÕßÇк϶ȷdz£¸ß¡£
k8s ºÍÏÖÔÚ¿ªÔ´ÉçÇø½áºÏ·Ç³£ºÃ£¬°üÀ¨¼à¿ØÈÕÖ¾·½°¸ÉçÇøÒѾȡµÃÁËÏ൱³É¹û£¬Ôڴƽ̨µÄʱºòÕâ¸ö²¿·ÖÊ¡Á˺ܶàÈËÁ¦¡£
3¡¢»ùÓÚ Kubernetes µÄ AI ѵÁ·
AI ѵÁ·-Éú³ÉÑù±¾¼¯
Õû¸öµ÷¶È½ö½öÀûÓà k8s µ÷¶ÈûÓа취ºÜºÃµÄÂú×ãÒµÎñÐèÇó£¬ÎÒÃǶÔÑù±¾¼¯µÄÉú³ÉÓÐ SampleJobController
×öÕû¸öÑù±¾¼¯Éú³ÉÈÎÎñµÄµ÷¶È£¬Éú³ÉÈÎÎñ´Ó¶ÔÏó´æ´¢ºÍ mongo Êý¾Ý¿âÖжÁÈ¡ÊäÈëÊý¾Ý£¬²ú³öÒ»¸öÑù±¾¼¯ÊäÈëµ½
CEPH ´æ´¢¡£
AI ѵÁ·-Æô¶¯ÑµÁ·ÈÎÎñ

ÒÔÉÏÈÎÎñÍê³ÉÒÔºó»á´¥·¢Ò»¸ö Training Job Controller µÄѵÁ·ÈÎÎñ£¬Õâ¸öѵÁ·ÈÎÎñ´Ó¸Õ¸ÕµÄÑù±¾¼¯Àï¶ÁÈ¡Êý¾Ý£¬Í¬Ê±ÅäºÏË㷨ģÐͺÍѵÁ·²ÎÊý£¬¶ÔÓÚË㷨ģÐ͵ÄÈ¨ÖØ½øÐмÆË㣬×îºóѵÁ·Íê³ÉÒÔºóÔÙ½«ÐµÄË㷨ģÐÍÊä³öµ½
CEPH ´æ´¢µ±ÖУ¬Èç¹ûÆÀ¹ÀÏÂÀ´±È½ÏºÃµÄ»°¿ÉÒÔÉÏ´«µ½¶ÔÏó´æ´¢µ±ÖУ¬CEPH Õⲿ·Ö´æ´¢×ÊÔ´¿ÉÒÔÊͷŵôÁË¡£
AI ѵÁ·-ʹÓà CEPH ´æ´¢
ʹÓÃµÄ CEPH ´æ´¢ÑµÁ· AI ѵÁ·ÈÎÎñ³¡¾°Ö÷ÒªÓÐÈý¸öºÃ´¦£º
µÚÒ»£¬Êý¾Ý¹æÄ£¿ÉÒÔÖ§³Ö·Ç³£´ó£¬×î´óÑù±¾¼¯¿ÉÒÔ´ïµ½Ò»¸öÑù±¾¼¯ 10T Êý¾Ý£¬ÐèÒª¶ÁÈ¡Êý¾Ý·þÎñÒ»¶¨ÐèÒªÓÐÒ»¸öÍøÂç¹²ÏíÀ´Ö§³Å£¬ÕâÑùÔÚÎïÀí»ú·¢Éú¹ÊÕÏʱ£¬pod
Ôڱ𴦱»ÖØÆôºóÈÔÈ»ÄÜ·ÃÎÊ֮ǰµÄÊý¾Ý£»
µÚ¶þ£¬ CEPH ´æ´¢ÊÇ·Ö²¼Ê½´æ´¢£¬Ë®Æ½À©Õ¹ÐԷdz£Á¼ºÃ£¬ÑµÁ·¼¶¹æÄ£ÉÏÉýÒÔºó¿ÉÒԺܿìËٵĽøÐÐˮƽÀ©Õ¹£»
µÚÈý£¬¶Áд¿ØÖÆ£¬Kubernetes Ò»¸ö¶ÀÕ¼µÄ¶ÁдºÍ¶à¸ö Pod ͬʱ¶ÁÈ¡µÄÄ£ÐÍ£¬ÊÊÓÃÓÚѵÁ·Ä£Ð͵ÄÕû¸öÁ÷³Ì£¬°üÀ¨Ö®Ç°Ñù±¾¼¯Éú³ÉÓÐÒ»¸öÑù±¾Éú³É£¬Ò»µ©Íê³ÉÒÔºó¿ÉÒÔ½øÈëÖ»¶Áģʽ£¬¶à¸öÈÎÎñͬʱ¶ÁÈ¡½øÐв¢·¢ÑµÁ·¡£CEPH
ʹÓùý³ÌÖÐÎÒÃǰѻý¼«µØ¸Ä½ø»ØÀ¡¸øÉçÇø£¬±ÈÈç˵ ImageFormat2 µÄÖ§³Ö£¬»¹ÓÐ k8s ¶Ô CEPH
µ÷¶ÈÐèÒªÓÐÒ»¸ö Provisioner È¥Ö§³ÖµÄ£¬ÏÖÔÚÕû¸öÉçÇøÑݽø·½ÏòÏ£Íû½«ÕâЩ´æ´¢ Provisioner
È«²¿±ä³É¶ÀÁ¢²¿ÊðµÄÐÎʽ£¬±ãÓÚËüµÄÉý¼¶À©Õ¹¡£
GPU×ÊÔ´¹æ»®²ÉÓÃNode Label+Node Selector£¬¶ÔѵÁ·ÈÎÎñ½øÐе÷¶È£¬ÎÒÃÇµÄ GPU
¿¨¿ÉÄÜÓв»Í¬µÄÐͺţ¬¶Ô²»Í¬ÑµÁ·ÈÎÎñ»áÓÐÐͺÅÉÏµÄÆ«ºÃ£¬Õâ¸öʱºò¿ÉÒÔΪÿһ̨»úÆ÷ÉÏ×°µÄ¾ßÌåÐͺŵÄÏÔ¿¨°ïÖúËü´òÉÏÒ»¸ö±êÇ©£¬Ö®ºó½øÐÐѵÁ·ÈÎÎñµ÷¶ÈµÄʱºò¿ÉÒÔʹÓÃ
Node Selector ½«Õâ¸öÈÎÎñµ÷¶ÈÉÏÈ¥¡£
¹ØÓÚ×ÊÔ´·½ÃæµÄ£¬Kubernetes ÌṩÁË±È½ÏºÃµÄ Limits+request ×ÊÔ´·ÖÅäÄ£ÐÍ£¬Limits
±íʾÕâ¸ö Pod ×î¶àʹÓöàÉÙ×ÊÔ´£¬Request ÊÇ˵Ҫ½«Õâ¸öÈÎÎñµ÷¶ÈÆðÀ´×îÉÙÐèÒª¶àÉÙ×ÊÔ´£¬Ä¿Ç°¶ÔÓÚ
GPU ÕâÑùµÄÄ£ÐÍûÓа취ºÜºÃµÄ¹¤×÷£¬ÎÒÃÇȱÉÙÒ»¸öÓÐЧµÄ»úÖÆ¼à¿Ø GPU ʹÓöàÉÙ£¬ÏÞÖÆ¶Ô GPU
ʹÓ㬶ÔÓÚ CPU ºÍ Memory ¿ÉÒÔÓÐЧµÄʹÓÃÕâÑùµÄÄ£ÐÍ£¬½øÐкÏÀí³¬Âô£¬Ìá×ÊÔ´µÄÀûÓÃÂÊ¡£
¹ØÓÚ Nvidia GPU Driver£¬ÑµÁ·ÈÎÎñÐèÒªÔÚ Pod µ±ÖÐʹÓþßÌåÏÔ¿¨µÄÇý¶¯£¬Ã¿Ò»Ì¨»úÆ÷°²×°²»
ͬÐͺŵÄÏÔ¿¨Çý¶¯°æ±¾Ò²ÊDz»Ò»ÑùµÄ£¬µ«ÊÇÎÒÃÇ Pod ²¢²»¹ØÐÄÕâ¸ö°æ±¾£¬Ö»Êǵ÷¶Èµ½Õą̂»úÆ÷ÉϾÍÐèÒªÕą̂»úÆ÷É϶ÔÓ¦ÐͺÅÏÔ¿¨µÄÇý¶¯£¬ÎҾͿÉÒÔͨ¹ý
k8s µÄ Hostpath ·½Ê½¹ÒÔØµ½ PodÉÏÈ¥£¬´ò°ü¾µÏñµÄʱºòÍêÈ«²»ÐèÒª¹ØÐÄ GPU Çý¶¯Õâ¸öÊÂÇé¡£
ÎïÀí»ú¼à¿Ø
»ùÓÚ Prometheus Node Exporter
»ñÈ¡ CPU¡¢ÄÚ´æ¡¢´ÅÅÌ¡¢ÍøÂçά¶ÈÐÅÏ¢
ÈÝÆ÷¼à¿Ø
kubelet ÄÚǶ cadvisor
¼à¿Ø×¢²á
Prometheus ´Ó kubernetes apiserver »ñÈ¡Ðè¼à¿ØµÄ×ÊÔ´
GPU ¼à¿Ø
GPU ʹÓÃÂÊ
ÏÖ´æÊ¹ÓÃÂÊ
GPUºËÐÄʹÓÃÂÊ
¹ØÓÚ¼à¿ØºÍÈÕÖ¾·½°¸²ÉÓõÄÊÇ Prometheus£¬Ëü±¾ÉíÌṩµÄ Prometheus Node Exporter
¿ÉÒԺܺõİïÖúÎÒÃǹØ×¢Õû¸ö¼¯ÈºÀïÎïÀí»ú½á¹¹µÄÐÅÏ¢£¬kubelet ÀïÒѾ¼¯³É cadvisor °ïÖúÎÒÃÇÌṩÈÝÆ÷ÄÚ²¿µÄ¼à¿ØÐÅÏ¢£¬ÎÒÃÇ»¹ÔÚÉÏÃæ×öÁËÒ»¸ö¸Ä½ø¾ÍÊǽ«
GPU µÄ¼à¿ØÐÅÏ¢Ìí¼Óµ½¼à¿Ø·½°¸µ±ÖÐÈ¥£¬²¢ÇÒ¹±Ï׸øÉçÇø¡£
¹ØÓÚÈÕÖ¾·½°¸ÎÒÃDzÉÓÃÁËÓÉÆßÅ£×ÔÖ÷Ñз¢µÄ·ÖƬµÄ Elastic Search ×ÔÑÐµÄ Sharding
¼¯Èº£¬³ÐÔØÁËĿǰËùÓÐÆßÅ£µÄÒµÎñÊý¾ÝÒÔ¼°°üÀ¨Íⲿ¿Í»§µÄÊý¾Ý£¬°Ñ Elastic Search ÔËάµÄ¹¤×÷ÍêÈ«½»¸¶¸øÆßÅ£
pandora ÈÕÖ¾´æ´¢·ÖÎöƽ̨¡£


4¡¢Ò»´Î²È¿Ó¾Àú
½ÓÏÂÀ´·ÖÏíÒ»ÏÂÎÒÃÇÔÚÔËάµ±ÖÐÅöµ½µÄÎÊÌ⣬×îºóÔì³ÉºÜÑÏÖØºó¹ûµÄʹʣ¬Ê×ÏȽéÉÜҪһϠk8s ʹÓà CEPH
´æ´¢ÊÇÔõÑùÒ»¸ö¹ý³Ì£¿ÎÒÃÇÖªµÀ CEPH ´æ´¢ÊÇͨ¹ý CEPH µÄ RBD ÃüÁ½« CEPH µÄ image
attach µ½ÄãµÄËÞÖ÷»ú³ÉΪһ¸ö¿éÉ豸£¬k8s ½«Õâ¸öÉ豸 Mount µ½ Kubelet µÄPulgins
Îļþ¼ÐÏÂÃæ£¬ÔÙ´Îͨ¹ý Mount rbind µÄ·½Ê½°ó¶¨µ½¶ÔÓ¦ÐèÒªµÄ POD µÄĿ¼Ï£¬ÕâÊÇ CEPH
image °ó¶¨µ½ POD µÄ¹ý³Ì¡£ mount rbind ÊǹÒÔØÃüÁ±¾ÉíÊÇÓÐÈýÖÖģʽ£º Shared¡¢slave¡¢private£¬k8s
ÔÚʹÓõÄʱºòÊÇ 1.6£¬½ö½öÖ§³Ö Private ģʽ¡£
ÒòΪÕâÑùÒ»¸öÔÒòµ¼ÖÂÁ˹ÊÕϵķ¢Éú£ºÎÒÃÇÓÐÒ»¸öÈÝÆ÷ A ÒѾÔËÐÐÆðÀ´ÁË£¬ÊÇÖ»¶ÁµÄ·½Ê½¹ÒÔØÁËÒ»¸ö´æ´¢£¬½ÓÏÂÀ´
Node Exporter Òª½øÐÐ¼à¿Ø²É¼¯£¬ÎªÁË»ñȡijЩ¼à¿ØÊý¾Ý£¬»áÒÔ Make-private ¹ÒÔØÕû¸öËÞÖ÷»ú¸ùĿ¼£¬¸Õ²ÅÌáµ½ÒѾÓÐÈÝÆ÷
A ½« RBD Volume ¹ÒÔÚÆðÀ´ÁË£¬ÊƱØÖ®ºóµÄ¹ÒÔØÒ²½«Õâ¸ö´øµ½ÁË Promethus µÄ Node
Exporter£¬µ¼ÖÂÁË A ÈÝÆ÷ÔËÐÐÍê³ÉÒÔºó½«ÈÝÆ÷Ïú»ÙÁË£¬Ð¶ÔØ RBD Volume ³É¹¦£¬µ«ÊÇ
RBD umap ʧ°Ü£¬Ö÷ÒªÊÇÒòΪ Node-Exporter ÈÔÔÚÔÚ RO ¹ÒÔØ¡£ÎÒÃÇҲûÓз¢ÏÖÕâ¸öÎÊÌâ¡£

ÓÖÒ»¸öеÄÈÝÆ÷ B ÒªÊÔͼ¶ÁÈ¡Õâ¸ö Volume µÄʱºò£¬·¢ÏÖÕâ¸ö Volume Õâ¸öÉ豸ÒѾ´æÔÚÎïÀí»úÉÏ£¬ÔÙ´ÎÒÔ¶ÁÈ¡µÄ·½Ê½¹ÒÔØ£¬µ¼ÖÂÁ˹ÒÔØÊ§°Ü£¬k8s
ÔÚÕâÖÖÇé¿öÏ»áÊÔͼ»ñÈ¡É豸µÄÎļþϵͳÐÅÏ¢£¬ÒòΪÎÒÃÇ֮ǰ·¸µÄÁíÍâÒ»¸ö´íÎ󣬻ñÈ¡ÎļþÐÅÏ¢¾Íʧ°ÜÁË£¬k8s
»ñÈ¡ÎļþÐÅÏ¢Êǿյ쬴¦Àí·½Ê½Ò²±È½Ï¼òµ¥´Ö±©£¬ÈÏΪ»ñȡʧ°ÜÕâ¸öÅ̾ÍÊÇûÓиñʽ»¯¹ýµÄ£¬¾Í´¥·¢Á˸ñʽ»¯£¬°ÑÎÒÃÇ֮ǰѵÁ·ºÃµÄÊý¾ÝÖ±½ÓÈ«²¿¸ñʽ»¯µôÁË¡£
Õû¸ö¹ÊÕÏÔÒò¾ÍÊÇ Node-exporter ¹ÒÔØ¸ùĿ¼·½Ê½±È½ÏΣÏÕ£¬µ¼ÖÂÁËÖ®ºó¹ÒÔØ ceph ¾íʧ°Ü¡£¹ÒÔØ¾íʧ°ÜÕâÔ±¾²»ÊÇÒ»¸öÌ«ÑÏÖØµÄÎÊÌ⣬ҲÊÇÒòΪÎÒÃÇ֮ǰ²¿ÊðÉϵÄÎÊÌâµ¼ÖÂÁË
ceph ·þÎñ¶ËºÍ¿Í»§¶Ë°æ±¾²»Ò»Ö£¬ÖÂʹ»ñÈ¡ÎļþÐÅϢʧ°Ü£¬±¾Éí k8s ¶ÔÕâÖÖÇé¿ö´¦Àí·½Ê½¼òµ¥´Ö±©£¬ÖÖÖÖÔÒò·ÅÔÚÒ»Æðµ¼ÖÂÁËÕû¸öʹʵķ¢Éú¡£·´Ë¼Õâ´Îʹʣ¬Ö÷ÒªÓм¸¸öµã£º
µÚÒ»£¬²¿ÊðÁ÷³ÌÐèÒª¹Ì»¯£¬ÒòΪÎÒÃÇÿ´Î²¿Êð¶¼ÒªÐèÒªÈËÊÖ¹¤²Ù×÷£¬Ò»Ð©ÅäÖÃÎļþ¶¼Êǵ±Ê±Ð޸ijöÀ´µÄ£¬Òª°ÑÕû¸ö²¿ÊðÁ÷³Ì¹Ì»¯ÏÂÀ´£¬ÔÚ²¿ÊðÍê³ÉÒÔºóÒª¼ì²éÏàÓ¦µÄ°æ±¾¡£

µÚ¶þ£¬¼´±ãÄãÕæÕý·¸ÁË´íÎóµÄʱºò£¬ÐèÒªÓÐÒ»¸öÍêÕûµÄ»úÖÆ°Ñϵͳ×é¼þµÄÈÕ־ץȡ³öÀ´£¬ÅäÖóɸ澯µÄÐÎʽ¼°Ê±µÄ¸ú½ø£¬ÒòΪ»Ø¹ËÅŲéµÄʱºòÓÉÓÚ°æ±¾²»Ò»ÖµÄÎÊÌâÒѾÔÚÈÕ³£¹¤×÷ÖвúÉúÁË´íÎóÈÕÖ¾£¬µ«ÊÇÎÒÃDz¢Ã»ÓÐÒýÆðÖØÊÓ£¬Ò»Ö±µ½Ê¹ʷ¢Éú£¬·¢ÉúÊý¾Ý¶ªµôµÄʱºò»Ø¹ýÍ·À´¿´µÄʱºò²Å·¢ÏÖÕâ¸öÎÊÌâ¡£

µÚÈý£¬¶ÔÓÚ Kubernetes ʹÓÃ֪ʶÁ˽ⲻ¹»£¬ÕâЩ֪ʶҲÊǶÔÕû¸öʹʸú½øµÄʱºò£¬·¢ÏÖ Kubernetes
´¦ÀíÂß¼ÊÇÕâ¸öÑù×ӵġ£ÍŶӶÔÓÚ Kubernetes Ïà¹Ø´¦ÀíÂß¼Òª½øÐÐÊáÀí£¬±ÜÃâһЩDZÔÚµÄÒþ»¼¡£×îÖÕÎÒÃÇͨ¹ýʹÓöþ½øÖÆÔÚÎïÀí»ú²¿Êð
Node Exporter µÄ·½Ê½À´ÔÝʱ»º½âÕâ¸öÎÊÌâ¡£
5¡¢½ÓÏÂÀ´µÄ¹¤×÷
**µÚÒ»£¬×ÔÅäÖÃ¼à¿Ø¡£**Prometheus ¶Ô k8s µÄÖ§³ÖÊDZȽϺõģ¬ÏÖÔÚÒѾ֧³Ö×Ô¶¯´Ó k8s
µÄ APIServer Àï»ñÈ¡ Service£¬·¢ÏÖÐèҪץÁ¿µÄ Service ×Ô¶¯×¥È¡£¬Õâ¸öÒѾӦÓõ½ÏµÍ³×é¼þµ±ÖУ¬µ«ÊǶÔÓÚÎÒÃǵÄAIѵÁ··þÎñ£¬×Ô¶¯ÅäÖÃµÄ¼à¿Ø¹¤×÷»¹Ã»ÓÐÂäʵ£¬ÕâÊÇÎÒÃǽÓÏÂÀ´ÒªÍêÉÆµÄÒ»²½¡£
µÚ¶þ£¬·Ö²¼Ê½ÑµÁ·¡£ÎÒÃÇÕû¸öѵÁ·Ä£ÐÍ»¹Êǵ¥»ú°æµÄģʽ£¬¶ÔÓÚÒ»¸ö±È½ÏÍêÕûµÄÉî¶Èѧϰƽ̨ÐèÒªÓÐÒ»¸ö·Ö²¼Ê½Ñ§Ï°µÄ·½Ê½£¬Ò²ÐèÒªÈÝÆ÷ÍŶӺÍ
AI ÍŶÓÒ»ÆðÊáÀíÕû¸öÒµÎñÁ÷³Ì´Ó¶øÈ¥Ö§³ÅÕâÑùÒ»¸öѵÁ··½Ê½¡£
|