±à¼ÍƼö: |
±¾ÎÄÏò´ó¼Ò½éÉÜ¿ª·¢¹ý³ÌÖУ¬¾³£»áÐèÒª¶ÔÎÒÃÇ¿ª·¢µÄ³ÌÐò×öÐÔÄÜ·ÖÎö£¬ÓкܶàÐÔÄÜ·ÖÎöµÄ¹¤¾ß£¬ºÜ¶àÓïÑÔ¶¼ÌṩÁ˲»Í¬µÄprofiling¹¤¾ß£¬ÕâЩ¹¤¾ßºÜÓÐÓã¬ÌṩÁ˳ÌÐòÔËÐеÄÔʼ¼Ç¼Êý¾Ý,Ï£Íû¶ÔÄúµÄѧϰÓÐËù°ïÖú¡£
±¾ÎÄÀ´×ÔÓÚcsdn£¬ÓÉ»ðÁú¹ûÈí¼þDelores±à¼¡¢ÍƼö¡£ |
|
¾ßÊÖ»úµÄÊý¾Ý±È½ÏÔʼ£¬ÍùÍù»¹ÐèҪһЩ¸ü½øÒ»²½µÄ·ÖÎö£¬²ÅÄܶ¨Î»ÎÊÌâ¡£
SplunkÊÇÒ»¸ö¿ÉÒÔÔËÐÐÔÚ²»Í¬Æ½Ì¨ÉϵĻúÆ÷Êý¾ÝµÄʵʱÔËάƽ̨£¬Ëùν»úÆ÷Êý¾Ý£¬¾ÍÊÇÖ¸»úÆ÷²úÉúµÄÊý¾Ý£¬ÆäÖÐÒ»¸ö³£¼ûµÄ³¡¾°¾ÍÊÇÈÕÖ¾¡£¶ÔÓÚ¹ã´ó³ÌÐòÔ±À´Ëµ£¬·ÖÎöÈÕÖ¾ÊÇÒ»¸ö·Ç³£³£¼û£¬¶øÇÒ·±ËöµÄ¹¤×÷£¬¶øÇҺܶàʱºò£¬±ØÐëͨ¹ýÈÕÖ¾À´¶Ô³ÌÐò½øÐе÷ÊÔ£¬ÀýÈç¶àÏ̵߳ÄÇé¿ö¡£¼ÇµÃÒÔǰΪÁ˼¸°ÙÕ×»òÕß¼¸GµÄÈÕÖ¾½øÐзÖÎö£¬²»µÃ²»Ð´ÁËlogViewerÀ´·ÖÎö¡£ÏÖÔÚÓÐÁËSplunk£¬ÕæµÄ¼«´óµÄ¼ò»¯Á˶ÔÈÕÖ¾·ÖÎöµÄ¹¤×÷¡££¨×¢SplunkÃâ·Ñ°æÖ§³ÖÿÌì500MµÄÈÕÖ¾Êý¾Ý£¬³¬¹ýÕâ¸ö¶î¶ÈÐèÒªÊÕ·Ñ£©
ͨ¹ýÈÕÖ¾½øÐÐÐÔÄܲâÊÔÊǷdz£³£¼ûµÄ£¬´«Í³µÄÒ²ÊÇÔÚÒª·ÖÎöµÄ´úÂë´¦£¬×¢ÈëÐÔÄÜÈÕÖ¾£¬È»ºóÔÚ³ÌÐòÔËÐк󣬶ÔдÈëµÄÐÔÄÜÊý¾Ý½øÐзÖÎö¡£Ê¹ÓÃSplunk£¬·½·¨ÊÇÒ»ÑùµÄ£¬µ«ÊÇÓÐÒÔÏÂÃ÷ÏԵĸĽø
SplunkÌṩ´óÁ¿ÓѺõķÖÎöÃüÁîºÍͼ±í£¬ÎÞÐèÁíÐпª·¢·ÖÎöÈÕÖ¾µÄ³ÌÐò
Splunk¿ÉÒÔʵʱµÄ¶ÔÓ¦ÓóÌÐò×÷·ÖÎö£¬¿ÉÒÔÔÚ³ÌÐòµÄÔËÐйý³ÌÖУ¬Ò»±ßÔËÐУ¬Ò»±ß·ÖÎö
ÎÒÏÂÃæ¾ÙÒ»¸öÎÒÅöµ½µÄÀý×Ó¡£
ÎÒÒª·ÖÎöµÄ³ÌÐòÊÇÒ»¸ö´ÓAWS CloudWatchÊÕ¼¯Êý¾ÝµÄPython³ÌÐò¡£ÊÕ¼¯Êý¾ÝʹÓõÄÊÇAWSÌṩµÄRestful API (Boto),ΪÁ˸ü¸ßЧµÄÊÕ¼¯Êý¾Ý£¬³ÌÐòʹÓöà¸öÏß³ÌÀ´µ÷ÓÃRestful API µÄQuery½Ó¿Ú¡£ÎÒÏ£Íûͨ¹ýÐÔÄÜÈÕÖ¾Á˽âÿһ¸öÇëÇó´ó¸ÅµÄºÄʱ£¬ÒÔ¾ö¶¨Ê¹ÓöàÉÙ¸öÏß³ÌÊýºÍ¶ÔÓ¦µÄ²É¼¯¼ä¸ô¡£
Ê×ÏÈ£¬ÐèҪдÈÕÖ¾£º
logger.log(logging.DEBUG,
"PerfLog=QueryStart" )## Query Code Goes Heredo_query_aws()##
Query Completelogger.log(logging.DEBUG, "PerfLog=QueryEnd,
Query Result) |
×¢ÒâʹÓÃName=ValueµÄÐÎʽ¿ÉÒÔ°ïÖúSplunkÔÚËÑË÷ʱ£¬ÌáȡҪ·ÖÎöµÄ×ֶΡ£
È»ºóÔËÐгÌÐò£¬³ÌÐòÔËÐÐÒÔºó»áÉú³ÉÈÕÖ¾Îļþ£¬°Ñ¸ÃÈÕÖ¾Îļþµ¼Èëµ½Splunk£¬¿ªÊ¼·ÖÎö¡£

µã»÷Add Data°´Å¥£¬È»ºó¸úËæSplunkµÄÖ¸µ¼£¬Ñ¡ÔñA file or directory of files. µ¼ÈëÄãµÄÈÕÖ¾Îļþ£¬µ¼Èë¹ý³ÌÖУ¬Splunk»áÒªÇó¸øÄãµÄÈÕÖ¾ÎļþÃüÃûÒ»¸ösourcetype£¬ÎÒÓõÄÊÇ¡°cloud_watch_debug¡±
µ¼ÈëºÃÒÔºó¾Í¿ÉÒÔ¿ªÊ¼ËÑË÷ÁË¡£
ÔÚËÑË÷¿òÖÐÊäÈë
sourcetype="cloud_watch_debug¡± |
Splunk»áʵʱµÄ·µ»ØËùÓеÄÈÕÖ¾Îļþ£¬²¢°´Ê±¼ä½âÎöΪһ¸ö¸öµÄʼþ¡£

SplunkµÄSPL£¨Splunk Search Language£©ÊÇÒ»¸öÀàËÆSQLºÍUNIX CommandµÄ×ÛºÏÌ壬¿ÉÒÔ¶ÔÊý¾Ý½øÐÐËÑË÷£¬·ÖÎö£¬Í³¼Æ£¬Éú³Éͼ±í£¬Ö§³Ö¹ÜµÀ£¬Ê¹ÓÃÆðÀ´·Ç³£·½±ã£¬½¨Òé´ó¼Òͨ¹ý¹Ù·½ÎĵµÀ´Á˽⡣
ÎÒô½ñÌìÒª×öµÄÊÇÐÔÄÜ·ÖÎö£¬ÄÇôÎÒ¾ÍÊÇҪͳ¼ÆÒ»Ï£¬·¢Á˶àÉÙ¸öquery£¬Ã¿Ò»¸öqueryÓÃÁ˶àÉÙʱ¼ä¡£
ÿһÌõÈÕÖ¾µÄÄÚÈÝ´óÖÂÈçÏÂ
2014-08-11 10:52:40,587
DEBUG pid=3742 tid=QueryWorkerThread-1 file=aws_cloudwatch.py: _main_work_loop:469
| PerfLog = Qu |
SplunkÄܹ»ÌáÈ¡³ö´óÁ¿µÄÐÅÏ¢ºÍ×ֶΣ¨field£©£¬°üÀ¨Ê¼þ£¬pid£¬tid£¬fileµÈµÈ£¬»¹ÓÐÎÒÃÇÔÚÈÕÖ¾ÖмÓÈëµÄ×Ö¶ÎPerfLog¡£
ÏëÒªÖªµÀÿһ¸ö²éѯËù»¨·ÑµÄʱ¼ä£¬¿ÉÒÔͨ¹ýSplunkÌṩµÄtransactionÃüÁî¡£
sourcetype=cloud_watch_debug | transaction tid startswith="QueryStart" endswith="QueryEnd"
tid±íʾÿһ¸ötransactionÐèÒªÓÐÏàͬµÄtid£¬Ò²¾ÍÊÇ˵ͬһ¸öÏß³Ì
startwithºÍendwith±íʾtransactionµÄÆäʵºÍ½áÊø±êÖ¾
¸ÃÃüÁî·µ»ØËùÓеÄqueryµÄtransaction

È»ºóÎÒÃǾͿÉÒÔͳ¼ÆÃ¿Ò»¸ötransactionËùÓõÄʱ¼ä
sourcetype=cloud_watch_debug
|
transaction tid startswith="QueryStart" endswith="QueryEnd"
| stats sum(duration), count, avg(duration),max(duration), min(duration) |
statsÃüÁîÓÃÓÚ¶ÔÊý¾Ý½øÐÐͳ¼Æ
durationÊÇSplunk¶ÔtransactionÉú³ÉµÄʼþ¿ç¶È
sum,count,avg,max,minÊÇͳ¼ÆÃüÁî
ÔËÐнá¹ûÈçÏ£º

³ÌÐòÒ»¹²·¢ËÍÁË111075¸öcloudwatchµÄÇëÇó£¬×îÂýµÄÐèÒª2.5Ã룬×î¿ìµÄ0.06Ã룬ƽ¾ù´ó¸Å0.11Ãë¡£
ÎÒ»¹ÏëÖªµÀqueryµÄºÄÊ±ËæÊ±¼äµÄ±ä»¯£¬ÎÒ¿ÉÒÔÉú³ÉÒ»¸ötimechart
sourcetype=cloud_watch_debug
| transaction
tid startswith="QueryStart" endswith="QueryEnd" |
timechart avg(duration) |
½á¹ûÈçÏ£¨×î½ü1Сʱ£©£º

ͨ¹ý¸Ã·ÖÎöÔÚ¹ýÈ¥µÄÒ»¸öСʱÀï10£º30ºÍ11£º00Ö®ºóµÄÊ®·ÖÖÓʱ¼ä¶Î£¬ºÄʱÂÔÓÐÉÏÉý£¬´ó¸Å·åÖµ0.2Ãë¡£
×ܽ᣺
SplunkµÄÈÕÖ¾·ÖÎö¹¦Äܷdz£Ç¿´ó£¬¶øÇÒ500MµÄÃâ·Ñ°æ»ù±¾Äܹ»Âú×ã´ó²¿·Ö³ÌÐòÔ±¶Ô³ÌÐòÈÕÖ¾µÄ·ÖÎöÒªÇó¡£ÓÐЧµØÊ¹ÓÃSplunkÀ´½øÐÐÈÕÖ¾·ÖÎö£¬¿ÉÒÔ×öµ½Ê°빦±¶£¬Ð¡»ï°éÃÇ¿ìÀ´ÊÔÓðɣ¡
|