±à¼ÍƼö: |
À´Ô´ÍøÂ磬elasticsearch
ÊÇÒ»¸ö¿ªÔ´·Ö²¼Ê½ËÑË÷ÒýÇæ£¬logstash£ºÊÇÒ»¸öÍêÈ«¿ªÔ´µÄ¹¤¾ß£¬kibana£ºÊÇÒ»¸ö¿ªÔ´ºÍÃâ·ÑµÄ¹¤¾ß£¬±¾ÎĽéÉÜÁËÓÃELKµÄÈÕÖ¾²É¼¯¡£ |
|
¸ÅÒª
´ø×ÅÎÊÌâÈ¥¿´½Ì³Ì£º
²»ÊÇÓÃlogstashÀ´¼àÌýÎÒÃǵÄÈÕÖ¾£¬ÎÒÃÇ¿ÉÒÔʹÓÃlogbackÅäÖÃÀ´Ê¹ÓÃTCP appenderͨ¹ýTCPÐÒ齫ÈÕÖ¾·¢Ë͵½Ô¶³ÌLogstashʵÀý¡£
ÎÒÃÇ¿ÉÒÔʹÓÃLogstashÖ¸Ïò¶à¸öÈÕÖ¾Îļþ¡£
ÎÒÃÇ¿ÉÒÔÔÚlogstashÅäÖÃÎļþÖÐʹÓøü¸´ÔӵĹýÂËÆ÷£¬ÒÔ¸ù¾ÝÐèÒªÖ´Ðиü¶à²Ù×÷¡£
ÎÒÃÇ¿ÉÒÔʹÓÃÔ¶³ÌELK¼¯ÈºÖ¸ÏòÎÒÃǵÄÈÕÖ¾Îļþ£¬»òÕß½«ÈÕÖ¾ÍÆÈ룬ÕâÔÚ½«ËùÓÐÓ¦ÓóÌÐò²¿Êðµ½ÔÆÖÐʱ»ù±¾ÉÏÊDZØÐèµÄ¡£
ÔÚlogstashÖд´½¨²»Í¬µÄË÷Òýģʽ¡£
ͨ¹ýʹÓÃ΢·þÎñ£¬ÎÒÃÇÒѾÄܹ»¿Ë·þÐí¶àÒÅÁôÎÊÌ⣬²¢ÇÒËüÔÊÐíÎÒÃÇ´´½¨Îȶ¨µÄ·Ö²¼Ê½Ó¦ÓóÌÐò£¬²¢¶Ô´úÂ룬ÍŶӹæÄ££¬Î¬»¤£¬·¢²¼ÖÜÆÚ£¬ÔƼÆËãµÈ½øÐÐËùÐèµÄ¿ØÖÆ¡£µ«ËüÒ²ÒýÈëÁËһЩÌôÕ½¡£ÆäËûÁìÓò£¬ÀýÈç·Ö²¼Ê½ÈÕÖ¾¹ÜÀíºÍ²é¿´ÔÚÐí¶à·þÎñÖзֲ¼µÄÍêÕûÊÂÎñµÄÈÕÖ¾ºÍÒ»°ãµÄ·Ö²¼Ê½µ÷ÊÔµÄÄÜÁ¦¡£
ʵ¼ÊÉÏ£¬ÌôÕ½ÔÚÓÚ΢·þÎñÊÇÏ໥¸ôÀëµÄ£¬ËüÃDz»¹²Ïí¹«¹²Êý¾Ý¿âºÍÈÕÖ¾Îļþ¡£Ëæ×Å΢·þÎñÊýÁ¿µÄÔö¼ÓÒÔ¼°ÎÒÃÇʹÓÃ×Ô¶¯»¯³ÖÐø¼¯³É¹¤¾ßʵÏÖÔÆ²¿Ê𣬵±ÎÒÃÇÓöµ½ÈκÎÎÊÌâʱ£¬·Ç³£ÓбØÒª¶Ô×é¼þ½øÐÐһЩµ÷ÊÔ¡£
¸Ðл¿ªÔ´Ô˶¯¡£ÎÒÃÇÒѾӵÓÐÁËһϵÁй¤¾ß£¬Èç¹ûÒ»ÆðʹÓÿÉÒÔ·¢»ÓħÁ¦¡£Ò»×éÈç´ËÊÜ»¶ÓµÄ¹¤¾ßÊÇElastic
Search£¬LogstashºÍKibana - Ò»Æð³ÆÎªELK Stack ¡£ËüÃÇÓÃÓÚʵʱËÑË÷£¬·ÖÎöºÍ¿ÉÊÓ»¯ÈÕÖ¾Êý¾Ý¡£
ELK Stack
ElasticsearchÊÇÒ»¸ö»ùÓÚJSONµÄ·Ö²¼Ê½ËÑË÷ºÍ·ÖÎöÒýÇæ£¬×¨ÎªË®Æ½¿ÉÀ©Õ¹ÐÔ£¬×î¸ß¿É¿¿ÐÔºÍÒ×¹ÜÀíÐÔ¶øÉè¼Æ¡£
LogstashÊÇÒ»¸ö¶¯Ì¬Êý¾ÝÊÕ¼¯¹ÜµÀ£¬¾ßÓпÉÀ©Õ¹µÄ²å¼þÉú̬ϵͳºÍÇ¿´óµÄElasticsearchÐͬ×÷Óá£
Kibanaͨ¹ýUI ÌṩÊý¾Ý¿ÉÊÓ»¯¡£
ELK Stack¼Ü¹¹
Logstash¸ù¾ÝÎÒÃÇÉèÖõĹýÂËÌõ¼þ´¦ÀíÓ¦ÓóÌÐòÈÕÖ¾Îļþ£¬²¢½«ÕâЩÈÕÖ¾·¢Ë͵½Elasticsearch¡£Í¨¹ýKibana£¬ÎÒÃÇ¿ÉÒÔÔÚÐèҪʱ²é¿´ºÍ·ÖÎöÕâЩÈÕÖ¾¡£

ELKÅäÖÃ
ËùÓÐÕâÈý¸ö¹¤¾ß¶¼»ùÓÚJVM£¬ÔÚ¿ªÊ¼°²×°Ö®Ç°£¬ÇëÑéÖ¤JDKÊÇ·ñÒÑÕýÈ·ÅäÖ᣼ì²é±ê×¼JDK 1.8°²×°£¬JAVA_HOME²¢ÇÒPATHÒѾÍê³ÉÉèÖá£
Elasticsearch
´Ó´ËÏÂÔØÒ³ÃæÏÂÔØ×îа汾µÄElasticsearch ²¢½«Æä½âѹËõµ½ÈκÎÎļþ¼ÐÖС£
bin\elasticsearch.bat´ÓÃüÁîÌáʾ·ûÔËÐС£
ĬÈÏÇé¿öÏ£¬Ëü½«´Óhttp£º// localhost£º9200
Kibana
´ÓÏÂÔØÒ³ÃæÏÂÔØ×îеķ¢Ðа沢½âѹËõµ½ÈκÎÎļþ¼ÐÖС£
config/kibana.ymlÔÚ±à¼Æ÷Öдò¿ª²¢ÉèÖÃelasticsearch.urlΪָÏòÄúµÄElasticsearchʵÀý¡£ÔÚÎÒÃǵÄÀý×ÓÖУ¬ÎÒÃǽ«Ê¹Óñ¾µØÊµÀýÈ¡Ïû×¢ÊÍelasticsearch.url:
"http://localhost:9200"
bin\kibana.bat´ÓÃüÁîÌáʾ·ûÔËÐС£
³É¹¦Æô¶¯ºó£¬Kibana½«Æô¶¯Ä¬È϶˿Ú5601£¬Kibana UI½«ÔÚhttp£º// localhost£º5601ÉÏÌṩ
Logstash
´ÓÏÂÔØÒ³ÃæÏÂÔØ×îеķ¢Ðа沢½âѹËõµ½ÈκÎÎļþ¼ÐÖС£
logstash.conf¸ù¾ÝÅäÖÃ˵Ã÷´´½¨Ò»¸öÎļþ¡£ÔÚʵ¼ÊÑÝʾʱ¼äÄÚ£¬ÎÒÃǽ«ÔٴνøÐо«È·ÅäÖá£
ÏÖÔÚÔËÐÐbin/logstash -f logstash.confÒÔÆô¶¯logstash
ELK¶ÑջδÆô¶¯²¢ÕýÔÚÔËÐС£ÏÖÔÚÎÒÃÇÐèÒª´´½¨Ò»Ð©Î¢·þÎñ²¢Ö¸ÏòAPIÈÕ־·¾¶µÄlogstash¡£
´´½¨Î¢·þÎñ
´´½¨Spring BootÏîÄ¿
ÈÃÎÒÃÇʹÓÃspring boot´´½¨Ò»¸öÓ¦ÓóÌÐò£¬ÒÔËõ¶Ì¿ª·¢Ê±¼ä¡£Çë°´ÕÕÕâЩ²½ÖèÆô¶¯´Ë·þÎñ¡£
Ìí¼ÓREST¶Ëµã
Ìí¼ÓÒ»¸öRestControllerÀà»á±©Â¶Ò»Ð©¶ËµãÒ»Ñù/elk£¬/elkdemo£¬/exception¡£Êµ¼ÊÉÏÎÒÃÇÖ»»á²âÊÔ¼¸¸öÈÕÖ¾Óï¾ä£¬Òò´Ë¿ÉÒÔ¸ù¾ÝÄúµÄÑ¡ÔñËæÒâÌí¼Ó/ÐÞ¸ÄÈÕÖ¾¡£
package com.example.howtodoinjava.elkexamplespringboot;
import java.io.PrintWriter;
import java.io.StringWriter;
import java.util.Date;
import org.apache.log4j.Level;
import org.apache.log4j.Logger;
import org.springframework.beans .factory.annotation.Autowired;
import org.springframework.boot .SpringApplication;
import org.springframework.boot .autoconfigure.SpringBootApplication;
import org.springframework .context.annotation.Bean;
import org.springframework .core.ParameterizedTypeReference;
import org.springframework .http.HttpMethod;
import org.springframework .web.bind.annotation.RequestMapping;
import org.springframework .web.bind.annotation.RestController;
import org.springframework .web.client.RestTemplate;
@SpringBootApplication
public class ElkExampleSpringBootApplication {
public static void main(String[] args) {
SpringApplication.run (ElkExampleSpringBootApplication.class,
args);
}
}
@RestController
class ELKController {
private static final Logger LOG = Logger.getLogger (ELKController.class.getName());
@Autowired
RestTemplate restTemplete;
@Bean
RestTemplate restTemplate() {
return new RestTemplate();
}
@RequestMapping(value = "/elkdemo")
public String helloWorld() {
String response = "Hello user ! " +
new Date();
LOG.log(Level.INFO, "/elkdemo - >
" + response);
return response;
}
@RequestMapping(value = "/elk")
public String helloWorld1() {
String response = restTemplete.exchange ("http://localhost:8080/elkdemo",
HttpMethod.GET, null, newParameterizedTypeReference()
{
}).getBody();
LOG.log (Level.INFO, "/elk - > "
+ response);
try {
String exceptionrsp = restTemplete.exchange ("http://localhost:8080/exception",
HttpMethod.GET, null,new ParameterizedTypeReference()
{
}).getBody();
LOG.log(Level.INFO, "/elk trying to print
exception - > " + exceptionrsp);
response = response + " === " + exceptionrsp;
} catch (Exception e) {
// exception should not reach here. Really bad
practice :)
}
return response;
}
@RequestMapping (value = "/exception")
public String exception() {
String rsp = "";
try {
int i = 1 / 0;
// should get exception
} catch (Exception e) {
e.printStackTrace();
LOG.error(e);
StringWriter sw = new StringWriter();
PrintWriter pw = new PrintWriter(sw);
e.printStackTrace (pw);
String sStackTrace = sw.toString(); // stack trace
as a string
LOG.error(" Exception As String :: - >
"+sStackTrace);
rsp = sStackTrace;
}
return rsp;
}
} |
ÅäÖÃSpringÒýµ¼¼Ç¼
application.propertiesÔÚresourcesÎļþ¼ÐÏ´ò¿ª²¢Ìí¼ÓÒÔÏÂÅäÖÃÌõÄ¿¡£
logging.file=elk-example.log
spring.application.name = elk-example |
Ñé֤΢·þÎñÉú³ÉµÄÈÕÖ¾
ͨ¹ýä¯ÀÀhttp£º// localhost£º8080 / elk£¬mvn clean installʹÓÃÃüÁîjava
-jar target\elk-example-spring-boot-0.0.1-SNAPSHOT.jarºÍ²âÊÔÀ´Ö´ÐÐ×îÖÕµÄmaven¹¹½¨²¢Æô¶¯Ó¦ÓóÌÐò¡£
²»Òªº¦Å¿´µ½ÆÁÄ»ÉϵĴó¶ÑÕ»¸ú×Ù£¬ÒòΪÓÐÒâʶµØ¿´¿´ELKÈçºÎ´¦ÀíÒì³£ÏûÏ¢¡£
תµ½Ó¦ÓóÌÐò¸ùĿ¼²¢ÑéÖ¤ÊÇ·ñelk-example.logÒÑ´´½¨ÈÕÖ¾Îļþ£¬²¢¶Ô¶ËµãÖ´Ðм¸´Î·ÃÎÊ£¬²¢ÑéÖ¤ÈÕÖ¾ÎļþÖÐÊÇ·ñÌí¼ÓÁËÈÕÖ¾¡£
LogstashÅäÖÃ
ÎÒÃÇÐèÒª´´½¨Ò»¸ölogstashÅäÖÃÎļþ£¬ÒÔ±ãËü¼àÌýÈÕÖ¾Îļþ²¢½«ÈÕÖ¾ÏûÏ¢ÍÆË͵½µ¯ÐÔËÑË÷¡£ÒÔÏÂÊÇʾÀýÖÐʹÓõÄlogstash
ÅäÖã¬Çë¸ù¾ÝÄúµÄÉèÖøü¸ÄÈÕ־·¾¶¡£
input {
file {
type => "java"
path => "F:/Study/eclipse_workspace_mars/elk-example
-spring-boot/elk-example.log"
codec => multiline {
pattern => "^%{YEAR}-%{MONTHNUM}-%{MONTHDAY}
%{TIME}.*"
negate => "true"
what => "previous"
}
}
}
filter {
#If log line contains tab character followed by
'at' then we will tag that entry as stacktrace
if [message] =~ " \tat" {
grok {
match => [" message", "^(\tat)"]
add_tag => ["stacktrace"]
}
}
grok {
match => [ "message", "(?<timestamp>%{YEAR}-% {MONTHNUM}-%{MONTHDAY}
%{TIME}) %{LOGLEVEL:level} %{NUMBER:pid} --- \[(?<thread>[A-Za-z0-9-]+)\]
[A-Za-z0-9.]*\.(?<class>[A-Za-z0-9#_]+)\s*:\s+(?<logmessage>.*)",
"message", "(?<timestamp>%{YEAR}-%{MONTHNUM}-%{MONTHDAY}
%{TIME}) %{LOGLEVEL:level} %{NUMBER:pid} --- .+?
:\s+(?<logmessage>.*)"
]
}
date {
match => [ "timestamp" , "yyyy-MM-dd
HH:mm:ss.SSS" ]
}
}
output {
stdout {
codec => rubydebug
}
# Sending properly parsed log events to elasticsearch
elasticsearch {
hosts => ["localhost:9200"]
}
} |
KibanaÅäÖÃ
Ôڲ鿴KibanaÖеÄÈÕ־֮ǰ£¬ÎÒÃÇÐèÒªÅäÖÃË÷Òýģʽ¡£ÎÒÃÇ¿ÉÒÔÅäÖÃlogstash-*ΪĬÈÏÅäÖá£ÎÒÃÇ×ÜÊÇ¿ÉÒÔÔÚlogstash¶Ë¸ü¸Ä´ËË÷Òýģʽ²¢ÔÚKibanaÖнøÐÐÅäÖá£Îª¼òµ¥Æð¼û£¬ÎÒÃǽ«Ê¹ÓÃĬÈÏÅäÖá£
Ë÷Òýģʽ¹ÜÀíÒ³ÃæÈçÏÂËùʾ¡£Í¨¹ýÕâÖÖÅäÖã¬ÎÒÃǽ«KibanaÖ¸ÏòÄúÑ¡ÔñµÄElasticsearchË÷Òý¡£LogstashʹÓÃÃû³ÆÄ£Ê½´´½¨Ë÷Òý¡£logstash-YYYY.MM.DDÎÒÃÇ¿ÉÒÔÔÚKibana¿ØÖÆÌ¨http£º//
localhost£º5601 / app / kibanaÖÐÖ´ÐÐËùÓÐÕâЩÅäÖã¬È»ºóתµ½×ó²àÃæ°åÖеÄManagementÁ´½Ó¡£

ÑéÖ¤ELK
ÏÖÔÚ£¬µ±ËùÓÐ×é¼þ¶¼Æô¶¯²¢ÔËÐÐʱ£¬ÈÃÎÒÃÇÑéÖ¤Õû¸öÉú̬ϵͳ¡£
תµ½Ó¦ÓóÌÐò²¢²âÊԶ˵㼸´ÎÒÔ±ãÉú³ÉÈÕÖ¾£¬È»ºóתµ½Kibana¿ØÖÆÌ¨£¬¿´¿´ÈÕÖ¾ÊÇ·ñÕýÈ·¶ÑµþÔÚKibanaÖУ¬»¹ÓÐÐí¶à¶îÍâµÄ¹¦ÄÜ£¬±ÈÈçÎÒÃÇ¿ÉÒÔ¹ýÂË£¬²é¿´ÄÚÖõIJ»Í¬Í¼±íµÈ¡£
ÒÔÏÂÊÇKibanaÖÐÉú³ÉµÄÈÕÖ¾µÄÊÓͼ¡£

|