Äú¿ÉÒÔ¾èÖú£¬Ö§³ÖÎÒÃǵĹ«ÒæÊÂÒµ¡£

1Ôª 10Ôª 50Ôª





ÈÏÖ¤Â룺  ÑéÖ¤Âë,¿´²»Çå³þ?Çëµã»÷Ë¢ÐÂÑéÖ¤Âë ±ØÌî



  ÇóÖª ÎÄÕ ÎÄ¿â Lib ÊÓÆµ iPerson ¿Î³Ì ÈÏÖ¤ ×Éѯ ¹¤¾ß ½²×ù Model Center   Code  
»áÔ±   
   
 
     
   
 ¶©ÔÄ
  ¾èÖú
ELK ÈÕÖ¾²É¼¯ ʵս½Ì³Ì
 
  1997  次浏览      27
 2019-1-15 
 
±à¼­ÍƼö:
À´Ô´ÍøÂ磬elasticsearch ÊÇÒ»¸ö¿ªÔ´·Ö²¼Ê½ËÑË÷ÒýÇæ£¬logstash£ºÊÇÒ»¸öÍêÈ«¿ªÔ´µÄ¹¤¾ß£¬kibana£ºÊÇÒ»¸ö¿ªÔ´ºÍÃâ·ÑµÄ¹¤¾ß£¬±¾ÎĽéÉÜÁËÓÃELKµÄÈÕÖ¾²É¼¯¡£

¸ÅÒª

´ø×ÅÎÊÌâÈ¥¿´½Ì³Ì£º

²»ÊÇÓÃlogstashÀ´¼àÌýÎÒÃǵÄÈÕÖ¾£¬ÎÒÃÇ¿ÉÒÔʹÓÃlogbackÅäÖÃÀ´Ê¹ÓÃTCP appenderͨ¹ýTCPЭÒ齫ÈÕÖ¾·¢Ë͵½Ô¶³ÌLogstashʵÀý¡£

ÎÒÃÇ¿ÉÒÔʹÓÃLogstashÖ¸Ïò¶à¸öÈÕÖ¾Îļþ¡£

ÎÒÃÇ¿ÉÒÔÔÚlogstashÅäÖÃÎļþÖÐʹÓøü¸´ÔӵĹýÂËÆ÷£¬ÒÔ¸ù¾ÝÐèÒªÖ´Ðиü¶à²Ù×÷¡£

ÎÒÃÇ¿ÉÒÔʹÓÃÔ¶³ÌELK¼¯ÈºÖ¸ÏòÎÒÃǵÄÈÕÖ¾Îļþ£¬»òÕß½«ÈÕÖ¾ÍÆÈ룬ÕâÔÚ½«ËùÓÐÓ¦ÓóÌÐò²¿Êðµ½ÔÆÖÐʱ»ù±¾ÉÏÊDZØÐèµÄ¡£

ÔÚlogstashÖд´½¨²»Í¬µÄË÷Òýģʽ¡£

ͨ¹ýʹÓÃ΢·þÎñ£¬ÎÒÃÇÒѾ­Äܹ»¿Ë·þÐí¶àÒÅÁôÎÊÌ⣬²¢ÇÒËüÔÊÐíÎÒÃÇ´´½¨Îȶ¨µÄ·Ö²¼Ê½Ó¦ÓóÌÐò£¬²¢¶Ô´úÂ룬ÍŶӹæÄ££¬Î¬»¤£¬·¢²¼ÖÜÆÚ£¬ÔƼÆËãµÈ½øÐÐËùÐèµÄ¿ØÖÆ¡£µ«ËüÒ²ÒýÈëÁËһЩÌôÕ½¡£ÆäËûÁìÓò£¬ÀýÈç·Ö²¼Ê½ÈÕÖ¾¹ÜÀíºÍ²é¿´ÔÚÐí¶à·þÎñÖзֲ¼µÄÍêÕûÊÂÎñµÄÈÕÖ¾ºÍÒ»°ãµÄ·Ö²¼Ê½µ÷ÊÔµÄÄÜÁ¦¡£

ʵ¼ÊÉÏ£¬ÌôÕ½ÔÚÓÚ΢·þÎñÊÇÏ໥¸ôÀëµÄ£¬ËüÃDz»¹²Ïí¹«¹²Êý¾Ý¿âºÍÈÕÖ¾Îļþ¡£Ëæ×Å΢·þÎñÊýÁ¿µÄÔö¼ÓÒÔ¼°ÎÒÃÇʹÓÃ×Ô¶¯»¯³ÖÐø¼¯³É¹¤¾ßʵÏÖÔÆ²¿Ê𣬵±ÎÒÃÇÓöµ½ÈκÎÎÊÌâʱ£¬·Ç³£ÓбØÒª¶Ô×é¼þ½øÐÐһЩµ÷ÊÔ¡£

¸Ðл¿ªÔ´Ô˶¯¡£ÎÒÃÇÒѾ­ÓµÓÐÁËһϵÁй¤¾ß£¬Èç¹ûÒ»ÆðʹÓÿÉÒÔ·¢»ÓħÁ¦¡£Ò»×éÈç´ËÊÜ»¶Ó­µÄ¹¤¾ßÊÇElastic Search£¬LogstashºÍKibana - Ò»Æð³ÆÎªELK Stack ¡£ËüÃÇÓÃÓÚʵʱËÑË÷£¬·ÖÎöºÍ¿ÉÊÓ»¯ÈÕÖ¾Êý¾Ý¡£

ELK Stack

ElasticsearchÊÇÒ»¸ö»ùÓÚJSONµÄ·Ö²¼Ê½ËÑË÷ºÍ·ÖÎöÒýÇæ£¬×¨ÎªË®Æ½¿ÉÀ©Õ¹ÐÔ£¬×î¸ß¿É¿¿ÐÔºÍÒ×¹ÜÀíÐÔ¶øÉè¼Æ¡£

LogstashÊÇÒ»¸ö¶¯Ì¬Êý¾ÝÊÕ¼¯¹ÜµÀ£¬¾ßÓпÉÀ©Õ¹µÄ²å¼þÉú̬ϵͳºÍÇ¿´óµÄElasticsearchЭͬ×÷Óá£

Kibanaͨ¹ýUI ÌṩÊý¾Ý¿ÉÊÓ»¯¡£

ELK Stack¼Ü¹¹

Logstash¸ù¾ÝÎÒÃÇÉèÖõĹýÂËÌõ¼þ´¦ÀíÓ¦ÓóÌÐòÈÕÖ¾Îļþ£¬²¢½«ÕâЩÈÕÖ¾·¢Ë͵½Elasticsearch¡£Í¨¹ýKibana£¬ÎÒÃÇ¿ÉÒÔÔÚÐèҪʱ²é¿´ºÍ·ÖÎöÕâЩÈÕÖ¾¡£

ELKÅäÖÃ

ËùÓÐÕâÈý¸ö¹¤¾ß¶¼»ùÓÚJVM£¬ÔÚ¿ªÊ¼°²×°Ö®Ç°£¬ÇëÑéÖ¤JDKÊÇ·ñÒÑÕýÈ·ÅäÖ᣼ì²é±ê×¼JDK 1.8°²×°£¬JAVA_HOME²¢ÇÒPATHÒѾ­Íê³ÉÉèÖá£

Elasticsearch

´Ó´ËÏÂÔØÒ³ÃæÏÂÔØ×îа汾µÄElasticsearch ²¢½«Æä½âѹËõµ½ÈκÎÎļþ¼ÐÖС£

bin\elasticsearch.bat´ÓÃüÁîÌáʾ·ûÔËÐС£

ĬÈÏÇé¿öÏ£¬Ëü½«´Óhttp£º// localhost£º9200

Kibana

´ÓÏÂÔØÒ³ÃæÏÂÔØ×îеķ¢Ðа沢½âѹËõµ½ÈκÎÎļþ¼ÐÖС£

config/kibana.ymlÔڱ༭Æ÷Öдò¿ª²¢ÉèÖÃelasticsearch.urlΪָÏòÄúµÄElasticsearchʵÀý¡£ÔÚÎÒÃǵÄÀý×ÓÖУ¬ÎÒÃǽ«Ê¹Óñ¾µØÊµÀýÈ¡Ïû×¢ÊÍelasticsearch.url: "http://localhost:9200"

bin\kibana.bat´ÓÃüÁîÌáʾ·ûÔËÐС£

³É¹¦Æô¶¯ºó£¬Kibana½«Æô¶¯Ä¬È϶˿Ú5601£¬Kibana UI½«ÔÚhttp£º// localhost£º5601ÉÏÌṩ

Logstash

´ÓÏÂÔØÒ³ÃæÏÂÔØ×îеķ¢Ðа沢½âѹËõµ½ÈκÎÎļþ¼ÐÖС£

logstash.conf¸ù¾ÝÅäÖÃ˵Ã÷´´½¨Ò»¸öÎļþ¡£ÔÚʵ¼ÊÑÝʾʱ¼äÄÚ£¬ÎÒÃǽ«ÔٴνøÐо«È·ÅäÖá£

ÏÖÔÚÔËÐÐbin/logstash -f logstash.confÒÔÆô¶¯logstash

ELK¶ÑջδÆô¶¯²¢ÕýÔÚÔËÐС£ÏÖÔÚÎÒÃÇÐèÒª´´½¨Ò»Ð©Î¢·þÎñ²¢Ö¸ÏòAPIÈÕ־·¾¶µÄlogstash¡£

´´½¨Î¢·þÎñ

´´½¨Spring BootÏîÄ¿

ÈÃÎÒÃÇʹÓÃspring boot´´½¨Ò»¸öÓ¦ÓóÌÐò£¬ÒÔËõ¶Ì¿ª·¢Ê±¼ä¡£Çë°´ÕÕÕâЩ²½ÖèÆô¶¯´Ë·þÎñ¡£

Ìí¼ÓREST¶Ëµã

Ìí¼ÓÒ»¸öRestControllerÀà»á±©Â¶Ò»Ð©¶ËµãÒ»Ñù/elk£¬/elkdemo£¬/exception¡£Êµ¼ÊÉÏÎÒÃÇÖ»»á²âÊÔ¼¸¸öÈÕÖ¾Óï¾ä£¬Òò´Ë¿ÉÒÔ¸ù¾ÝÄúµÄÑ¡ÔñËæÒâÌí¼Ó/ÐÞ¸ÄÈÕÖ¾¡£

package com.example.howtodoinjava.elkexamplespringboot;

import java.io.PrintWriter;
import java.io.StringWriter;
import java.util.Date;

import org.apache.log4j.Level;
import org.apache.log4j.Logger;
import org.springframework.beans .factory.annotation.Autowired;
import org.springframework.boot .SpringApplication;
import org.springframework.boot .autoconfigure.SpringBootApplication;
import org.springframework .context.annotation.Bean;
import org.springframework .core.ParameterizedTypeReference;
import org.springframework .http.HttpMethod;
import org.springframework .web.bind.annotation.RequestMapping;
import org.springframework .web.bind.annotation.RestController;
import org.springframework .web.client.RestTemplate;

@SpringBootApplication
public class ElkExampleSpringBootApplication {

public static void main(String[] args) {
SpringApplication.run (ElkExampleSpringBootApplication.class, args);
}
}

@RestController
class ELKController {
private static final Logger LOG = Logger.getLogger (ELKController.class.getName());

@Autowired
RestTemplate restTemplete;

@Bean
RestTemplate restTemplate() {
return new RestTemplate();
}

@RequestMapping(value = "/elkdemo")
public String helloWorld() {
String response = "Hello user ! " + new Date();
LOG.log(Level.INFO, "/elkdemo - > " + response);

return response;
}

@RequestMapping(value = "/elk")
public String helloWorld1() {

String response = restTemplete.exchange ("http://localhost:8080/elkdemo", HttpMethod.GET, null, newParameterizedTypeReference() {
}).getBody();
LOG.log (Level.INFO, "/elk - > " + response);

try {
String exceptionrsp = restTemplete.exchange ("http://localhost:8080/exception", HttpMethod.GET, null,new ParameterizedTypeReference() {
}).getBody();
LOG.log(Level.INFO, "/elk trying to print exception - > " + exceptionrsp);
response = response + " === " + exceptionrsp;
} catch (Exception e) {
// exception should not reach here. Really bad practice :)
}

return response;
}

@RequestMapping (value = "/exception")
public String exception() {
String rsp = "";
try {
int i = 1 / 0;
// should get exception
} catch (Exception e) {
e.printStackTrace();
LOG.error(e);

StringWriter sw = new StringWriter();
PrintWriter pw = new PrintWriter(sw);
e.printStackTrace (pw);
String sStackTrace = sw.toString(); // stack trace as a string
LOG.error(" Exception As String :: - > "+sStackTrace);

rsp = sStackTrace;
}

return rsp;
}
}

 

ÅäÖÃSpringÒýµ¼¼Ç¼

application.propertiesÔÚresourcesÎļþ¼ÐÏ´ò¿ª²¢Ìí¼ÓÒÔÏÂÅäÖÃÌõÄ¿¡£

logging.file=elk-example.log
spring.application.name = elk-example

 

Ñé֤΢·þÎñÉú³ÉµÄÈÕÖ¾

ͨ¹ýä¯ÀÀhttp£º// localhost£º8080 / elk£¬mvn clean installʹÓÃÃüÁîjava -jar target\elk-example-spring-boot-0.0.1-SNAPSHOT.jarºÍ²âÊÔÀ´Ö´ÐÐ×îÖÕµÄmaven¹¹½¨²¢Æô¶¯Ó¦ÓóÌÐò¡£

²»Òªº¦Å¿´µ½ÆÁÄ»ÉϵĴó¶ÑÕ»¸ú×Ù£¬ÒòΪÓÐÒâʶµØ¿´¿´ELKÈçºÎ´¦ÀíÒì³£ÏûÏ¢¡£

תµ½Ó¦ÓóÌÐò¸ùĿ¼²¢ÑéÖ¤ÊÇ·ñelk-example.logÒÑ´´½¨ÈÕÖ¾Îļþ£¬²¢¶Ô¶ËµãÖ´Ðм¸´Î·ÃÎÊ£¬²¢ÑéÖ¤ÈÕÖ¾ÎļþÖÐÊÇ·ñÌí¼ÓÁËÈÕÖ¾¡£

LogstashÅäÖÃ

ÎÒÃÇÐèÒª´´½¨Ò»¸ölogstashÅäÖÃÎļþ£¬ÒÔ±ãËü¼àÌýÈÕÖ¾Îļþ²¢½«ÈÕÖ¾ÏûÏ¢ÍÆË͵½µ¯ÐÔËÑË÷¡£ÒÔÏÂÊÇʾÀýÖÐʹÓõÄlogstash ÅäÖã¬Çë¸ù¾ÝÄúµÄÉèÖøü¸ÄÈÕ־·¾¶¡£

input {
file {
type => "java"
path => "F:/Study/eclipse_workspace_mars/elk-example -spring-boot/elk-example.log"
codec => multiline {
pattern => "^%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{TIME}.*"
negate => "true"
what => "previous"
}
}
}

filter {
#If log line contains tab character followed by 'at' then we will tag that entry as stacktrace
if [message] =~ " \tat" {
grok {
match => [" message", "^(\tat)"]
add_tag => ["stacktrace"]
}
}

grok {
match => [ "message",
"(?<timestamp>%{YEAR}-% {MONTHNUM}-%{MONTHDAY} %{TIME}) %{LOGLEVEL:level} %{NUMBER:pid} --- \[(?<thread>[A-Za-z0-9-]+)\] [A-Za-z0-9.]*\.(?<class>[A-Za-z0-9#_]+)\s*:\s+(?<logmessage>.*)",
"message",
"(?<timestamp>%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{TIME}) %{LOGLEVEL:level} %{NUMBER:pid} --- .+? :\s+(?<logmessage>.*)"
]
}


date {
match => [ "timestamp" , "yyyy-MM-dd HH:mm:ss.SSS" ]
}
}

output {

stdout {
codec => rubydebug
}

# Sending properly parsed log events to elasticsearch
elasticsearch {
hosts => ["localhost:9200"]
}
}

KibanaÅäÖÃ

Ôڲ鿴KibanaÖеÄÈÕ־֮ǰ£¬ÎÒÃÇÐèÒªÅäÖÃË÷Òýģʽ¡£ÎÒÃÇ¿ÉÒÔÅäÖÃlogstash-*ΪĬÈÏÅäÖá£ÎÒÃÇ×ÜÊÇ¿ÉÒÔÔÚlogstash¶Ë¸ü¸Ä´ËË÷Òýģʽ²¢ÔÚKibanaÖнøÐÐÅäÖá£Îª¼òµ¥Æð¼û£¬ÎÒÃǽ«Ê¹ÓÃĬÈÏÅäÖá£

Ë÷Òýģʽ¹ÜÀíÒ³ÃæÈçÏÂËùʾ¡£Í¨¹ýÕâÖÖÅäÖã¬ÎÒÃǽ«KibanaÖ¸ÏòÄúÑ¡ÔñµÄElasticsearchË÷Òý¡£LogstashʹÓÃÃû³ÆÄ£Ê½´´½¨Ë÷Òý¡£logstash-YYYY.MM.DDÎÒÃÇ¿ÉÒÔÔÚKibana¿ØÖÆÌ¨http£º// localhost£º5601 / app / kibanaÖÐÖ´ÐÐËùÓÐÕâЩÅäÖã¬È»ºóתµ½×ó²àÃæ°åÖеÄManagementÁ´½Ó¡£

ÑéÖ¤ELK

ÏÖÔÚ£¬µ±ËùÓÐ×é¼þ¶¼Æô¶¯²¢ÔËÐÐʱ£¬ÈÃÎÒÃÇÑéÖ¤Õû¸öÉú̬ϵͳ¡£

תµ½Ó¦ÓóÌÐò²¢²âÊԶ˵㼸´ÎÒÔ±ãÉú³ÉÈÕÖ¾£¬È»ºóתµ½Kibana¿ØÖÆÌ¨£¬¿´¿´ÈÕÖ¾ÊÇ·ñÕýÈ·¶ÑµþÔÚKibanaÖУ¬»¹ÓÐÐí¶à¶îÍâµÄ¹¦ÄÜ£¬±ÈÈçÎÒÃÇ¿ÉÒÔ¹ýÂË£¬²é¿´ÄÚÖõIJ»Í¬Í¼±íµÈ¡£

ÒÔÏÂÊÇKibanaÖÐÉú³ÉµÄÈÕÖ¾µÄÊÓͼ¡£

   
1997 ´Îä¯ÀÀ       27
Ïà¹ØÎÄÕÂ

»ùÓÚEAµÄÊý¾Ý¿â½¨Ä£
Êý¾ÝÁ÷½¨Ä££¨EAÖ¸ÄÏ£©
¡°Êý¾Ýºþ¡±£º¸ÅÄî¡¢ÌØÕ÷¡¢¼Ü¹¹Óë°¸Àý
ÔÚÏßÉ̳ÇÊý¾Ý¿âϵͳÉè¼Æ ˼·+Ч¹û
 
Ïà¹ØÎĵµ

GreenplumÊý¾Ý¿â»ù´¡Åàѵ
MySQL5.1ÐÔÄÜÓÅ»¯·½°¸
ijµçÉÌÊý¾ÝÖÐ̨¼Ü¹¹Êµ¼ù
MySQL¸ßÀ©Õ¹¼Ü¹¹Éè¼Æ
Ïà¹Ø¿Î³Ì

Êý¾ÝÖÎÀí¡¢Êý¾Ý¼Ü¹¹¼°Êý¾Ý±ê×¼
MongoDBʵս¿Î³Ì
²¢·¢¡¢´óÈÝÁ¿¡¢¸ßÐÔÄÜÊý¾Ý¿âÉè¼ÆÓëÓÅ»¯
PostgreSQLÊý¾Ý¿âʵսÅàѵ