Äú¿ÉÒÔ¾èÖú£¬Ö§³ÖÎÒÃǵĹ«ÒæÊÂÒµ¡£

1Ôª 10Ôª 50Ôª





ÈÏÖ¤Â룺  ÑéÖ¤Âë,¿´²»Çå³þ?Çëµã»÷Ë¢ÐÂÑéÖ¤Âë ±ØÌî



  ÇóÖª ÎÄÕ ÎÄ¿â Lib ÊÓÆµ iPerson ¿Î³Ì ÈÏÖ¤ ×Éѯ ¹¤¾ß ½²×ù Model Center   Code  
»áÔ±   
   
 
     
   
 ¶©ÔÄ
  ¾èÖú
×î»ðʵʱ´óÊý¾Ý OLAP ¼¼ÊõÔ­ÀíºÍʵ¼ù
 
×÷ÕߣºÁõ÷èÙš ·¢²¼ÓÚ£º2017-10-10
  5726  次浏览      27
 

DruidÊÇʲô

DruidÔÚ´óÊý¾ÝÁìÓòÒѾ­²»ÊÇÐÂÈËÁË£¬Òò´Ë¿ÉÄܺܶà¶ÁÕß¶¼ÒѾ­Ìý˵¹ýDruid£¬ÉõÖÁÓùýDruid£¬µ«ÊÇδ±ØÃ¿¸öÈ˶¼ÕæÕýÇåÎúµØÁ˽âDruidµ½µ×ÊÇʲô£¬ÒÔ¼°ÔÚʲôÇé¿öÏ¿ÉÒÔÓÃDruid¡£Í¬Ê±£¬ÎªÁ˱ÜÃâ´ó¼ÒÌýÁ˰ëÌ죬ȴһֱÏÝÔÚ¸÷ÖÖϸ½ÚÖе«ÈÔÈ»²»ÖªµÀµ½µ×ÔÚÌýʲô¶«Î÷£¬ÎÒÃÇ»¹ÊÇÓбØÒªÔÚ¿ªÊ¼µÄʱºòÏÈ×ÜÌå̸һ̸Druidµ½µ×ÊÇʲô¡£

¼òµ¥À´Ëµ£¬Druid ÊÇÒ»¸ö·Ö²¼Ê½µÄ¡¢Ö§³Öʵʱ¶àάOLAP·ÖÎöµÄÊý¾Ý´¦Àíϵͳ¡£Ëü¼ÈÖ§³Ö¸ßËÙµÄÊý¾ÝʵʱÉãÈë´¦Àí£¬Ò²Ö§³ÖʵʱÇÒÁé»îµÄ¶àάÊý¾Ý·ÖÎö²éѯ¡£Òò´ËDruid×î³£Óõij¡¾°¾ÍÊÇ´óÊý¾Ý±³¾°Ï¡¢Áé»î¿ìËٵĶàάOLAP·ÖÎö¡£ ÁíÍ⣬Druid»¹ÓÐÒ»¸ö¹Ø¼üµÄÌØµã£ºËüÖ§³Ö¸ù¾Ýʱ¼ä´Á¶ÔÊý¾Ý½øÐÐÔ¤¾ÛºÏÉãÈëºÍ¾ÛºÏ·ÖÎö£¬Òò´ËÒ²ÓÐÓû§¾­³£ÔÚÓÐʱÐòÊý¾Ý´¦Àí·ÖÎöµÄ³¡¾°ÖÐÓõ½Ëü¡£

DruidÓû§ÈºÄܹ»Ñ¸ËÙ·¢Õ¹µÄÒ»¸öÔ­ÒòÊÇËüÔÚ´óÊý¾Ý±³¾°Ï¼¯ÈºÒÀÈ»¾ß±¸ÓÅÐãµÄÐÔÄܺͿÉÀ©Õ¹ÐÔ¡£Imply.io¹«Ë¾ÊÇDruid´´Ê¼ÈË´´°ìµÄ¹«Ë¾£¬ÎÒÃÇ¿ÉÒÔÏÈͨ¹ýËü¹«²¼µÄһЩ¼¯ÈºÐÔÄܽéÉÜͼƬÀ´¸ÅÀÀÒ»ÏÂDruid¼¯ÈºµÄÒ»Ð©ÌØµã¡£

Êý¾ÝʵʱÏû·ÑÄÜÁ¦£º

¼¯ÈºµÄÊý¾Ý¹æÄ££º

²éѯËÙ¶È£º

²éѯµÄ²¢·¢Á¿£º

²»ÄÑ¿´³ö£¬Druid¼¯ÈºÔÚ´óÊý¾Ý³¡¾°Ï±íÏÖÓÅÒì¡£

½ÓÏÂÀ´£¬ÎÒÃÇҲ˳±ãÁÄÒ»ÁÄDruidµÄÀúÊ·¡£Druid µ¥´ÊÀ´Ô´ÓÚÎ÷·½¹ÅÂÞÂíµÄÉñ»°ÈËÎï, ÖÐÎij£³£·­Òë³ÉµÂ³ÒÁ¡£´«Ëµ Druid ½ÌÊ¿¾«Í¨Õ¼²·, ¶Ô¼ÀìëÀñÒÇһ˿²»¹¶,Ò²Éó¤ÓÚÌìÎÄÀú·¨¡¢Ò½Ò©¡¢ÌìÎĺÍÎÄѧµÈ¡£Í¬Ê±, Druid Ò² ÊÇÖ´·¨Õß¡¢Ò÷ÓÎÊ«È˺Í̽ÏռҵĴúÃû´Ê¡£ÃÀ¹ú¹ã¸æ¼¼Êõ¹«Ë¾ MetaMarkets ÓÚ 2011 Äê´´½¨ÁË Druid ÏîÄ¿, ²¢ÇÒÓÚ 2012 Ä꿪ԴÁË Druid ÏîÄ¿¡£ Druid Éè¼ÆÖ®³õµÄÏë·¨¾ÍÊÇΪ·ÖÎö¶øÉú,ËüÔÚ´¦ÀíÊý¾ÝµÄ¹æÄ£¡¢Êý¾Ý´¦ÀíµÄʵʱÐÔ·½Ãæ, ±È´«Í³µÄ OLAP ϵͳÓÐÁËÏÔÖøµÄÐÔÄܸĽø, ¶øÇÒÓµ±§Ö÷Á÷µÄ¿ªÔ´Éú̬, °üÀ¨ Hadoop µÈ¡£¶àÄêÒÔÀ´, Druid Ò»Ö±ÊǷdz£»îÔ¾µÄ¿ªÔ´ÏîÄ¿¡£ ÁíÍâ, °¢Àï°Í°ÍÒ²Ôø´´½¨¹ýÒ»¸ö¿ªÔ´ÏîÄ¿½Ð×÷ Druid(¼ò³Æ°¢Àï Druid), ËüÊÇÒ»¸öÊý¾Ý¿âÁ¬½Ó³ØµÄÏîÄ¿¡£°¢Àï Druid ºÍ±¾ÊéÌÖÂÛµÄ Druid ûÓÐÈκιØÏµ,ËüÃǽâ¾öÍêÈ«²»Í¬µÄÎÊÌâ¡£

Druid µÄ¹Ù·½ÍøÕ¾ÊÇ http://druid.io¡£

¼¼ÊõÑ¡Ð͵Ä˼¿¼

¸÷¸ö¹«Ë¾ÔÚ¼¼ÊõÑ¡Ð͵Äʱºò¶¼»á»õ±ÈÈý¼Ò£¬Òò´ËÔÚ¿¼²ìDruidµÄʱºòÒ²Ò»¶¨»áÎÊΪʲôÐèҪѡÔñDruid¡£ÔÚ±¾¶ÎÂäÖУ¬ÎÒÃǽ«»á¶ÔÓû§¾­³£»áÄÃÀ´ºÍDruid×ö±È½ÏµÄ×é¼þÖÐµÄÆäÖм¸¸öÀ´×öһЩ¼òµ¥µÄ˵Ã÷¡£µ«ÊÇÔڱȽÏ֮ǰ£¬ÎÒÐèÒª²ûÃ÷Ò»¸ö»ù±¾µÄ¹Ûµã£ºÕâЩ²»Í¬×é¼þ»ò¼¼ÊõÓи÷×ÔµÄÌØµãºÍÊÊÓ÷¶Î§£¬²¢Ã»ÓÐÒ»¸ö¼¼Êõ¾ÍÄܳ¹µ×Ìæ´úÆäËü¼¼ÊõÕâÖÖ˵·¨£¬¶ø±¾ÎľͽöÄá°´óÊý¾ÝÁ¿ÏµÄʵʱ¶àάOLAP·ÖÎö¡±ÕâÒ»³¡¾°À´×ö±È½Ï¡£

´«Í³¹ØÏµÐÍÊý¾Ý¿â£º

´«Í³¹ØÏµÐÍÊý¾Ý¿â£¨RDBMS£©ÀúÊ·¾ÃÔ¶¡¢¼¼Êõ³ÉÊ죬ӦÓÃÕ߹㷺£¬Òò´Ë²ÉÓõÄѧϰ³É±¾ºÍ¼¼Êõ·çÏÕÒ²Ïà¶Ô½ÏµÍ¡£µäÐ͵ÄRDBMSÓÐOracle¡¢DB2ºÍMySQLµÈ¡£´ÓÀíÂÛÉϽ²£¬ÒòΪRDBMSÄܱ£Ö¤Êý¾ÝµÄÁ¢¼´Ò»ÖÂÐÔ£¬Òò´ËÌØ±ðÊʺÏOLTPµÄ³¡¾°¡£È»¶øÓÉÓÚ×ÔÉíµÄÌØµãËùÏÞ£¬RDBMSÔÚ¿ÉÀ©Õ¹ÐÔ·½ÃæÈ´±íÏÖÇ·¼Ñ£¬ºÜÄÑÇáÒ׺͵ͳɱ¾µÍ½øÐÐÏßÐÔÀ©Õ¹ÒÔ´¦Àí¸ü´óµÄÊý¾ÝÁ¿¡£¼´±ãÏÖÔÚ´ó¶àÊý¾Ý¿â¿ÉÒÔͨ¹ý·Ö¿â·Ö±íµÈ·½·¨À´Ê¹×Ô¼º¿ÉÒÔÈÝÄɸü¶àµÄÊý¾Ý£¬µ«ÊÇÕâÖÖ·½Ê½Ò²´æÔÚ׏ÜÀí¸´ÔӺͳɱ¾¸ß°ºµÈÃ÷ÏԶ̰壬Òò´Ëʵ¼ÊÓ¦ÓÃÖÐÒ²´æÔÚ×ÅÊý¾ÝÁ¿µÄÌ컨°å¡£Òò´Ë£¬¿¼ÂǼȶ¨µÄ³¡¾°¡°´óÊý¾ÝÁ¿ÏµÄʵʱ¶àάOLAP·ÖÎö¡±£¬RDBMSÊ×ÏȾͺÜÄÑ¿ç¹ý¡°´óÊý¾ÝÁ¿¡±ÕâÒ»¹Ø¡£

Massively Parallel Processing Êý¾Ý¿â£º

´ó¹æÄ£²¢Ðд¦Àí£¨MPP£¬massively parallel processing£©Êý¾Ý¿âµÄ´ú±íÊÇTeradata¡¢Greenplum¡¢VerticaºÍImpalaµÈ¡£ËüÃǵÄÓŵãÊÇÊʺϽ«RDBMSµÄÓ¦ÓÃÀ©Õ¹µ½¼¯Èº·¶Î§£¬ÒÔ´¦Àí¸ü´óµÄÊý¾ÝÁ¿£¬Í¬Ê±¼Ì³ÐÁËRDBMSµÄºÜ¶àÓŵ㡣Ȼ¶ø£¬ËüÒÀȻû·¨´Ó¸ù±¾ÉÏÂú×ã´¦Àíº£Á¿Êý¾ÝµÄÐèÇó£¬ÒòΪËüµÄϵͳÐÔÄܺÜÄÑËæ¼¯ÈºµÄÀ©Õ¹Ò»Ö±ÏßÐÔÔö³¤£¬ËùÒÔÆä¼¯ÈºÀ©Õ¹ÐÔÓÐÏÞ£»¶øÇÒËüµÄ¼¯ÈºÈÝ´íÐÔÒ²ÓÐÏÞ£¬±ÈÈçµ±ÆäRaid´ÅÅ̳ö´íºó¿ÉÄܻᵼÖÂÆä½ÚµãÏìÓ¦ËٶȱäÂý¡£

ElasticSearch£º

ElasticSearch¼¸ºõÒѾ­Êǵ±ÏÂʵʱÉϵÄËÑË÷·þÎñÆ÷±ê×¼£¬¶øËüÓëLogstashºÍKibanaµÄ´îÅ䣨¼ò³ÆELK£©Ò²ÊÇÓ¦ÓúܹãµÄÌ×¼þ×éºÏ¡£ElasticSearchÓкÜÇ¿µÄÎĵµË÷ÒýºÍÈ«ÎļìË÷µÄÄÜÁ¦£¬Ò²Ö§³Ö·á¸»µÄ²éѯ¹¦ÄÜ£¨°üÀ¨¾ÛºÏ²éѯ£©£¬²¢ÇÒÀ©Õ¹ÐԺã¬Òò´ËºÜ¶àÓû§Ò²³£³£ÔÚÄÃElasticsearchºÍDruid×ö±È½Ï¡£È»¶ø£¬ÔÚÎÒÃǵļȶ¨³¡¾°Ï£¬ElasticsearchÓÐ×ÅÆäÃ÷ÏԵĶ̰壺²»Ö§³ÖÔ¤¾ÛºÏ£»×éºÏ²éѯÐÔÄÜÇ·¼Ñ£»µÈµÈ¡£

OpenTSDB£º

OpenTSDBÊÇÒ»¿îÓÅÐãµÄʱÐòÊý¾Ý¿â£¬ÇÒ»ùÓÚÓû§¹ã·ºµÄHBaseÊý¾Ý¿â£¬Òò´ËÓÐ׎϶àµÄ¿Í»§Èº¡£ËüµÄÓÅÊÆÔÚÓÚ²éѯËٶȿ졢À©Õ¹ÐԺã¬ÇÒschemaless¡£È»¶ø£¬ËüÒ²ÓÐһЩȱµã£º²éѯµÄά¶È×éºÏÊýÁ¿ÐèÒªÌáǰȷ¶¨ºÃ£¬¼´Í¨¹ý´æ´¢ÖеÄtag×éºÏÀ´È·¶¨£¬Òò´Ëȱ·¦ÁËÁé»îÐÔ£»Êý¾ÝÈßÓà¶È´ó£»»ùÓÚHBase£¬¶ÔÓÚÔËάÈËÔ±ÄÜÁ¦ÒªÇó½Ï¸ß£»µÈµÈ¡£

Kylin£º

Kylin³ö×ÔeBay£¬²¢ÇÒÊÇApacheµÄж¥¼¶ÏîÄ¿£¬ÊÇÒ»¿î·Ç³£²»´íµÄ·Ö²¼Ê½OLAPϵͳ¡£ËüµÄÓÅÊÆÔÚÓÚÏîÄ¿À´×ÔÓڷḻµÄʵ¼ù¡¢·¢Õ¹Ç°¾°¹âÃ÷¡¢Ö±½ÓÀûÓÃÁ˳ÉÊìµÄHadoopÌåϵ£¨HBaseºÍHiveµÈ£©¡¢cubeÄ£ÐÍÖ§³Ö½ÏºÃ¡¢²éѯËÙ¶È¿ìËٵȡ£µ«ËüÒ²ÔÚһЩ·½ÃæÓÐ×Å×Ô¼ºµÄ¾ÖÏÞ£¬±ÈÈ磺²éѯµÄά¶È×éºÏÊýÁ¿ÐèÒªÌáǰȷ¶¨ºÃ£¬²»Êʺϼ´Ï¯²éѯ·ÖÎö£»Ô¤¼ÆËãÁ¿´ó£¬×ÊÔ´ÏûºÄ¶à£»¼¯ÈºÒÀÀµ½Ï¶à£¬ÈçHBaseºÍHiveµÈ£¬ÊôÓÚÖØÁ¿¼¶·½°¸£¬Òò´ËÔËά³É±¾Ò²½Ï¸ß£»µÈµÈ¡£

Druid»ù´¡ÖªÊ¶

Êý¾Ý´æ´¢¸ñʽ

Druid×Ô¼ºÓÐ×ÅÊý¾Ý´æ´¢µÄÂß¼­ºÍ¸ñʽ£¨Ö÷ÒªÊÇDataSourceºÍSegment£©£¬¶øÇÒÆä¼Ü¹¹ºÍʵÏÖÒ²ÊÇÎ§ÈÆ×ÅËüÃǶø½øÐеģ¬Òò´ËÎÒÃÇÓбØÒª¶ÔÆäÊý¾Ý´æ´¢¸ñʽ×öһЩ½éÉÜ¡£

Ê×ÏÈ£¬ÈôÓ봫ͳµÄ¹ØÏµÐÍÊý¾Ý¿â¹ÜÀíϵͳ(RDBMS)×ö±È½Ï£¬Druid µÄ DataSource ¿ÉÒÔÀí½âΪ RDBMS Öеıí(Table)¡£ÕýÈçÇ°ÃæÕ½ÚÖÐËù½éÉܵÄ,DataSource µÄ½á¹¹°üº¬ÒÔϼ¸¸ö·½Ãæ¡£

  • ʱ¼äÁÐ(TimeStamp)£º±íÃ÷ÿÐÐÊý¾ÝµÄʱ¼äÖµ£¬Ä¬ÈÏʹÓà UTC ʱ¼ä¸ñʽÇÒ¾«È·µ½ºÁÃë¼¶±ð¡£Õâ¸öÁÐÊÇÊý¾Ý¾ÛºÏÓ뷶Χ²éѯµÄÖØÒªÎ¬¶È¡£
  • ά¶ÈÁÐ(Dimension)£º ¶ÈÀ´×ÔÓÚOLAPµÄ¸ÅÄÓÃÀ´±êʶÊý¾ÝÐеĸ÷¸öÀà±ðÐÅÏ¢¡£
  • Ö¸±êÁÐ(Metric)£ºÖ¸±ê¶ÔÓ¦ÓÚOLAP¸ÅÄîÖеÄFact£¬ÊÇÓÃÓھۺϺͼÆËãµÄÁÐÕâЩָ±êÁÐͨ³£ÊÇһЩÊý×Ö£¬¼ÆËã²Ù×÷ͨ³£°üÀ¨ Count¡¢Sum ºÍ Mean µÈ¡£

ÎÞÂÛÊÇʵʱÊý¾ÝÏû·Ñ»¹ÊÇÅúÁ¿Êý¾Ý´¦Àí£¬Druid ÔÚ»ùÓÚ DataSource ½á¹¹´æ´¢Êý¾Ýʱ¼´¿ÉÑ¡Ôñ¶ÔÈÎÒâµÄÖ¸±êÁнøÐоۺÏ(Roll Up)²Ù×÷¡£Ïà¶ÔÓÚÆäËûʱÐòÊý¾Ý¿â£¬Druid ÔÚÊý¾Ý´æ´¢Ê±±ã¿É¶ÔÊý¾Ý½øÐоۺϲÙ×÷ÊÇÆäÒ»´óÌØµã, ²¢ÇÒ¸ÃÌØµãʹµÃ Druid ²»½öÄܹ»½ÚÊ¡´æ´¢¿Õ¼ä£¬¶øÇÒÄܹ»Ìá¸ß¾ÛºÏ²éѯµÄЧÂÊ¡£

DataSource ÊÇÒ»¸öÂß¼­¸ÅÄSegment È´ÊÇÊý¾ÝµÄʵ¼ÊÎïÀí´æ´¢¸ñʽ£¬Druid ÕýÊÇͨ¹ý Segment ʵÏÖÁ˶ÔÊý¾ÝµÄºá×ÝÏòÇиî(Slice and Dice)²Ù×÷¡£´ÓÊý¾Ý°´Ê±¼ä·Ö²¼µÄ½Ç¶ÈÀ´¿´, ͨ¹ý²ÎÊý segmentGranularity µÄÉèÖã¬Druid ½«²»Í¬Ê±¼ä·¶Î§ÄÚµÄÊý¾Ý´æ´¢ÔÚ²»Í¬µÄ Segment Êý¾Ý¿éÖУ¬Õâ±ãÊÇËùνµÄÊý¾ÝºáÏòÇиÕâÖÖÉè¼ÆÎª Druid ´øÀ´Ò»¸öÏÔ¶øÒ×¼ûµÄÓŵ㣺°´Ê±¼ä·¶Î§²éѯÊý¾Ýʱ£¬½öÐèÒª·ÃÎʶÔӦʱ¼ä¶ÎÄÚµÄÕâЩ Segment Êý¾Ý¿é,¶ø²»ÐèÒª½øÐÐÈ«±íÊý¾Ý·¶Î§²éѯ,ÕâʹЧÂʵõ½Á˼«´óµÄÌá¸ß¡£

ͬʱ£¬ÔÚ Segment ÖÐÒ²ÃæÏòÁнøÐÐÊý¾ÝѹËõ´æ´¢£¬Õâ±ãÊÇËùνµÄÊý¾Ý×ÝÏòÇи¶øÇÒ¶ÔSegment ÖеÄά¶ÈÁÐʹÓÃÁË Bitmap ¼¼Êõ¶ÔÆäÊý¾ÝµÄ·ÃÎʽøÐÐÁËÓÅ»¯¡£ÆäÖУ¬Druid»áΪÿһά¶ÈÁд洢ËùÓÐÁÐÖµ¡¢´´½¨×ֵ䣨ÓÃÀ´´æ´¢ËùÓÐÁÐÖµ¶ÔÓ¦µÄID£©ÒÔ¼°ÎªÃ¿Ò»¸öÁÐÖµ´´½¨ÆäbitmapË÷ÒýÒÔ°ïÖú¿ìËÙ¶¨Î»ÄÄЩÐÐÓµÓиÃÁÐÖµ¡£

»ùÓÚʵʱ½ÚµãµÄ¼Ü¹¹

Druid×îÔçµÄ¼Ü¹¹ÊÇ»ùÓÚʵʱ½Úµã£¨Realtime Node£©µÄ¡£ËüµÄÌØµãÊÇÓÐרÃŵÄʵʱ½ÚµãÀ´¸ºÔðÏû·ÑÊý¾Ý¡£

Druid×ÔÉí°üº¬ÒÔÏ 4 Àà½Úµã¡£

  • ʵʱ½Úµã(RealtimeNode)£º¼´Ê±ÉãÈëʵʱÊý¾Ý£¬ÒÔ¼°Éú³ÉSegmentÊý¾ÝÎļþ¡£
  • ÀúÊ·½Úµã(HistoricalNode)£º¼ÓÔØÒÑÉú³ÉºÃµÄÊý¾ÝÎļþ£¬ÒÔ¹©Êý¾Ý²éѯ¡£
  • ²éѯ½Úµã(Broker Node)£º¶ÔÍâÌṩÊý¾Ý²éѯ·þÎñ£¬²¢Í¬Ê±´Óʵʱ½ÚµãÓëÀúÊ·½Úµã²éѯÊý¾Ý,ºÏ²¢ºó·µ»Ø¸øµ÷Ó÷½¡£
  • Эµ÷½Úµã(CoordinatorNode)£º¸ºÔðÀúÊ·½ÚµãµÄÊý¾Ý¸ºÔؾùºâ£¬ÒÔ¼°Í¨¹ý¹æÔò(Rule) ¹ÜÀíÊý¾ÝµÄÉúÃüÖÜÆÚ¡£

ͬʱ£¬¼¯Èº»¹°üº¬ÒÔÏÂÈýÀàÍⲿÒÀÀµ¡£

  • ÔªÊý¾Ý¿â(Metastore)£º´æ´¢Druid¼¯ÈºµÄÔ­Êý¾ÝÐÅÏ¢£¬±ÈÈçSegmentµÄÏà¹ØÐÅÏ¢£¬Ò»°ãÓà MySQL »ò PostgreSQL¡£
  • ·Ö²¼Ê½Ð­µ÷·þÎñ(Coordination)£ºÎªDruid¼¯ÈºÌṩһÖÂÐÔЭµ÷·þÎñµÄ×é¼þ£¬Í¨³£ÎªZookeeper¡£
  • Êý¾ÝÎļþ´æ´¢¿â(DeepStorage)£º´æ·ÅÉú³ÉµÄSegmentÊý¾ÝÎļþ£¬²¢¹©ÀúÊ·½ÚµãÏÂÔØ ¶ÔÓÚµ¥½Úµã¼¯Èº¿ÉÒÔÊDZ¾µØ´ÅÅÌ£¬¶ø¶ÔÓÚ·Ö²¼Ê½¼¯ÈºÒ»°ãÊÇ HDFS »ò NFS¡£

´ÓÊý¾ÝÁ÷תµÄ½Ç¶ÈÀ´¿´£¬Êý¾Ý´Ó¼Ü¹¹Í¼µÄ×ó²à½øÈëϵͳ,·ÖΪʵʱÁ÷Êý¾ÝÓëÅúÁ¿Êý¾Ý¡£ÊµÊ±Á÷Êý¾Ý»á±»ÊµÊ±½ÚµãÏû·Ñ,È»ºóʵʱ½Úµã½«Éú³ÉµÄ Segment Êý¾ÝÎļþÉÏ´«µ½Êý¾ÝÎļþ´æ´¢¿â£»¶øÅúÁ¿Êý¾Ý¾­¹ý Druid ¼¯ÈºÏû·Ñºó»á±»Ö±½ÓÉÏ´«µ½Êý¾ÝÎļþ´æ´¢¿â¡£Í¬Ê±£¬²éѯ½Úµã»áÏìÓ¦ÍⲿµÄ²éѯÇëÇ󣬲¢½«·Ö±ð´Óʵʱ½ÚµãÓëÀúÊ·½Úµã²éѯµ½µÄ½á¹ûºÏ²¢ºó·µ»Ø¡£

»ùÓÚIndexing ServiceµÄ¼Ü¹¹

ÔÚ»ùÓÚIndexing ServiceµÄ¼Ü¹¹ÖУ¬ÊµÊ±½Úµã±»·ÏÆúµôÁË£¬×ª¶ø´úÖ®µÄOverlord½ÚµãºÍMiddleManager½Úµã¡£ÆäÖУ¬Overlord½Úµã×÷ΪË÷Òý·þÎñµÄÖ÷½Úµã,¶ÔÍ⸺Ôð½ÓÊÕÈÎÎñÇëÇó,¶ÔÄÚ¸ºÔð½«ÈÎÎñ·Ö½â²¢Ï·¢µ½´Ó½Úµã¼´MiddleManager½ÚµãÉÏ¡£¶øMiddleManager½Úµã¾ÍÊÇË÷Òý·þÎñµÄ¹¤×÷½Úµã,¸ºÔð½ÓÊÕͳÖνڵã·ÖÅäµÄÈÎÎñ,È»ºóÆô¶¯Ïà¹Ø¿à¹¤£¨Peon£©¼´¶ÀÁ¢µÄ JVM À´Íê³É¾ßÌåµÄÈÎÎñ¡£ÕâÑùµÄ¼Ü¹¹Êµ¼ÊÓë Hadoop Yarn ºÜÏñ¡£

½ÏÖ®»ùÓÚʵʱ½ÚµãµÄ¼Ü¹¹£¬»ùÓÚIndexing ServiceµÄ¼Ü¹¹Óв»ÉÙÓŵ㣺³ýÁ˶ÔÊý¾ÝÄܹ»Óà pull µÄ·½Ê½Íâ,»¹Ö§³Ö push µÄ·½Ê½£»²»Í¬ÓÚÊÖ¹¤±àдÊý¾ÝÏû·ÑÅäÖÃÎļþµÄ·½Ê½£¬¿ÉÒÔͨ¹ý API µÄ±à³Ì·½Ê½À´Áé»î¶¨ÒåÈÎÎñÅäÖ㻿ÉÒÔ¸üÁé»îµØ¹ÜÀíÓëʹÓÃϵͳ×ÊÔ´£»¿ÉÒÔÍê³É Segment ¸±±¾ÊýÁ¿µÄ¿ØÖÆ£»Äܹ»Áé»îÍê³É¸ú Segment Êý¾ÝÎļþÏà¹ØµÄËùÓвÙ×÷£¬ÈçºÏ²¢¡¢É¾³ý Segment Êý¾ÝÎļþµÈ¡£

²éѯ

DruidµÄĬÈÏÌṩÁËHTTP REST ·ç¸ñµÄ²éѯ½Ó¿Ú¡£Ò»¸öµäÐ굀 curl ÃüÁîÈçÏÂ: curl -X POST ':/druid/v2/?pretty' -H 'Content-Type:application /json' -d @ ¡£

ÆäÖУ¬queryablehost:port Ϊ²éѯ½ÚµãµÄ IP µØÖ·ºÍ¶Ë¿Ú£»queryjson_ leΪPOSTµ½²éѯ½ÚµãµÄ²éѯÇëÇó¡£

ΪÁ˱ãÓÚÓû§½øÐÐÁé»îµÄOLAP·ÖÎö£¬DruidÌṩÁ˷ḻµÄ²éѯ½Ó¿Ú£¬±ÈÈç¹ýÂËÆ÷¡¢¾ÛºÏÆ÷ºÍºóÖÃ¾ÛºÏÆ÷µÈ¡£ÓÉÓÚÕⲿ·ÖÄÚÈݱȽϺÃÀí½â£¬¶øÇÒDruid¹ÙÍøÉÏÓÐÏêϸµÄ½éÉÜ£¬Òò´Ë±¾ÎľͲ»ÔÙ¹ý¶à˵Ã÷£¬ÕâÀï½ö¾ÛÒ»¸öʵ¼ÊµÄ²éѯÀý×ÓÈçÏ£º

{
"queryType": "timeseries",
"dataSource": "visitor_statistics",
"granularity": "all",
"filter": {
"type": "and",
"fields": [
{
"type": "selector",
"dimension": "host",
"value": "www.mejia.wang"
},
{
}
]
},
"aggregations": [
{
"type": "longSum",
"name": "pv",
"fieldName": "counbt"
},
{
},
{
"type": "hyperUnique",
"name": "new_visitor_count",
"fieldName": "new_visit_count"
},
{
}
],
"postAggregations": [
{
"type": "arithmetic",
"name": "new_visitor_rate",
"fn": "/",
"fields": [
{
"type": "hyperUniqueCardinality",
"fieldName": "new_visitor_count"
},
{
"fieldName": "visitor_count"
}
]
},
{
"type": "arithmetic",
"name": "click_rate",
"fn": "/",
"fields": [
{
"type": "hyperUniqueCardinality",
"fieldName": "click_visitor_count"
},
{
"type": "hyperUnique",
"name": "click_visitor_count",
"fieldName": "click_visit_count""type": "hyperUniqueCardinality",
"type": "hyperUniqueCardinality",
"fieldName": "visitor_count"
}
]
}
],
"intervals": [
"2016-08-07T00:00:00+08:00/2016-09-05T23:59:59+08:00"
]
}

·µ»Ø½á¹û¿ÉÄÜÈçÏ£º

["
{
"timestamp": "2016-08-27T16:00:00.000Z",
"result": {
"pv": 30.000000061295435,
"visit_count": 5.006113467958146,
"new_visitor_count": 0,
"click_visitor_count": 5.006113467958146,
"new_visitor_rate": 0,
"click_rate": 1,
}
}
]

·á¸»µÄ¸¨Öú¹¦ÄÜ

Druid³ýÁË»ù±¾µÄºËÐÄÊý¾ÝÏû·ÑºÍ²éѯ¹¦ÄÜÍ⣬»¹ÌṩÁ˷ḻµÄ¸¨Öú¹¦ÄÜ£¬ÒÔ°ïÖúÓû§¸üºÃµØ»ùÓÚDruidÍê³ÉÊý¾Ý´¦Àí¹¤×÷¡£±¾Îļòµ¥Áоټ¸¸ö£º

DataSketches aggregator£º½üËÆ¼ÆËãCOUNT DISTINCTµÈÎÊÌ⣬ÈçPV¡¢Áô´æµÈ£»¾«¶È¿Éµ÷½Ú¡£

Multi-value dimensions£º¶ÔÓÚͬһά¶ÈÁУ¬ÔÊÐí²»Í¬ÐÐÓµÓв»Í¬ÊýÁ¿µÄÊý¾ÝÖµ£ºÕâʹµÃDruidÒ²Äܹ»ÓÐÀàËÆschemalessµÄ¹¦ÄÜ¡£

{"timestamp": "2011-01-12T00:00:00.000Z", "tags": ["t1","t2","t3"]} #row1 {"timestamp": "2011-01-13T00:00:00.000Z", "tags": ["t3","t4","t5"]} #row2

Router Node£ºÐ­ÖúBroker NodeʵÏÖ¸ºÔؾùºâ¡£

Query Caching£º°ïÖúÌáÉý²éѯÐÔÄÜ£¬¿ÉÒÔÓÃÔÚHistorical Node»òBroker NodeÉÏ¡£

ʵ¼ù

×îºó£¬½éÉÜһϱÊÕßĿǰÔÚ±±¾©ÔƲâÐÅÏ¢¼¼ÊõÓÐÏÞ¹«Ë¾£¨¼´Testin¹«Ë¾£©µÄAB²âÊÔ²úÆ·ÖеÄDruidʵ¼ù¾­Ñé¡£

ÏÂͼÊÇÎÒÃÇAB²âÊÔ²úÆ·µÄ¼Ü¹¹Í¼£¬¿ÉÒÔ¿´µ½DruidÔÚÆäÖÐÖ÷Òª¸ºÔð¶àάʵʱOLAP·ÖÎö¡£

ÎÒÃÇʹÓõÄÊÇDruid Indexing ServiceµÄ¼Ü¹¹£¬¶øÇÒʹÓÃÁËKafka Indexing ServiceµÄ¹¦ÄÜ£¬ÕâʹµÃÎÒÃÇ¿ÉÒԺܷ½±ãµØ±£Ö¤ÁËDruid²»¶ªÆú³¬¹ýʱ¼ä´°¿ÚµÄÊý¾Ý¡£

´ÓʹÓÃЧ¹ûÀ´¿´£¬Druid¸³ÓèÁËÎÒÃDzúƷʵʱµÄÊý¾ÝÏû·ÑºÍ²éѯ£¬Ê¹µÃÓû§¿ÉÒÔÔÚÃë¼¶±ã¿´µ½×îеÄÊý¾Ý·ÖÎö½á¹û£¬ÈçÏÂͼ£º

ͬʱ£¬DruidʹµÃÎÒÃÇ¿ÉÒÔ½øÐÐÁé»îµÄ¶àάʵʱ·ÖÎö£º

   
5726 ´Îä¯ÀÀ       27
Ïà¹ØÎÄÕÂ

»ùÓÚEAµÄÊý¾Ý¿â½¨Ä£
Êý¾ÝÁ÷½¨Ä££¨EAÖ¸ÄÏ£©
¡°Êý¾Ýºþ¡±£º¸ÅÄî¡¢ÌØÕ÷¡¢¼Ü¹¹Óë°¸Àý
ÔÚÏßÉ̳ÇÊý¾Ý¿âϵͳÉè¼Æ ˼·+Ч¹û
 
Ïà¹ØÎĵµ

GreenplumÊý¾Ý¿â»ù´¡Åàѵ
MySQL5.1ÐÔÄÜÓÅ»¯·½°¸
ijµçÉÌÊý¾ÝÖÐ̨¼Ü¹¹Êµ¼ù
MySQL¸ßÀ©Õ¹¼Ü¹¹Éè¼Æ
Ïà¹Ø¿Î³Ì

Êý¾ÝÖÎÀí¡¢Êý¾Ý¼Ü¹¹¼°Êý¾Ý±ê×¼
MongoDBʵս¿Î³Ì
²¢·¢¡¢´óÈÝÁ¿¡¢¸ßÐÔÄÜÊý¾Ý¿âÉè¼ÆÓëÓÅ»¯
PostgreSQLÊý¾Ý¿âʵսÅàѵ