Ծ֧ǵĹҵ

10Ԫ 50Ԫ





֤룺  ֤,?ˢ֤



  ֪ Ŀ Lib Ƶ Code iProcess γ ֤ ѯ   ɳ֮·  
Ա   
 
   
 
  
ÿ15ƪ
ı
׷
 
 
     
   
 
 
ݺ֮ıȽϷ
 
Դ51CTO ڣ2017-3-14
381     ۣ      
 

ݺ֮иͬĹؼʣֲʽһִӼȨ¢ϵȥĻת䡣

ֲʽ洢HDFS vs.

ݣҪӦԺͿĴ洢ҪײӲܹļϵͳԼ۱ҪڴͳܹŴ洢ȸGFSHadoopHDFS춨˴ݴ洢Ļ⣬ݶԴ洢һսǶݸʽӦڴݵײĴ洢㲻ֻHDFSHBaseKuduȴ洢ܹ

DZرҵĵײ㼼ܹڱһȥĻķֲʽ˱Ϊһֳġʽݽṹͨжڵ㹲ͬݵļͼ¼һ֤ϢЧԡһ˵Ҳһضݿ⼼ȥĻݿڰȫݷԣܶҵʿ䷢չΪǶл벹䡣

ֲʽ㣺MapReduce vs. ʶ

ݵķھܼͼ㣬Ҫ޴ķֲʽڵȡݴ͸߿ɿǹؼGoogleHadoopMapReduceֲַʽ㼼Ĵͨӷڵչϵͳܴ(Scale Out)ڳɱͿչ϶о޴ơڣ㣬ݻ㡢ͼ㡢ʵʱ㡢ѯȼܡ

Ĺʶƣзֲʽ֮ôɹʶͨ㷨ɺ͸ݣȥ϶һ¼Чԣ϶ֶΣҲǷֹ۸ĵֶΡҪֲͬĹʶƣڲͬӦóЧʺͰȫ֮ȡƽ⡣ԱرΪõǡ֤(Proof Of WorkPOW)ֻڿȫ51%ļ˽ڵ£пαһڵļ¼

ITչķַֺϺ

һITչҲֳϾñط֣־ñغϡֲʽ

ڣʵһһʹãǼлġΪʹһ̨ͻܹͬʱΪͻṩIBM˾⻯˼룬ʹöͻͬʱʹͬһ̨ͻʱͺָ˶Сͻʱָõļʽ㡣

СͻPCʱعһһʹã豸Ѿɢǧ򻧡뻥ʱC/Sģ͵Ŀͻ˺ͷǷֲʽ㣬ֻ֮仹Ƿɢġ

Ƽʱֱͳһܿڿͻ˺ͷķֲʽ֮ϣ֮Ҳʼ˷ֲʽЭͬΪЭͬҲΪһּʽļ

ʱƼΪݻʩҲʹôݵĺ˼ƼһСMapReduceֽзֲʽ㣬Ȼ󽫽ϲӶʵϢϷ

Ǵϵķֲʽϵͳ

ʲô˼ֲĴ˳?

ǴʷѰҴ𰸡

ҵҪУϣͨƷʵָõĿƺ͸ߵ󡣵ŲƷжȵIJϵͳԽԽӣʵֵѶԽԽ󣬹ͨ͹ijɱҲԽԽߣձòá

Ҫֹרҵרҵ£Ϳɵġ̸ֹֹۡʹлһÿԱΪͬʱֲ壬ֹŦν֮лŽᡯ

ֲʽĵǻ˼롣ƷֽܱⲢֲͬĽڵȥɣڵ֮ͨʵֲֹͨʽϵͳеһЩڵΪҵϵijɹ³ΪлĽڵ㣬ʱĸı䣬սһֵķֲʽڡ

кͷֲǹ׵ˣκΰIJƷҵͼġлŽᡱ

֮һЩͬ㣬һ֮IJͬ㡣

ڲͬʱ̴֮һμ

ڲͬ

Gartner Hype Cycle

(The Hype Cycle)ѯ˾GartnerԤ¿Ƽijݱٶȼʱߡ

ݡ롰Gartnerġߡеij

2011꣬ݡһϰλڼѿڵ½׶ΣʱͳΪBig Data and Extreme Information Processing and Management(ݡͼϢ͹)2012һ2013꼸ﵽ˹ڶ塣2014»2015꿪ʼݡͻȻʧɽΪGartnerԴݵĶλѴӡˡתΪǰݶҵѴҪΪսԺġ

Զԣֱ2016ŵһγڡߡУֱӽ롰ڡܵݡ͡ڽ׶δͬԼ5ҵIJࡣ

Ҫ?

ͨݼ㹻㹻ӣºôͳķʽܳصϢ޵ģ롰ݡ׼úԶм죺

ṹvsǽṹǽṹϽĿ飬ָͨɵ͵ĽṹݣҪĸǷǽṹ;

vsϣϵͳΪ֤ȫԣϢԶģصϢϷ;

ֱvsӣϵͳһݿ⣬ָǶݵȷھһּӵ;

ѧvsݣͼѧ˵š뼴ɡͼ˵;

vsԣ(˱ӵߣڴͳڻĹ˺ţ˱)ǸԻ;

ܷ?

һֲʽϵͳ˵CAP(CAP theorem)ֱ³(Brewer's theorem)ָһֲʽϵͳͬʱ㣺

һ(Consistence)ڷֲʽϵͳеݱݣͬһʱǷֵͬ

(Availability)ڼȺһֽڵϺ󣬼ȺǷӦͻ˵Ķд

(Partition tolerance)ȺеijЩڵ޷ϵ󣬼ȺǷܼз

ڵǰӲ϶ӳٶ⣬ԷDZҪʵֵġ仰˵CAPDZһ(C)Ϳ(A)֮Ȩ⡣

嵽ʹ˵һ(C)ȡ(A)ͷ(P)ģȴȱ֤һ(C)

໥֮

ͨCAP֪ʹݵ޷ȫҪԾ峡ڶȡ᷽Ƴϵͳ

һֲɴ۸ĵġȫʷķֲʽݿ洢޴ݼϰÿһʽ׵ȫʷӦѸٷչݹģԽԽ󣬲ͬҵ񳡾ںϻһݹģͷḻԡ

ԡȫԺͲɴ۸ԣøݱųƽݵĺĿ׷ʹݴӲɼסͨԼÿһ¼ϣʹݵǰδеǿα飬Ҳ֤ݷȷԺھЧ

ܹһ淶ݵʹãϸȨΧݽͨͻϢµݺͨƣγɡữݡļֵת磬ƶγɻȫ򻯵ݽ׳

ṩ˱ԣͳƷ߱ݴ洢Чķݵļֵʹÿռ䡣

+ڴʹ

ݵļ̬ٻţûĸܽе⣬ܽҲһΧڣʹSparkFlinkȡǿ͸ԡȫԵij£֮ءڴݵϵͳʹʹݲܱӡ޸ĺɾȻʱ޶ȵġ

ʱ䡢ΪᣬгĿǰóݵķΧгΪһֺܺõIJ

磬ڴ浵ʷݣΪDzܱ޸ĵģǿԶԴHashʱ֮ϡδijһʱ̣Ҫ֤ԭʼݵʵʱԶԶӦͬHashóĴͬģ˵ûб۸ĹġߣֻԻݺͽֻҪݴôӦԵǽƹϵͳԴġ

ͨѴϣеݸмֵҲôݵԤʵΪжǶ־ʱĻʯ

   
 
 
ĵ ؿγ



Ǹݿ
ݿƾ̸
ݿƹ
ݿܽ
ݿܵż
ݿܵ
ݿŻ
ݿϵͳܵϵ
ݿŻ
߼ݿܹʦ
ݲֿھ
Hadoopԭܵ
 

APPƹ֮ù߽ݷ
Hadoop Hivesql﷨
Ӧö༶ģʽ֧ź
HBase ϸ
HBaseϸ
Spark̬Դ
...   

HadoopSparkݼܹ
Hadoopԭ߼ʵ
HadoopԭӦŻ
ϵӦ
ݵļʵ
Sparkݴ
γ...   

IT˾ androidƽ̨ʵ
Android
ijԴҵ Android
ij칫˾ AndroidIOSӦ
Linuxں
Ĭ Ƕʽܹ
Ƕʽܹ
...   
 
 
 
 
 
ÿ2ĵ/Ƶ
ɨ΢Ŷά붩
ļ¿
ÿ300Դ
 
 

| ϵ | ICP10020922 110108001071