澶ф暟鎹鐞嗗钩鍙版槸浼佷笟浜や粯寮忓ぇ鏁版嵁寮鏀惧钩鍙帮紝涓昏浣滅敤鏄兘澶熷疄鐜板ぇ鏁版嵁绠＄悊濂椾欢鐨勮嚜鍔ㄥ寲閮ㄧ讲銆佸惎鍋溿侀厤缃互鍙婄洃鎺ч泦缇ょ姸鎬侊紝鍚屽钩鍙版彁渚涘绉嶆帴鍏ユ墜娈靛皢浼佷笟鍐呴儴鎵鏈夌粨鏋勫寲鍜岄潪缁撴瀯鍖栨暟鎹繘琛屾暣鍚堬紝涓轰紒涓氬湪杩愮淮銆佺爺鍙戙佷骇鍝併佽繍钀ャ佺瓑澶氫釜閮ㄩ棬鍙婄鐞嗗眰鎻愪緵缁熶竴鐨勬绱佸垎鏋愬拰鎸栨帢鐨勬湇鍔°

       闆嗙兢鏈嶅姟鍖呮嫭锛 HDFS銆乊ARN銆丮apreduce2銆 Spark銆乀ez銆丯agios銆丟anglia 銆丠ive銆丠Base銆丼qoop銆乑ooKeeper銆丠ugeTable绛夈傚悓鏃讹紝閫氳繃鎺ュ彛鍙疄鐜颁簩娆″紑鍙戯紝浠ユ弧瓒充笟鍔℃繁鍏ュ畾鍒堕渶姹傦紝璁╀紒涓氭暟鎹簲鐢ㄤ竴鐩簡鐒躲

        骞冲彴鐗圭偣鍦ㄧ粺涓鐨勫垎甯冨紡瀛樺偍涔嬩笂鏁版嵁骞冲彴涓婇氳繃YARN鎻愪緵缁熶竴鐨勮祫婧愮鐞嗚皟搴︼紝缁撳悎LDAP涓嶬erberos鎻愪緵瀹屽鐨勬潈闄愮鐞嗘帶鍒讹紝涓嶅悓鐨勯儴闂ㄤ互鍙婁娇鐢ㄧ鎴峰彲浠ユ寜闇鍒涘缓璁＄畻闆嗙兢璁块棶鍏舵巿鏉冩暟鎹紝鍖呮嫭鍩轰簬Spark璁＄畻妗嗘灦鐨凷QL绫荤粺璁″垎鏋愬簲鐢ㄤ笌鏁版嵁鎸栨帢绫诲簲鐢紝鎴栬呭熀浜嶮apReduce璁＄畻妗嗘灦鐨勫簲鐢ㄣ傚悓鏃讹紝骞冲彴閫氳繃浣跨敤Hbase缁撳悎Hive/Phoenix/Impala/SparkSQL锛屼负鐢ㄦ埛鎻愪緵鍩轰簬SQL鐨勯珮骞跺彂鐨勬煡璇互鍙婂垎鏋愯兘鍔涖

         鍦ㄦ暟鎹泦鎴愪笌浜ゆ崲鏂归潰锛屽钩鍙版彁渚涚粺涓鏁版嵁绠￠亾鏈嶅姟锛屾敮鎸佸疄鏃朵笌闈炲疄鏃剁殑缁撴瀯/闈炵粨鏋勫寲鐨勬暟鎹鍏ヤ笌浜ゆ崲鑳藉姏锛屽寘鎷現lume鎻愪緵娴烽噺鏁版嵁鏂囦欢鐨勮仛鍚堟眹鎬诲埌HDFS鐨勫姛鑳斤紝Sqoop鎻愪緵涓庡叧绯诲瀷鏁版嵁搴撶殑鏁版嵁浜ゆ崲浠ュ強Kafka娑堟伅闃熷垪闆嗙兢鎺ユ敹瀹炴椂娴佹暟鎹紝瀹炴椂鎽勫彇鍏崇郴鍨嬫暟鎹簱澧為噺浜嬪姟鏁版嵁锛圕DC锛夈

          姝ゅ锛孴DP鏁版嵁骞冲彴鎻愪緵瀹炴椂鐨勬祦澶勭悊鑳藉姏锛岄氳繃娑堟伅闃熷垪Kafka鎺ユ敹瀹炴椂鏁版嵁娴侊紝鍋氬埌鏁版嵁涓嶄涪涓嶉噸锛岄氳繃Stream鍩轰簬SparkStreaming鎻愪緵绫讳技涓庢壒澶勭悊绯荤粺鐨勮绠楄兘鍔涖佸仴澹с佹墿灞曟х殑鍚屾椂锛屽皢鏁版嵁鏃跺欢闄嶄綆鑷崇绾х敋鑷虫绉掔骇銆