HadoopConceptsNote
Introduction
1.
Chapter 1: Data Flow
1.1.
Data Locality Optimization
1.2.
Shuffle and Sort Phase
2.
Chapter 2: Data Read and Write
2.1.
Read
2.2.
Network Topology
2.3.
Write
2.4.
Coherency Model
3.
Chapter 3: YARN and MapReduce
3.1.
Anatomy of a YARN Application Run
3.2.
Resource Request
3.3.
Application Lifespan
3.4.
Differences between YARN and MR1
3.5.
Scheduling in YARN
4.
Hadoop I/O
4.1.
Data Integrity
4.2.
Compression
Powered by
GitBook
A
A
Serif
Sans
White
Sepia
Night
Share on Twitter
Share on Google
Share on Facebook
Share on Weibo
Share on Instapaper
HadoopConceptsNote
Network Topology
Node n1, rack r1, data center d1
4 cases
Distance(d1/r1/n1, d1/r1/n1) = 0
Distance(d1/r1/n2, d1/r1/n1) = 2
Distance(d1/r2/n1, d1/r1/n1) = 4
Distance(d2/r1/n1, d1/r1/n1) = 6