ISSUE-64. Hadoop 3.0 NN drops replicas silently. by jojochuang · Pull Request #65 · linkedin/dynamometer · GitHub
Skip to content

ISSUE-64. Hadoop 3.0 NN drops replicas silently. - #65

Open
jojochuang wants to merge 1 commit into
linkedin:masterfrom
jojochuang:ISSUE-64
Open

jojochuang wants to merge 1 commit into
linkedin:masterfrom
jojochuang:ISSUE-64

Conversation

@jojochuang

Copy link
Copy Markdown

Fixes #64
This PR changes blockMap of dyno DN from hash map to tree map. The fix was verified internally at Cloudera. Additional stress test will be needed to understand the performance impact of this change. But it is functionally correct for Hadoop 3.0/CDH5.7.

@xkrogen

xkrogen commented Oct 23, 2018

Copy link
Copy Markdown
Collaborator

@jojochuang

Copy link
Copy Markdown
Author

Hi Eric,
Apologies -- my github communication goes into my personal email account and I didn't see this one.

Why is the comparator a Comparator instead of Comparator?

Good point. It should be a Block. The code was based on the HDFS-9260.

Performance -- we've not compared the performance yet since our focus is on CDH, and CDH clusters won't even run for long without this patch. I thought about using FoldedTreeSet (implemented in HDFS-9260) but then opted fora simpler, more well known data structure.

@xkrogen

xkrogen commented Jan 4, 2019

Copy link
Copy Markdown
Collaborator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Hadoop 3.0 NN drops replicas silently

2 participants