POC of code level optimization recommendation for Hive#618
POC of code level optimization recommendation for Hive#618pralabhkumar wants to merge 10 commits into
Conversation
pralabhkumar
changed the title
| import org.codehaus.jettison.json.JSONObject; | ||
|
|
||
|
|
||
| public class AzkabanJarvisCodeExtractor implements CodeExtractor { |
There was a problem hiding this comment.
Jarvis is internal to LinkedIn. This should not go to open source ideally. Definitely the URL should be in config.
Can we modify it as a JSON code fetcher where the hierarchy of which node to fetch can be configured.
Or move this code to internal repo.
There was a problem hiding this comment.
Moved URL to configuration
| @@ -0,0 +1,121 @@ | |||
| package com.linkedin.drelephant.analysis.code.impl; | |||
There was a problem hiding this comment.
Add file header here and elsewhere
|
|
||
| // Retrieve information from job configuration like scheduler information and store them into result. | ||
| InfoExtractor.loadInfo(result, data); | ||
| if (result.queueName.toLowerCase().equals("ump_normal") || result.queueName.toLowerCase().equals("ump_hp")) { |
There was a problem hiding this comment.
Giving out internal queue configuration. At a minimum make this configurable. Better model the conditional execution as an interface? Applies elsewhere too
|
@varunsaxena As mentioned in the description , this is still work in progress . Please don't review it now , as there will be considerable changes in the PR |
pralabhkumar
changed the title
|
@varunsaxena Can u please review it |
|
@mkumar1984 Please review the same |
2 participants
This PR is to add the code level optimization recommendation(as Heuristics) to Dr Elephant. This PR is specifically for Hive . For more details , please see the following design doc .
Design Doc
Testing
unit test cases are written and End to End testing is done on EI .