java - 创建集群需要 InstanceProfile

Question

我试图从 Eclipse 运行 Elastic MapReduce，但无法这样做。

我的代码如下:

public class RunEMR {

    /**
     * @param args
     */
    public static void main(String[] args) {
        // TODO Auto-generated method stub
         AWSCredentials credentials = new BasicAWSCredentials("xxxx","xxxx");
            AmazonElasticMapReduceClient emr = new AmazonElasticMapReduceClient(credentials);

            StepFactory stepFactory = new StepFactory();

            StepConfig enableDebugging = new StepConfig()
                .withName("Enable Debugging")
                .withActionOnFailure("TERMINATE_JOB_FLOW")
                .withHadoopJarStep(stepFactory.newEnableDebuggingStep());

            StepConfig installHive = new StepConfig()
                .withName("Install Hive")
                .withActionOnFailure("TERMINATE_JOB_FLOW")
                .withHadoopJarStep(stepFactory.newInstallHiveStep());

            StepConfig hiveScript = new StepConfig().withName("Hive Script")
                .withActionOnFailure("TERMINATE_JOB_FLOW")
                .withHadoopJarStep(stepFactory.newRunHiveScriptStep("s3://mywordcountbuckett/binary/WordCount.jar"));

            RunJobFlowRequest request = new RunJobFlowRequest()
                .withName("Hive Interactive")
                .withSteps(enableDebugging, installHive)
                .withLogUri("s3://mywordcountbuckett/")
                .withInstances(new JobFlowInstancesConfig()
                    .withEc2KeyName("xxxx")
                    .withHadoopVersion("0.20")
                    .withInstanceCount(3)
                    .withKeepJobFlowAliveWhenNoSteps(true)
                    .withMasterInstanceType("m1.small")
                    .withSlaveInstanceType("m1.small"));

            RunJobFlowResult result = emr.runJobFlow(request);



    }

}

我得到的错误是:

Exception in thread "main" com.amazonaws.AmazonServiceException: InstanceProfile is required for creating cluster. (Service: AmazonElasticMapReduce; Status Code: 400; Error Code: ValidationException; Request ID: 7a96ee32-9744-11e5-947d-65ca8f7db0a5

我已经尝试了几个小时但无法修复它。有谁知道怎么做？

Answer 1

我遇到了同样的异常InstanceProfile is required for creating cluster。

必须像下面这样设置服务角色和工作流程角色

aRunJobFlowRequest.setServiceRole("EMR_DefaultRole")
aRunJobFlowRequest.setJobFlowRole("EMR_EC2_DefaultRole")

在那之后我就没事了。

---

AWS Document for EMR IAM Roles说

AWS Identity and Access Management (IAM) roles provide a way for IAM users or AWS services to have certain specified permissions and access to resources. For example, this may allow users to access resources or other services to act on your behalf. You must specify two IAM roles for a cluster: a role for the Amazon EMR service (service role), and a role for the EC2 instances (instance profile) that Amazon EMR manages.

所以异常消息中的单词 InstanceProfile 可能意味着 a role for the EC2 instances (instance profile) 在文档中，但我在指定 后通过了该异常作业流角色。有点奇怪。