gpt4 book ai didi

apache-spark - Livy REST Spark java.io.FileNotFoundException:

转载 作者:行者123 更新时间:2023-12-02 20:26:25 24 4
gpt4 key购买 nike

我是BigData的新手,我曾尝试用apache Livy调用spark作业。
与提交命令行工作正常。有天赋的我有异常(exception)

  • 命令行:

    curl -X POST --data '{"file": "/user/romain/spark-examples.jar", "className": "org.apache.spark.examples.SparkPi"}' -H 'Content-Type: application/json' http://localhost:8998/batches

  • Livy日志:
  • 2019-06-01 00:43:19,160 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    Exception in thread "main" java.io.FileNotFoundException: File hdfs://localhost:9000/home/spark-2.4.3-bin-hadoop2.7/examples/jars/spark-examples_2.11-2.4.3.jar does not exist.
    at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:795)
    .......org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

    最佳答案

    与@ cricket_007注释有关,由execute hadoop fs -copyFromLocal命令行解决

    关于apache-spark - Livy REST Spark java.io.FileNotFoundException:,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/56414713/

    24 4 0
    Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
    广告合作:1813099741@qq.com 6ren.com