gpt4 book ai didi

scala - 从 sbt 运行 spark-shell

转载 作者:行者123 更新时间:2023-12-04 01:42:38 24 4
gpt4 key购买 nike

获取 spark shell 的默认方式似乎是 to download the distribution from the website 。然而,this spark issue 提到它可以通过 sbt 安装。我找不到这方面的文档。在使用 spark-sqlspark-core 的 sbt 项目中,没有找到 spark-shell 二进制文件。

你如何从 sbt 运行 spark-shell

最佳答案

从以下网址:
https://bzhangusc.wordpress.com/2015/11/20/use-sbt-console-as-spark-shell/

If you already using Sbt for your project, it’s very simple to setup Sbt Console to replace Spark-shell command.Let’s start from the basic case. When you setup the project with sbt, you can simply run the console as sbt console

Within the console, you just need to initiate SparkContext and SQLContext to make it behave like Spark Shell


scala> val sc = new org.apache.spark.SparkContext("localhell")
scala> val sqlContext = new org.apache.spark.sql.SQLContext(sc)

关于scala - 从 sbt 运行 spark-shell,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/56733204/

24 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com