apache-spark - spark中的Driver和Application manager有什么区别

Question

我无法弄清楚 Spark 驱动程序和应用程序主程序之间有什么区别。基本上是运行应用程序的职责，谁做什么？
在客户端模式下，客户端机器具有驱动程序，应用程序主节点在集群节点之一中运行。在集群模式下，客户端没有任何驱动程序和应用程序主节点在同一节点(集群节点之一)中运行。
driver 和 app master 的具体操作是什么？
引用:

Spark Driver memory and Application Master memory

Spark yarn cluster vs client - how to choose which one to use?

Answer 1

根据 Spark 文档
Spark 驱动器:

The Driver(aka driver program) is responsible for converting a userapplication to smaller execution units called tasks and then schedulesthem to run with a cluster manager on executors. The driver is alsoresponsible for executing the Spark application and returning thestatus/results to the user.

Spark Driver contains various components – DAGScheduler,TaskScheduler, BackendScheduler and BlockManager. They are responsiblefor the translation of user code into actual Spark jobs executed onthe cluster.

应用程序大师在哪里

The Application Master is responsible for the execution of a singleapplication. It asks for containers from the Resource Scheduler(Resource Manager) and executes specific programs on the obtained containers.Application Master is just a broker that negotiates resources with the Resource Manager and then after getting some container it make sure to launch tasks(which are picked from scheduler queue) on containers.

简而言之，Driver 程序会将您的自定义逻辑转换为阶段、作业和任务……并且您的应用程序主程序将确保从 RM 获得足够的资源，并确保检查您在容器中运行的任务的状态。
正如您在提供的引用文献中已经说过的那样，客户端和集群模式之间的唯一区别是
在客户端，模式 司机 将在我们执行/运行 spark 应用程序/作业的机器上运行，AM 在集群节点之一中运行。
(和)
在集群模式驱动程序运行在应用程序主程序中，这意味着应用程序有更多的责任。
引用 :
https://luminousmen.com/post/spark-anatomy-of-spark-application#:~:text=The%20Driver(aka%20driver%20program,status%2Fresults%20to%20the%20user.
https://www.edureka.co/community/1043/difference-between-application-master-application-manager#:~:text=The%20Application%20Master%20is%20responsible,class )%20on%20the%20obtained%20containers。