spark-sql --master yarn --deploy-mode cluster
时间: 2023-09-30 20:09:19 浏览: 114
Spark环境搭建——on yarn集群模式
As an AI language model, I don't have access to your cluster or environment. However, I can explain the meaning of this command.
This command is used to run Spark SQL on a YARN cluster in distributed mode. YARN is a resource manager in Hadoop that manages resources and schedules tasks. The --master yarn option specifies YARN as the cluster manager, and the --deploy-mode cluster option specifies that the Spark job should be submitted to the cluster in cluster mode, which means that the driver program runs on one of the nodes in the cluster, and the executors run on other nodes.
The spark-sql command is used to run SQL queries on data stored in Spark. It is a Spark application that provides a SQL-like interface to Spark data. When you run this command, Spark SQL will be deployed to the YARN cluster and will run the SQL queries against the data stored in the cluster.
阅读全文