hadoop - 向 dataproc sqoop 提交作业时出现问题

Question

我对 apache sqoop 很陌生。我在 GCP 中创建了 dataproc 集群并尝试使用以下命令提交 sqoop 作业:

gcloud dataproc jobs submit hadoop --cluster cluster123 --class=org.apache.sqoop.Sqoop --import -Dmapreduce.job.user.classpath.first=true --jars=gs://bucket123/sqoop-1.4.7-hadoop260.jar,gs://bucket123/mssql-jdbc-8.4.1.jre11.jar --connect="jdbc:sqlserver://server_ip:1433/db_name" --username=sql_user --password-file=gs://bucket123/password.txt --target-dir gs://bucket123/mssql-output-folder/ --table=table123

得到以下错误:

    ERROR: (gcloud.dataproc.jobs.submit.hadoop) unrecognized arguments:
  --import (did you mean '--format'?)
  -Dmapreduce.job.user.classpath.first=true
  --connect=jdbc:sqlserver://server_ip:1433/table123 (did you mean '--project'?)
  --username=sqlserver (did you mean '--cluster'?)
  --password-file=gs://bucket123/password.txt (did you mean '--flags-file'?)
  --target-dir
  gs://bucket123/mssql-output-folder/
  --table=test1 (did you mean '--labels'?)
  To search the help text of gcloud commands, run:
  gcloud help -- SEARCH_TERMS

Answer 1

在命令中添加作业参数之前，您需要放置“--”。
尝试这个:

gcloud dataproc jobs submit hadoop \
  --cluster cluster123 \
  --class=org.apache.sqoop.Sqoop \
  --jars=gs://bucket123/sqoop-1.4.7-hadoop260.jar,gs://bucket123/mssql-jdbc-8.4.1.jre11.jar \
  -- \
  --import -Dmapreduce.job.user.classpath.first=true --connect="jdbc:sqlserver://server_ip:1433/db_name" --username=sql_user --password-file=gs://bucket123/password.txt --target-dir gs://bucket123/mssql-output-folder/ --table=table123