sqoop import --connect jdbc:mysql://zhaosai:3306/mydb --username root --password jqe6b6 --table news --target-dir /user/news --fields-terminated-by “;” --hive-import --hive-table news -m 1出现错误Warning: /opt/programs/sqoop-1.4.7.bin__hadoop-2.6.0/../hbase does not exist! HBase imports will fail. Please set $HBASE_HOME to the root of your HBase installation. Warning: /opt/programs/sqoop-1.4.7.bin__hadoop-2.6.0/../hcatalog does not exist! HCatalog jobs will fail. Please set $HCAT_HOME to the root of your HCatalog installation. Warning: /opt/programs/sqoop-1.4.7.bin__hadoop-2.6.0/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. Warning: /opt/programs/sqoop-1.4.7.bin__hadoop-2.6.0/../zookeeper does not exist! Accumulo imports will fail. Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation. 23/06/10 16:18:23 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7 23/06/10 16:18:23 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 23/06/10 16:18:23 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 23/06/10 16:18:23 INFO tool.CodeGenTool: Beginning code generation Sat Jun 10 16:18:23 CST 2023 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. 23/06/10 16:18:24 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM news AS t LIMIT 1 23/06/10 16:18:24 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM news AS t LIMIT 1 23/06/10 16:18:24 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/programs/hadoop-2.7.6 注: /tmp/sqoop-root/compile/84ba419f00fa83cb5d16dba722729d01/news.java使用或覆盖了已过时的 API。 注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。 23/06/10 16:18:25 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/84ba419f00fa83cb5d16dba722729d01/news.jar 23/06/10 16:18:25 WARN manager.MySQLManager: It looks like you are importing from mysql. 23/06/10 16:18:25 WARN manager.MySQLManager: This transfer can be faster! Use the --direct 23/06/10 16:18:25 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path. 23/06/10 16:18:25 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql) 23/06/10 16:18:25 ERROR tool.ImportTool: Import failed: No primary key could be found for table news. Please specify one with --split-by or perform a sequential import with '-m 1'.
时间: 2024-04-01 08:36:47 浏览: 164
这段代码是用 Sqoop 工具从 MySQL 数据库中导入数据到 Hadoop 分布式文件系统,并在 Hive 中创建表。但是在导入过程中出现了错误,错误提示为“Import failed: No primary key could be found for table news. Please specify one with --split-by or perform a sequential import with '-m 1'.”,意思是在导入表 news 的过程中没有找到主键,需要使用 --split-by 参数指定一个主键或者使用 -m 1 进行串行导入。
相关问题
sqoop import --connect jdbc:mysql://zhaosai:3306/mydb --username root --password jqe6b6 --table news --target-dir /user/news --fields-terminated-by “;” --hive-import --hive-table news -m 1
这是一个Sqoop命令,用于将MySQL中的news表导入到Hive中。具体参数如下:
--connect:指定JDBC连接字符串。
--username:指定连接用户名。
--password:指定连接密码。
--table:指定要导入的表。
--target-dir:指定导入后的HDFS目录。
--fields-terminated-by:指定字段间的分隔符。
--hive-import:表示导入到Hive。
--hive-table:指定导入到Hive的表名。
-m:指定并发导入的任务数。
在执行此命令之前,请确保已安装并配置好Sqoop。
检查错误sqoop import --connect jdbc:mysql://zhaosai:3306/mydb --username root --password jqe6b6 --table news --target-dir /user/news --fields-terminated-by “;” --hive-import --hive-table news -m 1
在这个命令中,参数 `--fields-terminated-by` 后面的分隔符使用了中文的双引号 `";"`,应该改为英文的双引号 `";"`。即命令应该改为:
```
sqoop import --connect jdbc:mysql://zhaosai:3306/mydb --username root --password jqe6b6 --table news --target-dir /user/news --fields-terminated-by ";" --hive-import --hive-table news -m 1
```
另外,`--hive-table` 指定的表名需要在 Hive 中已经存在。如果不存在,需要先创建。而且,如果 `--hive-import` 指定了,那么 `--target-dir` 就不需要指定了,因为 sqoop 会自动将数据导入到 Hive 的表中。
阅读全文