β

问题记录:[Sqoop]

孤岛旭日的个人空间 304 阅读

Sqoop在导入MySQL数据时遇到Timestamp列为空时报错,解决方法是:在JDBC连接后加上?zeroDateTimeBehavior=convertToNull

Error: java.io.IOException: SQLException in nextKeyValue
    at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
    at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:556)
    at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
    at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
    at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.sql.SQLException: Value '0000-00-00' can not be represented as java.sql.Date
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1073)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:987)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:982)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:927)
    at com.mysql.jdbc.ResultSetRow.getDateFast(ResultSetRow.java:141)
    at com.mysql.jdbc.ByteArrayRow.getDateFast(ByteArrayRow.java:238)
    at com.mysql.jdbc.ResultSetImpl.getDate(ResultSetImpl.java:2185)
    at com.mysql.jdbc.ResultSetImpl.getDate(ResultSetImpl.java:2147)
    at org.apache.sqoop.lib.JdbcWritableBridge.readDate(JdbcWritableBridge.java:115)
    at com.cloudera.sqoop.lib.JdbcWritableBridge.readDate(JdbcWritableBridge.java:87)
    at party_org.readFields(party_org.java:441)
    at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:244)
    ... 12 more

Sqoop在导入MySQL数据时遇到Timestamp列为空时报错,解决方法是:在JDBC连接后加上?zeroDateTimeBehavior=convertToNull 例如:

sqoop import --driver com.mysql.jdbc.Driver --connect jdbc:mysql://192.168.4.48:3306/xdgc?zeroDateTimeBehavior=convertToNull --username root --password <...> --table party_org --hive-import --hive-table ori.party_org --incremental lastmodified --check-column last_update_time --last-value '2014-01-01 00:00:00.000'

参考:http://stackoverflow.com/questions/11133759/0000-00-00-000000-can-not-be-represented-as-java-sql-timestamp-error

作者:孤岛旭日的个人空间
孤岛旭日的博客
原文地址:问题记录:[Sqoop], 感谢原作者分享。