V2EX = way to explore
V2EX 是一个关于分享和探索的地方
现在注册
已注册用户请  登录
V2EX 提问指南
xiaoheijw
V2EX  ›  问与答

hive 使用 show databases 报错

  •  1
     
  •   xiaoheijw · 2017-09-05 17:10:41 +08:00 · 1706 次点击
    这是一个创建于 2397 天前的主题,其中的信息可能已经有所发展或是发生改变。

    请教各位大神,问题如下:

    hive> show databases;
    FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop.util.NativeCrc32.nativeComputeChunkedSumsByteArray(II[BI[BIILjava/lang/String;JZ)V
    

    谷歌到一个一样的错误,但是作者的回答不懂是什么,说 native 的问题。不知道 native 是什么 答案链接: https://ask.hellobi.com/question/21542 错误日志如下:

    2017-09-05T08:48:21,374 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: Acquired the compile lock.
    2017-09-05T08:48:21,375 DEBUG [main] conf.VariableSubstitution: Substitution is on: show databases
    2017-09-05T08:48:21,376  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: Compiling command(queryId=root_20170905084821_10312949-18c9-439a-9573-015e66e827fe): show databases
    2017-09-05T08:48:21,378 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] parse.ParseDriver: Parsing command: show databases
    2017-09-05T08:48:21,380 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] parse.ParseDriver: Parse Completed
    2017-09-05T08:48:21,402  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: Semantic Analysis Completed
    2017-09-05T08:48:21,404 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] lazy.LazySerDeParameters: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe initialized with: columnNames=[database_name] columnTypes=[string] separator=[[B@70d63e05] nullstring=  lastColumnTakesRest=false timestampFormats=null
    2017-09-05T08:48:21,405  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, comment:from deserializer)], properties:null)
    2017-09-05T08:48:21,409 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] lazy.LazySerDeParameters: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe initialized with: columnNames=[database_name] columnTypes=[string] separator=[[B@485e13d7] nullstring=  lastColumnTakesRest=false timestampFormats=null
    2017-09-05T08:48:21,410  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] exec.ListSinkOperator: Initializing operator LIST_SINK[0]
    2017-09-05T08:48:21,411 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] lazy.LazySerDeParameters: org.apache.hadoop.hive.serde2.DelimitedJSONSerDe initialized with: columnNames=[] columnTypes=[] separator=[[B@43984213] nullstring=  lastColumnTakesRest=false timestampFormats=null
    2017-09-05T08:48:21,411 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] exec.ListSinkOperator: Initialization Done 0 LIST_SINK
    2017-09-05T08:48:21,412 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] exec.ListSinkOperator: Operator 0 LIST_SINK initialized
    2017-09-05T08:48:21,412 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] exec.ListSinkOperator: Initialization Done 0 LIST_SINK done is reset.
    2017-09-05T08:48:21,412  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase
    2017-09-05T08:48:21,412 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(HiveConf, )=0, flushCache_()=9}
    2017-09-05T08:48:21,413  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: Completed compiling command(queryId=root_20170905084821_10312949-18c9-439a-9573-015e66e827fe); Time taken: 0.037 seconds
    2017-09-05T08:48:21,413  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager
    2017-09-05T08:48:21,414  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: Executing command(queryId=root_20170905084821_10312949-18c9-439a-9573-015e66e827fe): show databases
    2017-09-05T08:48:21,415  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode
    2017-09-05T08:48:21,449 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] sqlstd.SQLStdHiveAuthorizationValidator: Obtained following objects in  filterListCmdObjects [Object [type=DATABASE, name=default], Object [type=DATABASE, name=resultdb], Object [type=DATABASE, name=test]] for user root. Context Info: QueryContext [commandString=null, forwardedAddresses=null]
    2017-09-05T08:48:21,450  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] exec.DDLTask: results : 3
    FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop.util.NativeCrc32.nativeComputeChunkedSumsByteArray(II[BI[BIILjava/lang/String;JZ)V
    2017-09-05T08:48:21,468 ERROR [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] exec.DDLTask: java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCrc32.nativeComputeChunkedSumsByteArray(II[BI[BIILjava/lang/String;JZ)V
    	at org.apache.hadoop.util.NativeCrc32.nativeComputeChunkedSumsByteArray(Native Method)
    	at org.apache.hadoop.util.NativeCrc32.calculateChunkedSumsByteArray(NativeCrc32.java:86)
    	at org.apache.hadoop.util.DataChecksum.calculateChunkedSums(DataChecksum.java:430)
    	at org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunks(FSOutputSummer.java:202)
    	at org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:163)
    	at org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:144)
    	at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.close(ChecksumFileSystem.java:412)
    	at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72)
    	at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106)
    	at org.apache.hadoop.io.IOUtils.cleanup(IOUtils.java:244)
    	at org.apache.hadoop.io.IOUtils.closeStream(IOUtils.java:261)
    	at org.apache.hadoop.hive.ql.exec.DDLTask.showDatabases(DDLTask.java:2515)
    	at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:439)
    	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
    	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
    	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2182)
    	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1838)
    	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1525)
    	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1236)
    	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1226)
    	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
    	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
    	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
    	at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
    	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
    	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:498)
    	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
    
    2017-09-05T08:48:21,469 ERROR [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop.util.NativeCrc32.nativeComputeChunkedSumsByteArray(II[BI[BIILjava/lang/String;JZ)V
    2017-09-05T08:48:21,469 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: Shutting down query show databases
    2017-09-05T08:48:21,469  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] metadata.Hive: Dumping metastore api call timing information for : execution phase
    2017-09-05T08:48:21,469 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(HiveConf, )=0, getAllDatabases_()=34}
    2017-09-05T08:48:21,470  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: Completed executing command(queryId=root_20170905084821_10312949-18c9-439a-9573-015e66e827fe); Time taken: 0.056 seconds
    2017-09-05T08:48:21,470 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: Shutting down query show databases
    2017-09-05T08:48:21,470 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] exec.ListSinkOperator: close called for operator LIST_SINK[0]
    2017-09-05T08:48:21,470  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] exec.ListSinkOperator: Closing operator LIST_SINK[0]
    2017-09-05T08:48:21,471 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] exec.ListSinkOperator: 0 Close done
    2017-09-05T08:48:21,471 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Context: Deleting result file: file:/tmp/hive/local/4dcf2a35-1dad-4899-bcab-f25a85a6778c/hive_2017-09-05_08-48-21_377_4100251675857995130-1/-local-10000
    2017-09-05T08:48:21,473 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Context: Deleting scratch dir: file:/tmp/hive/local/4dcf2a35-1dad-4899-bcab-f25a85a6778c/hive_2017-09-05_08-48-21_377_4100251675857995130-1
    
    目前尚无回复
    关于   ·   帮助文档   ·   博客   ·   API   ·   FAQ   ·   我们的愿景   ·   实用小工具   ·   2996 人在线   最高记录 6543   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 31ms · UTC 08:17 · PVG 16:17 · LAX 01:17 · JFK 04:17
    Developed with CodeLauncher
    ♥ Do have faith in what you're doing.