JAVA高级面试进阶训练营视频教程

admin

 

　　本人，苦心多时，历经磨难和心血，与大家共同攻克问题难关！

 

问题一: 

　　FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient

　参考：http://blog.csdn.net/l1028386804/article/details/51566262

　解决办法：

　　进入所安装的Hive的conf目录，找到hive-site.xml，（若没修改，则是hive-default.xml.template）。

<property>

  <name>hive.metastore.schema.verification</name>

  <value>true</value>

   <description>

   Enforce metastore schema version consistency.

   True: Verify that version information stored in metastore matches with one from Hive jars.  Also disable automatic

         schema migration attempt. Users are required to manully migrate schema after Hive upgrade which ensures

         proper metastore schema migration. (Default)

   False: Warn if the version information stored in metastore doesn't match with one from in Hive jars.

   </description>

</property>

 

 

改为

<property>

  <name>hive.metastore.schema.verification</name>

  <value>false</value>

   <description>

   Enforce metastore schema version consistency.

   True: Verify that version information stored in metastore matches with one from Hive jars.  Also disable automatic

         schema migration attempt. Users are required to manully migrate schema after Hive upgrade which ensures

         proper metastore schema migration. (Default)

   False: Warn if the version information stored in metastore doesn't match with one from in Hive jars.

   </description>

</property>

 

　　然后，再重启hive，进入即可。

 　　或者  在hive安装目录下的bin目录下，./hive -hiveconf hive.root.logger=DEBUG,console

 

　　同时，最近还有人来问我。出现如下问题SeesionHiveMetaStoreClient

　　先按照我的问题一的思路去解决，若解决不了，看看你的hdfs-site.xml和core-site.xml是否正确。（因为有人问我，最后帮解决发现是这个问题）

见。如下博客

hive权威安装出现的不解错误！（完美解决）两种方法都可以

 　　和

http://www.cnblogs.com/braveym/p/6685045.html

 

 

 

 

 

 

 

　　问题二：

　　　　Hive报错为：There are 2 datanode(s) running and no node(s) are excluded in this operation.

　　详细报错为：

 org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /tmp/hive/zhou/ed69eb4e-855a-4b18-a498-345e8685533a/hive_2015-12-14_05-32-03_335_9179215496953152524-1/-mr-10004/f8d2e938-b439-495f-b886-801c98f660da/map.xml could only be replicated to 0 nodes instead of minReplication (=1).  There are 2 datanode(s) running and no node(s) are excluded in this operation.

　　参考：http://jingyan.baidu.com/article/7082dc1c65a76be40a89bd09.html

　　解决方法：

　　　　　　出现此类报错主要原因是datanode存在问题，要么硬盘容量不够，要么datanode服务器down了。检查datanode，重启Hadoop即可解决。

 

 

　　问题三：

　　　hive保持，　　Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.CopyTask

　　参考：http://blog.sina.com.cn/s/blog_75353ff40102v0d3.html

　　解决：

　　　　　　./hive -hiveconf hive.root.logger=DEBUG,console

　　　　准备通过控制台输出来定位出错位置，没想到这样设置后，运行程序就成功了。自己比较莫名其妙。

　　　　总之还是要养成常看错误日志解决问题。

 

　　　问题四：

 

Caused by: MetaException(message:Hive metastore database is not initialized. Please use schematool (e.g. ./schematool -initSchema -dbType ...) to create the schema.  If needed, don't forget to include the option to auto-create the underlying database in your JDBC connection string (e.g. ?createDatabaseIfNotExist=true for mysql))  

这是由于没有初始化Hive元数据的数据库，默认情况下，Hive的元数据保存在了内嵌的derby数据库里。  

执行如下命令方可  

　在hive安装目录下，执行　./bin/schematool -initSchema -dbType derby  

 

　　参考： http://fengshulin.iteye.com/blog/2320359

 

　　问题五：

Exception in thread "main"java.lang.RuntimeException: java.lang.IllegalArgumentException:java.net.URISyntaxException: Relative path in absolute URI:${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D

 

解决方案：

在hive 安装目录下，创建一个临时的IO文件iotmp。比如hive的安装目录是/home/hadoop/app/hive

然后将路径配置到hive-site.xml文件的以下参数中：

< property> < name>hive.querylog.location< /name> < value>/home/hadoop/app/hive/iotmp< /value> < description>Location of Hive run time structured log file< /description> < /property> < property> < name>hive.exec.local.scratchdir< /name> < value>/home/hadoop/app/hive/iotmp< /value> < description>Local scratch space for Hive jobs< /description> < /property> < property> < name>hive.downloaded.resources.dir< /name> < value>/home/hadoop/app/hive/iotmp< /value> < description>Temporary local directory for added resources in the remote file system.< /description> < /property>


问题六：
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

　　解决办法：

[hadoop@HadoopSlave1 hive-1.2.1]$ hive --service metastore &

 　　如果实在不行，就，再参考这篇。https://my.oschina.net/AlbertHa/blog/339022

 　　再实在不行，见http://www.aboutyun.com/thread-11131-1-1.html




问题七:

[hadoop@HadoopSlave1 hive-1.2.1]$ bin/hive

Logging initialized using configuration in jar:file:/home/hadoop/app/hive-1.2.1/lib/hive-common-1.2.1.jar!/hive-log4j.properties
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
... 8 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
... 14 more
Caused by: MetaException(message:Got exception: java.io.IOException No FileSystem for scheme: hfds)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:1213)
at org.apache.hadoop.hive.metastore.Warehouse.getFs(Warehouse.java:106)
at org.apache.hadoop.hive.metastore.Warehouse.getDnsPath(Warehouse.java:140)
at org.apache.hadoop.hive.metastore.Warehouse.getDnsPath(Warehouse.java:146)
at org.apache.hadoop.hive.metastore.Warehouse.getWhRoot(Warehouse.java:159)
at org.apache.hadoop.hive.metastore.Warehouse.getDefaultDatabasePath(Warehouse.java:177)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB_core(HiveMetaStore.java:600)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
... 19 more
[hadoop@HadoopSlave1 hive-1.2.1]$

 




将hadoop中share/hadoop/yarn/lib路径下jline包换成hive中匹配的jar包

 

 

欢迎大家，加入我的微信公众号：大数据躺过的坑     免费给分享

 

 

 

同时，大家可以关注我的个人博客：

   http://www.cnblogs.com/zlslch/   和     http://www.cnblogs.com/lchzls/ 

   详情请见：http://www.cnblogs.com/zlslch/p/7473861.html

 

 

　　人生苦短，我愿分享。本公众号将秉持活到老学到老学习无休止的交流分享开源精神，汇聚于互联网和个人学习工作的精华干货知识，一切来于互联网，反馈回互联网。
　　目前研究领域：大数据、机器学习、深度学习、人工智能、数据挖掘、数据分析。 语言涉及：Java、Scala、Python、Shell、Linux等 。同时还涉及平常所使用的手机、电脑和互联网上的使用技巧、问题和实用软件。 只要你一直关注和呆在群里，每天必须有收获

 

       以及对应本平台的QQ群：161156071（大数据躺过的坑）