pyspark查询基于hbase的hive external表异常：Class org.apache.hadoop.hive.hbase.HBaseSerDe not found

近期准备上线了一个基于HBase的hive，但是通过spark查询这个hive表的时候出现了如下异常：

19/12/21 15:27:18 ERROR log: error in initSerDe: java.lang.ClassNotFoundException Class org.apache.hadoop.hive.hbase.HBaseSerDe not found
java.lang.ClassNotFoundException: Class org.apache.hadoop.hive.hbase.HBaseSerDe not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2409)
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:442)
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:435)
        at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:280)
        at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:262)
        at org.apache.hadoop.hive.ql.metadata.Table.getColsInternal(Table.java:632)
        at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:615)
        at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$getTableOption$1$$anonfun$apply$7.apply(HiveClientImpl.scala:379)

出错的原因是没有找到HBaseSerDe，这个类是在hive-hbase-handler的jar包中，具体目录对应CDH版本的位置在/opt/cloudera/parcels/CDH/jars目录下面

[root@data2 jars]# pwd
/opt/cloudera/parcels/CDH/jars
[root@data2 jars]# ls -l|grep hive-hbase
-rw-r--r-- 1 root root    119117 Jul 19 01:55 hive-hbase-handler-2.1.1-cdh6.3.0.jar

在spark目录下建立一个ln指向这个jar包，问题解决

[root@data3 ~]# cd /opt/cloudera/parcels/CDH/lib/spark/jars
[root@data3 jars]#  ln -s /opt/cloudera/parcels/CDH/jars/hive-hbase-handler-2.1.1-cdh6.3.0.jar hive-hbase-handler-2.1.1-cdh6.3.0.jar