Thursday, March 13, 2014

Hive - Install Parquet into Hive

@ 1. Get the source download
$ git clone https://github.com/Parquet/parquet-mr.git
@ 2. Change the tag
$ git checkout parquet-1.0.0
@ 3. You only need to command the following like this.
$ mvn install -rf :parquet-hive -e -DskipTests=true
$ mv ./parquet-hive-1.0.0.jar /usr/local/hive/lib/

@ 4. You need the library under the lib
$ cd /usr/local/hive/lib
$ for f in parquet-avro parquet-cascading parquet-column parquet-common parquet-encoding parquet-generator parquet-hadoop parquet-hive parquet-pig parquet-scrooge parquet-test-hadoop2 parquet-thrift
> do
> curl -O https://oss.sonatype.org/service/local/repositories/releases/content/com/twitter/${f}/1.2.5/${f}-1.2.5.jar
> done
> curl -O https://oss.sonatype.org/service/local/repositories/releases/content/com/twitter/parquet-format/1.0.0/parquet-format-1.0.0.jar
 

Thank you
http://cmenguy.github.io/blog/2013/10/30/using-hive-with-parquet-format-in-cdh-4-dot-3/

No comments:

Post a Comment