Monday, March 17, 2014

Troubleshooting - When a data is putted into the Hive.

@  The permission is cause that the following is error.
@ You'd better change date on the log file.
--------------------------------------------------------------------
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
(88 bytes are truncated)


Thursday, March 13, 2014

Hive - Install Parquet into Hive

@ 1. Get the source download
$ git clone https://github.com/Parquet/parquet-mr.git
@ 2. Change the tag
$ git checkout parquet-1.0.0
@ 3. You only need to command the following like this.
$ mvn install -rf :parquet-hive -e -DskipTests=true
$ mv ./parquet-hive-1.0.0.jar /usr/local/hive/lib/

@ 4. You need the library under the lib
$ cd /usr/local/hive/lib
$ for f in parquet-avro parquet-cascading parquet-column parquet-common parquet-encoding parquet-generator parquet-hadoop parquet-hive parquet-pig parquet-scrooge parquet-test-hadoop2 parquet-thrift
> do
> curl -O https://oss.sonatype.org/service/local/repositories/releases/content/com/twitter/${f}/1.2.5/${f}-1.2.5.jar
> done
> curl -O https://oss.sonatype.org/service/local/repositories/releases/content/com/twitter/parquet-format/1.0.0/parquet-format-1.0.0.jar
 

Thank you
http://cmenguy.github.io/blog/2013/10/30/using-hive-with-parquet-format-in-cdh-4-dot-3/

OpenStack - How to install (I don't try it yet)

@How to install
http://naleejang.tistory.com/68

Wednesday, March 12, 2014

Hive - Install on CentOs

@1. Download the hive(user:root, place:/usr/local/src)
$ wget http://ftp.riken.jp/net/apache/hive/stable/hive-0.11.0.tar.gz

@ 2.Next you need to unpack the tarball(user:root, place:/usr/local/src)
$ tar xvf ./hive-0.11.0.tar.gz

@ 3. Move the hive unpacked(user:root, place:/usr/local/src)
$ mv ./hive-0.11.0 ../hive

@ 4. Change own(user:root, place:/usr/local)
$ chown -R hadoop.hadoop ./hive/

@5. Set the environment variable(user:root, place:/usr/local)
$ vim /etc/profile
export HIVE_HOME=/usr/local/hive
@OR (user:hadoop, place:/home/hadoop)
export HIVE_HOME=/usr/local/hive
export PATH=$HIVE_HOME/bin:$PATH


@6. You must create /tmp and /hive
$ $HADOOP_HOME/bin/hadoop fs -mkdir       /tmp
$ $HADOOP_HOME/bin/hadoop fs -mkdir       /hive
$ $HADOOP_HOME/bin/hadoop fs -chmod g+w   /tmp
$ $HADOOP_HOME/bin/hadoop fs -chmod g+w   /hive

@7. You must change the path on Hive(user:hadoop, place:/usr/local/hive/conf)
$ cp ./hive-default.xml.template ./hive-default.xml
# /user/hive/warehouse -> /hive
<property>
  <name>hive.metastore.warehouse.dir</name>
  <value>/hive</value>
  <description>location of default database for the warehouse</description>
</property>


@8. Execute hive
$ $HIVE_HOME/bin/hive

Tuesday, March 11, 2014

Link - Heartbeat

@ I want to install and use Heartbeat.
http://linux-ha.org/wiki/Main_Page

Friday, March 7, 2014

Git - Install Gitolite on the CentOs

$ ./install -to /usr/local/gitolite/bin
Can't locate Time/HiRes.pm in @INC (@INC contains: /home/git/gitolite/src/lib /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at /home/git/gitolite/src/lib/Gitolite/Common.pm line 74.
BEGIN failed--compilation aborted at /home/git/gitolite/src/lib/Gitolite/Common.pm line 74.
Compilation failed in require at ./install line 15.
BEGIN failed--compilation aborted at ./install line 15.
-----------------------------------------------------------------------------
@You should install the following Perl module. 
$ yum install perl-Time-HiRes

Monday, March 3, 2014

Link - Make a server for git

@How to install git for server.
http://opentutorials.org/course/303/2291