Wednesday, January 29, 2014

Maven - Install Jar files into the local system without the test.

$ mvn clean install -DskipTests

Eclipse - Making configuration to compile the project existed in the local system.

@Go to the project
$ cd /Users/name/git/project
$ vim ./.project
@Add the following this.
----------------------------------------------------------------------------------------
        <buildSpec>
                <buildCommand>
                        <name>org.eclipse.jdt.core.javabuilder</name>
                        <arguments>
                        </arguments>
                </buildCommand>
                <buildCommand>
                        <name>org.eclipse.m2e.core.maven2Builder</name>
                        <arguments>
                        </arguments>
                </buildCommand>
        </buildSpec>
        <natures>
                <nature>org.eclipse.jdt.core.javanature</nature>
                <nature>org.eclipse.m2e.core.maven2Nature</nature>
        </natures>
----------------------------------------------------------------------------------------

@If there is not the [.classpath] file, 
@You might need to edit in the properties > Java Build Path > Libraries > JRE System Library in the case
$ vim ./.classpath
----------------------------------------------------------------------------------------
<classpathentry kind="con" path="org.eclipse.m2e.MAVEN2_CLASSPATH_CONTAINER"/>
----------------------------------------------------------------------------------------

Link - Hbase on Java

@ It is a very good site to Hbase on Java.
http://www.ne.jp/asahi/hishidama/home/tech/apache/hbase/Scan.html

@ How to make schema design.
http://wiki.gurubee.net/display/DEVSTUDY/NoSQL+HBase

@ Material for Hbase
http://www.google.co.kr/url?sa=t&rct=j&q=&esrc=s&source=web&cd=3&ved=0CD4QFjAC&url=http%3A%2F%2Fwww.biworld.co.kr%2Ffiles%2Fmc%2Fboard%2F0%2F416%2Fhadoop%25EA%25B8%25B0%25EB%25B0%2598_%25EB%25B9%2585%25EB%258D%25B0%25EC%259D%25B4%25ED%2584%25B0_%25EA%25B8%25B0%25EC%2588%25A0%25EC%258B%25A4%25EB%25AC%25B4%25EA%25B3%25BC%25EC%25A0%2595_hbase_ver2.0.ppt&ei=kIoNU8bnGMG-lQX0u4CgAg&usg=AFQjCNEXEfN-TQpTzI2wxmfC2DgqXLqoLw&sig2=NAfU6N1ktkXy7sB9GIj2Kw&bvm=bv.61965928,d.aGc&cad=rjt

Monday, January 27, 2014

Troubleshooting - Maven

 @If you get an error the following error(or OutOfMemory) in Maven Compile and Test.
-----------------------------------------------------------------------------------------
Exception in thread "main" java.lang.StackOverflowError
    at sun.nio.cs.UTF_8.updatePositions(UTF_8.java:77)
    at sun.nio.cs.UTF_8$Encoder.encodeArrayLoop(UTF_8.java:564)
    at sun.nio.cs.UTF_8$Encoder.encodeLoop(UTF_8.java:619)
    at java.nio.charset.CharsetEncoder.encode(CharsetEncoder.java:561)
    at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:271)
    at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
    at java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
    at java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
    at java.io.PrintStream.write(PrintStream.java:526)
    at java.io.PrintStream.print(PrintStream.java:669)
    at java.io.PrintStream.println(PrintStream.java:806)
    at org.apache.maven.cli.PrintStreamLogger.info(PrintStreamLogger.java:111)
    at org.codehaus.plexus.logging.AbstractLogger.info(AbstractLogger.java:51)
    at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:478)
    at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:481)
-----------------------------------------------------------------------------------------
@Solution
In your mvn.sh add this as the first line after the copyright and other comments.
#----------------------------------------------------------------------------------------- 
export MAVEN_OPTS="-Xms128m -Xmx1024m -Xss128m"
#----------------------------------------------------------------------------------------- 

Thursday, January 16, 2014

Link - Machine Learning

@Machine Learning
http://www.realtechsupport.org/UB/MRIII/papers/MachineLearning/Alppaydin_MachineLearning_2010.pdf
http://www.john-foreman.com/data-smart-book.html
http://www.packtpub.com/machine-learning-with-r/book
http://work.caltech.edu/telecourse.html

@There is another thing.
http://www.okstate.edu/sas/v8/saspdf/ets/chap7.pdf

Tuesday, January 14, 2014

Git - How to fork your own repository in Github

@You should fork from the your original project in advance.

@Make a clone of your project on your machine.
$ git clone https://github.com/YOURNAME/forkedrepo.git

@Add an upstream remote to your original repository.
$ git remote add upstream git@aaa.cccc.local:ORIGINALNAME/original.git

@Pull down a copy of the original repository to your new repository.
$ git fetch upstream






 


















$ git merge upstream/master



















Or, an easier way:
$ git pull upstream master

@Upload the fresh copy of your new repository back up to git
$ git push origin master 


Thursday, January 9, 2014

Hive - How to stop a job in Hive.

$ hadoop job -kill <job_id>.

Hive - A QL for NULL

SELECT count(distinct s1.uu_id)
FROM (
    SELECT distinct uu_id FROM table1
    WHERE dt >= "2014-01-01" AND dt < "2014-01-02"
   ) s1
   LEFT OUTER JOIN (
    SELECT distinct uu_id FROM table1
    WHERE dt >= "2014-01-01" AND dt < "2014-01-02"
    AND activity = 'spend'
) s2
  ON (s1.uu_id = s2.uu_id)
JOIN (
    SELECT distinct uu_id FROM table1
    WHERE dt = "2014-01-02"
    AND activity = 'spend'
) s3
  ON (s1.uu_id = s3.uu_id)
WHERE s2.uu_id IS NULL

Monday, January 6, 2014

Hbase - If you don't know a ROW name.

@Just  Execute this.
hbase(main):007:0>  scan 't1', {LIMIT => 5}
@Add Filter on the raw name.
hbase(main):007:0> scan 't1', {FILTER => "PrefixFilter('aaaa')", LIMIT => 5}

Thursday, January 2, 2014

MySkillsheets - Skills and Experiences.

System engineer and programmer with wide set of skill and experiences.

** Skill and Experiences **
* Development :
  e-commerce, blog, game API, distributed system, webmail(CDO2.0),
  education for children with tablet(Korea). Smartphone payment system(Korea).
* Recent interests :
  Real time aggregation, Big data, Vert.x and Gradle.
* Big data experience :
  Configured Hadoop, Hbase, Flume, Zookeeper.
  Developed applications for serching logs.
  Built an internal Itunes Rss ranking tool for the producer team.
* Java experience :
  Developed the batch to manage threads for ping service based.
  Developed all services based on Spring and Mybatis(iBatis).
  Developed to manage threads for services.
  Developed distributed system web services.
* Middleware setup :
  Tomcat, Apache, Memached, Zookeeper, ActiveMQ(blog).
* Watch system setup :
  MRTG, Mumin, Mon, Nagios.
* Web server setup :
  Production environment setup, optimization and automization.
* Database setup :
  Mysql(5.5), Hbase, Oracle(PLsql), MongoDB, tokyo tyrant, Neo4j.
* Server setup :
  Kernel Parameters, Dns, Sudo, OpenSSH.
* Team management and leadership :
  Experience managing a team of engineers and designer in Korea and Japen.
* API/shared library :
  Notification Push Iphone API, Invite system API, Iphone games API.
  Development and maintenance experiences.
* Release tool :
  Developed shell script to release production and staging.
  Jenkins experience for svn, git, gradle.
* Source code management(SCM) system :
  Lot of experience SVN, CSV and GIT.
* Joined studying group :
  hadoop, hive, zookeeper.
* Development tool :
  Eclipse(Plugin), Xcode
* Frame Work :
  Spring3, Struts2, Cocos2d