Monday, April 27, 2015

Linux - You may get the following warning message when you login in Linux.


PC-4096:~ A11260$ ssh userId@172.xxx.xxx.1
Last login: Wed Apr 22 05:12:38 2015 from 172.17.90.xx
-bash: warning: setlocale: LC_CTYPE: cannot change locale (UTF-8): No such file or directory

■ The following check box should be not checked.
 @ A selected data might be garbled in your Mysql.

Thursday, April 16, 2015

Gradle - Quick start web - new!

@ This is new version
@ If you want to develop your project in localhost,
     I recommend JettyRun in Gradle.

■ It can make your project deploy automatically.
■  File>New>Project
■  Gradle>Gradle Project


   ■ Project name: gradleJetty
   ■ Sample project: Java Quickstart


■  Open the build.gradle and Add the follew plugins
  (Delete sourceCompatibility = 1.5)
   ->apply plugin: 'war'
     apply plugin: 'jetty'
     sourceCompatibility = '1.7'
     targetCompatibility = '1.7'


■  Make the directory webapp


■  Make the index file index.html


■  Move to the project home
$ gradle jettyRun


■  Can access the follow URL
http://localhost:8080/gradleJetty

■  If you want to change http port, add this into the build.gradle
jettyRun {   httpPort = 9000
  reload = 'automatic'
  scanIntervalSeconds = 2
  // Change http://localhost:9000/gradleJetty for http://localhost:9000 
  contextPath = ''
}

■  You need to change the Classpath compiled like this.

@ Enjoy to develop your project

Nginx - How to install nginx on your mac

■ If brew is not installed in your mac, need to execute the following command.
$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

■ how to install nginx on mac 
$ brew install nginx 

■ Start
$ sudo nginx 

■ Stop
$ sudo -s nginx stop

■ Check
http://localhost:8080

■ Just test the configuration file as grammar
$ sudo nginx -t

■ To load config changes, type:
$ sudo nginx -s reload

Tuesday, April 14, 2015

Https - How to generate crt and key files.

@ Change the current directory.
$ cd /usr/local/etc/nginx
$ mkdir ./ssl

@ Generate SSL key, crt files.[openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /usr/local/etc/nginx/ssl/nginx.key -out /usr/local/etc/nginx/ssl/nginx.crt]
$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /usr/local/etc/nginx/ssl/nginx.key -out /usr/local/etc/nginx/ssl/nginx.crt
Generating a 2048 bit RSA private key
.........................................+++
..............................................................+++
writing new private key to '/usr/local/etc/nginx/ssl/nginx.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:jp
State or Province Name (full name) [Some-State]:tokyo
Locality Name (eg, city) []:shibuya
Organization Name (eg, company) [Internet Widgits Pty Ltd]:ca
Organizational Unit Name (eg, section) []:test
Common Name (e.g. server FQDN or YOUR name) []:test.io
Email Address []:kim_woongjoon@cyberagent.co.jp

openssl: This is the basic command line tool for creating and managing OpenSSL certificates, keys, and other files.

req: This subcommand specifies that we want to use X.509 certificate signing request (CSR) management. The "X.509" is a public key infrastructure standard that SSL and TLS adheres to for its key and certificate management. We want to create a new X.509 cert, so we are using this subcommand.

-x509: This further modifies the previous subcommand by telling the utility that we want to make a self-signed certificate instead of generating a certificate signing request, as would normally happen.

-nodes: This tells OpenSSL to skip the option to secure our certificate with a passphrase. We need Nginx to be able to read the file, without user intervention, when the server starts up. A passphrase would prevent this from happening because we would have to enter it after every restart.

-days 365: This option sets the length of time that the certificate will be considered valid. We set it for one year here.

-newkey rsa:2048: This specifies that we want to generate a new certificate and a new key at the same time. We did not create the key that is required to sign the certificate in a previous step, so we need to create it along with the certificate. The rsa:2048 portion tells it to make an RSA key that is 2048 bits long.

-keyout: This line tells OpenSSL where to place the generated private key file that we are creating.

-out: This tells OpenSSL where to place the certificate that we are creating.

Monday, April 13, 2015

How to install play framework on mac

@ Download and decompress the zip file.
$ http://downloads.typesafe.com/play/2.2.6/play-2.2.6.zip

@ Change the current directory for the existing Play application directory.
$ cd /Users/user_account/play

@ Generate new project[enter]
$ ./play new HelloWorld
       _
 _ __ | | __ _ _  _
| '_ \| |/ _' | || |
|  __/|_|\____|\__ /
|_|            |__/

play 2.2.6 built with Scala 2.10.3 (running Java 1.7.0_45), http://www.playframework.com

The new application will be created in /Users/user_account/play-2.2.6/HelloWorld

What is the application name? [HelloWorld]

>   

@ Select the option[2]
Which template do you want to use for this new application?

  1             - Create a simple Scala application
  2             - Create a simple Java application

> 2

@ Completed
OK, application HelloWorld is created.

Have fun!

@ Changed the current directory.
$ cd HelloWorld/

@ Execute play.[play]
$ play
Getting org.scala-sbt sbt 0.13.0 ...
:: retrieving :: org.scala-sbt#boot-app
	confs: [default]
	43 artifacts copied, 0 already retrieved (12440kB/261ms)
[info] Loading project definition from /Users/user_account/play-2.2.6/HelloWorld/project
[info] Set current project to HelloWorld (in build file:/Users/kim_woongjoon/play-2.2.6/HelloWorld/)
       _
 _ __ | | __ _ _  _
| '_ \| |/ _' | || |
|  __/|_|\____|\__ /
|_|            |__/

play 2.2.6 built with Scala 2.10.3 (running Java 1.7.0_45), http://www.playframework.com

> Type "help play" or "license" for more information.
> Type "exit" or use Ctrl+D to leave this console.

[HelloWorld] $ 

@ Run HelloWorld project[run]
[HelloWorld] $ run
[info] Updating {file:/Users/user_account/play-2.2.6/HelloWorld/}helloworld...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.

--- (Running the application from SBT, auto-reloading is enabled) ---

[info] play - Listening for HTTP on /0:0:0:0:0:0:0:0:9000

(Server started, use Ctrl+D to stop and go back to the console...)

@ Connect http://localhost:9000/ on your browser.

Thursday, April 9, 2015

Flume - How to install Flume NG 1.5.2

■ Download Flume NG 1.5.2(/usr/local/src/)
$ git clone https://git-wip-us.apache.org/repos/asf/flume.git flume
$ cd flume
$ git checkout trunk

or
$ wget http://mirror.nexcess.net/apache/flume/1.5.2/apache-flume-1.5.2-bin.tar.gz

■ Decompress it
$ tar xvf ./apache-flume-1.5.2-bin.tar.gz
$ mv ./apache-flume-1.5.2-bin ../flume

■ Change owner file
$ chown -R hadoop.hadoop ./flume/


■ Change the file's name
$ cd  /usr/local/flume/conf
$ mv ./flume-env.sh.template ./flume-env.sh
$ mv ./flume-conf.properties.template ./flume.conf

Set environment variables 
$ vim  /usr/local/flume/conf/flume-env.sh

JAVA_HOME=/usr/local/java
FLUME_CLASSPATH="/usr/local/flume/lib"

Set configuration file 
$ vim  /usr/local/flume/conf/flume-env.sh

agent1.channels = ch1
# Define a memory channel called ch1 on agent1
agent1.channels.ch1.type = memory
agent1.channels.ch1.capacity = 100000
agent1.channels.ch1.transactionCapacity = 1000

# Define an Avro source called avro-source1 on agent1 and tell it
# to bind to xxx.xxx.xxx.xxx:41414. Connect it to channel ch1.
agent1.sources = avro-source1
agent1.sources.avro-source1.channels = ch1
agent1.sources.avro-source1.type = avro
agent1.sources.avro-source1.bind = 192.168.11.25
agent1.sources.avro-source1.port = 41414

agent1.sinks = hdfs-sink1
agent1.sinks.hdfs-sink1.type = hdfs
agent1.sinks.hdfs-sink1.channel = ch1
agent1.sinks.hdfs-sink1.hdfs.path = hdfs://192.168.11.23:9000/home/hadoop/data/flume/%Y%m%d/%H
agent1.sinks.hdfs-sink1.hdfs.filePrefix = ch1
agent1.sinks.hdfs-sink1.hdfs.inUseSuffix = .txt
agent1.sinks.hdfs-sink1.hdfs.fileType = DataStream
agent1.sinks.hdfs-sink1.hdfs.rollCount = 0
agent1.sinks.hdfs-sink1.hdfs.rollInterval = 1200
agent1.sinks.hdfs-sink1.hdfs.writeFormat = text
agent1.sinks.hdfs-sink1.hdfs.rollSize = 0
agent1.sinks.hdfs-sink1.hdfs.rollCount=1000000
agent1.sinks.hdfs-sink1.hdfs.batchSize = 10
agent1.sinks.hdfs-sink1.hdfs.threadsPoolSize=10

■ Execute Flume
$ bin/flume-ng agent -n agent1 -c conf -f conf/flume.conf
or
$ bin/flume-ng agent -n agent1 --conf ./conf/ -f conf/flume.conf -Dflume.root.logger=DEBUG,console

Monday, April 6, 2015

Https - Configure SSL/HTTPS support on Apache Tomcat 7

■ Configure SSL/HTTPS support on Tomcat 7
 -(sample-test.sample.local -> sample)
 -(JKS -> Java KeyStore)
$ /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/bin/keytool -genkey -alias sample -keyalg RSA -keystore /Users/kim_joon/tomcat/conf/sample.jks
Enter keystore password:   # -> sample
Re-enter new password: # -> sample
What is your first and last name?
  [Unknown]:  Joon Kim
What is the name of your organizational unit?
  [Unknown]:  sample
What is the name of your organization?
  [Unknown]:  sample
What is the name of your City or Locality?
  [Unknown]:  tokyo
What is the name of your State or Province?
  [Unknown]:  shibuya
What is the two-letter country code for this unit?
  [Unknown]:  jp
Is CN=Joon Kim, OU=sample, O=sample, L=tokyo, ST=shibuya, C=jp correct?
  [no]:  yes

Enter key password for <sample>
(RETURN if same as keystore password):                                                    # -> testtest
Re-enter new password:        # -> testtest


■ For confirming a JKS file
$ /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/bin/keytool -list -keystore /Users/kim_joon/tomcat/conf/sample.jks
Enter keystore password:     # -> sample

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 1 entry

cyberagent, Apr 6, 2015, PrivateKeyEntry,
Certificate fingerprint (SHA1): 9F:36:89:8E:B5:06:86:20:B3:47:BF:E4:94:87:0F:C0:FF:87:A9:C4

■ Configure the server.xml in Tomcat7.
 @Before
    <!-- $TOMCAT_HOME/conf/server.xml -->
    <!--
    <Connector port="8443" protocol="org.apache.coyote.http11.Http11Protocol"
               maxThreads="150" SSLEnabled="true" scheme="https" secure="true"
               clientAuth="false" sslProtocol="TLS" />
    -->
 @After
    <!-- $TOMCAT_HOME/conf/server.xml -->
    <Connector port="8443" protocol="HTTP/1.1"
               maxThreads="150" SSLEnabled="true" scheme="https" secure="true"
               clientAuth="false" sslProtocol="TLS"
               keystoreFile="conf/sample.jks"
               keystoreType="JKS"
               keystorePass="sample"            
               keyPass="sample"/>

Friday, April 3, 2015

Hbase - How to install Hbase

@ Download
$ wget http://ftp.riken.jp/net/apache/hbase/stable/hbase-1.0.0-bin.tar.gz

@ uncompress on the [/usr/local/src]
$ tar xvf ./hbase-1.0.0-bin.tar.gz
$ mv ./hbase-1.0.0 ../hbase
$ chown -R hadoop.hadoop ./hbase/
$ cd ./hbase/conf

@ Set configuration
$ vim ./hbase-env.sh
export JAVA_HOME=/usr/local/java
export HBASE_MANAGES_ZK=false

@ Set environment variables 
$ vim ./.bash_profile
export HBASE_HOME=/usr/local/hbase
export PATH=$HBASE_HOME/bin:$PATH

@Make a public key (on hadoop user)
$ ssh-keygen -t rsa

@Copy a public key into authorized_keys in other server (on hadoop user)

@ Make a folder hbase( on Name Server)
$ cd /usr/local/hadoop/bin
$ ./hadoop fs -mkdir /hbase

@ If there are 3 server on the zookeeper.
$ vim ./hbase-site.xml
# Execute start-hbase.sh on the master Hbase <----
# You have to execute the zookeeper first

@Check URL
http://192.168.11.23:60010/master-status?filter=handler

Zookeeper - How to install zookeeper

@ DownLoad zookeeper.
@ http://ftp.riken.jp/net/apache/zookeeper/
$ cd /usr/local/src/
$ wget http://ftp.riken.jp/net/apache/zookeeper/stable/zookeeper-3.4.6.tar.gz

@ Decompress zookeeper-3.4.5.tar.gz on the [/usr/local/src]
$ tar xvf ./zookeeper-3.4.6.tar.gz
$ mv ./zookeeper-3.4.6 ../zookeeper
$ chown -R hadoop.hadoop /usr/local/src/zookeeper
$ cd /usr/local/zookeeper/conf/
$ mv ./zoo_sample.cfg ./zoo.cfg

@ Set configuration
$ vim ./zoo.cfg

@ If you want Running Replicated ZooKeeper.
@ Add  the following settings into zoo.cfg
server.1=192.168.11.24:2888:3888
server.2=192.168.11.25:2888:3888
server.3=192.168.11.26:2888:3888

@ Make a tar file with zookeeper
@ Copy the zookeeper.tar to each server

@ start
$ bin/zkServer.sh start
@ If this error occur, set myId to every server.
2015-04-03 01:22:15,509 [myid:] - ERROR [main:QuorumPeerMain@85] - Invalid config, exiting abnormally
org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: Error processing /usr/local/zookeeper/bin/../conf/zoo.cfg
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:123)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:101)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
Caused by: java.lang.IllegalArgumentException: /tmp/zookeeper/myid file is missing
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(QuorumPeerConfig.java:350)
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:119)

@ First server
$ echo "1" > /tmp/zookeeper/myid
@ Second server
$ echo "2" > /tmp/zookeeper/myid
@ Third server
$ echo "3" > /tmp/zookeeper/myid

http://youngdeok-k.com/entry/7-%EC%A3%BC%ED%82%A4%ED%8D%BC-zookeeper-%ED%99%9C%EC%9A%A9-ndash-%EB%B6%84%EC%82%B0%EC%84%9C%EB%B2%84-%EA%B5%AC%ED%98%84-2%ED%8E%B8

Thursday, April 2, 2015

Spring Security - If you will not use a login page, how to use it?

@If you use other authentication system like OpenID Connect on Spring Security,
@I recommend the following code. Because it doesn't need the login processing.

try {
    // Must be called from request filtered by Spring Security, 
    // otherwise SecurityContextHolder is not updated
    UsernamePasswordAuthenticationToken token = 
     new UsernamePasswordAuthenticationToken("userId", "password");
    token.setDetails(new WebAuthenticationDetails(request));
    Authentication authentication = defaultAuthenticationProvider.authenticate(token);
    logger.debug("Logging in with [{}]", authentication.getPrincipal());
    SecurityContextHolder.getContext().setAuthentication(authentication);
} catch (Exception e) {
    SecurityContextHolder.getContext().setAuthentication(null);
    logger.error("Failure in autoLogin", e);
}


@I will post a full sample coding in the near future.
I referred to the Stackoverflow.
http://stackoverflow.com/questions/4664893/how-to-manually-set-an-authenticated-user-in-spring-security-springmvc Thank you to Kevin Stembridge