Run mvn package with multiple modules containing independent pom files without public repository

Recently i need to build a java application which must be built with dependencies from several different modules, each having their own pom files that packages them into jar. As these files are not distributed from a public repository such as Maven Central and I do not reuse repository system such as Nexus in this case. The modules are in their own folders (with names such as "SK-Utils" "SK-Statistics" "SK-DOM" "OP-Core" "OP-Search" "ML-Core" "ML-Tune" "ML-Clustering" "ML-Trees"). What makes it complicated is that these modules actually have their dependencies specified on each other. E.g., ML-Core depends on "SK-DOM"and "SK-Utils" to build and run unit testing. Running these independent modules using IntelliJ IDE is ok. However, the modules failed to build when build using command lines such as "mvn package". Therefore i wrote a bash scripts which put in the same directory containing the independent modules. The bash script basically run "mvn package" using pom file in each module, then install them to the local repository. The bash script "mvn-build.sh" looks like the following:

#!/usr/bin/env bash
dirArray=( "SK-Utils" "SK-Statistics" "SK-DOM" "OP-Core" "OP-Search" "ML-Core" "ML-Tune" "ML-Clustering" "ML-Trees")
for dirName in "${dirArray[@]}"
do
 echo $dirName
 cd $dirName

 jarPath="target/$dirName-0.0.1-SNAPSHOT-jar-with-dependencies.jar"

 if [ -d $jarPath ]; then
     chmod 777 $jarPath
 fi

    mvn package




    mvn install:install-file -Dfile=$jarPath -DgroupId=com.meme -DartifactId=$dirName -Dpackaging=jar -Dversion=0.0.1-SNAPSHOT


    cd ..
done

Just run the above script using command such as "sudo ./mvn-build.sh" from its folder should build the multiple module project.

Note that each module should have a plugin like below specified so that the "jar-with-dependencies" jar will be generated.

<build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-install-plugin</artifactId>
                <version>2.5.2</version>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-assembly-plugin</artifactId>
                <version>2.6</version>
                <configuration>
                    <descriptorRefs>
                        <descriptorRef>jar-with-dependencies</descriptorRef>
                    </descriptorRefs>
                </configuration>

                <executions>
                    <execution>
                        <id>make-assembly</id> 
                        <phase>package</phase> 
                        <goals>
                            <goal>single</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

One more note is that if you are running on environment such as centos linux and encounter "mvn: command not found" when executing "sudo ./mvn-build.sh", it may be due to the fact that the PATH environment variable not in the sudoer, in this case, just run

> sudo env "PATH=$PATH" ./mvn-build.sh

Create a passwordless SSH login to remote computers to automate launch of a zookeeper cluster

Passwordless SSH Login can be useful in situations, for example, when one likes to write a bash script that automatically setup or launch a zookeeper cluster. Suppose we want to ssh login to a computer at 192.168.2.4 without a password from a our client computer at 192.168.2.2, we can do so by creating a RSA pair of authentication key at 192.168.2.2 terminal:

> ssh-keygen -t rsa

In the terminal prompt, just press the ENTER key multiple times for each prompt. Once this is done, a public key rsa_id.pub is stored in the .ssh folder under the user root folder. Now ssh login to the 192.168.2.4 (password required at this point) to create the .ssh folder under the user root folder on that computer:

> ssh username@192.168.2.4 mkdir .ssh

The last step is to append the .ssh/rsa_id.pub file (which is the public key) from 192.168.2.2 to the .ssh/authorized_keys in 192.168.2.4 (password required at this point):

> cat .ssh/id_rsa.pub | ssh username@192.168.2.4 'cat >> .ssh/authorized_keys'

Now we can ssh login to 192.168.2.4 from the client 192.168.2.2 without password. For example, run the following command in the client 192.168.2.2 terminal:

> ssh username@192.168.2.4

Now we are ready to write a bash script to launch the zookeeper cluster which consists of the following networked computers:

192.168.2.1
192.168.2.3
192.168.2.4

First make sure that the zookeeper is setup in the cluster properly (follow instructions from this link: http://czcodezone.blogspot.sg/2014/11/setup-zookeeper-in-cluster.html and for storm at this link http://czcodezone.blogspot.sg/2014/11/setup-storm-in-cluster.html) and make sure the above steps of passwordless SSH login is repeated for 192.168.2.1 and 192.168.2.3 so that user can log into these two computers without password from the client 192.168.2.2. Now on the terminal of 192.168.2.2, create a start_zookeeper_cluster.sh script:

> touch start_zookeeper_cluster.sh
> gedit start_zookeeper_cluster.sh

In the start_zookeeper_cluster.sh that opens, enter the following lines:

ssh username@192.168.2.1 $ZK_HOME/bin/zkServer.sh start
ssh username@192.168.2.3 $ZK_HOME/bin/zkServer.sh start
ssh username@192.168.2.4 $ZK_HOME/bin/zkServer.sh start

Save and close the start_zookeeper_cluster.sh, and run the following command on 192.168.2.2 client terminal to make it executable:

> chmod +x ./start_zookeeper_cluster.sh

Now run the script to start the zookeeper cluster:

> ./start_zookeeper_cluster.sh

We can also write a stop_zookeeper_cluster.sh, start_storm_cluster.sh, stop_storm_cluster.sh in a similar way to automate the process.