Continuum vs. Hudson and Triggering Hudson Builds with a Subversion Post Commit Hook Script
My experience is that initially I setup Continuum as our Continuous Integration Server with Archiva as the repository manager. Both of these are developed by the maven people.
I later switched to Hudson with Nexus Maven repository manager. I was having problems keeping Archiva up on tomcat and I read about Nexus in the book "Maven the Definitive Guide". Nexus runs on Jetty. I highly suggest that book to anyone working at all with Maven. I wish that book existed when I first started using Maven. I read about Hudson in the book "Java Power Tools" and I also recommend that book to anyone coding in Java. Continuous Integration has completely changed the way I work and I cant imagine developing code without it. Once you have worked with Continuous Integration you will not be able to go without it on anything but the simplest projects
Hudson is an awesome piece of work. It has numerous plugins.
https://hudson.dev.java.net/
I have previously setup Continuum and I think that it is a great product as well. I think that Continuum does have its strengths, mainly that the people who developed it did so with Maven in mind. I think that Hudson deals with Maven but that it wasn't done from the ground up with Maven in mind and you see some deficiencies in Hudson's handling of build order, and maven dependent projects and with multi-level poms. Also Hudson can do parallel builds however when you have dependent projects in the same hudson isntance you are limited to a single build executor otherwise you get transient jar exceptions. Bascially hudson doesnt know not to delete a dependent jar when one project is building.
In Continuum there are build schedules which essentially are build configurations and you can apply those to multiple projects. Hudson does not yet have this from what I can see, although I expect it will shortly. Its a pretty glaring deficiency when you have one build configuraiton that you have to essentially copy to 5 projects in hudson.
So why then you may ask do I use Hudson over continuum, when all of my projects are built with maven. Simply, it has cooler plugins, functionality for reporting and it was vastly easier to install and to upgrade than continuum. Continuum uses a database and hudson only uses a HUDSON_HOME directory for persistent storage. When I first tried to upgrade continuum it was really painful, hudson was simply undeploy war , deploy new war and that was it. That was a major plus in my mind. Also the book Java Power Tools somewhat sold me on Hudson and I wanted to try it out. Hudson has great reporting plugins for cobertura, pmd, findbugs etc. Some of which I have only scratched the surface on.
With continuum you setup a schedule to essentially poll the Source Control Management system (SCM aka for us Subversion). With the schedules you specify the build order so everthing builds in a specific order. In hudson you can also specify the build order , however I dont know why Hudson cant seem to always figure out the appropriate build order from the maven pom itself. You can also set up Hudson to poll the SCM (subversion) or to build periodically (every hour for isntance). With hudson you can also trigger the builds remotely through a URL.
http://hudson.gotdns.com/wiki/display/HUDSON/Building+a+software+project#Buildingasoftwareproject-Configuringautomaticbuilds
What this allows you to do is the following. Setup a subversion post-commit hook script which calls the hudson url.
What this is composed of
1. A subversion post-commit hook, This script fires every time a file is commited (checked in) via subversion
In your subversion repository directory SVN_ROOT/repoisitory/hooks you will add a script if its not there.
This script does three things
1. Sends an email about the commit
2. updates our issue trackign system (Eventum, and yes I would prefer JIRA)
3. triggers the hudson builds
-------------begin file
#
# The post-commit hook is invoked after a commit. Subversion runs
# this hook by invoking a program (script, executable, binary, etc.)
# named 'post-commit' (for which this file is a template) with the
# following ordered arguments:
#
# [1] REPOS-PATH (the path to this repository)
# [2] REV (the number of the revision just committed)
#
# The default working directory for the invocation is undefined, so
# the program should set one explicitly if it cares.
#
# Because the commit has already completed and cannot be undone,
# the exit code of the hook program is ignored. The hook program
# can use the 'svnlook' utility to help it examine the
# newly-committed tree.
#
# On a Unix system, the normal procedure is to have 'post-commit'
# invoke other programs to do the real work, though it may do the
# work itself too.
#
# Note that 'post-commit' must be executable by the user(s) who will
# invoke it (typically the user httpd runs as), and that user must
# have filesystem-level permission to access the repository.
#
# On a Windows system, you should name the hook program
# 'post-commit.bat' or 'post-commit.exe',
# but the basic idea is the same.
#
# The hook program typically does not inherit the environment of
# its parent process. For example, a common problem is for the
# PATH environment variable to not be set to its usual value, so
# that subprograms fail to launch unless invoked via absolute path.
# If you're having unexpected problems with a hook program, the
# culprit may be unusual (or missing) environment variables.
#
# Here is an example hook script, for a Unix /bin/sh interpreter.
# For more examples and pre-written hooks, see those in
# the Subversion repository at
# http://svn.collab.net/repos/svn/trunk/tools/hook-scripts/ and
# http://svn.collab.net/repos/svn/trunk/contrib/hook-scripts/
REPOS="$1"
REV="$2"
/usr/lib/subversion/tools/hook-scripts/commit-email.pl "$REPOS" "$REV" --from subversion@yourcompany.com svn-commits@yourcompany.com
#log-commit.py --repository "$REPOS" --revision "$REV"
#Sinner added 8/6/08
/usr/bin/php /usr2/svn_root/java/hooks/process_svn_commits.php "$REPOS" "$REV"
#Sinner added 11/20/08 schedules hudson build
/usr/java/default/bin/java -jar /usr2/svn_root/java/hooks/svnhooks-1.0-SNAPSHOT-jar-with-dependencies.jar $REPOS $REV
--------------end file
2. An executable jar (maven assembly with plugins) which has a java class which logins via ssh to my subversion server calls a subversion command to get information about the commit. The reason I did it with ssh was mainly cause I developed it on a different machine than on the subversion server and the subversion command I need to run doesnt work remotely.
When this is actually run it is using ssh on the same local machine, however for development purposes this seemed quickest to me. I used the same jsch dependency that the ant scp task uses. Also this class makes sure that the name of the project is one of the projects built in hudson.
I created a simple maven project from the standard maven project archetype. I used this pom.
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.cms</groupId>
<artifactId>svnhooks</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<name>svnhooks</name>
<url>http://maven.apache.org</url>
<parent>
<artifactId>cms-parent</artifactId>
<groupId>com.cms</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<build>
<plugins>
<plugin>
<!--
NOTE: We don't need a groupId specification because the group is
org.apache.maven.plugins ...which is assumed by default.
-->
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>com.cms.svnhooks.HudsonPostCommitHook</mainClass>
</manifest>
</archive>
</configuration>
<executions>
<execution>
<id>make-assembly</id> <!-- this is used for inheritance merges -->
<phase>package</phase> <!-- append to the packaging phase. -->
<goals>
<goal>attached</goal> <!-- goals == mojos -->
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.jcraft</groupId>
<artifactId>jsch</artifactId>
<version>0.1.38</version>
</dependency>
</dependencies>
</project>
There is only one class in this project.
com.cms.svnhooks.HudsonPostCommitHook
----- begin class
package com.cms.svnhooks;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.InputStream;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import org.apache.log4j.Logger;
import com.jcraft.jsch.Channel;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.Session;
import com.jcraft.jsch.UserInfo;
/**
* @author Robert Sinner Nov 19, 2008
* to run from Subversion post-commit script
java -jar svnhooks-1.0-SNAPSHOT-jar-with-dependencies.jar repositoryPath revision
java -jar svnhooks-1.0-SNAPSHOT-jar-with-dependencies.jar /usr2/svn_root/java 2175
*/
public class HudsonPostCommitHook
{
private static final Logger log = Logger.getLogger(HudsonPostCommitHook.class);
String NEW_LINE = "\r";
String ENCODING = "UTF-8";
String HUDSON_SERVER = "yourhudsonserver:8080";
String HUDSON_TOKEN = "yourtoken";
String HOST = "yoursvnserver";
String USER = "yoursvnuser";
String PASSWORD = "yoursvnpassword";
Integer PORT = 22;
static List<String> hudsonProjects = new ArrayList<String>();
static{
hudsonProjects.add("cms-parent");
hudsonProjects.add("CMSShared");
hudsonProjects.add("FirestormExt");
hudsonProjects.add("stdcore");
hudsonProjects.add("cmssql");
hudsonProjects.add("aircore");
hudsonProjects.add("airweb");
}
/**
* @param args
*/
public static void main( String[] args )
{
try {
HudsonPostCommitHook hook = new HudsonPostCommitHook();
if(args == null || args.length != 2)
{
throw new RuntimeException("Args Length should be 2 (repositoryPath, revision) and it is not!");
}
String repositoryPath = args[0];
String revision = args[1];
hook.notifyHudson(repositoryPath, revision);
} catch (Exception e) {
log.error(e.getMessage());
}
}
public void notifyHudson(String repositoryPath, String revision) throws Exception
{
JSch jsch=new JSch();
Session session=jsch.getSession(USER, HOST, PORT);
UserInfo ui=new MyUserInfo();
session.setUserInfo(ui);
session.connect(30000);
Channel channel=session.openChannel("shell");
//svnlook dirs-changed /usr2/svn_root/java --revision 2175
String command = "svnlook dirs-changed "+repositoryPath+" --revision "+revision+ NEW_LINE;
log.info("command="+command);
InputStream in = new ByteArrayInputStream(command.getBytes(ENCODING));
channel.setInputStream(in);
ByteArrayOutputStream out = new ByteArrayOutputStream();
channel.setOutputStream(out);
channel.connect();
Thread.currentThread().sleep(1000);
String commandResponse = out.toString();
//log.info("commandResponse="+commandResponse);
String[] directoriesChanged = commandResponse.split(NEW_LINE);
Set<String> changedProjects = new HashSet<String>();
for (String directoryChanged : directoriesChanged) {
if(directoryChanged.contains("svnlook")
|| directoryChanged.contains("-revision")
|| directoryChanged.contains("ciserveradmin")
)
{
continue;
}
//log.info("directoryChanged="+directoryChanged);
//has to start from 1 since there is a carriage return in there
String projectName = directoryChanged.substring(1, directoryChanged.indexOf("/"));
changedProjects.add(projectName);
}
for (String changedProject: changedProjects) {
//log.info("changedProject='"+changedProject+"'");
if(isHudsonProject(changedProject))
{
scheduleHudsonBuild(changedProject);
}
}
channel.disconnect();
session.disconnect();
}
/**
* Convenience method for sorting and binarysearching through array of strings for a string
* using java Arrays class
* @param arrayToSortAndSearch
* @param searchForString
* @return
* @author Robert Sinner
*/
public static boolean listContainsString(List<String> listToSortAndSearch, String searchForString)
{
Collections.sort(listToSortAndSearch);
int index = Collections.binarySearch(listToSortAndSearch,searchForString);
//log.info("index of the search key="+index);
if(index >= 0)
{
return true;
}
else{
return false;
}
}
public boolean isHudsonProject(String projectName) throws Exception
{
if(listContainsString(hudsonProjects, projectName))
{
return true;
}
else {
return false;
}
}
public void scheduleHudsonBuild(String projectName) throws Exception
{
// Create a URL for the desired page
String urlString = "http://"+HUDSON_SERVER+"/hudson/job/"+projectName+"/build?token="+HUDSON_TOKEN;
URL url = new URL(urlString);
HttpURLConnection urlConnection = (HttpURLConnection)url.openConnection();
int httpResponseCode = urlConnection.getResponseCode();
log.info("scheduleHudsonBuild("+projectName+") httpResponseCode="+httpResponseCode);
if (httpResponseCode == HttpURLConnection.HTTP_OK)
{
//for speed not necessary to read the entire response
// BufferedReader in = new BufferedReader(new InputStreamReader(urlConnection.getInputStream()));
// String inputLine;
// while ((inputLine = in.readLine()) != null)
// {
// log.info(inputLine);
// }
// in.close();
}
else {
throw new Exception("Response Code was not OK for url "+urlString+" was "+httpResponseCode);
}
}
public class MyUserInfo implements UserInfo{
public String getPassword(){
return PASSWORD;
}
public boolean promptYesNo(String str){
return true;//accept RSA key
}
public String getPassphrase()
{
return null;
}
public boolean promptPassphrase(String message)
{
return true;
}
public boolean promptPassword(String message){
return true;
}
public void showMessage(String message){
log.info(message);
}
}
}
------ end class
3. Configuring Hudson to allow remote builds with an authentication token.
Under your projects configuration page in hudson find the build triggers section and check Trigger Builds Remotely (e.g. from Scripts) and enter in a authentication token (a url parameter that is attached to authenticate the requestor of the build).
The nice thing about this is it is truly continuous integration testing, as soon as the code is checked into subversion the hudson build is kicked off if there is a problem then hudson emails the appropriate people.
The http remote build trigger of course does mean that subversion and hudson are more tightly coupled than if hudson was just polling the subversion server. However I have noticed with polling in hudson and some of its deficiencies with Maven that you will get false test failures. This leads developers to think that hudson is crying wolf and then they ignore the results of the CI server and that defeats the entire purpose of having Continuous Integration. What should happen is that if the CI server has an error developers should stop whatever they are doing and fix that error. So false test failures are not good for promoting this.
Hope this helps somebody. Please leave comments.
I later switched to Hudson with Nexus Maven repository manager. I was having problems keeping Archiva up on tomcat and I read about Nexus in the book "Maven the Definitive Guide". Nexus runs on Jetty. I highly suggest that book to anyone working at all with Maven. I wish that book existed when I first started using Maven. I read about Hudson in the book "Java Power Tools" and I also recommend that book to anyone coding in Java. Continuous Integration has completely changed the way I work and I cant imagine developing code without it. Once you have worked with Continuous Integration you will not be able to go without it on anything but the simplest projects
Hudson is an awesome piece of work. It has numerous plugins.
https://hudson.dev.java.net/
I have previously setup Continuum and I think that it is a great product as well. I think that Continuum does have its strengths, mainly that the people who developed it did so with Maven in mind. I think that Hudson deals with Maven but that it wasn't done from the ground up with Maven in mind and you see some deficiencies in Hudson's handling of build order, and maven dependent projects and with multi-level poms. Also Hudson can do parallel builds however when you have dependent projects in the same hudson isntance you are limited to a single build executor otherwise you get transient jar exceptions. Bascially hudson doesnt know not to delete a dependent jar when one project is building.
In Continuum there are build schedules which essentially are build configurations and you can apply those to multiple projects. Hudson does not yet have this from what I can see, although I expect it will shortly. Its a pretty glaring deficiency when you have one build configuraiton that you have to essentially copy to 5 projects in hudson.
So why then you may ask do I use Hudson over continuum, when all of my projects are built with maven. Simply, it has cooler plugins, functionality for reporting and it was vastly easier to install and to upgrade than continuum. Continuum uses a database and hudson only uses a HUDSON_HOME directory for persistent storage. When I first tried to upgrade continuum it was really painful, hudson was simply undeploy war , deploy new war and that was it. That was a major plus in my mind. Also the book Java Power Tools somewhat sold me on Hudson and I wanted to try it out. Hudson has great reporting plugins for cobertura, pmd, findbugs etc. Some of which I have only scratched the surface on.
With continuum you setup a schedule to essentially poll the Source Control Management system (SCM aka for us Subversion). With the schedules you specify the build order so everthing builds in a specific order. In hudson you can also specify the build order , however I dont know why Hudson cant seem to always figure out the appropriate build order from the maven pom itself. You can also set up Hudson to poll the SCM (subversion) or to build periodically (every hour for isntance). With hudson you can also trigger the builds remotely through a URL.
http://hudson.gotdns.com/wiki/display/HUDSON/Building+a+software+project#Buildingasoftwareproject-Configuringautomaticbuilds
What this allows you to do is the following. Setup a subversion post-commit hook script which calls the hudson url.
What this is composed of
1. A subversion post-commit hook, This script fires every time a file is commited (checked in) via subversion
In your subversion repository directory SVN_ROOT/repoisitory/hooks you will add a script if its not there.
This script does three things
1. Sends an email about the commit
2. updates our issue trackign system (Eventum, and yes I would prefer JIRA)
3. triggers the hudson builds
-------------begin file
#
# The post-commit hook is invoked after a commit. Subversion runs
# this hook by invoking a program (script, executable, binary, etc.)
# named 'post-commit' (for which this file is a template) with the
# following ordered arguments:
#
# [1] REPOS-PATH (the path to this repository)
# [2] REV (the number of the revision just committed)
#
# The default working directory for the invocation is undefined, so
# the program should set one explicitly if it cares.
#
# Because the commit has already completed and cannot be undone,
# the exit code of the hook program is ignored. The hook program
# can use the 'svnlook' utility to help it examine the
# newly-committed tree.
#
# On a Unix system, the normal procedure is to have 'post-commit'
# invoke other programs to do the real work, though it may do the
# work itself too.
#
# Note that 'post-commit' must be executable by the user(s) who will
# invoke it (typically the user httpd runs as), and that user must
# have filesystem-level permission to access the repository.
#
# On a Windows system, you should name the hook program
# 'post-commit.bat' or 'post-commit.exe',
# but the basic idea is the same.
#
# The hook program typically does not inherit the environment of
# its parent process. For example, a common problem is for the
# PATH environment variable to not be set to its usual value, so
# that subprograms fail to launch unless invoked via absolute path.
# If you're having unexpected problems with a hook program, the
# culprit may be unusual (or missing) environment variables.
#
# Here is an example hook script, for a Unix /bin/sh interpreter.
# For more examples and pre-written hooks, see those in
# the Subversion repository at
# http://svn.collab.net/repos/svn/trunk/tools/hook-scripts/ and
# http://svn.collab.net/repos/svn/trunk/contrib/hook-scripts/
REPOS="$1"
REV="$2"
/usr/lib/subversion/tools/hook-scripts/commit-email.pl "$REPOS" "$REV" --from subversion@yourcompany.com svn-commits@yourcompany.com
#log-commit.py --repository "$REPOS" --revision "$REV"
#Sinner added 8/6/08
/usr/bin/php /usr2/svn_root/java/hooks/process_svn_commits.php "$REPOS" "$REV"
#Sinner added 11/20/08 schedules hudson build
/usr/java/default/bin/java -jar /usr2/svn_root/java/hooks/svnhooks-1.0-SNAPSHOT-jar-with-dependencies.jar $REPOS $REV
--------------end file
2. An executable jar (maven assembly with plugins) which has a java class which logins via ssh to my subversion server calls a subversion command to get information about the commit. The reason I did it with ssh was mainly cause I developed it on a different machine than on the subversion server and the subversion command I need to run doesnt work remotely.
When this is actually run it is using ssh on the same local machine, however for development purposes this seemed quickest to me. I used the same jsch dependency that the ant scp task uses. Also this class makes sure that the name of the project is one of the projects built in hudson.
I created a simple maven project from the standard maven project archetype. I used this pom.
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.cms</groupId>
<artifactId>svnhooks</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<name>svnhooks</name>
<url>http://maven.apache.org</url>
<parent>
<artifactId>cms-parent</artifactId>
<groupId>com.cms</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<build>
<plugins>
<plugin>
<!--
NOTE: We don't need a groupId specification because the group is
org.apache.maven.plugins ...which is assumed by default.
-->
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>com.cms.svnhooks.HudsonPostCommitHook</mainClass>
</manifest>
</archive>
</configuration>
<executions>
<execution>
<id>make-assembly</id> <!-- this is used for inheritance merges -->
<phase>package</phase> <!-- append to the packaging phase. -->
<goals>
<goal>attached</goal> <!-- goals == mojos -->
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.jcraft</groupId>
<artifactId>jsch</artifactId>
<version>0.1.38</version>
</dependency>
</dependencies>
</project>
There is only one class in this project.
com.cms.svnhooks.HudsonPostCommitHook
----- begin class
package com.cms.svnhooks;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.InputStream;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import org.apache.log4j.Logger;
import com.jcraft.jsch.Channel;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.Session;
import com.jcraft.jsch.UserInfo;
/**
* @author Robert Sinner Nov 19, 2008
* to run from Subversion post-commit script
java -jar svnhooks-1.0-SNAPSHOT-jar-with-dependencies.jar repositoryPath revision
java -jar svnhooks-1.0-SNAPSHOT-jar-with-dependencies.jar /usr2/svn_root/java 2175
*/
public class HudsonPostCommitHook
{
private static final Logger log = Logger.getLogger(HudsonPostCommitHook.class);
String NEW_LINE = "\r";
String ENCODING = "UTF-8";
String HUDSON_SERVER = "yourhudsonserver:8080";
String HUDSON_TOKEN = "yourtoken";
String HOST = "yoursvnserver";
String USER = "yoursvnuser";
String PASSWORD = "yoursvnpassword";
Integer PORT = 22;
static List<String> hudsonProjects = new ArrayList<String>();
static{
hudsonProjects.add("cms-parent");
hudsonProjects.add("CMSShared");
hudsonProjects.add("FirestormExt");
hudsonProjects.add("stdcore");
hudsonProjects.add("cmssql");
hudsonProjects.add("aircore");
hudsonProjects.add("airweb");
}
/**
* @param args
*/
public static void main( String[] args )
{
try {
HudsonPostCommitHook hook = new HudsonPostCommitHook();
if(args == null || args.length != 2)
{
throw new RuntimeException("Args Length should be 2 (repositoryPath, revision) and it is not!");
}
String repositoryPath = args[0];
String revision = args[1];
hook.notifyHudson(repositoryPath, revision);
} catch (Exception e) {
log.error(e.getMessage());
}
}
public void notifyHudson(String repositoryPath, String revision) throws Exception
{
JSch jsch=new JSch();
Session session=jsch.getSession(USER, HOST, PORT);
UserInfo ui=new MyUserInfo();
session.setUserInfo(ui);
session.connect(30000);
Channel channel=session.openChannel("shell");
//svnlook dirs-changed /usr2/svn_root/java --revision 2175
String command = "svnlook dirs-changed "+repositoryPath+" --revision "+revision+ NEW_LINE;
log.info("command="+command);
InputStream in = new ByteArrayInputStream(command.getBytes(ENCODING));
channel.setInputStream(in);
ByteArrayOutputStream out = new ByteArrayOutputStream();
channel.setOutputStream(out);
channel.connect();
Thread.currentThread().sleep(1000);
String commandResponse = out.toString();
//log.info("commandResponse="+commandResponse);
String[] directoriesChanged = commandResponse.split(NEW_LINE);
Set<String> changedProjects = new HashSet<String>();
for (String directoryChanged : directoriesChanged) {
if(directoryChanged.contains("svnlook")
|| directoryChanged.contains("-revision")
|| directoryChanged.contains("ciserveradmin")
)
{
continue;
}
//log.info("directoryChanged="+directoryChanged);
//has to start from 1 since there is a carriage return in there
String projectName = directoryChanged.substring(1, directoryChanged.indexOf("/"));
changedProjects.add(projectName);
}
for (String changedProject: changedProjects) {
//log.info("changedProject='"+changedProject+"'");
if(isHudsonProject(changedProject))
{
scheduleHudsonBuild(changedProject);
}
}
channel.disconnect();
session.disconnect();
}
/**
* Convenience method for sorting and binarysearching through array of strings for a string
* using java Arrays class
* @param arrayToSortAndSearch
* @param searchForString
* @return
* @author Robert Sinner
*/
public static boolean listContainsString(List<String> listToSortAndSearch, String searchForString)
{
Collections.sort(listToSortAndSearch);
int index = Collections.binarySearch(listToSortAndSearch,searchForString);
//log.info("index of the search key="+index);
if(index >= 0)
{
return true;
}
else{
return false;
}
}
public boolean isHudsonProject(String projectName) throws Exception
{
if(listContainsString(hudsonProjects, projectName))
{
return true;
}
else {
return false;
}
}
public void scheduleHudsonBuild(String projectName) throws Exception
{
// Create a URL for the desired page
String urlString = "http://"+HUDSON_SERVER+"/hudson/job/"+projectName+"/build?token="+HUDSON_TOKEN;
URL url = new URL(urlString);
HttpURLConnection urlConnection = (HttpURLConnection)url.openConnection();
int httpResponseCode = urlConnection.getResponseCode();
log.info("scheduleHudsonBuild("+projectName+") httpResponseCode="+httpResponseCode);
if (httpResponseCode == HttpURLConnection.HTTP_OK)
{
//for speed not necessary to read the entire response
// BufferedReader in = new BufferedReader(new InputStreamReader(urlConnection.getInputStream()));
// String inputLine;
// while ((inputLine = in.readLine()) != null)
// {
// log.info(inputLine);
// }
// in.close();
}
else {
throw new Exception("Response Code was not OK for url "+urlString+" was "+httpResponseCode);
}
}
public class MyUserInfo implements UserInfo{
public String getPassword(){
return PASSWORD;
}
public boolean promptYesNo(String str){
return true;//accept RSA key
}
public String getPassphrase()
{
return null;
}
public boolean promptPassphrase(String message)
{
return true;
}
public boolean promptPassword(String message){
return true;
}
public void showMessage(String message){
log.info(message);
}
}
}
------ end class
3. Configuring Hudson to allow remote builds with an authentication token.
Under your projects configuration page in hudson find the build triggers section and check Trigger Builds Remotely (e.g. from Scripts) and enter in a authentication token (a url parameter that is attached to authenticate the requestor of the build).
The nice thing about this is it is truly continuous integration testing, as soon as the code is checked into subversion the hudson build is kicked off if there is a problem then hudson emails the appropriate people.
The http remote build trigger of course does mean that subversion and hudson are more tightly coupled than if hudson was just polling the subversion server. However I have noticed with polling in hudson and some of its deficiencies with Maven that you will get false test failures. This leads developers to think that hudson is crying wolf and then they ignore the results of the CI server and that defeats the entire purpose of having Continuous Integration. What should happen is that if the CI server has an error developers should stop whatever they are doing and fix that error. So false test failures are not good for promoting this.
Hope this helps somebody. Please leave comments.
Re: Continuum vs. Hudson and Triggering Hudson Builds with a Subversion Post Commit Hook Script
Hi Rob, Good post.
I'm wondering why you wanted to trigger from svn instead of just have a short polling period in Hudson?
Also, there are lots of "Maven people", so you could say that Nexus is also developed by Maven people, since we have 3 Maven PMC and 3 additional Maven committers working on Nexus and Maven core.
We also use Hudson and have blogged about our grid and best practices with Hudson and Maven:
<br><a href="http://www.sonatype.com/people/tag/hudson//">http://www.sonatype.com/people/tag/hudson/</a>
I'm wondering why you wanted to trigger from svn instead of just have a short polling period in Hudson?
Also, there are lots of "Maven people", so you could say that Nexus is also developed by Maven people, since we have 3 Maven PMC and 3 additional Maven committers working on Nexus and Maven core.
We also use Hudson and have blogged about our grid and best practices with Hudson and Maven:
<br><a href="http://www.sonatype.com/people/tag/hudson//">http://www.sonatype.com/people/tag/hudson/</a>
Re: Continuum vs. Hudson and Triggering Hudson Builds with a Subversion Post Commit Hook Script
With polling we were getting transient jar exceptions alot. I have the following structure.
1. cross client project dependency
2. client specific project dependency
3. client specific war project
Lets say I have polling setup to poll project 1 at 0 minutes on the hour and then project 2 at 5 minutes after the hour and then project 3 at ten minutes after the hour.
What happens if someone commits code to project 1 and then to project 2 and then to project 3 subsequently at 7 minutes after the hour ?
Well what I have seen happen is that project 3 basically gets run in hudson at 10 minutes after the hour, and fails, this is a false negative test. You will get one false negative the subsequent build will actually correct itself . What this causes developers to do is to ignore hudsons first build failure thinking it is crying wolf. I really dislike false negative test failures because developers will then start to ignore continuous integration results and not be as reactive to them immediately.
With the scenario above and the post commit script what happens if someone commits code to project 1 and then to project 2 and then to project 3 subsequently at 7 minutes after the hour ? Well project 1 is built first then project 2 then project 3 is this results in no false negative test failures.
With the subversion post commit hook and a single executor with hudson , false negatives are severely decreased. There is still one case where we can continue to get false negatives and this happens project 3 is already in the build queue with a single excecutor. I would like hudson to be smart enough that if project 1 and 2 are added to the build queue after project 3 that it actually runs project 1 and 2 first and then project 3, currently hudson doesn't appear smart enough to figure this out from the maven dependency graph that this should happen.
In any case the short answer to the question is that we see less false negative test failures with the more truly continuous nature of the subversion post commit hook script. However this does couple the SCM more closely to the build server. Hope that answers your question.
R
S
1. cross client project dependency
2. client specific project dependency
3. client specific war project
Lets say I have polling setup to poll project 1 at 0 minutes on the hour and then project 2 at 5 minutes after the hour and then project 3 at ten minutes after the hour.
What happens if someone commits code to project 1 and then to project 2 and then to project 3 subsequently at 7 minutes after the hour ?
Well what I have seen happen is that project 3 basically gets run in hudson at 10 minutes after the hour, and fails, this is a false negative test. You will get one false negative the subsequent build will actually correct itself . What this causes developers to do is to ignore hudsons first build failure thinking it is crying wolf. I really dislike false negative test failures because developers will then start to ignore continuous integration results and not be as reactive to them immediately.
With the scenario above and the post commit script what happens if someone commits code to project 1 and then to project 2 and then to project 3 subsequently at 7 minutes after the hour ? Well project 1 is built first then project 2 then project 3 is this results in no false negative test failures.
With the subversion post commit hook and a single executor with hudson , false negatives are severely decreased. There is still one case where we can continue to get false negatives and this happens project 3 is already in the build queue with a single excecutor. I would like hudson to be smart enough that if project 1 and 2 are added to the build queue after project 3 that it actually runs project 1 and 2 first and then project 3, currently hudson doesn't appear smart enough to figure this out from the maven dependency graph that this should happen.
In any case the short answer to the question is that we see less false negative test failures with the more truly continuous nature of the subversion post commit hook script. However this does couple the SCM more closely to the build server. Hope that answers your question.
R
S
