It's just as important to have good development processes as it is a good system architecture
ByEric J. Bruno
May 19, 2010
URL:http://www.ddj.com/tools/224900379
Eric J. Bruno is a contributing editor to Dr. Dobb's. He can be contacted at eric@ericbruno.com.
For an application's long-term success, it's just as important tohave good development processes as it is a good system architecture. Inmy experience, even the best development organizations -- those whomanage their design and coding tasks well -- tend to leave applicationdeployment and management until the end. This includes the proper use ofa source code repository such as CVS, where multiple developmentprojects may be working in parallel. Additionally, the source coderepository should be used, along with a scripted procedure, to pull anddeploy code to production environments.
Too many organizations rely on developers or systemadministrators to build software releases by hand, and then manuallydeploy them to the appropriate servers in production. This manualprocess applies to both new software releases, as well as theprovisioning of new servers. Further, they also rely on developers orother build specialists to manage parallel programming projects in thesame code base, again by hand. These approaches are fraught with dangerbecause they rely on manual processes, as well as specializedknowledge only a few people in the organization may possess. To remedythis, I suggest the following:
Before getting into the details, let's cover some basics that,hopefully, you're already following. When developing code for anysystem, it's best to adhere to the following development processprincipals:
Although I use CVS for examples in this article, most of theconcepts should apply to other source-code repositories as well, be itSubversion, Git, or Mercurial. Let's examine the procedure for handlingparallel application development first.
Based on experience on multiple projects in the past, I'vesettled upon a CVS method of working that seems to work best. Theprocess is based on code branching. It's important to note that I'veseen projects go very wrong through the improper use if branching, soI've come up with a recipe that works well.
Even if you don't maintain parallel programming teams, there maybe times when you'll need to work on two versions of the sameapplication at one time. For instance, when a software release isdeployed to production, it's important that you be able to get to theexact source code and all supporting files that were used for thatrelease, even if coding has progressed since then, in case bugs arefound in production.
To achieve this, development is done in the HEAD (sometime calledthe "tip") of the source code repository, and each release cycle mustbe performed in its own branch. This branch, uniquely labeled torepresent the particular release being tested/deployed, is created whendevelopment is complete, and a comprehensive quality-assurance (QA)cycle is about to begin prior to its release to production.
Also note that there is only one active source code branch at anyone time. This rule should be strictly followed. The branch is createdwhen development is "feature complete," and a QA cycle is about tobegin. The following steps discuss this procedure in more detail:
Because the code in a branch is merged into the HEAD when itpasses QA and is released, the HEAD is sure to have all of the codechanges (bug fixes) made during the QA cycle, even if development hasprogressed in the HEAD for the next release. Also, if a problem arisesin an earlier release, because the branches are not deleted or removedin any way, the precise code base is always available for every releaseever made. This guarantees that you can recreate any revision of yourapplication at any point in the future.
This process works well for a one development group working onone release at a time, as well as multiple development groups working onmultiple releases at a time. The most efficient approach to followinvolves two parallel development groups, where you align the releasesso that when one development group is in a QA/bug fix/release cycle, thesecond development group is coding the next release. As a result, ifyou plan each release to be roughly equal in terms size and developmenttime, the second development group should be entering QA as the firstdevelopment group deploys its release and begins coding the nextversion.
This swapping of roles between the development groups (when oneis coding, the other is testing/bug fixing) ensures the most efficientuse of resources, and that the release frequency is increased. Thisprocess has the following advantages:
Next, let's dive deeper into improvements to the softwaredeployment process. These improvements build from the process we justexamined. The example we'll follow is for enterprise (hosted)applications, but the process should apply to any type of software.
The CVS development and branching procedures outlined in theprevious section are important to ensure that the exact code andconfiguration for every release can be retrieved when needed. It alsoprovides an efficient process for parallel development groups to follow.However, it's also important that software deployment be doneefficiently, and without error. To make this process painless anderror-free, the solution I use is to combine Ant scripts with CVS, usingthe proper software release label.
Ant scripts that take as parameters the task/software name, thesoftware's CVS label, and the server name to deploy to, make for aquick, easy, and precise deployment process. For example, look at thefollowing command:
> ant deploy.xml -Dcvsuser=ebruno -Dlabel=MYAPP_2_234_build_012 -Denvironment_name=prodserver_01 deploy_app
In this command, the first parameter is the Ant script; next isthe CVS username to use; next is the CVS label to use to pull therelease; next is the server to deploy to; and the final parameter is theant task itself. In this case, the Ant task, deploy_app, clearly indicates that we are deploying software.
Let's take a closer look at the Ant scripts themselves, alongwith the shell script that makes it easier to pass in all of thoseparameters. For instance, the shell script in Example 1 sets the path, classpath,and Ant options (such as heap size) to run Ant. It also collects theparameters (CVS user, CVS label, and the environment name), and callsthe Ant script to execute the deployment.
#! /bin/shANT_HOME=/usr/local/antJAVA_HOME=/usr/javaANT_OPTS="-Xmx512m"PATH=${PATH}:${ANT_HOME}/binCLASSPATH=$CLASSPATH:$ANT_HOME/lib:$ANT_HOME/lib/jakarta-ant-1.4.1-optional.jarexport ANT_HOME JAVA_HOME ANT_OPTS PATH CLASSPATHCVSUSER=$1RELEASE=$2## ex: qa, prodserver_01, prodserver_02, etc.ENVIRONMENT=$3## true or falseFORCE_CHECKOUT=$4ant -buildfile deploy.xml -Dcvsuser=$CVSUSER -Dlabel=$RELEASE -Denvironment_name=$ENVIRONMENT -Dforce_checkout=$FORCE_CHECKOUT deploy_app
The parameters passed into this shell script are set asenvironment variables in the line that executes the ant command at theend. These are read in as part of the Ant script, as shown here (seeListing 1 for the entire script):
...<property environment="env"/><property name="cvsuser" value="${env.CVSUSER}" /><property name="label" value="${env.LABEL}" /><property name="environment_name" value="${env.ENVIRONMENT_NAME}" /><property name="force_checkout" value="${env.FORCE_CHECKOUT}" />...
The properties listed here are set with the parameters entered inthe shell script and specified as -D command-line parameters whencalling Ant. There are other properties, such as the CVS project name,and the name of application archive file being deployed, that arehard-coded in this example. You'll need to replace these with thecorrect names for your project, or you can pass them as additionalenvironment variables if you choose.
<project name="My Application Release" default="init" basedir="."><!--Read some values set in the environment (as -D parameters)--><property environment="env"/><property name="cvsuser" value="${env.CVSUSER}" /><property name="label" value="${env.LABEL}" /><property name="environment_name" value="${env.ENVIRONMENT_NAME}" /><property name="force_checkout" value="${env.FORCE_CHECKOUT}" /><!--The following could also be set from the script used to run deploy.xmlThese are examples. Replace with your project's name in CVS, etc.--><property name="cvsproject" value="MYAPP"/><property name="ear" value="myapp_ear"/><!--These params store backup location (to store current release)and the directory to deploy the new release to--><property name="application_dir" location="/apps"/><property name="backup_dir" location="/apps/backups"/><property name="release_dir" location="/apps/releases"/><property name="cvs_checkout_dest" value="${release_dir}/${label}" /><target name="init"><echo message="CVS User is ${cvsuser}"/><!-- create the timestamp --><tstamp><format property="backup.date" pattern="yyyy.MM.dd" locale="en"/></tstamp><!-- has backup been made? --><available file="${backup_dir}/${cvsproject}_bak_${backup.date}"type="dir" property="backup.present"/></target><target name="check_release"><!--Skip checkout of if already there and not forcing a new one--><condition property="skip.checkout"><and><available file="${cvs_checkout_dest}/${cvsproject}"type="dir" property="release.present"/><isfalse value="${force_checkout}" /></and></condition><echo message="skip checkout=${skip.checkout}" /></target><target name="cvs_checkout"depends="init,check_release"unless="skip.checkout"description="check out project from cvs"><!-- delete existing files before checking out --><delete dir="${cvs_checkout_dest}/${cvsproject}"quiet="true" failonerror="false" /><!-- perform the checkout (replace cvs server details with your own) --><echo message="Checking out release..."/><mkdir dir="${cvs_checkout_dest}" /><cvs command="checkout"cvsRoot=":pserver:${cvsuser}@192.168.1.2/repository/cvs"dest="${cvs_checkout_dest}"package="${cvsproject}/${ear}"tag="${label}"/><echo message="Removing CVS-specific files..."/><delete includeEmptyDirs="true"><fileset dir="${cvs_checkout_dest}" defaultexcludes="no"><include name="**/CVS/**" /></fileset></delete><echo message="Creating version.txt file..."/><!-- update version number --><propertyfile file="${cvs_checkout_dest}/${cvsproject}/${ear}/version.txt"><entry key="version" value="${label}" /></propertyfile></target><target name="backup"depends="init"unless="backup.present"description="backup old release"><echo message="Backing up old release..."/><mkdir dir="${backup_dir}/${cvsproject}_bak_${backup.date}" /><copy todir="${backup_dir}/${cvsproject}_bak_${backup.date}"><fileset dir="${application_dir}/${ear}"><!-- you can exclude directories here --><exclude name="data/**" /></fileset></copy></target><target name="deploy_app"depends="cvs_checkout,backup"description="deploy the application"><echo message="Deploying new release..."/><!-- copy application files --><copy todir="${application_dir}"preservelastmodified="true" overwrite="true"><fileset dir="${cvs_checkout_dest}/${cvsproject}" includes="${ear}/**" /></copy><!-- move environment properties --><copy todir="${application_dir}/properties"preservelastmodified="true" overwrite="true"><fileset dir="${application_dir}/${ear}/properties/${environment_name}" /></copy><!-- move environment scripts and make executable --><copy todir="${application_dir}/scripts"preservelastmodified="true" overwrite="true"><fileset dir="${application_dir}/scripts/${environment_name}/${server_type}" /></copy><chmod dir="${application_dir}/scripts" perm="ugo+x" includes="**/*"/></target></project>
The rest of the Ant script contains five targets:
Ant has built-in support for source code repositories such as CVS. For instance, the cvs_checkout target uses the following command to do its work:
<cvs command="checkout"cvsRoot=":pserver:${cvsuser}@192.168.1.2/repository/cvs"dest="${cvs_checkout_dest}"package="${cvsproject}/${ear}"tag="${label}"/>
Remember to replace the CVS server entry with the address of youractual CVS server, and the path to your CVS repository. The packagename, tag name (label), and destination directory are all derived fromthe parameters you specified in the shell script earlier.
This sample Ant script assumes you're deploying a Java EEenterprise application archive (EAR) file. However, you can pull anynumber of files and file types. The deploy Ant target contains the entries to copy the files to the server, as shown here:
<!-- copy application files --><copy todir="${application_dir}"preservelastmodified="true" overwrite="true"><fileset dir="${cvs_checkout_dest}/${cvsproject}"includes="${ear}/**" /></copy>
The parameters in this Ant task are pretty self explanatory. Theexample script contains additional entries that copy other files, suchas properties files and execution scripts to start start and stop theserver, simply as an example. You can take this further, and createscripts that deploy the actual binaries and other files for your webserver, application server, database, and so on. This will enable you toprovision new servers identically when you need to, and then deployyour software to them from that point forward.
The success of a software project goes beyond good architectureand code; it relies on good processes and procedures throughout thedevelopment, testing, and deployment phases as well. This articleillustrates that maintaining good source-code repository practices whenit comes to release management and parallel development teams ispossible if a proven plan is followed. It also illustrates thatmaintaining Ant scripts toautomate the deployment of labeled releases from your source coderepository is not difficult. You can even use them to provision entireservers into your data center. In the end, having a reliable, anderror-free, deployment process will save you a lot of time andheadaches. Using the processes and scripts provided here in your ownprojects can help you achieve that.
联系客服