Tuesday, March 5, 2013

Automation tips and tricks February 2013

As promised I'm back with the summary of cool stuff that I've done with my team in Build & Release domain to help us deal with day to day problems in efficient & effective way. As I said this month was about creating tools/utilities that sounds very simple but overall their impact in productivity & agility of build release teams and tech verticals was awesome :).

Automated deployment of Artifacts : If you have ever worked with a set of maven based projects that are interdependent on each other, one of the major problem that you will face in such a setup is to have the latest dependencies in your local system. Here I'm assuming two things you would be using a Maven Repo to host the artifacts & the dependencies would be SNAPSHOT dependencies if their is active development going on dependencies as well. Now the manual way of making sure that maven repo will always have the latest SNAPSHOT version is that every-time somebody does change in the code-base he/she manually deploy that artifact to maven repo. What we have done is that for each & every project we have created a Jenkins job that check if code is checked in for a specific component & if so that component's SNAPSHOT version get's deployed to maven repo. The impact of these utilities jobs was huge as now all the developers doesn't have to focus on deploying their code to maven repo & also keeping track of who last committed the code was also not needed.

Log Parser Utility : We have done further improvement in our event based log analyzer utility. Now we also have a simple log parser utility through which we can parse the logs of a specific component & segregate the logs as per ERROR/WARN/INFO. Most importantly it is integrated with jenkins so you can go to jenkins select a component whose log needs to be analyzed, once analysis is finished the logs are segregated as per our configuration(in our case it is ERROR/WARN/INFO) after that in the left bar these segregations are shown with all the various instances of these categories and user can click on those links to go exactly at the location where that information is present in logs

Auto Code Merge : As I already told we have a team of around 100+ developers & a sprint cycle of 10 days and two sprints overlap each other for 5 days i.e first 5 days for development after tat code freeze is enforced and next 5 days are for bug fixing which means that at a particular point of time there are 3 parallel branches on which work is under progress one branch which is currently deployed in production second branch on which testing is happening and third branch on which active development is happening. You can easily imagine that merging these branches is a task in itself. What we have done is to create an automated code merge utility that tries to merge branches in a per-defined sequence if automatic merge is successful the merge proceeds for next set of branches otherwise a mail is sent to respective developers whose files are in conflict mode

Hope you will get motivated by these set of utilities & come up with new suggestions or point of improvements

5 comments:

Deepak Chauhan said...
This comment has been removed by the author.
Deepak Chauhan said...

I don't understand that why a Jenkins job was required to deploy the latest SNAPSHOT to maven repo? Did you give a try to maven update policy?

Sandeep Rawat said...

What is maven update policy

Deepak Chauhan said...

For SNAPSHOT versions you can set this policy to "always". Please ceck the given url.

http://stackoverflow.com/questions/3805329/how-does-the-updatepolicy-in-maven-really-work

But, now I understand your concern to create the job as you are not building on the location where maven repo exists. Am I right?

Sandeep Rawat said...

Deepak we had a slight different requirement. First of all our codebase is hosted on git repository & a single repository contains multiple components, now we needed an intelligent system that will upload the artifact of only that component which has some code commit in it. That's why we have built job in jenkins that first get's the latest code of a git repo after that check's which component is updated in the repo & after figuring out upload the artifact of that component only