Accelerate with Gerrit DevOps Analytics, in one click!

 

Accelerating your time to market while delivering high-quality products is vital for any company of any size. This fast pacing and always evolving world relies on getting quicker and better in the production pipeline of the products. The whole DevOps and Lean methodologies help to achieve the speed and quality needed by continuously improving the process in a so-called feedback loop. The faster the cycle, the quicker is the ability to achieve the competitive advantage to outperform and beat the competition.

It is fundamental to have a scientific approach and put metrics in place to measure and monitor the progress of the different actors in the whole software lifecycle and delivery pipeline.

Gerrit DevOps Analytics (GDA) to the rescue

We need data to build metrics to design our continuous improvement lifecycle around it. We need to juice information from all the components we use, directly or indirectly, on a daily basis:

  • SCM/VCS (Source and Configuration Management, Version Control System)
    how many commits are going through the pipeline?
  • Code Review
    what’s the lead time for a piece of code to get validated?
    How are people interacting and cooperating around the code?
  • Issue tracker (e.g. Jira)
    how long does it take the end-to-end lifecycle outside the development, from idea to production?

Getting logs from these sources and understanding what they are telling us is fundamental to anticipate delays in deliveries, evaluate the risk of a product release and make changes in the organization to accelerate the teams’ productivity. That is not an easy task.

Gerrit DevOps Analytics (aka GDA) is an OpenSource solution for collecting data, aggregating them based on different dimensions and expose meaningful metrics in a timely fashion.

GDA is part of the Gerrit Code Review ecosystem and has been presented during the last Gerrit User Summit 2018 at Cloudera HQ in Palo Alto. However, GDA is not limited to Gerrit and is aiming at integrating and processing any information coming from other version control and code-review systems, including GitLab, GitHub and BitBucket.

Case study: GDA applied to the Gerrit Code Review project

One of the golden rules of Lean and DevOps is continuous improvement: “eating your dog food” is the perfect way to measure the progress of the solution by using its outcome in our daily life of developing GDA.

As part of the Gerrit project, I have been working with GerritForge to create Open Source tools to develop the GDA dashboards. These are based on events coming from Gerrit and Git, but we also extract data coming from the CI system, the Issue tracker. These tools include the ETL, for the data extraction and the presentation of the data.

As you will see in the examples Gerrit is not just the code review tool itself, but also its plugins ecosystem, hence you might want to include them as well into any collection and processing of analytics data.

Wanna try GDA? You are just one click away.

We made the GDA more accessible to everybody, so more people can play with it and understand its potentials. We create the Gerrit Analytics Wizard plugin so you can have some insights in your data with just one click.

What you can do

With the Gerrit Analytics Wizard you can get started quickly and with only one click you can get:

  • Initial setup with an Analytics playground with some defaults charts
  • Populate the Dashboard with data coming from one or more projects of your choice

The full GDA experience

When using the full GDA experience, you have the full control of your data:

  • Schedule recurring data imports. It is just meant to run a one-off import of the data
  • Create a production ready environment. It is meant to build a playground to explore the potentials of GDA

What components are needed?

To run the Gerrit Analytics Wizard you need:

You can find here more detailed information about the installation.

One click to crunch loads of data

Once you have Gerrit and the GDA Analytics and Wizard plugins installed, chose the top menu item Analytics Wizard > Configure Dashboard.

You land on the Analytics Wizard and can configure the following parameters:

  • Dashboard name (mandatory): name of the dashboard to create
  • Projects prefix (optional): prefix of the projects to import, i.e.: “gerrit” will match all the projects that are starting with the prefix “gerrit”. NOTE: The prefix does not support wildcards or regular expressions.
  • Date time-frame (optional): date and time interval of the data to import. If not specified the whole history will be imported without restrictions of date or time.
  • Username/Password (optional): credentials for Gerrit API, if basic auth is needed to access the project’s data.

Sample dashboard analytics wizard page:

wizard.pngOnce you are done with the configuration, press the “Create Dashboard” button and wait for the Dashboard, tailored to your data, to be created (beware this operation will take a while since it requires to download several Docker images and run an ETL job to collect and aggregate the data).

At the end of the data crunching you will be presented with a Dashboard with some initial Analytics graphs like the one below:

dashboard-e1549490575330.png

You can now navigate among the different charts from different dimensions, through time, projects, people and Teams, uncovering the potentials of your data thanks to GDA!

What has just happened behind the scenes?

When you press the “Create Dashboard” button, loads of magic happens behind the scenes. Several Docker images will be downloaded to run an ElasticSearch and Kibana instance locally, to set up the Dashboard and run the ETL job to import the data. Here a sequence workflow to illustrate the chain of events is happening:

components.png

Conclusion

Getting insights into your data is so important and has never been so simple. GDA is an OpenSource and SaaS (Software as a Service) solution designed, implemented and operated by GerritForge. GDA allows setting up the extraction flows and gives you the “out-of-the-box” solution for accelerating your company’s business right now.

Contact us if you need any help with setting up a Data Analytics pipeline or if you have any feedback about Gerrit DevOps Analytics.

Fabio Ponciroli – Gerrit Code Review Contributor – GerritForge Ltd.

GitMinutes #30: Luca Milanesio on Gerrit Code Review

git-minutesMany thanks to Thomas Ferris Nicolaisen for inviting me to talk about Gerrit Code Review at GitMinutes.

It has been a very interesting discussion on the benefits of Code Review and how Gerrit can help out small and large companies embracing it.

The interview is available on-line at http://episodes.gitminutes.com/2014/07/gitminutes-30-luca-milanesio-on-gerrit.html, alternatively you can download and listen the 1h 27′ conversation on PodCast at https://itunes.apple.com/de/podcast/gitminutes-podcasts/id637843725?l=en.

Use the force Luca!

We started (of course!) talking about the [in]famous force push of 186 Jenkins repositories to GitHub, I was on the Top-10 HackersNews over 7h … so I was expecting the question to pop-up during the interview 🙂

My friend Alex Blewitt took the opportunity as well to forge a Star-Wars like headline for his InfoQ article on what happened.

Git adoption in the Enterprise, where all began

We moved the discussion to the foundation of my business on Git and Code Review and the reasons and challenges that an Enterprise company is facing when moving to Git. We went through the history on how LMIT started GitEnterprise.com and then focused on Gerrit Code Review based product and services for large Enterprises World-Wide: a niche and successful business nowadays.

GitHub or Gerrit? or both with GerritHub?

As I expected, we ended up comparing GitHub and Gerrit analysing the similarities and differences between the two. This topic has been presented as well in two conferences at Gerrit User Summit @GooglePlex – Mountain View CA and 33rd Degree.org Java Developers Conference in Krakow; slides are available at http://www.slideshare.net/lucamilanesio/gerrit-codereviewgit-hubplugin.

Gerrit has historically been considered as “more difficult” than GitHub: true in the past but not anymore today apart from the Web User-Experience CSS styling, much nicer and pleasent on GitHub. The availability of http://gerrithub.io allowed over 1,800 developers since October 2013 to get started with Gerrit in less than 5 minutes by watching an Gerrit Introductionary YouTube video: using it was then just 3 clicks away, no installation or configuration needed! The availability of an easy and accessible Public Cloud instance represents a big improvement in accessibility and usability of Gerrit.

For which teams is Gerrit the right choice?

We talked about the “typical learning curve” of people coming from previous version control systems, such as Subversion. Does it make sense to get started with Git and Gerrit at the same time? When is Gerrit needed and when is it going to provide most of its value?

I’ve covered the topic in the past webinars and talks: hands-on Webinars recordings are freely available on-line at:

The size of the project (in terms of number of people x number of repositories) is typically one of the key factors in Code Review adoption. Gerrit however can be used as well as a standalone OpenSource Git Server , even without leveraging its Code Review capabilities: this makes the choice of Gerrit a good first step towards a smoother Git adoption.

What are Gerrit Topics about?

We went through a very interesting discussion about “Gerrit Topic”, a feature that is not new to Gerrit but is sometimes forgotten besides its important and relevance for medium-large teams.

With the forthcoming support of multi-repositories atomic commits in Gerrit, it will be possible to merge multiple changes on multiple repositories at the same time for a single topic. This feature is not ready yet but coming hopefully in the near future and Google Gerrit Team developers and contributors are working on it.

The ability to make an atomic commit across multiple repositories will allow to have a more consistent Jenkins build process as well, with less broken builds because of interdependent changes on multiple components.

Who is using Gerrit today?

We talked about the adoption of Gerrit in the community, which is growing year after year. A lot of medium companies adopted Gerrit in the past, including Spotify side-by-side with GitHub.

The ability to “submit a change” to any project without the risk to break the build is definitely an incentive to encourage even more people to contribute to share the knowledge and improve the code base, without the risk of breaking anything or  forking the code. This is one of the reason that drove large OpenSource organisations such as the Eclipse Foundation and OpenStack to the adoption Gerrit Code Review in their tools platform.

How to embrace Code Review in a Team or Company?

We went through an interesting comparison / discussion of Agile Methodology vs. Code Review. Often Teams misunderstand and confuse the concept of “review” with “pair-programming”: the problem was well analysed in my book “Learning Gerrit Code Review” (available on Amazon.com at http://www.amazon.com/Learning-Gerrit-Code-Review-Milanesio/dp/1783289473). I defined the pair-programming as a dot in a time/people space: two developers writing a piece of code at the same time. This however does not exclude all the other points in the time/people space where multiple people at different times will read the code and provide their feedback: pair-programming is then a “specific example” of the “code review space”.

Because of the different perspectives (pair-programming is a dot whilst code-review is a “cloud of dots” in time/people space) they are not one exclusive of the other: they are equally important and both enable effective collective code ownership and knowledge sharing.

References and greetings.

It has been a very long but interesting discussion with Thomas and hope you’ll enjoy it.

See below the links of the resources we mentioned during the interview:

Thanks again to Thomas for his fantastic initiative: GitMinutes PodCast!

Luca Milanesio 

Continuous Development with GerritForge and Codenvy

On March 22nd, come see Codenvy CEO Tyler Jewell and Gerritforge CEO Luca Milanesio present at Google’s HQ in Mountain View, CA. They’ll cover Codenvy’s continuous delivery system for integrating code reviews, git, and SAAS developer environments in order to eliminate waste in the development workflow.

[…]

Read the full story at Codenvy Blog
[by Eric Cavazos]

Gerrit User Summit 2014 talks proposals

The list of talks proposed for the next forthcoming Gerrit User Summit in Mountain View has been published.

There are very interesting talks on ideas, extensions and case studies from large enterprises and projects: it is going to again an exciting rendez-vous for all of those interested in SCM, SDLC and Continuous Agile.

See below a distilled summary of the proposed topics:

  • Using Gerrit and Jenkins together for the LibreOffice OpenSource Project
  • How to manage and monitor Gerrit using JavaMelody
  • Extend the GitHub fork & pull-request model using Gerrit Code Review lifecycle and GerritHub.io
  • Extending Gerrit with scripting plugins (Groovy, Jython and Scala)
  • Continuous Development and Code Review with Codenvy
  • Large scale Gerrit installations with testimonials from OpenStack, Yahoo and Ericsson !
  • Integrating and using Gerrit in the Enterprise with CollabNet TeamForge
  • … and new talks are coming over !

Seats are running out quickly but there are still spaces available: you can register now for free to the Gerrit User Summit event:

See you soon at the Gerrit User Summit 2014 !

Go Agile with Git Part 3 of 3: Hands-On Lab with Gerrit & Jenkins

git+gerrit-webinar LM-20130208-Series-3 LS

Thank you to CollabNet Inc. for the opportunity to showcase Gerrit Code Review “in action” to a broad audience of over 120 attendees from all over the world ! During the last Session we have seen how to create your first Code-Review with Gerrit and getting automatic validation through Jenkins Gerrit-Trigger-Plugin.

NOTE: Unfortunately the session was broken in the middle of Q&A due to a technical issue with the Webinar conference tool: we invite you to comment on this blog by posting your additional questions, we will be happy to answer all of them.

GerritForge LLP is proud partner of CollabNet Inc. and invites you to replay the three webinars of the series:

  1. Workflows, Branching & Merging (View on demand)
    Learn the basic concepts of Git and Gerrit and see explore the power of merging and applying code from different branches thanks to recursive merge, rebase and cherry-pick.
  2. Peer Programming & Code Reviews (View on demand)
    See how Gerrit Code Review can enforce full code collaboration and collective ownership, similarly to peer programming you can achieve faster and better code delivery. Learn how to integrate Jenkins CI into the development process and have automatic validation to feedback into your code review workflow.
  3. Hands-On Lab with Gerrit & Jenkins (View on demand)
    Step-by-step hands-on with Gerrit to learn how to move your first steps with Code Review: contribute your change, trigger your Jenkins CI validation and learn how to understand the feedbacks and amend your code for achieving a successful merged change into the main branch. Learn how to manage multiple reviews when the code evolves.

See below the log of Question and Answers of the Gerrit Code Review webinar, brought to you by GerritForge LLP in order to give you the opportunity of getting more insight on Gerrit and trigger more discussion and feedback on Code Review.

Gerrit Code Revew – Q&A

Q: Is refs/for/master branch created for each user by gerrit?.
A: Gerrit automatically manages the “refs/for/<target-branch>” logical branches (not real branches on Git Server). Whenever a commit is pushed to a “refs/for/<target-branch>”, Gerrit capture the commit and creates (or add to) a Change for review  for the target-branch indicated. Different commits pushed by different users create different changes.

Q: would you reccommend using -SNAPSHOT in the POM’s version during development phase, if so when to remove it for prod releases?
A: Yes, it is a common and good practice to use -SNAPSHOT suffix for labelling maven artifacts that are still in development. On stable branches however it is recommended to edit (possibly via scripting) all the pom.xml and use a final version number, without the -SNAPSHOT suffix.

Q: Is it possible to share the job configurations from Jenkins
A: Jenkins has been integrated with Gerrit using the Git Plugin Ver. 1.1.18 plus Gerrit Trigger Plugin Ver. 2.5.2; Gerrit configuration can be found in the global Jenkins configuration screen:

Screen Shot 2013-02-18 at 08.25.49

Gerrit Trigger contains the SSH details to connect from Jenkins to Gerrit Server (gitent and /home/gitent are the CollabNet TeamForge 6.2 user and user home directory under which Jenkins CI is running on):

Screen Shot 2013-02-18 at 08.29.29

Jenkins Job configuration / Git settings, needed to instruct the Git plugin on the ref-spec and branch to clone when a new Gerrit Change is pushed:

Screen Shot 2013-02-18 at 08.35.43

 

Git plugin advanced options are specified:
1) wipe out the workspace before build (as changes could be taken from completely different branches every time)
2) Trigger a build from Gerrit events / changes.

Screen Shot 2013-02-18 at 09.07.47

 

Gerrit Trigger settings on the Jenkins Job:

Screen Shot 2013-02-18 at 09.09.58

Q: Is ammend command part of git? Can it be done via eclipse or some other IDE?
A: The “–amend” option allows to edit the last commit objects in HEAD. EGit (Eclipse Git plugin) allows to amend using  a specific icon on the top-right corner of the commit window.

egit-amend

Q: what code deployment tools are recommended with GIT/Gerrit/Jenkin teamforge solution
A: CollabNet recommends UC4 (See: http://www.collab.net/deploy). Please refer to a CollabNet tech sales representative for more information.


Thank you again for your attendance to all the three webinars and stay tuned as new Webinars and training material on Git and Gerrit is coming soon !

Go Agile with Git Part 2 of 3: Collective code ownership with Code Review

Gerrit Code Review

Thank you to all the attendees of the second webinar of the Series “Go Agile with Git” sponsored by CollabNet Inc.

Session was about an high level view of the Gerrit Code Review workflow and his benefits in encouraging collaboration around the code and collective ownership of the Software Project. Moreover we have seen how Code Review and Continuous Integration can bring the “early integration” together with the scoring and promotion of changes into mainstream branches.

GerritForge LLP is proud partner of CollabNet Inc. and invites you to replay the first two webinars and register to the last webinar of the series:

  1. Workflows, Branching & Merging (View on demand)
    Learn the basic concepts of Git and Gerrit and see explore the power of merging and applying code from different branches thanks to recursive merge, rebase and cherry-pick.
  2. Peer Programming & Code Reviews (View on demand)
    See how Gerrit Code Review can enforce full code collaboration and collective ownership, similarly to peer programming you can achieve faster and better code delivery. Learn how to integrate Jenkins CI into the development process and have automatic validation to feedback into your code review workflow.
  3. Hands-On Lab with Gerrit & Jenkins (February 12th) << Next Webinar
    Step-by-step hands-on with Gerrit to learn how to move your first steps with Code Review: contribute your change, trigger your Jenkins CI validation and learn how to understand the feedbacks and amend your code for achieving a successful merged change into the main branch. Learn how to manage multiple reviews when the code evolves.

See below the full log of Question and Answers of the Gerrit Code Review webinar, brought to you by GerritForge LLP in order to give you the opportunity of getting more insight on Gerrit and trigger more discussion and feedback on Code Review.

Gerrit Code Revew – Q&A

Q: Please stop selling and instead show us the features.
A: The initial two webinars focus was more on the concepts introduced by Git and Gerrit, alongside with their support in commercial tools such as CollabNet TeamForge ALM product: this may have been perceived as “product selling” rather than functional and conceptual introduction to Code Review; that was not really our intention.
The third Webinar of the series (Hands-On Lab with Gerrit & Jenkins – February 12th) includes a step-by-step hands-on lab on how to use Gerrit from both Web-UI and command line for managing the Code Review workflow in a real-life environment, integrated with Jenkins CI Gerrit Trigger plug-in for code Validation. We will use Gerrit 2.1.8 embedded in CollabNet TeamForge 6.2 for our lab exercises, however they same exercises can be repeated with any other version of Gerrit.

Q: what were the three branches you mentioned? (1: _____ branch, 2: stabilization branch, 3: ______ branch)
A: You typically have at least three branches on your Git repositories: 1. main development branch (master), 2. stabilisation and release branch, 3. production and maintenance branch. The level of access control and code-review workflow you may want to apply to different branches is potentially different: Gerrit Code Review allows the definition of ACLs on a per-branch basis, allowing to easily manage a more fine-grained access to contributors on development branch whilst putting more strict control and validation on stabilisation and production branches.

Q: I tried to POC Gerrict but fell at the 1st hurdle – the login to webiste was insisting on an OpenID logon – we run behind a strict firewall so OpenID did not work – is there a way to allow a simple logon process at least for the QA
A: Gerrit Code Review set-up typically requires to have an existing user registry for authenticating users: I suggest to use a simple plain LDAP configuration rather than OpenID (i.e. you could download ApacheDS to set-up a sample registry) or download the CollabNet TeamForge 6.2 VMware appliance that includes Gerrit 2.1.8 integrated with CollabNet user registry.
For a more recent Gerrit 2.5.1 binary distribution, you could download the GerritForge 2.5.1 binary distribution that includes a step-by-step setup procedure that provides: 1. creation of an initial administrative account without the need of any external user registries or LDAP, 2. initial configuration of groups and roles, 3. sample repositories ready to be used as templates.

Q: Can Gerrit be used in Linux and Microsoft OSes ?
A: Gerrit is a Java EE based application; can be hosted in any OSes that supports Java JDK 1.6 or later. There is still a binary dependency on Git executable in the Gerrit wrapper start-up shell: this can be simply resolved by porting the gerrit.sh to the target Operating System you want Gerrit to be hosted.
GerritForge 2.5.1 for instance is available for all major UNIX platforms, Mac OSX and Microsoft Windows Servers and is integrated as automatic native system service.

Q: How Gerrit was different from sonar and hudson integrated to SVN ?
A: Sonar and Hudson are excellent tools for executing of a number of quality checks every time that a commit is created. However they do not include the ability to provide change-promotion based on voting plus collaboration tracking in terms of ideas and comments to the code. They remain however excellent tools that can be used as complement to the Code Review process, for providing additional automated score on change-sets uploaded to Gerrit.

Q: Is there any good web based code review tool like Gerrit on Subversion?
A: There are definitely other tools on the market, for instance CollabNet TeamForge includes ReviewBoard whilst Atlassian has its own Crucible commercial product: both can be used on top of other SCMs including Subversion. They are not comparable to Gerrit though, because the underlying SCM is not powerful as Git: when managing code-reviews you do need the ability to amend your code history and merge changes back to the target branch when review is completed. Subversion and other SCMs do not provide that flexibility, as a consequence the code-review on top of it cannot provide all the functionalities provided out-of-the-box by Gerrit.

Q: Can we integrate Bugzilla and / or QC with Team Forge
A: CollabNet provides integration to HP QC with his connector architecture; potentially its CollabNet Connector Framework would allow integration with other SCMs such as Bugzilla.
ccf 4-3-09
If you would like to just integrate change-sets to issues, Gerrit includes a generic Issue-Tracker Connector framework that allows to connect to potentially any external Issue-Tracker. Bugzilla has been recently supported by WikiMedia and HP QC is currently in the GerritForge 2.6 pipeline: those plugins allow Gerrit to talk directly to QC and Bugzilla for associating code commits and code reviews to existing issues and linking their respective workflows together.
GerritForge4HPQC-Ver.2.6

Q: What are the major differences between Gerrit/Jenkins and existing enteprise application testing tools such as Junit/ANT or Oracle Application testing framework? Can they be integrated with Eclipse IDE for code reviewing and testing purposes?
A: Gerrit/Jenkins are not integrated out-of-the-box with Junit/ANT or Oracle Application Testing tools: main purpose of Gerrit is not to execute tests against the code but enforcing maximum visibility of changes in the Development Team and beyond.
Gerrit itself does not enforce the numbers and nature of checks against your code; however Gerrit can be potentially integrated with Junit/Ant and Oracle Application Testing tools to collect their feedback and track as score into a change-set review. From Gerrit 2.6 the code-review functions will be available even via RESTFul API: automation through Ant will be possible via scripting or custom tasks. Gerrit is already integrated with Eclipse IDE via Mylyn and its native Gerrit plugin: it allows to receive feedback in realtime on incoming code-reviews and display comments as annotations inside Eclipse standard code editor.

Screen Shot 2013-02-03 at 19.50.45

Q: Can we Integrate Maven and / or Ant with Team Forge
A: TeamForge provides a SOAP Interface and both Java and Python interface to access via external tools or scripting. This means that potentially both Maven and Ant can be used for accessing TeamForge artifacts.

Q: Can we block commits to the master branch and only use ref/for/master and force commits to gerrit code review ?
A: Yes, by using Gerrit fine-grained ACLs to refs/heads/master and allowing push for review to refs/for/refs/heads/master.

Q: Slide #28 (Gerrit workflow): What happens if master contains code conflicting the check-in of A2* (because this check-in was done prior to change A2*)? Can Gerrit auto-resolve this conflict?
A: No, as Gerrit does not know how to resolve automatically a conflict: Gerrit can try to merge the code anyway, depending on the Gerrit project settings, but if a conflict is detected, change-sets cannot merged back into the branch.

git+gerrit-webinar LM-20130126-Series-2-Final.workflow

Q: That overview graph (CI + CR with traffic lights, branches) you showed was very helpful, to see what you are talking about. Which tools were you saying that covered? Gerrit and Git? TeamForge?
A: Workflow shows the usage of Git, Gerrit Code review and Jenkins CI together to validate a Change-Set and merging it to master branch. Master branch is always with a “green” street light (Build is healthy) whilst the “red” and “amber” lights are only on the refs/for/master (code-review branch) and do not impact the rest of project team development. TeamForge is the ALM to connect everything together (Git, Gerrit and Jenkins) in a unique environment.

Q: Is there integrations between other CI with TeamForge?
A: TeamForge provides integration with Cruise Control and Maven as well. Potentially other Continuous Integration tools can be integrated with TeamForge by using the SOAP or Java and Python API.

Q: Just wondering number of paritcipants on call, to relate to % in polls. 
A: 138

Q: How would you recommend a small distributed group (4 devs, 2 testers) to sort of “try out” this complex approach you are promoting here, while continuing to maintain our daily work via Subversion?
A: Git allows to pull and push back to Subversion, using the “git svn” commands. You can dedicate a branch (i.e. trunk-review) to experiment Gerrit code-review workflow, whilst keeping the rest of the development on the standard trunk branch. Once code-review is completed and merged to trunk-review, branch can be pushed back to Subversion and tracked as a normal regular commit. Bear in mind that Subversion does not have all the power of Git on merging code together, you may need extra manual work then to bring the changes back into the main trunk.

Q: How does Teamforge run Gerrit? Can Gerrit run as a Windows service?
A: TeamForge runs Gerrit into a separate JVM, activated and managed in the same way as any other TeamForge daemons. Gerrit itself can be activated either via gerrit.sh wrapper script or as standard Java WAR application into a standalone Servlet Container (i.e. Tomcat). In this second form it can be technically executed into a Windows service, such as Apache Tomcat running into a Java Service Wrapper.

Q: We’re using TeamCity as CI, can it be used and integrated with Gerrit ?
A: In theory they can be integrated, using Gerrit SSH or RESTFul (from Gerrit 2.6) scripting. See this example from Timothy Basanov BLOG. Unfortunately TeamCity is not OpenSource and thus has a more limited community of plugin developers providing integrations for Git and Gerrit. Jenkins CI  has the world’s largest community of plugin developers for a Continuous Integration engine and already provides support for Gerrit Code Review workflow.

Q: Gerrit or Team Forge can help to migrate code from ClearCase to GIT?
A: Not really, they are not intended to drive migration from one SCM to another.


Thank you again for your attendance and don’t miss our next Webinar with CollabNet Inc, Tuesday, February 12, 10:00 AM – 11:00 AM PST.

Go Agile with Git Part 1 of 3: Workflows, Branching & Merging

git+gerrit-webinar LM-20130111-Series-1 LS final-LM-fixes

The first webinar of the CollabNet series “Go Agile with Git” had a great audience of over 200 attendees from all over the world !! A big THANK YOU to all the attendees for the great result and the great questions during the webinar.

Unfortunately not all of the questions could be answered during the Webinar because of the limited time available. You can find below all the Webinar questions and answers on Git and Agile Development … and feel free to post additional ones by commenting this post: you will definitely get our answer shortly on the blog.

GerritForge LLP is proud partner of CollabNet Inc. and invites you to replay the first webinar and register to the following two webinars of the series:

  1. Workflows, Branching & Merging (View on demand)
    Learn the basic concepts of Git and Gerrit and see explore the power of merging and applying code from different branches thanks to recursive merge, rebase and cherry-pick.
  2. Peer Programming & Code Reviews (January 29th) << Next webinar
    See how Gerrit Code Review can enforce full code collaboration and collective ownership, similarly to peer programming you can achieve faster and better code delivery. Learn how to integrate Jenkins CI into the development process and have automatic validation to feedback into your code review workflow.
  3. Hands-On Lab with Gerrit & Jenkins (February 12th)
    Step-by-step hands-on with Gerrit to learn how to move your first steps with Code Review: contribute your change, trigger your Jenkins CI validation and learn how to understand the feedbacks and amend your code for achieving a successful merged change into the main branch. Learn how to manage multiple reviews when the code evolves.

Workflows, Branching and Merging – Q&A

Q: Do you have any tool to migrate source code of one SCM tool to GIT ?
A: Git native distributions already include tools that allow to fetch code from external SVN repository and push to a Git, including their history and branches. For other SCMs (i.e. CVS, Perforce, TFS) there are OpenSource and commercial tools to migrate the full history of your repository into Git.

Q: Gerrit Dashboards already works?
A: Dashboards have been introduced in Gerrit in November 2012 hackathon and are one of the major features of Gerrit 2.6. For more information on the Gerrit roadmap see our previous post.

Q: how can use GIT to map it ClearCase UCM
A: Git is not really comparable to ClearCase UCM. Gerrit Code Review plus the additional ALM features provided by CollabNet TeamForge or the Enterprise Issue-Tracker integrations of GerritForge LLP, can really be mapped to ClearCase UCM concepts as both define the concept of “project” and “tracking” between code changes and work items into your development lifecycle. Git can be mapped to ClearCase whilst Gerrit and its Issue-Tracker integrations map to UCM with the role of associating one or more Git commits to a delivery or set of issues tracked in ClearQuest.

Q: will we cover Gerrit and AD integration? or Git AD integration ?
A: Git does not provide AD integration out-of-the-box, unless integrated and customised through an HTTP front-end (i.e. Apache) to resolve user credentials authentication. Gerrit provides AD and industry-standard LDAP support and will be covered in the second and third webinars in the series.

Q: are there any danger when auto-merging?
A: It depends on whether you have Gerrit triggering your Jenkins CI to “protect” from wrong auto-merges or if you just rely on nightly builds and standard QA validation: when Gerrit and Jenkins are integrated, even auto-mergine is not a risk as the consistency of the code-merge is validated. Typically a code-review workflow requires the change to be already “rebased” on the branch candidate to receive the change, otherwise you would run into the risk of validating a change on the wrong context.

Q: If someone commits a huge binary file by accident and pushes it to the origin/master, can that be undone or is it there to stay forever? Or is this also covered in history protection link?
A: Git always allows to “remove” commits permanently, using “git reset” or “git rebase -i” (interactive rebase) to remove or amend the commit containing the huge binary file. However Git keeps the binary blob into its objects until garbage collection will trigger an physical removal of all unreferenced objects, including the huge binary file. With CollabNet TeamForge history protection you have full control of when and what to remove completely from your Git history (in this case the huge binary file).

Q: Can roles in Gerrit be defined in ldap?  If so, what are the requirements? Can roles in Gerrit be defined in an external source?
A: Gerrit allows the usage of RBAC (role-based access control) through its group level permissions. Groups can be defined in LDAP and Gerrit Ver. 2.5 can use them and retrieve them dynamically using the its pluggable “group backend” infrastructure. Groups LDAP entry points need to be configured in gerrit.config in order to be fetched by Gerrit. Similarly other external sources can be plugged as “group backend” into Gerrit by implementing the connector interface to the external roles as Gerrit plugin. For more information on how to create a Gerrit plugin see our talk on “How to cook a Gerrit Plugin in 10 mins” presented @GooglePlex in Mountain View in November 2012.

Q: You mentioned Jenkins fits the best here. How do you compare it with Team City in this regard? 
A: TeamCity is a very advanced CI system and has introduced the concept of “test build” and “private builds” to prevent faulty code to break the team build, same battleground of Code Review to improve Team Agility without loosing control and code quality. Unfortunately TeamCity is not OpenSource and thus has a more limited community of plugin developers providing integrations for Git and Gerrit. Jenkins CI  has the world’s largest community of plugin developers for a Continuous Integration engine and already provides support for Gerrit Code Review workflow.

Q: How many people voted on Gerrit poll?
A: 183

Q: What is the minimum number of branches needed if two geographically seperated teams are sharing the same codebase and committing to the same codebase? e.g. production, master, dev1, de2, etc. How many should we maintain to be minimal?
A: Minimum number of branches is actually one, the master. Two repositories in two locations have already a “copy” of the “master” branch and then are effectively different branches that can be merged during the repository synchronisation process across locations. In practice it is recommended to have at least one active development branch (“master”) and then one branch per stable release, independently from the number of geographically separated teams. Additionally you may want to develop features in parallel using the “topic” branches and then having them merged into the “master” as they are validated through Code Review by the Teams.

Q: We are about to start the migration from SVN to GIT and are worried about the best way to train the developers – can you give any tips of the best approach
A: Git requires a lot of “day-by-day” support from the beginning, especially for Teams coming from a SVN experience. Concepts are different and sometimes SVN command names have a different meaning and behaviour in Git. It is recommended to carefully tune Gerrit permissions to avoid “dangerous” pushes  from “beginners” Team members (i.e. disabling forge identity and forced push) and start with a dedicated Git training to a group of “champions” that will be then spread across all of the Teams to provide support. Those “champions” will become then the “day-by-day” tutorials of the other Team members learning how to use Git in everyday work. P.S. Stick the “Git cheat sheet” on the wall of every Team room to enforce the Git development SCM workflow.

Q: How do users use git on windows systems? Is there a tool like Tortoise?
A: Yes and its name is TortoiseGit (no surprise here !) but it is only a GUI front-end to Git for Windows that needs to be installed as pre-requisite. However I do encourage the Teams to use the Git command line in order to first understand the concepts and then to learn how to get the best from the tool when they need it. For most common operations they can use TortoiseGit or any other Git client GUI tool (personally I use and like a lot GitX on MacOS) but they need to understand the underlying commands that are executed and generated as Git actions on the repository.

Q: How to find out which files someone has edited over history of project?  It is easy in a specific commit, but seems hard over the life of the project …
A: Git is a very powerful tool designed to provide full history inspection and code search. Each command is very sophisticated and flexible to perform even complex tasks. This specific task mentioned is not complex for Git, the following example provides a list and stat of lines changed by someone@somecompany.com over the current branch:
git log –author someone@somecompany.com –numstat

Q: Do you have any best practices to migrate from Subversion to GIT?
A: My best suggestion is to assure consistency of Subversion vs Git usernames and e-mails. You wouldn’t like to loose visibility and association of who made a commit in SVN and his identity in Git. In order to enforce consistency make sure that “forge identity” permission is disabled when start using Git after the initial migration, in order to keep your ex-SVN users always aligned with their own identity in Git. “Forge identity” can then be granted gradually as developers become more aware and familiar with the Git tool.

Q: C1 to C5 is like Team Branch. C6, C7 is like Dev Branch.

recursive-merge

A: C6-C7 is more like a “topic-branch” where one or more developers are working on a specific feature. Master is the main development branch.

Q: Why do we need a Contributor who is NOT a commiter?
A: Anybody in the Team or outside the Team (when authorised to do so) should be able to provide their ideas and possibly even their fixes to the code; however you do not necessarily want to allow everybody to commit and merge a change (potentially flawed) and break the Team master build, causing delay on your project sprint. Defining “contributors” you can allow everybody to provide their changes into the Team’s discussion and promote them through automatic validation (Jenkins CI builds) and collective code-review, avoiding the risk of breaking the build by unwanted or not validated changes. You can then differentiate them from the “Senior Team Members” (Committer) who have the technical ownership of the design and timelines of the Team Project.

Q: any comment on gerrit vs sonar code reviews?
A: Sonar is a fantastic infrastructure to trigger code quality checks but not necessarily to action on code promotion. Gerrit is the place where people decide what to do with regards to a change, whilst Sonar is only providing a feedback on “how the change looks like” without any active review action on it. However it would be possible to integrate Sonar feedback into Gerrit code-review lifecycle developing a Sonar plugin for Gerrit or simply getting Sonar feedback into Jenkins build and then providing a validation result to Gerrit through the Jenkins Gerrit plugin.

Q: In the Git merge slide, what’s the recursive option?
A: Recursive merge strategy (the default merge option in Git) allows automatically to detect complex branches history and merge them together by walking recursively into each branch history and looking by a common ancestor branch point.

Q: does Gerrit only works with Git?
A: Yes, mainly because the it needs all the power and ability of Git to merge and rebase changes once they have been reviewed, approved and then submitted. Additionally Gerrit uses Git ability to define custom “hidden branch namespaces” to store Security and Roles definition of the repository itself.

Q: will Gerrit work with Git Fusion? Do we need a mirror repository in that case?
A: Perforce Git Fusion, used as “blessed Git repository” is a Perforce instance that “talks” the Git protocol to allow cloning and pushing to it. However Gerrit uses JGit to access its Git repository on the Filesystem and cannot then use a Perforce backend. However Perforce Git Fusion could be used as synchronisation peer of a Gerrit repo, through the usage of the Git replication protocol.

Q: how does the distributed scm work in git when the development teams are spread across geographical locations?
A: You typically configure one Gerrit replicated instance per geographical site, in order to allow maximum performance on the “git pull” operation. However for minimising conflicts during geographical repository replication, it is important to perform “git push” to only one of the repositories replicas; all the others will get the changes eventually thanks to the Gerrit replication events being propagated.

Q: how does git auto resolve work, how does git know which files to merge first in order to keep all changes from all submits?
A: Git has a lot of powerful commands and hidden (or semi-hidden) functionalities, waiting to be discovered and leveraged !
I think you refer here to the “git automatic conflict resolution” (aka git rerere) described in this Git documentation link. Git stores the “history” on how a conflict was resolved in the past and then reuse the same info to resolve future conflicts. Functionality is normally disabled but can be turned on with the following command:
git config –global rerere.enabled true

Q: Can you please explain what’s ther diff between Recursive Merge vs Rebase? the diagrams look very similar
A: There is one fundamental difference between “merging” and “rebasing” two branches: in the first case the merge preserves the full history of the two branches and creates one extra commit containing the “merged code”; in the second case the rebase modifies the history of the branch by “replaying the changes” on top of the target branch, as you were in a time-machine and you would push changes into a future state and the branch point was never created in the past. In a nutshell, the “merge” can be reverted because doesn’t change the history (removing the merged commit would suffice) whilst a “rebase” cannot be reverted easily as the “rebased branch” history has been modified and reapplied in the future. Rebase however has the effect of “flattening up” the branch history and allowing a more concise and readable set of commits on a development branch.

Thank you again for your attendance and don’t miss our next Webinar with CollabNet Inc, Tuesday, January 29, 10:00 AM – 11:00 AM PST.