Go Agile with Git Part 1 of 3: Workflows, Branching & Merging

git+gerrit-webinar LM-20130111-Series-1 LS final-LM-fixes

The first webinar of the CollabNet series “Go Agile with Git” had a great audience of over 200 attendees from all over the world !! A big THANK YOU to all the attendees for the great result and the great questions during the webinar.

Unfortunately not all of the questions could be answered during the Webinar because of the limited time available. You can find below all the Webinar questions and answers on Git and Agile Development … and feel free to post additional ones by commenting this post: you will definitely get our answer shortly on the blog.

GerritForge LLP is proud partner of CollabNet Inc. and invites you to replay the first webinar and register to the following two webinars of the series:

  1. Workflows, Branching & Merging (View on demand)
    Learn the basic concepts of Git and Gerrit and see explore the power of merging and applying code from different branches thanks to recursive merge, rebase and cherry-pick.
  2. Peer Programming & Code Reviews (January 29th) << Next webinar
    See how Gerrit Code Review can enforce full code collaboration and collective ownership, similarly to peer programming you can achieve faster and better code delivery. Learn how to integrate Jenkins CI into the development process and have automatic validation to feedback into your code review workflow.
  3. Hands-On Lab with Gerrit & Jenkins (February 12th)
    Step-by-step hands-on with Gerrit to learn how to move your first steps with Code Review: contribute your change, trigger your Jenkins CI validation and learn how to understand the feedbacks and amend your code for achieving a successful merged change into the main branch. Learn how to manage multiple reviews when the code evolves.

Workflows, Branching and Merging – Q&A

Q: Do you have any tool to migrate source code of one SCM tool to GIT ?
A: Git native distributions already include tools that allow to fetch code from external SVN repository and push to a Git, including their history and branches. For other SCMs (i.e. CVS, Perforce, TFS) there are OpenSource and commercial tools to migrate the full history of your repository into Git.

Q: Gerrit Dashboards already works?
A: Dashboards have been introduced in Gerrit in November 2012 hackathon and are one of the major features of Gerrit 2.6. For more information on the Gerrit roadmap see our previous post.

Q: how can use GIT to map it ClearCase UCM
A: Git is not really comparable to ClearCase UCM. Gerrit Code Review plus the additional ALM features provided by CollabNet TeamForge or the Enterprise Issue-Tracker integrations of GerritForge LLP, can really be mapped to ClearCase UCM concepts as both define the concept of “project” and “tracking” between code changes and work items into your development lifecycle. Git can be mapped to ClearCase whilst Gerrit and its Issue-Tracker integrations map to UCM with the role of associating one or more Git commits to a delivery or set of issues tracked in ClearQuest.

Q: will we cover Gerrit and AD integration? or Git AD integration ?
A: Git does not provide AD integration out-of-the-box, unless integrated and customised through an HTTP front-end (i.e. Apache) to resolve user credentials authentication. Gerrit provides AD and industry-standard LDAP support and will be covered in the second and third webinars in the series.

Q: are there any danger when auto-merging?
A: It depends on whether you have Gerrit triggering your Jenkins CI to “protect” from wrong auto-merges or if you just rely on nightly builds and standard QA validation: when Gerrit and Jenkins are integrated, even auto-mergine is not a risk as the consistency of the code-merge is validated. Typically a code-review workflow requires the change to be already “rebased” on the branch candidate to receive the change, otherwise you would run into the risk of validating a change on the wrong context.

Q: If someone commits a huge binary file by accident and pushes it to the origin/master, can that be undone or is it there to stay forever? Or is this also covered in history protection link?
A: Git always allows to “remove” commits permanently, using “git reset” or “git rebase -i” (interactive rebase) to remove or amend the commit containing the huge binary file. However Git keeps the binary blob into its objects until garbage collection will trigger an physical removal of all unreferenced objects, including the huge binary file. With CollabNet TeamForge history protection you have full control of when and what to remove completely from your Git history (in this case the huge binary file).

Q: Can roles in Gerrit be defined in ldap?  If so, what are the requirements? Can roles in Gerrit be defined in an external source?
A: Gerrit allows the usage of RBAC (role-based access control) through its group level permissions. Groups can be defined in LDAP and Gerrit Ver. 2.5 can use them and retrieve them dynamically using the its pluggable “group backend” infrastructure. Groups LDAP entry points need to be configured in gerrit.config in order to be fetched by Gerrit. Similarly other external sources can be plugged as “group backend” into Gerrit by implementing the connector interface to the external roles as Gerrit plugin. For more information on how to create a Gerrit plugin see our talk on “How to cook a Gerrit Plugin in 10 mins” presented @GooglePlex in Mountain View in November 2012.

Q: You mentioned Jenkins fits the best here. How do you compare it with Team City in this regard? 
A: TeamCity is a very advanced CI system and has introduced the concept of “test build” and “private builds” to prevent faulty code to break the team build, same battleground of Code Review to improve Team Agility without loosing control and code quality. Unfortunately TeamCity is not OpenSource and thus has a more limited community of plugin developers providing integrations for Git and Gerrit. Jenkins CI  has the world’s largest community of plugin developers for a Continuous Integration engine and already provides support for Gerrit Code Review workflow.

Q: How many people voted on Gerrit poll?
A: 183

Q: What is the minimum number of branches needed if two geographically seperated teams are sharing the same codebase and committing to the same codebase? e.g. production, master, dev1, de2, etc. How many should we maintain to be minimal?
A: Minimum number of branches is actually one, the master. Two repositories in two locations have already a “copy” of the “master” branch and then are effectively different branches that can be merged during the repository synchronisation process across locations. In practice it is recommended to have at least one active development branch (“master”) and then one branch per stable release, independently from the number of geographically separated teams. Additionally you may want to develop features in parallel using the “topic” branches and then having them merged into the “master” as they are validated through Code Review by the Teams.

Q: We are about to start the migration from SVN to GIT and are worried about the best way to train the developers – can you give any tips of the best approach
A: Git requires a lot of “day-by-day” support from the beginning, especially for Teams coming from a SVN experience. Concepts are different and sometimes SVN command names have a different meaning and behaviour in Git. It is recommended to carefully tune Gerrit permissions to avoid “dangerous” pushes  from “beginners” Team members (i.e. disabling forge identity and forced push) and start with a dedicated Git training to a group of “champions” that will be then spread across all of the Teams to provide support. Those “champions” will become then the “day-by-day” tutorials of the other Team members learning how to use Git in everyday work. P.S. Stick the “Git cheat sheet” on the wall of every Team room to enforce the Git development SCM workflow.

Q: How do users use git on windows systems? Is there a tool like Tortoise?
A: Yes and its name is TortoiseGit (no surprise here !) but it is only a GUI front-end to Git for Windows that needs to be installed as pre-requisite. However I do encourage the Teams to use the Git command line in order to first understand the concepts and then to learn how to get the best from the tool when they need it. For most common operations they can use TortoiseGit or any other Git client GUI tool (personally I use and like a lot GitX on MacOS) but they need to understand the underlying commands that are executed and generated as Git actions on the repository.

Q: How to find out which files someone has edited over history of project?  It is easy in a specific commit, but seems hard over the life of the project …
A: Git is a very powerful tool designed to provide full history inspection and code search. Each command is very sophisticated and flexible to perform even complex tasks. This specific task mentioned is not complex for Git, the following example provides a list and stat of lines changed by someone@somecompany.com over the current branch:
git log –author someone@somecompany.com –numstat

Q: Do you have any best practices to migrate from Subversion to GIT?
A: My best suggestion is to assure consistency of Subversion vs Git usernames and e-mails. You wouldn’t like to loose visibility and association of who made a commit in SVN and his identity in Git. In order to enforce consistency make sure that “forge identity” permission is disabled when start using Git after the initial migration, in order to keep your ex-SVN users always aligned with their own identity in Git. “Forge identity” can then be granted gradually as developers become more aware and familiar with the Git tool.

Q: C1 to C5 is like Team Branch. C6, C7 is like Dev Branch.

recursive-merge

A: C6-C7 is more like a “topic-branch” where one or more developers are working on a specific feature. Master is the main development branch.

Q: Why do we need a Contributor who is NOT a commiter?
A: Anybody in the Team or outside the Team (when authorised to do so) should be able to provide their ideas and possibly even their fixes to the code; however you do not necessarily want to allow everybody to commit and merge a change (potentially flawed) and break the Team master build, causing delay on your project sprint. Defining “contributors” you can allow everybody to provide their changes into the Team’s discussion and promote them through automatic validation (Jenkins CI builds) and collective code-review, avoiding the risk of breaking the build by unwanted or not validated changes. You can then differentiate them from the “Senior Team Members” (Committer) who have the technical ownership of the design and timelines of the Team Project.

Q: any comment on gerrit vs sonar code reviews?
A: Sonar is a fantastic infrastructure to trigger code quality checks but not necessarily to action on code promotion. Gerrit is the place where people decide what to do with regards to a change, whilst Sonar is only providing a feedback on “how the change looks like” without any active review action on it. However it would be possible to integrate Sonar feedback into Gerrit code-review lifecycle developing a Sonar plugin for Gerrit or simply getting Sonar feedback into Jenkins build and then providing a validation result to Gerrit through the Jenkins Gerrit plugin.

Q: In the Git merge slide, what’s the recursive option?
A: Recursive merge strategy (the default merge option in Git) allows automatically to detect complex branches history and merge them together by walking recursively into each branch history and looking by a common ancestor branch point.

Q: does Gerrit only works with Git?
A: Yes, mainly because the it needs all the power and ability of Git to merge and rebase changes once they have been reviewed, approved and then submitted. Additionally Gerrit uses Git ability to define custom “hidden branch namespaces” to store Security and Roles definition of the repository itself.

Q: will Gerrit work with Git Fusion? Do we need a mirror repository in that case?
A: Perforce Git Fusion, used as “blessed Git repository” is a Perforce instance that “talks” the Git protocol to allow cloning and pushing to it. However Gerrit uses JGit to access its Git repository on the Filesystem and cannot then use a Perforce backend. However Perforce Git Fusion could be used as synchronisation peer of a Gerrit repo, through the usage of the Git replication protocol.

Q: how does the distributed scm work in git when the development teams are spread across geographical locations?
A: You typically configure one Gerrit replicated instance per geographical site, in order to allow maximum performance on the “git pull” operation. However for minimising conflicts during geographical repository replication, it is important to perform “git push” to only one of the repositories replicas; all the others will get the changes eventually thanks to the Gerrit replication events being propagated.

Q: how does git auto resolve work, how does git know which files to merge first in order to keep all changes from all submits?
A: Git has a lot of powerful commands and hidden (or semi-hidden) functionalities, waiting to be discovered and leveraged !
I think you refer here to the “git automatic conflict resolution” (aka git rerere) described in this Git documentation link. Git stores the “history” on how a conflict was resolved in the past and then reuse the same info to resolve future conflicts. Functionality is normally disabled but can be turned on with the following command:
git config –global rerere.enabled true

Q: Can you please explain what’s ther diff between Recursive Merge vs Rebase? the diagrams look very similar
A: There is one fundamental difference between “merging” and “rebasing” two branches: in the first case the merge preserves the full history of the two branches and creates one extra commit containing the “merged code”; in the second case the rebase modifies the history of the branch by “replaying the changes” on top of the target branch, as you were in a time-machine and you would push changes into a future state and the branch point was never created in the past. In a nutshell, the “merge” can be reverted because doesn’t change the history (removing the merged commit would suffice) whilst a “rebase” cannot be reverted easily as the “rebased branch” history has been modified and reapplied in the future. Rebase however has the effect of “flattening up” the branch history and allowing a more concise and readable set of commits on a development branch.

Thank you again for your attendance and don’t miss our next Webinar with CollabNet Inc, Tuesday, January 29, 10:00 AM – 11:00 AM PST.

GerritForge 2.5.1 released

Gerrit 2.5.1

New maintenance version of GerritForge has been released today and is available for download from http://gerritforge.com/download.

 

 

Our commitment: up-to-date with Gerrit Code Review, always !

GerritForge confirms its commitment to be aligned with Gerrit Code Review in order to guarantee to our customers the best up-to-date features and most critical security fixes.

Noteworthy fixes included in GerritForge 2.5.1.  

Gerrit 2.5.1 includes important fixes with regards to Git protocol over HTTP and Security.

  • Restriction to ‘set-project-parent’ to Gerrit Administrators. Previously the “door was left opened” for project owners that were trying to workaround the security check granting access to refs/meta/config branch.
  • Correct categorisation of Git over HTTP commands as “GIT operations” and not Web-UI. This was creating confusion on the way access control to repos was evaluated.
  • RequestCleanup exception on Server threads for Git over HTTP: this was creating random exception when using Git over HTTP and “connection reset” seen on the Git client side. Problem was not impacting Git over SSH.

Fixes and enhancements on GerritForge Enterprise plugins.

GerritForge 2.5.1 adds additional fixes and extensions to the Enterprise plugins, such as:

  • Additional documentation on the delete-project plugin.
  • New plugin to remove Gerrit internal groups.
  • Fixes on the GitBlit plugin on the repository names duplicates and Git clone URL is now populated correctly.
  • Additional support for non-UK and non-US Java default locales.

Wants to know more ?

Gerrit roadmap: a bright future ahead !

Gerrit roadmap for 2013 is really amazing:

Gerrit 2.6 – Feb/Mar 2013

  • Major enhancements to the Gerrit plugin architecture, including pluggable 3rd party authentication and user-registries.
  • Introduction of the Gerrit RESTFul API: this will allow easier integration for batch operations using simple CURL commands and more stable API available for integrating with plugins.
  • Project dashboards: for the first time Gerrit allows projects to have custom dashboards to show a set of selected data extracted from the project itself.

GerritForge 2.6 additional enterprise features

  • Additional issue-tracking association plugins, including CollabNet TeamForge, Rally, Serena and HP Quality Center.
  • Group backends for Atlassian Jira and CollabNet TeamForge, allowing to use existing ALM Roles in Gerrit access control lists.
  • Gitblit 1.2.0 plugin

Gerrit 2.7 – June/July 2013

  • Multi-master replication: allows multiple Gerrit Servers to run in parallel on the same set of Git and relational data-set replicated in different locations. Consistency and control of the replicas is automated and is working out-of-the-box. Users can push and pull to any of the Gerrit master replicas.

GerritForge 2.7 additional enterprise features

  • Multi-tenant support and multi-authentication domain for the same Gerrit Server
  • Delegation of Gerrit administration to Web-UI roles.

Stay in touch 😉

Do you want to know more or want to be always informed of what’s next ? Register your interest in our contacts page and we will always let you know what is going on on Git, Gerrit and the Enterprise !

GitEnterprise new face and new mission

Welcome to the new face of GitEnterprise.com !

GitEnterprise is different, not only in his new external facing web-site but also deep inside in his mission and focus.

We have been providing cloud and hosted services powered by gitent-scm.com to provide a superior support for repository management, users and groups security and full audit-trail for around 2 years: now is time to make a bit leap forward.

One size does not fit all enterprises.

Can one solution be good enough for companies of all sizes ? We thought that we could build that solution and we started proposing our product in both hosted and cloud deployments for all our customers: over 7000 users registered to gitent-scm.com and created more than 10,000 repositories.

First phase of Git adoption is very much about discovery and gitent-scm.com has been a straightforward way to explore some enterprise-features such as role-based access control and branch-level security.

Once companies become familiar with Git and its usage in the Enterprise, the adoption really starts with the identification and exploration of the functional and ICT Security requirements needed by the organisation: different companies do have different requirements and integration with different systems.

Individual contractors.

The smallest of the Enterprises is the one-man-band where a single contractor is engaged by different companies for different projects. No specific requirements are needed other than private repositories with regular security and backup; Git Server is best hosted in the Cloud as minimises maintenance and operations costs whilst reducing risks of personal hardware failure.

Even though gitent-scm.com could be a cost-effective solution, it is possibly too complicated as RBAC and branch-level security are not needed and could be seen as repository administration burden.

We designed a “all-in-one” Git Server solution to combine ease of administration and personal security and confidentiality: his name is Git-in-a-box and requires no installation or administration, just download and use it.

Hosted: get Git-in-a-box and execute on your favourite Server to have a fully features Git Server without hassles.

Cloud: Git-in-a-box can be used in any private Cloud Server and can use any sort of Cloud storage backup. When a FREE Cloud Solution is required, the only choice on the market is BitBucket.org that offers privacy and space at no charge.

Small to medium Enterprises.

Teams are typically co-located in a single Office and additional requirements start to become recommended if not mandatory depending on the industry sector.

Most companies can still have a Git Server hosted in the Cloud without breaking any specific company policy whilst others such as Finance Software Development companies have stricter requirements to address their internal auditing constraints.

Cloud: when cost reduction is the main focus and source-code is not subject to strict ICT Security requirements, gitent-scm.com can be a good compromise as it provides 10-users / 1 GByte FREE Subscription with added Role-based Access Control and Web-based audit-trail. Security administration is easy and users can be self-provisioned and invited to join the company domain later on.

Hosted: for small companies without specific security requirements, Git-in-a-box is still our favourite choice of simplicity and functionality. Companies working in the Finance sector anyway have additional ICT Security requirements that involve compliance with SOX / PCI regulations. GerritForge is a cost-effective solution to provide governance, audit and integration with existing ICT Security assets such as LDAP User-registry and password enforcement and producing audit-trails for compliance inspections and reports.

Large Enterprises.

Teams can be even huge and distributed over the globe: they include developers with different skills-set and level of maturity of the adoption of Git as SCM tool. Most of them are possibly coming from a Subversion or SourceSafe background and need a more user-friendly and safer interface to Git.

100% pure Cloud hosting is typically not an option whilst some of them may want to use a more flexible hybrid-cloud model combining a traditional data-center hosting with additional private clouds located in remote geographical locations to reduce latency on remote sites.

The following key requirements become mandatory for a corporate adoption:

  • Integration with Corporate User-registry such as corporate LDAP and Directory Services.
  • Single-Sign-On with existing authentication system and credentials.
  • Reuse of existing Corporate roles in the definition of the access control to Git repositories.
  • Git over HTTP/S for SSL and X.509 Certificate validation.
  • Full tamper-proof audit-trail of every operation triggered either via Web UI or Git client.
  • Delegation of control across your organisation.
  • High-availability of both Web/Git Services and data-storage.
  • Multi-site replication across all the Corporate offices.
  • 24×7 Support on the full Git stack.
  • Existing rock-solid user base and Corporate reference in the same Business area.
  • Integration with existing Corporate Workflow and Application Lifecycle Tools

Git was not designed to fulfil all of those Enterprise requirements, most Cloud or Hosted Services such as GitHub:Enterprise or Atlassian Stash provides coverage for only some of them such as LDAP integration and could be accepted in some departments with reserve from Corporate Security. gitent-scm.com and Git-in-a-box are definitely not the solution either as do not support full multi-site replication or workflow and ALM integration.

The only solution on the market is Gerrit Code Review: it has been designed and implemented by Google for the distributed and collaborative development of the Android OS, based on ashes of Guido Van Rossum’s experiment named Google Mondrian Code Review.

Gerrit Code Review is the only Git Enterprise tool adopted on a large scale by Companies such as SONY, Garmin, Qalcomm, SAP, Intel, HP, Deusche Bank and the list is growing on a daily basis.

Starting from Gerrit 2.5, announced @GooglePlex in Mountain View in November 2012, Gerrit is now extensible and can be integrated then with other tools and issue-tracking systems. We demoed the integration between GitBlit and Gerrit whilst providing full access to the OpenSource community to our Atlassian Jira integration.

GerritForge hosted is the Enterprise version of Google Gerrit Code Review with all the additional Enterprise Integrations you need to provide RBAC, Auditing and Application Lifecycle and Workflow.

GerritForge added the “missing pieces” for plugging it into a Corporate Environment:

  • Issue tracking association and enforcement
  • Workflow definition and automation
  • Tamper-proof audit-trail
  • 24×7 Support on the full Git and Gerrit software stack
  • Single-sign-on and 3rd party authentication plugins, such as CollabNet TeamForge.
  • Formal native packaging and installation procedure.
  • Access to ad-hoc patches for production issues even on older branches and releases.

We do believe so much in Gerrit that we have rebuilt all our product offering using Gerrit as foundation:

Git-in-a-box is a Gerrit 2.5 engine with an HTML5 / RESTFul API layer to provide ease-of-use and plug&play installation.

gitent-scm.com is a Gerrit 2.2.2 engine with a JavaEE WebUI for automating common administration tasks.

GerritForge is Google Gerrit, with the same core code-base and version numbering.

Want to know more on Google Gerrit ?

For an initial introduction, the Google Gerrit Code Review Wikipedia page is the best place to start.

More detailed Wiki and project information can be found at Google Gerrit Code Review home page.

Why Git Enterprise?

Someone may correctly ask why do we need another Git hosting solution, while we have such a wide choice around, like github or bitbucket (just to name few). The thing is that an enterprise solution is missing. In our everyday work in a big firm we are experiencing these issues:

  • there’s no real security or access control system available out of the box with git, and the level you can achieve is far away too coarse grained to be really useful
  • there’s no real way to easily administer a git repository in order to eventually manage anything, from users to privileges
  • integration with modern IDE, particularly Eclipse, is really disappointing, so that being productive is really difficult
  • there’s no real way go visualize the status of a central repository, which definitely happens when git is used in an enterprise environment
  • the freedom of Git sometimes bring your project, and your repository, in a kind of “cowboy” state that is far from acceptable by an enterprise

We appreciate all these issues, and we want to provide a fix for them, so that Git may be adopted widely from enterprises, our personal problems will be solved and, eventually, we will be able to get some money out of this.

Dream? Fact. Development is started and the code is near to be production ready. So please, you guys, that landed here in search for an enterprise solution, stay tuned.