Category Archives: code-review

Gerrit Summit 2016 is coming

Posted on by
Reply

google

Four weeks from now, the eighth edition of the Gerrit User Summit will open its door at Google HQ in Mountain View – CA, 12th-13th of November 2016.
It has been a long journey since the first GitTogether in 2008, and after the split between the Git[Hub Universe] summit and the traditional “unconference” style Gerrit event at Google’s, things have changed quite a lot. While Gerrit remained a 100% OpenSource user-centric project, GitHub has attracted $350M in VC, and they have been losing traction over the years to join the unconference-style events.

What’s new this year?

For the first time, the proposals of talks to the Gerrit User Summit are submitted in Gerrit directly (yeah!) on the summit/2016 repository.

The list of currently approved talks is available by searching for “status:merged project:summit/2016” (https://gerrit-review.googlesource.com/#/q/status:merged+project:summit/2016)
The talks awaiting review are under “status:open project:summit2016” (https://gerrit-review.googlesource.com/#/q/status:open+project:summit/2016)

How cool is that? I foresee already a Doodle plugin for Gerrit 😉

How to register for the User Summit?

Shawn Pearce has prepared a Registration Form for you to sign-up to the event:
https://goo.gl/forms/oeEnQweHl2noNSnn1

Once you access the Registration Form at the above URL, you need to sign-in with your Google Account credentials and then complete the following information:
– Your name
– Your Organisation
– Your previous attendance to the user summit
– Any dietary restrictions

The User Summit is FREE for EVERYONE, including novice users of Git and Gerrit Code Review, but you would need to register beforehand.
The Summit is a unique opportunity to learn about Gerrit new feature, contribute to the product roadmap with your needs and requirements and, most of all, network with other users to learn new use-cases where Gerrit can be very helpful.

How to submit my talk proposal?

Well, you need to demonstrate a good understanding and use of Gerrit Code Review if you want to teach and talk to other people about it! At the end of the day, if you want to talk about Gerrit you should be able to clone a repository and submit a patch to a project 🙂

If you need just a little help … see my “Diffy super super talk” example:

$ git clone https://gerrit.googlesource.com/summit/2016 && (cd 2016 && curl -Lo `git rev-parse --git-dir`/hooks/commit-msg https://gerrit-review.googlesource.com/tools/hooks/commit-msg ; chmod +x `git rev-parse --git-dir`/hooks/commit-msg)
$ cd 2016
$ cat - > sessions/my-amazing-talk.md
# My amazing talk at Gerrit User Summit

Hi folks, this is my super-duper-talk. You should be interested in it as I will unleash the dark force of Code Review Diffy Kung Fu Review Cuckoo.

*Diffy, Birds & CO. Inc.*
^D
$ git add sessions/my-amazing-talk.md && git commit -m "Diffy super-duper talk"
$ git push origin HEAD:refs/for/master

Talks highlights.

There are already some fascinating talks submitted and approved and more will undoubtedly come in the next couple of weeks. We will start sharing some highlights of what’s happening at the conference. Here is the overview of the first talks.

What’s new in Gerrit 2.12 and 2.13

Two major versions of Gerrit have been released since the last summit in 2015, and they contain significant improvements to the platform:

  • Topic submission workflow – aka Git commits across repositories  (v2.12).
    Group multiple changes in a “topic” and having them merged as a whole, even across multiple repositories, in a single submit operation.
  • GPG signed pushed verification (v2.12).
    Allows people to upload their GPG public keys into Gerrit and have them used to verify Git signed commits.
  • Large File Storage support (Git LFS) (v2.13).
    Gerrit finally supports the automatic management of large files outside the Git repository. The feature is fully pluggable and exposed via plugins. Amazon S3 and Local file system support are available at the moment, but more plugins are here to come on this feature.
  • Gerrit metrics (v2.13).
    Expose the internal metrics to external consumers. The feature is exposed for plugins to gather this data and send to external systems for analysis and visualization purposes. Graphite, ElasticSearch, and JMX plugins are available.
  • Hooks plugin (v2.13).
    Finally, the Gerrit hooks mechanism have been entirely externalized and implemented in a pluggable way. The legacy hooks have become a core plugin. However, you can now leverage the new extension to develop a new-generation of hooks by leveraging the new extension points provided.
  • New HTML5 UX with WebComponents – PolyGerrit preview (v2.13).
    The next generation of Gerrit UX based on Polymer Web components is available. Even though not complete, offers a sneak preview of what the new interface looks like and, if you like it as-is and is good enough for your use-cases, you can enable and start using it already. Both GWT and Polymer-based UX are using the same REST API, and thus the changes generated and reviewed with them are 100% interoperable.

There is more to come.

In the next few days we will keep on publishing the highlights of the topics coming at the Gerrit User Summit this year, stay tuned and REGISTER NOW at:
https://goo.gl/forms/oeEnQweHl2noNSnn1

The GerritForge Team.

Gerrit Upgraded with No Downtime

Posted on by
Reply

Screen Shot 2016-03-21 at 20.48.12

Zero DownTime success story.

From today at 08:06 GMT GerritHub users are served by our brand new infrastructure geo-located in Canada, Quebec, Beauharnois. It is the first time we applied a zero-downtime roll-out scheme, the PingDom uptime for the past 24h reported 100% uptime and 688 msec average response time for the page of the list of opened changes. The two response times spike on the above graphs are actually due to the old German infrastructure and happened before the start of the roll-out.

We can see the switch of the traffic to the new infrastructure from the increase of the overall response time (IP packets were routed from Germany to Canada causing extra hops); as the DNS propagation was spreading across the world, the overall number of hops gradually came back to normal.

Timeline of the events.

  • 08:00:00 GMT – Phase 1 – Set Gerrit READ-ONLY. All changes and Git repositories started to refuse push and updates.
  • 08:00:01 GMT – Phase 2 – Wait for pending replication to complete. Replication queue was empty; there was no need to wait.
  • 08:00:02 GMT – Phase 3 – Mirror DB and Git for the last time, delta-reindex, DB upgrade and Gerrit restart. It has been the longest part of the roll-out and lasted 5′ 32”, aligned with our estimates.
  • 08:05:34 GMT – Phase 4 – Cache warm-up. 20K projects, 8K accounts and 4.6K groups were pre-loaded in Gerrit. This step was optional but allowed us to redirect all the traffic without risks of causing thread spikes on the new infrastructure.
  • 08:06:23 GMT – Phase 5 – Redirect traffic to the new infrastructure.

Did anybody notice the rollout?

During the rollout the Git projects and Gerrit changes were read-only for 6′ and 23”. According to the logs, 493 Git/HTTP and 172 Git/SSH invocations were made and completed successfully: none of them failed.

What is the situation right now?

The new infrastructure public IP (192.99.233.76) has almost completed his DNS propagation around the world, the only countries not entirely covered are Australia and China. The rest of the world is coming directly to Canada avoid the German hops. Metrics are good, low CPU utilization and threads consumption compared to the old German infrastructure, symptom of the reduction of the execution and serving times and latency.

What’s next?

From now on we will continue to use this Blue/Green roll-out strategy, possibly improving in the ReadOnly window by introducing live distributed reindex and cache warm-up.

We fully commit to Zero-Downtime and Stability, the most valuable assets for our clients.

GerritHub and Zero-Downtime Upgrade

Posted on by
Reply

GerritHub gets bigger on Mon, 21 March 08:00 GMT


GerritHub has experienced unprecedented growth over the past two years. The November 2015 numbers presented at the Google User Summit in Mountain View – CA have been surpassed again, and we do need to make sure that our infrastructure is still capable of dealing with current and future users’ needs.

What is changing in GerritHub.io?

We are changing everything, from the version of Gerrit to the hardware, network and storage infrastructure. Data, DBMS, Indexes and cache, need to be upgraded and refreshed to make sure that the new systems are reflecting exacting the current production data and sessions.
We are changing as well the geo-location of our servers, from the current server farm in Germany (Bayern, Nuremberg – 100 MBps) to a new server farm in Canada (Quebec, Beauharnois – 1 GBps).

Why have so many changes?

We started to measure some significant delay in the Git and review operations on the old infrastructure, mainly due to three factors:

  1. More users, more repositories, more concurrency. Individuals, OpenSource projects and Businesses started using GerritHub.io for their mission-critical repositories, considering Gerrit the “source of truth” of their review workflow. We needed more horsepower, memory, storage and ability to scale even further.
  2. Bandwidth from USA and Far-east. The majority of people using GerritHub.io are from the other side of the Atlantic Ocean: this is typically not a problem from 7 AM to 3 PM … but after 4 PM the connectivity between Europe and the Americas becomes slow. Additionally, people using GerritHub.io from India, Japan, Australia and New Zealand experienced terrible slowdowns because of the excessive number of hops to reach Germany.
  3. Gerrit master is much faster. Based on the current data and metrics measured on GerritHub.io, we have contributed a lot of patches to reduce the overhead caused by Gerrit DB and lessen the number of SQL queries per minute. All those new improvements are on Gerrit master, and we need to catch-up with the “latest and greatest” version.

Will I experience any GerritHub.io outage?

Last time that GitHub needed to make a major upgrade, asked his 5M users to stop working for 23 minutes,. This translates to a loss of two millions of hours of continuous delivery lifecycle, equivalent to over 130 man/years, worth no less than eight millions dollars.
We are going to adopt a new Zero-Downtime Gerrit roll-out strategy to make sure that all those changes are not going to impact your day-by-day activity. If you were not reading this post you would possibly even not notice the “switch” from the old to the new infrastructure, apart from the increase in speed and bandwidth.

Zero-downtime GerritHub.io migration, step by step with the associated expected timings.

Phase 0 – Replication to the new Gerrit infrastructure. (- 1 month ago)
We started migrating everything one month ago, and the old and new infrastructure are working side-by-side, thanks to Gerrit master-slave replication. The new Gerrit servers are active as slaves and are read-only.

Phase 1 – Migration kick-off. (08:00 GMT)
We install a Gerrit plugin that rejects all the push to GerritHub.io repositories providing a courtesy message: “Gerrit is under maintenance, all projects are READ ONLY”. All the HTTP POST, PUT and DELETE are disable on the Gerrit REST-API.

Phase 2 – Wait for replication events to complete and migrate DB. (08:02 GMT)
Git repositories are continuously replicated, but we do need to make sure that the event queue is empty. Once that happens we schedule the last final DB migration to the new infrastructure.

Phase 3 – Gerrit DB upgrade and reindex (08:04 GMT)
New Gerrit server executes the final upgrade and off-line reindex of the latest received changes.

Phase 4 – Gerrit start-up and cache warm-up (08:05 GMT)
New Gerrit is restarted and the most critical Gerrit caches (projects, accounts and groups) are pre-loaded in memory. This allows the incoming traffic spike to avoid the collapse of used threads and makes the transition as smooth as possible without slowdowns.

Phase 5 – Traffic switch and DNS updates (08:06 GMT)
GerritHub.io redirects all incoming HTTPS and SSH traffic to the new infrastructure. Git pushes and HTTP PUT, POST and DELETE operations of the REST API are operational again and served by the new Gerrit infrastructure. GerritHub.io DNS is updated to the new Canadian IPs.

Phase 6 – New IPs gets propagate to all worldwide DNSs (+ 1 day)
Once all the DNSs in the world would have been updated, everyone will start going directly to the new infrastructure without further hops or redirection from Germany. Customers from USA, Canada, South America, Asia, Japan, New Zealand and Australia should see a significant reduction of the network latency and increase of GerritHub.io responsiveness.

Firewall and SSH considerations

Even if Gerrit server’s SSH key is not changing, some of you may see a warning similar to this when they push or pull over SSH:

Warning: the RSA host key for ‘review.gerrithub.io’ differs from the key for the IP address ‘148.251.77.70’

The warning message will also tell you which lines in your ~/.ssh/known_hosts need to change. Open that file in your favorite editor, remove or comment out those lines, then retry your push or pull.

Should your network have some strict firewall rules to access external sites, you may want to whitelist the IP of the new infrastructure WLB to: 192.99.233.76.

Follow GerritHub.io migration progress.

We will advertise the migration progress on Twitter at @GitEnterprise. Should you have any issue you can tweet us or contact GerritForge Customer Support at support@gerritforge.com.

GerritForge helps Gerrit 3.0 stability

Posted on by
1

Gerrit 3.0 plan announced: we need stabilisation now

Screen Shot 2015-11-23 at 09.52.19

Gerrit 3.0 plan and its NoteDB reviews have been officially announced at the Gerrit User Summit 2015. It is already available as an experimental feature in the current Gerrit master but it needs much more stability in order to be officially supported for production.
GerritForge decided to help and reuse its existing continuous integration system to validate every Gerrit patch set against the current and the new NoteDB review persistence back-end in order to avoid regressions during the 2.13 and 3.0 development

Pre-commit validation by GerritForge CI

If you have posted a patch to gerrit-review.googlesource.com in November, you may have hopefully received a Verified+1 from a strange user with a Diffy logo on the side.
GerritForge’s provided CI on gerrit-ci.gerritforge.com fetches automatically every patch-set pushed to gerrit-review.googlesource.com and triggers a slightly modified Gerrit build with the purpose of checking whether the code change introduces a regression or not. This may seem at first sight a quite normal Gerrit to Jenkins job integration, however implementing it on top of Google’s multi-master replicated installation was not a piece of cake.

Gerrit Trigger plugin limitations on multi-master setups

Jenkins has already an out-of-the-box integration with Gerrit provided by the Gerrit Trigger plugin maintained by Robert Sandell – Cloudbees. It leverages the Gerrit stream events through an SSH channel and make use of Gerrit REST API to action them according to the build result.
The Google’s Gerrit setup, however, is not a trivial one-node installation and is further limited by the security constraints of the Google infrastructure, which does not allow any incoming SSH connectivity.
Additionally all concept of “getting the events in a stream” isn’t going to work when events can come concurrently from multiple places at the same time: who is going to define the “global ordering” and how to put all those events in a single TCP/IP Socket? Even UDP would not work in this case because SSH channel requires confidentiality between two and only two peers.

Alternatives to SSH

During the hackathon, other approaches have been discussed by Shawn Pearce, including the use of HTTP WebSockets (or Cometd) for fetching events without the need of an SSH connection. Events are still distributed and generated by multiple masters all the time, and the Jenkins plugin would then have the onus of contacting all the Gerrit servers and keep a connection opened to all of them. This is clearly not going to work because the number of servers, their IPs and locations may change at any time and the solution would eventually be in danger of losing precious events.

Back to polling

The only solution we envisaged was to fall back to a polling logic where Jenkins ever 10 minutes is asking Gerrit “what’s new since last time we spoke?”. This solution goes against the main reason the Gerrit Trigger plugin was designed: avoiding SCM polling. It is, however, a much better and optimised polling strategy and let’s see why.

Query and then fetch

The typical Git SCM polling relies on fetching all references every poll interval and detect if new Git commits are available. This is notably slow and generates a huge overhead on the Git server. The approach we took is quite different and makes use of the Gerrit search capabilities that are way faster and more powerful than a simple Git fetch.
Jenkins first ask Gerrit the list of changes and associated commit-IDs involved in any event since the last polling time: the result may include patchsets that have been already built to avoid having any gaps between polling intervals. The search is fast and implemented in … you know, Google is a search company isn’t it?
Once the list of candidate commit-ids is identified, Jenkins goes through all of them and checks using the Gerrit REST-API:
– has it been build during my previous execution?
– has it been already accepted (or rejected) by me?
The Commit-IDs that results as not being checked before and not yet validated are then used to trigger a specific job parametrised on:
– Specific branch
– Specific change ref-spect
Fetching is performed avoiding any wildcard and the corresponding load on the Git server is minimum. Fetch (Git protocol) + build (using Buck) + test (unit + integration) + review feedback (REST API) is taking an average of 5 minutes, which is an amazing result if you consider the size of the Gerrit project and the typical slow speed of a default Jenkins Git fetch.

The bottom line

Using the query + fetch approach, which seemed a bit slow and old-fashioned at the beginning, was eventually very simple and successful. Instead of setting up SSH hostkey verification, key exchange and ad-hoc channels, the only configuration needed is a valid Gerrit user and the HTTPS endpoint URL, the same used for cloning the code.
The solution is much more reliable as SSH channels are notably unstable and consume server threads. The only drawback is the slight delay between the patch-set upload the start of the build (at max 10 minutes) which is acceptable in most cases.
Results
Since its roll-out more than 1200 patches have been checked and rated, a lot of potentially Gerrit regression avoided and more importantly we have prevented the NoteDB code to start diverging regarding stability from the current mainstream development.

How can re-trigger validation for a single change?

We have enabled anyone to trigger ad-hoc executions of the Gerrit validation flow using the following URL:
https://gerrit-ci.gerritforge.com/job/Gerrit-verifier-flow/build
This is a standard Jenkins parametrized build that request the change-id to be built, as either SHA1 or number. Once the job is triggered the build will be executed and the validation feedback applied to your change, regardless of the previous build or validation status.

GerritHub user-controlled GitHub Scopes

Posted on by
1

Nowadays people are very careful about privacy and user data: nobody grants access to their profile without checking first the possible consequences.
We want to give the user the ability always to know and control what level of access is given to their data: that’s why we improved the way you login in GerritHub.io.

GitHub scopes: what is it?

GitHub provides the authentication and access to user’s profile using a protocol called OAuth 2.0. When GerritHub is requesting a user to authenticate is then granted a set of permissions to operate on behalf of the user on their GitHub resources, which include:

  • User’s personal data (name, e-mails)
  • User’s membership to organisations and teams
  • User’s repositories

The set of permissions to access and operate on your data is also known as “Scope” in GitHub terms.

How is GerritHub helping me to control my access?

GerritHub has from today a new “Scope Selection” screen with two main objectives:

  1. Displaying your current scope and associated rights GerritHub has on your GitHub profile
  2. Giving you the ability to switch to a different “Scope” and consequently the rights that GerritHub has on your profile data

Screen Shot 2015-10-08 at 10.16.53

Transparency is good, but what is the practical added value?

There has been in the past a common complaint about GerritHub having too much or too little access to your GitHub profile:

  • Too much? Why GerritHub.io needs access to my e-mail address? Why does GerritHub need to see my public keys?
  • Too little? Why does GerritHub not show my private repositories in the import screen? How can I see my organisation membership in GerritHub project security screen?

With the ability now to visualise and change the current “Scope”, people can be more aware of why things are not showing up. They can make conscious decisions about how to change them with full transparency on the associated implications.

A common scenario: importing and accessing private GitHub Organisations, Teams and Repositories.

When you need to import an existing private GitHub project, you need to access information that is not publicly available:

  • Your membership to a private organisation
  • Your ownership of a Team structure
  • Ability to clone and push your private organisations’ repositories

There is now a special information box suggesting that you have the ability to change your “Scope” if you don’t see the organisations and repositories you want to import.

Screen Shot 2015-10-08 at 10.22.12

After changing the scope, you can then log in again and you will have an improved set of options to get more data and repositories from your GitHub account.

Like it? Will you use it on a daily basis?

We are eager to get your feedback on this new feature: Tell us what you think and let us know what you would change or add to the set of “Scope” permissions.

Gerrit Code Review and Jenkins Continuous Delivery Pipeline on BigData

Posted on by
Reply

Gerrit at the Jenkins User Conference 2015 – London

For the very first time, CloudBees organised a full User Conference in London and we have been very pleased to speak to present a real-life case-study of Continuous Integration and Continuous Delivery applied to a large-scale BigData Project.

See below a summary of the overall presentation published on the above YouTube video.

The trap of the BigData production phase

BigData has been historically used by data scientists in order to analyse data and extract  features that are relevant for the business. This has typically been a very interactive process happing mostly on “notebook-style” environments where almost everything, from ad-hoc queries and graphs, could have been edited and executed interactively. This early stage of the process is typically known as “exploration” or “prototype analysis” phase. Sometimes last only a few days but often is used as day-by-day modus operandi.

However when the exploration phase is over, projects needed to be rewritten or adapted using a programming language (Scala, Python or Java) and transformations and aggregations expressed in jobs. During the “production-isation” phase code needs to be properly written and tested to be suitable for production.

Many projects fall into the trap of reducing the “production phase” to a mere translation of notebooks (or spreadsheets) into Scala, Java or Python code, relying only on the manual analysis of the resulting data as unique testing methodology. The lack of software engineering practices generates complex monolithic code,  difficult to maintain, to understand and thus to validate: the agility of the initial “exploration” phase was then miserably lost in the translation into production code.

Why Continuous Delivery on BigData?

We have approached the development of BigData projects in a radically different way: instead of simply relying on existing tools, often not enough for setting up a proper Agile Delivery Pipeline, we introduced brand-new frameworks and applied them to the building blocks of a Continuous Delivery pipeline.

This is how Stefano Galarraga wrote started the ScaldingUnit project, aimed in de-composing the development of complex Scalding MapReduce jobs in simple and testable units.

We started then to benefit from the improved Agility and speed of delivery, giving constant feedback to data-scientists and delivering constant value to the Business stakeholders during the production phase. The talk presented at the Jenkins User Conference 2015 is smaller-scale show-case of the pipeline we created for our large clients.

Continuous Delivery Pipeline Building Blocks

In order to build a robust continuous delivery pipeline, we do need a robust code-base to start with: seems a bit obvious but is often forgotten. The only way to create a stable code-base,  collectively developed and shared across different [distributed] Teams, is to adopt a robust code review lifecycle.

Gerrit Code Review is the most robust and scalable collaboration system that allows distributed teams to submit their changes and provide valuable feedback about the building blocks of the BigData solution. Data scientists can participate as well during the early stage of the production code development, giving suggestions and insight on the solution whilst is still in progress.

Docker provided the pipeline with the ability to define a set of “standard disposable systems” to host the real-life components of the target runtime, from Oracle to a BigData CDH Cluster.

Jenkins Continuous Integration is the glue that allowed coordinating all the different actors of the pipeline, activating the builds based on the stream events received from Gerrit Code Review and orchestrating the activation of the integration test environments on Docker.

Mesos and Marathon managed all the physical resources to allow a balanced allocation of all the Docker containers across the cluster. Everything has been managed through Mesos / Marathon, including the Gerrit and Jenkins services.

Pipeline flow – Pushing a new change to Gerrit Code Review

The BigData pipeline starts when a new piece of code is changed on the local development environment. Typically developers test local changes using the IDE and the Hadoop “local mode” which allows the local machine to “simulate” the behaviour of the runtime cluster.

The local mode testing is typically good enough for running unit-tests but often is unable to detect problems (e.g. non-serialisable objects, compression, performance) that are likely to appear in the target BigData cluster only. Allowing to push a code change to a target branch without having tested on a real cluster represent a potential risk of breaking the continuous delivery pipeline.

Gerrit Code Review allows the change to be committed and pushed to the Server repository and built on Jenkins Continuous Integration before the code is actually merged into the master branch (pre-commit validation).

Pipeline flow – Build and Unit-tests execution

Jenkins uses the Gerrit Trigger Plugin to fetch the code currently under review (which is not on master but on an open change) and triggers the standard Scala SBT build. This phase is typically very fast and takes only a few seconds to complete and provide the first validation feedback to Gerrit Code Review (Verified +1).

Until now we haven’t done anything special of different than a normal git-flow based continuous integration: we pushed our code and we got it validated in Jenkins before merging it to master. You could actually implement the pipeline until this point using GitHub Pull Requests or similar.

Pipeline flow – Integration test automation with a real BigData Cloudera CDH Cluster

Instead of considering the change “good enough” after a unit-test validation phase and then automatically merging it, we wanted to go through a further validation on a real cluster. We have completely automated the provisioning of a fully featured Cloudera CDH BigData cluster for running our change under review with the real Hadoop components.

In a typical pipeline, integration tests in a BigData Cluster are executed *after* the code is merged, mainly because of the intrinsic latencies associated to the provisioning of a proper reproducible integration environment. How then to speed-up the integration phase without necessarily blocking the development of new features?

We introduced Docker with Mesos / Marathon to have a much more flexible and intelligent management of the virtual resources: without having to virtualise the Hardware we were able to spawn new Docker instances in seconds instead of minutes ! Additionally the provisioning was coordinated by the Docker Build Step Jenkins plugin to allow the orchestration of the integration tests execution and the feedback on Gerrit Code Review.

Whenever an integration test phase succeeded or failed, Jenkins would have then submitted an “Integrated +1/-1” feedback to the original Gerrit Code Review change that triggered the test.

Pipeline flow – Change submission and release

When a change has received the Verified+1 (build + unit-tests successful) and Integrated+1 (integration-tests successful) is definitely ready to be reviewed and submitted to the master branch. The additional commit triggers the final release build that tags the code and uploads it to Nexus ready to be elected for production.

Pipeline flow – Rollout to production

The decision to rollout to production with a new change is typically enabled by a continuous delivery pipeline but manually operated by the Business stakeholders. Even though we could *potentially* rollout every change, we did not want *necessarily* do that because of the associated business implications.

Our approach was then to publish to Nexus all the potential *candidates* to production and roll-them-out to a pre-production environment, ready to be assessed by Data-Scientists and Business in real-time. The daily job scheduler had a configuration parameter that simply allowed to “pointing” to the version of the code to run every day. In this way whatever is deployed to Nexus is potentially fully working in production and rollout or rollback a release is just a matter of changing a label in the daily job scheduler.

Summary

Building a Continuous Delivery Pipeline for BigData has been a lot of fun and improved the agility of the Business in rolling out changes more quickly without having to compromise on features or stability.

When using a traditional Continuous Integration pipeline, the different stages (build + unit-test, integration-tests, system-tests, rollout) are all happening on the target branch causing it to be amber or red at times: whenever tests are failing the pipeline need to be restarted from start and people are blocked.

By adopting a Code Review-driven Continuous Integration Pipeline we managed to get the best of both worlds, avoiding feature branches but still keeping the ability to validate the code at each stage of the pipeline and reporting it back to the original change and the associated developer without to compromise the stability of the target branch or introducing artificial and distracting feature branches.

Resources

The slides of the talk are published on SlideShare.

All the docker images used during the presentation are available on GitHub:

Gerrit Ver. 2.11 is now available for download

Posted on by
3

Gerrit version 2.11 is now available.

Release highlights:

  • Inline editing in the browser
  • Removal of the old change screen
  • Many improvements in the change screen UI

Release notes:
https://gerrit-documentation.storage.googleapis.com/ReleaseNotes/ReleaseNotes-2.11.html

Download:
https://gerrit-releases.storage.googleapis.com/gerrit-2.11.war

Documentation:
https://gerrit-documentation.storage.googleapis.com/Documentation/2.11/index.html

Native packages:
If you have already installed Gerrit 2.10 using them, you need to execute:

(on Debian / Ubuntu)

apt-get update & apt-get install gerrit

(on CentOS / RedHat)

yum clean all && yum install gerrit

If you have not configured yet the GerritForge / BinTray repositories, please follow the instructions at:
https://gitenterprise.me/2015/02/27/gerrit-2-10-rpm-and-debian-packages-available/

Yet again, Gerrit delivers new features and cutting-edge technology improvements for Git and Code Review for the Enterprise!

GitMinutes #30: Luca Milanesio on Gerrit Code Review

Posted on by
Reply

git-minutesMany thanks to Thomas Ferris Nicolaisen for inviting me to talk about Gerrit Code Review at GitMinutes.

It has been a very interesting discussion on the benefits of Code Review and how Gerrit can help out small and large companies embracing it.

The interview is available on-line at http://episodes.gitminutes.com/2014/07/gitminutes-30-luca-milanesio-on-gerrit.html, alternatively you can download and listen the 1h 27′ conversation on PodCast at https://itunes.apple.com/de/podcast/gitminutes-podcasts/id637843725?l=en.

Use the force Luca!

We started (of course!) talking about the [in]famous force push of 186 Jenkins repositories to GitHub, I was on the Top-10 HackersNews over 7h … so I was expecting the question to pop-up during the interview 🙂

My friend Alex Blewitt took the opportunity as well to forge a Star-Wars like headline for his InfoQ article on what happened.

Git adoption in the Enterprise, where all began

We moved the discussion to the foundation of my business on Git and Code Review and the reasons and challenges that an Enterprise company is facing when moving to Git. We went through the history on how LMIT started GitEnterprise.com and then focused on Gerrit Code Review based product and services for large Enterprises World-Wide: a niche and successful business nowadays.

GitHub or Gerrit? or both with GerritHub?

As I expected, we ended up comparing GitHub and Gerrit analysing the similarities and differences between the two. This topic has been presented as well in two conferences at Gerrit User Summit @GooglePlex – Mountain View CA and 33rd Degree.org Java Developers Conference in Krakow; slides are available at http://www.slideshare.net/lucamilanesio/gerrit-codereviewgit-hubplugin.

Gerrit has historically been considered as “more difficult” than GitHub: true in the past but not anymore today apart from the Web User-Experience CSS styling, much nicer and pleasent on GitHub. The availability of http://gerrithub.io allowed over 1,800 developers since October 2013 to get started with Gerrit in less than 5 minutes by watching an Gerrit Introductionary YouTube video: using it was then just 3 clicks away, no installation or configuration needed! The availability of an easy and accessible Public Cloud instance represents a big improvement in accessibility and usability of Gerrit.

For which teams is Gerrit the right choice?

We talked about the “typical learning curve” of people coming from previous version control systems, such as Subversion. Does it make sense to get started with Git and Gerrit at the same time? When is Gerrit needed and when is it going to provide most of its value?

I’ve covered the topic in the past webinars and talks: hands-on Webinars recordings are freely available on-line at:

The size of the project (in terms of number of people x number of repositories) is typically one of the key factors in Code Review adoption. Gerrit however can be used as well as a standalone OpenSource Git Server , even without leveraging its Code Review capabilities: this makes the choice of Gerrit a good first step towards a smoother Git adoption.

What are Gerrit Topics about?

We went through a very interesting discussion about “Gerrit Topic”, a feature that is not new to Gerrit but is sometimes forgotten besides its important and relevance for medium-large teams.

With the forthcoming support of multi-repositories atomic commits in Gerrit, it will be possible to merge multiple changes on multiple repositories at the same time for a single topic. This feature is not ready yet but coming hopefully in the near future and Google Gerrit Team developers and contributors are working on it.

The ability to make an atomic commit across multiple repositories will allow to have a more consistent Jenkins build process as well, with less broken builds because of interdependent changes on multiple components.

Who is using Gerrit today?

We talked about the adoption of Gerrit in the community, which is growing year after year. A lot of medium companies adopted Gerrit in the past, including Spotify side-by-side with GitHub.

The ability to “submit a change” to any project without the risk to break the build is definitely an incentive to encourage even more people to contribute to share the knowledge and improve the code base, without the risk of breaking anything or  forking the code. This is one of the reason that drove large OpenSource organisations such as the Eclipse Foundation and OpenStack to the adoption Gerrit Code Review in their tools platform.

How to embrace Code Review in a Team or Company?

We went through an interesting comparison / discussion of Agile Methodology vs. Code Review. Often Teams misunderstand and confuse the concept of “review” with “pair-programming”: the problem was well analysed in my book “Learning Gerrit Code Review” (available on Amazon.com at http://www.amazon.com/Learning-Gerrit-Code-Review-Milanesio/dp/1783289473). I defined the pair-programming as a dot in a time/people space: two developers writing a piece of code at the same time. This however does not exclude all the other points in the time/people space where multiple people at different times will read the code and provide their feedback: pair-programming is then a “specific example” of the “code review space”.

Because of the different perspectives (pair-programming is a dot whilst code-review is a “cloud of dots” in time/people space) they are not one exclusive of the other: they are equally important and both enable effective collective code ownership and knowledge sharing.

References and greetings.

It has been a very long but interesting discussion with Thomas and hope you’ll enjoy it.

See below the links of the resources we mentioned during the interview:

Thanks again to Thomas for his fantastic initiative: GitMinutes PodCast!

Luca Milanesio 

Heartbleed: GitEnterprise and GerritHub are safe

Posted on by
Reply

heartbleedA few days ago a large part of the Word Wide Web has been found vulnerable to the heartbleed bug in OpenSSL.

What is the vulnerability about?

The vulnerability is effectively a bug in all the versions of OpenSSL from Ver. 1.0.1 to 1.0.2. In reality a lot of web-sites are either using the older and still popular OpenSSL 0.9.8 or they have already upgraded to the latest patched version of OpenSSL and thus are NOT vulnerable to heartbleed.

 

Are you passwords safe ?

In a nutshell yes when they are posted or exchanged with a server that is not vulnerable to this attach:

  • GitEnterprise (gitent-scm.com) has never used any OpenSSL 1.0.1-1.0.2 (see: https://www.ssllabs.com/ssltest/analyze.html?d=gitent%2dscm.com) and thus is not vulnerable: you can keep your existing password as they are safe.
  • GerritHub (gerrithub.io) has been vulnerable for only 5 days and then has been upgraded (see https://www.ssllabs.com/ssltest/analyze.html?d=gerrithub.io). However GerritHub DOES NOT exchange passwords over the Internet but rely on your existing GitHub session through OAuth Token authentication. This means that during the 5 days of vulnerability your account has NOT been at risk on GerritHub.

What about GitHub ?

Unfortunately GitHub has been vulnerable (see https://github.com/blog/1818-security-heartbleed-vulnerability) but the problem has been resolved or is under resolution right now as the nodes get upgraded.

We do recommend then to change your GitHub password in order to be sure that any previous credentials potentially stolen would not impact the security of your account and repositories.

GerritHub relies on GitHub OAuth, so is GerritHub at risk as well ?

In real terms the answer is “potentially yes”: if a potential attacker had been stolen your GitHub password, he could have initiated a login on your behalf and then accessed GerritHub as well.

How can I strengthen my GitHub  security ?

GitHub already support today the two-factor authentication (see https://help.github.com/articles/about-two-factor-authentication): if you have this extra security enabled, nobody other than you can ever access your account, even if they could have potentially stolen your password.

Can I have a GerritHub account secured independetly from GitHub ?

Not yet, however we are working on an advanced security option for the private GerritHub accounts. We will offer for a monthly extra fee:

  • Access to your GitHub private Teams and Repositories
  • Extra scripting functionality to hook Gerrit events on the server side
    (commit validation, issue tracking association, …)
  • Integration with Atlassian Jira or BugZilla
  • Integration with BuildHive from CloudBees for Continuous Integration
  • Extra enterprise account protection for GerritHub.io accounts (additional password / X.509 Certificates)

Wow, that is amazing ! When can I get GerritHub private edition ?

We are currently in public beta stage, you can start using the implemented features for FREE during the trial by logging in to GerritHub using the URL:

https://review.gerrithub.io/login?scope=scopesPrivate

Can I provide suggestions and give feedback during the public beta trial ?

Yes, you are very welcome to provide your feedback and we are very opened to adjust the development of GerritHub private features to your needs !

For problems and getting support:
http://gerritforge.com/support

For suggestions and feedback, please use the Gerrit Code Review forum:
https://groups.google.com/forum/#!forum/repo-discuss

Is GerritHub OpenSource ?

Absolutely YES: GerritHub is based on Gerrit Code Review 2.10-SNAPSHOT master with a selected set of enterprise plugins:

  • GitHub plugin
  • Codenvy plugin
  • ITS-Jira plugin
  • Scripting provider plugin
  • SingleUserGroup plugin
  • Download commands plugin
  • Replication plugin
  • Gravatar plugin
  • Review notes plugin

If you want to directly review and contribute to Gerrit, you are welcome to the developers and contributors community !

 

 

-2 days to the Gerrit User Summit 2014

Posted on by
Reply

The Gerrit User Summit 2014 is about to start in only 2 days: it is going to be a two days of exciting news and innovations on the world of Code Review. There are names from the largest industries in the world that have adopted the Code Review workflow in large enterprise environments: Google, SAP, SonyMobile, Ericsson, IBM, Garmin, HP, CollabNet, GerritForge, Codenvy, Eclipse Foundation and LibreOffice.

During all this week there is a special promotional discount on the Learning Gerrit Code Review book. Additionally, for the attendees of the conference, there will be a limited number of signed paperback copies available at the session “Gerrit or GitHub? Take both !”

Learning-Gerrit-Code-Review-QRCodeIn order to redeem the book promotion, scan the QR code and enter one of the following PROMO-CODEs:

Book PROMO-CODE: LGCRB20
eBook PROMO-CODE: LGCReB20

 

 

The Gerrit User Summit Agenda has been published yesterday and has a lot of very interesting talks and announcements:

Day 1 – Friday 21st of March

  • What’s new in Gerrit 2.8 (David Pursehouse – Gerrit maintainer – SonyMobile)
  • Scaling Gerrit at Ericsson (Patrick Renaud, Vladimir Cantiru, Hugo Ares – Ericsson)
  • Monitoring Gerrit (Doug Kelly – Garmin)
  • Browsing Repository Content with Gerrit’s REST API (Simon Kaegi – IBM)
  • Gerrit@LibreOffice (David Ostrovsky – LibreOffice)
  • Gerrit plugins made easy with Scripting (Luca Milanesio – GerritForge)
  • The Angular revolution in Gerrit! (Dariusz Luksza – CollabNet)

The day 1 would end with a very interesting Q&A with the Gerrit User Community about the features they would like to see coming up in the next forthcoming releases!

Day 2 – Saturday 22nd of March

  • 2014 Roadmap (Shawn Pearce – Gerrit project founder, Google)
  • Gerrit@SAP (Edwin Kempin – Gerrit Code Review maintainer – SAP)
  • Integrating CLA and Origin checks with Gerrit (Denis Roy – Eclipse Foundation)
  • Guiding Diffy to the Enterprise land (Dariusz Luksza, Eryk Szymanski – CollabNet)
  • Collaboration at Scale: The Openstack CI toolbox (Khai Do – HP)
  • Gerrit or GitHub? Take Both! (Luca Milanesio – GerritForge)
  • Diffy gets Enterprise grade (Dariusz Luksza, Eryk Szymanski – CollabNet)
  • Continuous Development with Gerrit (Tyler Jewell & Luca Milanesio – Codenvy & GerritForge)

The day 2 will end with a meet-up with food and drinks sponsored and organised by Codenvy where the Gerrit Community can discuss and exchange their post-Summit impressions and ideas on the future of Code Review.

It is going to be again a huge leap forward for the Code Review community and the Git and Gerrit projects improvement !