2021: What’s cooking in GerritForge

Our mission is to improve the Gerrit Code Review platform, with the Open-Source community and in full transparency. Everything we do is published, is open, can be analysed and scrutinised by any member of the community at any time.

In this article, we are publishing what GerritForge is planning to do in the forthcoming 2021, a pivotal year that will see the entire world bouncing back and looking at the future of Technology with a different light.

Goal: cloud-native Gerrit Code Review

Since our initial post in 2019 about our plans for the future, we have been making steady progress toward a unified global Gerrit platform, highly available and distributed across the globe. Since then, a lot of steps have been made, many of them happened in 2020.

Thanks to SAP and GerritForge, we have now two Open-Source standard deployments on the Cloud:

  1. K8s-Gerrit – Helm-charts for deploying Gerrit Code Review to a Kubernetes Cluster (beta)
  2. AWS-Gerrit – CloudFormation templates for creating a production-like setup of Gerrit HA and Multi-Site to ECS on the AWS platform.

Even if Gerrit can deployed now into a Cloud environment, that does not mean that Gerrit is cloud-native. The main requirements for making an application cloud-native is captured by the 12-factor principles. We will be tackling some of them in our 2021 roadmap.

Cloud-native backing services

Thanks to the advances in the Gerrit Multi-Site plugin in 2020, it is today possible to have a geographically distributed cluster of Gerrit servers around the globe. The services that are needed for coordinating the events across Gerrit nodes and ensuring global consistency are:

  1. Event-broker (Kafka)
  2. Global ref-db (Zookeeper)

The price to pay for a global Gerrit Multi-Site is currently quite high, because one may argue that actually managing a cluster of Kafka brokers or Zookeeper nodes can be even more challenging that managing Gerrit itself.

A cloud-native application should be fully independent from its backing services and be opened to swapping one specific implementation (e.g. Kafka) with a serviced off-the-shelf solution (e.g. AWS Kinesis, GCP PubSub).

As GerritForge we are joining the forces with the Gerrit Code Review community in 2021 for refactoring the Gerrit Event system and making it more Cloud-native, using open-standards for representing and publishing events across the network.

The immediate actions are:

  1. Build a new protobuf-based events representation in Gerrit v3.4
  2. Abstract the serialisation of events from their transport
  3. Implement different events brokers and listeners, other than the Kafka. (e.g. NATS)
  4. Provide a AWS Kinesis-stream event broker connector
  5. Provide a GCP PubSub broker connector in cooperation with Google

Gerrit performance across sites

Making a stateful application like Gerrit Code Review a truly cloud-native and distributed platform brings some challenges in terms of latency and performance.

The current Gerrit Multi-Site architecture still relies heavily on the traditional (push) replication plugin, which has been working nicely for over a decade for a Gerrit primary/replica setup. When the repository size increases and the need for global consistency becomes more stringent, relying on a traditional async git push is not enough anymore: you need a lot more tools at your service.

GerritForge has introduced in 2020 the pull-replication plugin, which enables to invert the replication logic: instead of having one Gerrit server to push everything to everyone else, you just tell the other nodes of what’s new in the repository, and it will be their responsibility to fetch only the stuff that has changed.

In 2021 we are going to improve the plugin with native JGit client protocol v2 support and features that will go well beyond the limits of the git protocol:

  1. BLOB and ref-update on remote git pulls. The remote Gerrit node can receive directly the content of what has changed, directly in its payload. That allows to avoid the expensive git pull operation especially for NoteDb related updates.
  2. Event broker integration for git pulls. Instead of relying on REST-API, the pull replication plugin can produce pull replication events in the event broker.

The implementation of those improvements will allow to drastically reduce the incidence of the traditional (push) replication activity and make the Gerrit user experience more seamless across sites.

CI integration

Integrate Gerrit in more seamlessly with other cloud-native services, such as CI/CD. Gerrit v3.4 is planning to introduce a brand-new style of integration which would not require anymore any 3rd party adaptation or API support.

GerritForge is committed to endorse this new module and allow Gerrit v3.4 to be fully connected with Jenkins on day-1 and potentially other cloud-based CI/CD systems.

Git reftable: provide more performance for mono-repos

GerritForge will experiment and test at scale the introduction of Google’s Git reftable, already included from Gerrit v3.2 onwards, to assess its performance impact on large mono-repos, with hundreds of thousands of refs.

Have your say on Gerrit future

This is our GerritForge shopping list for 2021, wide open for discussion, contributions, amendments, and voting.

Why should you care about what we do? Simply because the union of intents and cross-pollination of ideas make the Open-Source products and community stronger: if we join forces in building a solution that is a common interest for all of us, we will all benefit from it.

Your feedback is welcome and precious, feel free to have your say and comment this post with your ideas, likes and suggestions on how to make them even better.

The GerritForge Team.

GerritForge 2020 year in Review

Dear Gerrit Code Review fellow,

It has been a challenging year, a strange year, for everyone. Most of us confined in our homes and working remotely, finding new ways of dealing with old problems.

Still, we believe the Gerrit Community as a whole has demonstrated an outstanding level of resilience in the face of exceptional difficulties. As far as we are concerned, we have seen no lesser activity, interest, new projects, compared to the previous years. For this, we are thankful to the community we are so strongly proud of being a part of.

In sharing our most sincere vows for happy festivities and a fruitful new year, we wanted to take the opportunity to share with you some of the achievements that made this 2020 worth getting through.

Top-Ten achievements in 2020

1. GerritForge has confirmed to be the world’s largest non-Google contributor to the Gerrit Code Review project.

GerritForge contributed almost a thousand changes merged, over 47 different components in the past 12 months. That is an outstanding achievement confirming the commitment and dedication of the company to the Gerrit Code Review platform and community.

2. Improved Gerrit DevOps Analytics with the ability to process Changes hashtags

The Gerrit DevOps Analytics platform continued to expand its possibilities, with the full support of the parsing of the NoteDb changes and the extractions of precious meta-data, such as the changes hashtags.

3. Driven two major releases of Gerrit Code Review

GerritForge helped driving the release of two major Gerrit Code Review version: Gerrit v3.2 – 1st June and with Ericsson making the Gerrit v3.3 – 1st December. GerritForge is also providing the CI/CD pipeline for the build and validation of the releases and helped the migration to Java 11.

4. Released major security fixes for the whole Gerrit Community, of all Gerrit versions dating back to v2.14

GerritForge coordinated together with Ericsson the release of critical security fixes across 6 different versions of the Gerrit Code Review platform, showing a continued commitment to a secure and scalable adoption of Gerrit in the enterprise.

5. Brand-new Gerrit module cache-chroniclemap, a new high-performance persistent cache in Gerrit v3.3

GerritForge continued its efforts in improving Gerrit’s scalability and performance with the development of a brand-new persistent cache backend powered by ChronicleMap, showing unprecedented low latency and performance on Gerrit v3.3.

6. Introduced the world-first official Open-Source AWS-based Gerrit Code Review Deployment

In a year of remote working and flexible environments, GerritForge provided its contribution to the whole community to help adopting Gerrit in the cloud. The aws-gerrit project is a brand-new production-ready AWS deployment now available for everyone and fully Open-Source. The new project is based on the GitOps best-practices and has been successfully adopted for the testing and validation of Gerrit v3.3.

7. New read-write scalability for Gerrit in High-Availability

GerritForge’s mission to scalability and high-availability (HA) plugin continued with the ability to scale horizontally with multiple R/W Gerrit Servers behind the same load balancer.

8. Improved reliability and flexibility of Gerrit Multi-Site

The Gerrit Multi-Site (MS) plugin has received exciting improvements with the support for geographically distributed R/W Gerrit servers across the globe.

9. Brand-new pull-replication plugin for improving latency and performance of large mono-repos replication.

The need to have Gerrit multi-site brought the request to have faster replication, especially for large mono-repos. GerritForge introduced a new pull-replication plugin project that, used in conjunction with Git Protocol v2, assure top-notch performance in replicating repositories with large number of refs.

10. Helped large-scale OpenSource Community migrating to Gerrit v3

The Eclipse Foundation and the OpenDev platform (by OpenStack), upgraded to the latest version of Gerrit Code Review, thanks to the continuous help and support from GerritForge and the rest of the Gerrit Code Review community.


All the best, Stay Safe, and we will shake hands again soon!

The GerritForge Team

Continuous Development with GerritForge and Codenvy

On March 22nd, come see Codenvy CEO Tyler Jewell and Gerritforge CEO Luca Milanesio present at Google’s HQ in Mountain View, CA. They’ll cover Codenvy’s continuous delivery system for integrating code reviews, git, and SAAS developer environments in order to eliminate waste in the development workflow.

[…]

Read the full story at Codenvy Blog
[by Eric Cavazos]