Category Archives: gcp

ARM-64 welcomes Gerrit Code Review

Posted on by
Reply

It has been seven years since we first introduced the native packages for Gerrit Code Review. At that time, Docker was just two years old and was starting to pick up momentum: the native packages found their first incarnation in the Gerrit Code Review base on images published on DockerHub.

Since then, Gerrit has always been released as both a “plain vanilla” war file and the equivalent “native packaging” flavors, including the base Docker images.

In the beginning, the release managers mainly used the Gerrit Docker images to simulate pre-configured test environments for testing purposes. However, the adoption spiked up when Kubernetes and AWS Container Services (aka ECS) started to become mainstream.

Nowadays, the Gerrit base images on Dockerhub are pulled millions of times and used as the baseline for creating Enterprise-grade setups, like in the aws-gerrit recipes.

x86 Native Packaging

Even though it wasn’t initially thought off, the native packages ended up mainly in x86-based containers, even though Gerrit is a 100% pure-Java application. How was that possible?

  • Dependencies: gerrit.sh requires some basic OS services to work properly, including the native git command line.
  • Build & testing: the build of gerrit.war and associated testing was always performed on x86, also due to the restrictions of the platform supported by Bazel at that time.

ARM-64 takes off

The ARM processor comes from the historical legacy of Acorn Computers Ltd, a British-born company of the 80s, well known for the BBC Micro project. Back in the 90s, a new joint venture was created with Apple Computer and VLSI Technology to design a brand-new Advanced Risc Machine (aka ARM).

Over time, ARM has created first a 32-bit architecture and subsequently a 64-bit architecture processor, well known and appreciated for its low-power consumption and performance efficiency.

Whilst Intel, with its x86 architecture, dominated the desktop and the server market, the ARM architecture found its natural market space in the low-power devices and, more recently, the mobile phones market.

Apple goes ARM-64, everywhere

Apple initially adopted the ARM architecture on its mobile phones and portable devices, licensing the instruction set to build its microchips in 2010. Ten years later, Apple announced it was also transitioning all the other desktop devices to its ARM-based silicon.

Nowadays, nearly all Apple laptops are on ARM and the other devices are migrating to that too. The applications are rapidly adapting to the new instruction set and getting rebuilt quickly, gaining in both performance and increased battery life.

Cloud services are moving to ARM-64 too.

The move from x86 to ARM-64 brings noticeable advantages in terms of costs and energy efficiency has been recently amplified by the increase in the costs of electricity triggered by the spiked wholesale prices of oil, gas, and other fossil fuel.

GCloud (see T2A offering ) and AWS (see EC2 Graviton) offer now ARM-64 based VMs that promise a healthy 60% reduction of cloud hosting costs and higher throughput and performance per $.

Gerrit has always put high performance and low operating costs at the top of its agenda, allowing companies to keep their development teams highly productive with minimum dedicated staff looking after the system.

Now it is also time to reduce the Gerrit Code Review Cloud hosting costs and increase efficiency while preserving all the benefits that Gerrit brings.

Introducing ARM-64 native packages for Gerrit Code Review

Gerrit Code Review native packages are now available for ARM-64 on all its recent releases, from v3.4.8 onwards. If you are currently running on x86, you can easily switch to ARM-64 transparently by just rebuilding from the base image. All the DockerHub images have been repackaged and republished:

Gerrit Native packages on bare-metal

If you are running Gerrit native packages on bare-metal, you would need to setup the GerritForge’s updated RPM repository:

rpm -i https://gerritforge.com/gerritforge-repo-1-4.noarch.rpm

On Debian, just refresh the repository cache with ‘apt-get update‘ and the new packages will automatically show-up and be available for setup.

Because this is a repackaging of the existing Gerrit release, the 4th number of the package name has been updated, e.g., gerrit-3.4.8-2 is the package name associated with the ARM-64 compatible package.

Ready to switch to ARM-64?

The Gerrit maintainers have already switched to ARM-64 with the recent wave of MacBook upgrades: all the non-EOL Gerrit releases are typically developed on Apple M1 or M2 which are based on ARM-64 rather than x86.

The need to reduce costs and the overall carbon footprint of the Gerrit project would most likely drive the CI agents’ switch to ARM-64 very soon.

Are you ready to switch to ARM-64 to reduce your operating costs and help the planet by reducing your Gerrit consumption?

Q&A

Q1. Are Gerrit on x86 and ARM-64 compatible in terms of data?

Yes, because the underlying data structure is platform-independent and is all based on Git, which is already interoperable cross-platform.

Q2. Can I mix&match x86 and ARM-64 Gerrit nodes?

Yes, by adopting the Gerrit multi-site setup, you can configure and host different sites on different architectures.

Q3. Will Gerrit on x86 continue to be supported?

Yes, because we want to allow the Gerrit administrator to choose the most suitable platform for his users and compatible with his Company’s requirements.

Luca Milanesio – GerritForge
Gerrit Code Review Maintainer
ESC Member
Gerrit Code Review Release Manager

2022 GOALS for Gerrit

Posted on by
Reply

The year 2021 has been a challenging one because of the COVID-19 global emergency; nevertheless, the Gerrit Code Review project has continued to deliver what the community expected:

GerritForge delivered on the promise of making Gerrit more cloud-native, with a particular focus on AWS, the platform that most users have adopted for running in the cloud. GoogleCloud has also been our focus, assuring a cloud-neutral approach to Gerrit and providing support for events over PubSub.

Focusing on Gerrit unique values

A successful product focuses on what makes it unique and innovative, compared to anything else in the world.

We believe that the key aspects that make Gerrit Code Review THE platform of choice for developing software based on Git repositories are:

  • Large-scale
    Gerrit is THE best platform for developing large-scale projects, huge monorepos, and a large number of changes and refs.
  • Maximum availability
    Large organizations and communities of developers need a platform that is always available, anywhere, anytime, 24×7, and 365 days a year.
  • Performance
    The need to work remotely poses multiple issues, one of them being the increase of network latency. Gerrit multi-site distribution of the repositories and reviews allows anyone, anywhere in the world, to clone, push and review at optimal latency and performance.
  • Quality of tracing of reviews
    Gerrit is based on single-commit code reviews, a winning approach in terms of review accuracy and supporting changes chains, and full traceability of the entire review history and workflow.

Many popular Git code-review tools exist in the Open-Source community; Gerrit is the winning choice when scale, availability, performance, and quality do matter.

GerritForge goals for improving Gerrit in 2022

Scale Gerrit beyond limits

GerritForge and the rest of the community have worked hard to identify the bottlenecks of large mono-repos with Gerrit. Some of them can be mitigated by keeping the Git repository lean and organized, despite the massive amount of push traffic and reviews coming from large teams.

We want to focus on improving at least ten times the following KPIs, without having a significant impact on the overall system performance:

  • Number of changes and refs in a repository: millions of changes and tens of millions of refs
  • Size of the repository: hundreds of GBs

GerritForge will step up its involvement in the JGit project in 2022 and introduce many innovations, some of them already implemented in the C-Git implementation:

  • JGit support for multi-pack index
  • Revamp of JGit cache, allowing the pluggability of high-performance implementations
  • Improvement of JGit bitmaps for large number of refs
  • Support for high-performance large storage systems
  • Introduction of new performance metrics
  • Replace Prolog with native submit rules in the owners plugin

99.999% up-time

GerritForge maintains a free service known as GerritHub.io to demonstrate what Gerrit can do and achieve. GerritHub.io is the most advanced and reliable Open-Source vanilla Gerrit deployment, apart from Google’s.

GerritHub.io uptime in 2021 – checked and reported by PIngdom.com

We achieved an astonishing 99.99% SLA in 2021; we want to push the GerritHub.io uptime further to 99.999%, reducing the annual downtime to just 315s.

In order to reach a five-nines uptime, we will work on:

  • Granular probing and health-checks
  • Advanced repositories performance monitoring and alerting
  • Gerrit limits and deadlines
  • RCAs
  • Multi-site improvements

Goal #3: Increase 1000x times the Gerrit replication performance

GerritForge has presented the innovating pull-replication plugin at the Gerrit Virtual User Summit 2021, showing that it is possible to replicate Git commits and changes meta-data across the globe with msec latency. The pull-replication plugin technology and speed is going to be improved and made available and Open-Sourced to anyone and match and outperform the traditional replication plugin features.

Join the 2022 endeavor


We need YOU and the Gerrit community’s help and support in this 2022 endeavor.

GerritForge has already increased his Team of contributors working on the project, including three Gerrit maintainers and two Gerrit release managers. However, Gerrit’s success is in the cooperation, contribution, and ideas of the whole community of contributors, Gerrit admins, and users.

Let us know what you think about our goals. We are happy to cooperate and work with anyone sharing the same values and goals.

2022 is the year where Gerrit Code Review is pushed beyond its limits even further, making it the MOST innovative tool for large-scale repositories and teams worldwide.

2021: What’s cooking in GerritForge

Posted on by
Reply

Our mission is to improve the Gerrit Code Review platform, with the Open-Source community and in full transparency. Everything we do is published, is open, can be analysed and scrutinised by any member of the community at any time.

In this article, we are publishing what GerritForge is planning to do in the forthcoming 2021, a pivotal year that will see the entire world bouncing back and looking at the future of Technology with a different light.

Goal: cloud-native Gerrit Code Review

Since our initial post in 2019 about our plans for the future, we have been making steady progress toward a unified global Gerrit platform, highly available and distributed across the globe. Since then, a lot of steps have been made, many of them happened in 2020.

Thanks to SAP and GerritForge, we have now two Open-Source standard deployments on the Cloud:

  1. K8s-Gerrit – Helm-charts for deploying Gerrit Code Review to a Kubernetes Cluster (beta)
  2. AWS-Gerrit – CloudFormation templates for creating a production-like setup of Gerrit HA and Multi-Site to ECS on the AWS platform.

Even if Gerrit can deployed now into a Cloud environment, that does not mean that Gerrit is cloud-native. The main requirements for making an application cloud-native is captured by the 12-factor principles. We will be tackling some of them in our 2021 roadmap.

Cloud-native backing services

Thanks to the advances in the Gerrit Multi-Site plugin in 2020, it is today possible to have a geographically distributed cluster of Gerrit servers around the globe. The services that are needed for coordinating the events across Gerrit nodes and ensuring global consistency are:

  1. Event-broker (Kafka)
  2. Global ref-db (Zookeeper)

The price to pay for a global Gerrit Multi-Site is currently quite high, because one may argue that actually managing a cluster of Kafka brokers or Zookeeper nodes can be even more challenging that managing Gerrit itself.

A cloud-native application should be fully independent from its backing services and be opened to swapping one specific implementation (e.g. Kafka) with a serviced off-the-shelf solution (e.g. AWS Kinesis, GCP PubSub).

As GerritForge we are joining the forces with the Gerrit Code Review community in 2021 for refactoring the Gerrit Event system and making it more Cloud-native, using open-standards for representing and publishing events across the network.

The immediate actions are:

  1. Build a new protobuf-based events representation in Gerrit v3.4
  2. Abstract the serialisation of events from their transport
  3. Implement different events brokers and listeners, other than the Kafka. (e.g. NATS)
  4. Provide a AWS Kinesis-stream event broker connector
  5. Provide a GCP PubSub broker connector in cooperation with Google

Gerrit performance across sites

Making a stateful application like Gerrit Code Review a truly cloud-native and distributed platform brings some challenges in terms of latency and performance.

The current Gerrit Multi-Site architecture still relies heavily on the traditional (push) replication plugin, which has been working nicely for over a decade for a Gerrit primary/replica setup. When the repository size increases and the need for global consistency becomes more stringent, relying on a traditional async git push is not enough anymore: you need a lot more tools at your service.

GerritForge has introduced in 2020 the pull-replication plugin, which enables to invert the replication logic: instead of having one Gerrit server to push everything to everyone else, you just tell the other nodes of what’s new in the repository, and it will be their responsibility to fetch only the stuff that has changed.

In 2021 we are going to improve the plugin with native JGit client protocol v2 support and features that will go well beyond the limits of the git protocol:

  1. BLOB and ref-update on remote git pulls. The remote Gerrit node can receive directly the content of what has changed, directly in its payload. That allows to avoid the expensive git pull operation especially for NoteDb related updates.
  2. Event broker integration for git pulls. Instead of relying on REST-API, the pull replication plugin can produce pull replication events in the event broker.

The implementation of those improvements will allow to drastically reduce the incidence of the traditional (push) replication activity and make the Gerrit user experience more seamless across sites.

CI integration

Integrate Gerrit in more seamlessly with other cloud-native services, such as CI/CD. Gerrit v3.4 is planning to introduce a brand-new style of integration which would not require anymore any 3rd party adaptation or API support.

GerritForge is committed to endorse this new module and allow Gerrit v3.4 to be fully connected with Jenkins on day-1 and potentially other cloud-based CI/CD systems.

Git reftable: provide more performance for mono-repos

GerritForge will experiment and test at scale the introduction of Google’s Git reftable, already included from Gerrit v3.2 onwards, to assess its performance impact on large mono-repos, with hundreds of thousands of refs.

Have your say on Gerrit future

This is our GerritForge shopping list for 2021, wide open for discussion, contributions, amendments, and voting.

Why should you care about what we do? Simply because the union of intents and cross-pollination of ideas make the Open-Source products and community stronger: if we join forces in building a solution that is a common interest for all of us, we will all benefit from it.

Your feedback is welcome and precious, feel free to have your say and comment this post with your ideas, likes and suggestions on how to make them even better.

The GerritForge Team.