New year, new JGit contributions coming

We have shared GerritForge’s goals for improving Gerrit in 2022: most of them will include significant contributions to JGit, the Java-based engine powering Gerrit Code Review’s support for Git data-format and protocol.

GerritForge will contribute many more changes to JGit during 2022, all focused on improving the functionality and performance of large mono-repos. All changes will go through the formal review through Eclipse Foundation’s JGit Gerrit Project.

Lack of knowledge and reviews

The JGit project has suffered from major losses in the past few years, which is clearly shown by the list of top-contributors vs. their recent 12 months activity. I have tried running a “git blame” against all the JGit code-base, which is a heuristic (therefore a rough approximation) of which part of the JGit code has been last written/edited.

  1. (49998 LOC) – Shawn Pearce
  2. (37854 LOC) – Thomas Wolf
  3. (31417 LOC) – Matthias Sohn
  4. (13593 LOC) – David Pursehouse
  5. (13200 LOC) – Christian Halstrick

See below the number of contributions (excluding merges and trivial changes) of the above 5 maintainers in the past 12 months:

  1. Shawn Pearce – 0 changes – He sadly passed away in 2018 
  2. Thomas Wolf – 86 changes
  3. Matthias Sohn – 128 changes
  4. David PurseHouse – 0 changes
  5. Christian Halstrick – 0 changes

The above stats show that the currently active maintainers (Thomas and Matthias) appear in the bit blame of 69k out of the total 390k LOCs.

Thomas and Matthias are doing a fantastic job in keeping up as much as possible with the incoming changes’ pace and reviewing them at their best. At times, though, the incoming change may touch parts of the code they are less familiar with and, therefore, would require more eyes or more time to review.

Breaking the vicious circle

The JGit project is in a dangerous vicious circle.

  1. Incoming changes would take longer to get reviewed and merged.
  2. The lengthy reviews cause detriment to contributors that may lose interest in following up contributions or upload new changes.
  3. The lack of contributions and merged changes would keep the pressure on current maintainers, which fuel point 1. again.

How can GerritForge help break this vicious circle, provide meaningful contributions, and get them merged fast and with proper and thorough reviews?

Keeping the pace of contributions is key to avoid detriment: GerritForge will therefore create a “dev branch” of JGit. All the GerritForge’s contributions to JGit master branch will be part of the dev branch and will go through a rigorous code-review and E2E validation cycle, including the Gatling tests for Gerrit.

Two-steps validation workflow

GerritForge’s workflow for validating JGit changes with Gerrit
  1. A new change is uploaded to the Eclipse Foundation JGit project.
  2. The normal Eclipse Foundation’s CI verification builds the change and, if passes all tests, provides a Verified +1
  3. One of the JGit maintainers, or members of the GerritForge’s contributors, can provide a Code-Review +1 score with the additional description “Approved for dev
  4. The special “Approved for dev” description triggers the cherry-pick of the Change onto the GerritForge’s JGit dev branch
  5. The new change for review on the JGit dev branch triggers the creation of a Change on gerrit-review.googlesource.com/gerrit with the update of the JGit submodule pointing to the open change.
  6. The JGit submodule update Change triggers the current E2E validation using the Gatling tests, developed and hosted by GerritForge. If all tests are passing, the Gerrit change receives a Verified +1.
  7. The cherry-picked Change on JGit dev branch receives a Verified +1
  8. The cherry-picked Change is merged to the JGit dev branch
  9. The merge of the cherry-picked Change is notified on the original Change with a Code-Review +1 score with description “Merged in dev”.
  10. One of the JGit maintainers can finalise the review and, if all is good, provides the final Code-Review +2 and merge the change on JGit master.

NOTE: The above workflow will only apply to the upcoming changes on the master branch, where we do need to innovate and implement new features at a faster pace. We have to plans to apply the workflow to any stable branches.

Plus and minuses

There are good things on the above workflow, however, there are also risks:

  • Complexity: the secondary dev branch will undergo an E2E validation process with Gerrit, which is obviously complex and it may break at times.
  • Danger of forking: if the JGit maintainers would veto the Change at step 10 the changes already merged in dev would make effectively dev and master branches diverge, which isn’t a good thing and it should be avoided as much as possible.

The augmented lifecycle, which also involves Gerrit E2E tests with Gatling, has also many advantages:

  • Additional E2E validation: incoming changes on JGit would involve an E2E validation with Gerrit against the suite of E2E Gatling tests, which is good feedback and gives more confidence in merging code also on less known parts of the JGit codebase.
  • Increased velocity: speedup validation of new incoming changes and getting them merged to the JGit dev branch, without impacting the pace and quality of reviews from the current JGit maintainers.
  • Gerrit edge release: allows to have a downloadable Gerrit change that includes the JGit dev branch, allowing canary deployments and see how Gerrit behaves with the latest and greatest of JGit code.

From a JGit project’s perspective, the flow of incoming changes will have an additional E2E validation, which is always a good thing. Additionally, it will bring more contributions and inspiration for new innovative changes on the project and attracting more and more talent.

Ready to gear-up contributions on JGit?

The workflow proposed is a starting point; however, we are committed to giving it a go and seeing how it would work in practice and if it will be enough to gear up the contributions to the JGit project.

2022 GOALS for Gerrit

The year 2021 has been a challenging one because of the COVID-19 global emergency; nevertheless, the Gerrit Code Review project has continued to deliver what the community expected:

GerritForge delivered on the promise of making Gerrit more cloud-native, with a particular focus on AWS, the platform that most users have adopted for running in the cloud. GoogleCloud has also been our focus, assuring a cloud-neutral approach to Gerrit and providing support for events over PubSub.

Focusing on Gerrit unique values

A successful product focuses on what makes it unique and innovative, compared to anything else in the world.

We believe that the key aspects that make Gerrit Code Review THE platform of choice for developing software based on Git repositories are:

  • Large-scale
    Gerrit is THE best platform for developing large-scale projects, huge monorepos, and a large number of changes and refs.
  • Maximum availability
    Large organizations and communities of developers need a platform that is always available, anywhere, anytime, 24×7, and 365 days a year.
  • Performance
    The need to work remotely poses multiple issues, one of them being the increase of network latency. Gerrit multi-site distribution of the repositories and reviews allows anyone, anywhere in the world, to clone, push and review at optimal latency and performance.
  • Quality of tracing of reviews
    Gerrit is based on single-commit code reviews, a winning approach in terms of review accuracy and supporting changes chains, and full traceability of the entire review history and workflow.

Many popular Git code-review tools exist in the Open-Source community; Gerrit is the winning choice when scale, availability, performance, and quality do matter.

GerritForge goals for improving Gerrit in 2022

Scale Gerrit beyond limits

GerritForge and the rest of the community have worked hard to identify the bottlenecks of large mono-repos with Gerrit. Some of them can be mitigated by keeping the Git repository lean and organized, despite the massive amount of push traffic and reviews coming from large teams.

We want to focus on improving at least ten times the following KPIs, without having a significant impact on the overall system performance:

  • Number of changes and refs in a repository: millions of changes and tens of millions of refs
  • Size of the repository: hundreds of GBs

GerritForge will step up its involvement in the JGit project in 2022 and introduce many innovations, some of them already implemented in the C-Git implementation:

  • JGit support for multi-pack index
  • Revamp of JGit cache, allowing the pluggability of high-performance implementations
  • Improvement of JGit bitmaps for large number of refs
  • Support for high-performance large storage systems
  • Introduction of new performance metrics
  • Replace Prolog with native submit rules in the owners plugin

99.999% up-time

GerritForge maintains a free service known as GerritHub.io to demonstrate what Gerrit can do and achieve. GerritHub.io is the most advanced and reliable Open-Source vanilla Gerrit deployment, apart from Google’s.

GerritHub.io uptime in 2021 – checked and reported by PIngdom.com

We achieved an astonishing 99.99% SLA in 2021; we want to push the GerritHub.io uptime further to 99.999%, reducing the annual downtime to just 315s.

In order to reach a five-nines uptime, we will work on:

  • Granular probing and health-checks
  • Advanced repositories performance monitoring and alerting
  • Gerrit limits and deadlines
  • RCAs
  • Multi-site improvements

Goal #3: Increase 1000x times the Gerrit replication performance

GerritForge has presented the innovating pull-replication plugin at the Gerrit Virtual User Summit 2021, showing that it is possible to replicate Git commits and changes meta-data across the globe with msec latency. The pull-replication plugin technology and speed is going to be improved and made available and Open-Sourced to anyone and match and outperform the traditional replication plugin features.

Join the 2022 endeavor


We need YOU and the Gerrit community’s help and support in this 2022 endeavor.

GerritForge has already increased his Team of contributors working on the project, including three Gerrit maintainers and two Gerrit release managers. However, Gerrit’s success is in the cooperation, contribution, and ideas of the whole community of contributors, Gerrit admins, and users.

Let us know what you think about our goals. We are happy to cooperate and work with anyone sharing the same values and goals.

2022 is the year where Gerrit Code Review is pushed beyond its limits even further, making it the MOST innovative tool for large-scale repositories and teams worldwide.