How Git and Gerrit are Re-Tooling for the Age of AI

A Special Report from the Gerrit User Summit 2025

First, a huge thank you to the OpenInfra Foundation for hosting this event in Paris. Their invitation to have the Gerrit User Summit join the rest of the community set the stage for a truly collaborative and impactful gathering.

Paris last weekend wasn’t just a conference; it was a reunion. Fourteen years after the last GitTogether at Google’s Mountain View HQ, the “Git and Gerrit, together again” spirit was electric.

On October 18-19, luminaries from the early days (Scott, Martin, Luca, and many others) reconvened, sharing the floor with the new generation of innovators. The atmosphere was intense, filled with the same collaborative energy of 2011, but focused on a new set of challenges. The core question: how to evolve Git and Gerrit for the next decade of software development, a future dominated by AI, massive scale, and an urgent demand for smarter workflows.

Here are the key dispatches from the summit floor.

A Historic Reunion, A Shared Future

This event was a powerful reminder that the open-source spirit of cross-pollination is alive and well. The discussions were invigorated by the “fresh air” from new-school tools like GitButler and Jujutsu (JJ), which are fundamentally rethinking the developer experience.

In a significant show of industry-wide collaboration, we were delighted to have GitLab actively participating. Patrick’s technical presentation on the status of reftable was a highlight, but his engagement in discussions on collaborative solutions moving forward with the Gerrit community truly set the tone. It’s clear that the challenges ahead are shared by all platforms, and the solutions will be too.

Scaling Git in the Age of AI

The central theme was scale. In this rapidly accelerating AI era, software repositories are growing at an unprecedented rate across all platforms—Gerrit, GitHub, and GitLab alike. This isn’t a linear increase; it’s an explosion, and it’s pushing SCM systems to their breaking point.

The consensus was clear: traditional vertical and horizontal scaling is no longer enough. The community is now in a race to explore new techniques—from the metal up—to improve performance, slash memory usage, and make core Git operations efficient at a scale we’ve never seen before. This summit was a rare chance for maintainers from different ecosystems to align on these shared problems and forge collaborative paths to solutions.

Dispatches from the Front Lines: NVIDIA and Qualcomm

This challenge isn’t theoretical. We heard powerful testimonials from industry giants NVIDIA and Qualcomm, who are on the front lines of the AI revolution.

They shared fascinating and sobering insights into the repository explosion they are actively managing. Their AI workflows—encompassing massive datasets, huge model binaries, and unprecedented CI/CD activity—are generating data on a scale that is stressing even the most robust SCM systems. Their presentations detailed the unique challenges and innovative approaches they are pioneering to tackle this data gravity, providing invaluable real-world context that fueled the summit’s technical deep dives.

Beyond the Pull Request: The Quest for a ‘Commit-First’ World

One of the most passionate debates centered on the developer workflow itself. The wider Git community increasingly recognizes that the traditional, monolithic “pull request” model is ill-suited to the “change-focused” code review that platforms like Gerrit have championed for years.

The benefits of a change-based workflow, cleaner history, better hygiene, and higher-quality atomic changes—are driving a growing interest in standardizing a persistent Change-ID for each commit. This would make structured, atomic reviews a first-class citizen in Git itself. The collaboration at the summit between the Gerrit community, GitButler, JJ, and other Git contributors on defining this standard was a major breakthrough.

This shift is being powered by tools like GitButler and JJ, which are built on a core philosophy: Workflow Over Plumbing. Modifying commits, rebasing, and resolving conflicts remain intimidating hurdles for many developers. The Git command line can be complex and unintuitive. These new tools abstract that complexity away, guiding users through commit management in a way that feels natural. The result is faster iteration, higher confidence, and a far better developer experience.

AI and the Evolving Craft of Code Review

Finally, no technical summit in 2025 would be complete without a deep dive into AI. The arrival of AI-assisted coding is fundamentally shifting the dynamic between author and reviewer.

Engineers at the summit expressed a cautious optimism. On one hand, AI is a powerful tool to accelerate reviews, improve consistency, and bolster safety. On the other, everyone is aware of the trade-offs. Carelessly used, AI-generated code can weaken knowledge sharing, blur IP boundaries, and erode a team’s deep, institutional understanding of its own codebase.

The challenge going forward is not to replace the human in the loop, but to strengthen the craft of collaborative review by integrating AI as a true co-pilot.

A Path to 100x Scale: The GHS Initiative

The most forward-looking discussions at the summit centered on how to achieve the massive scale required. One of the most promising solutions presented was GHS (Git-at-High-Speed). This innovative approach is not just an incremental improvement; it’s a strategic initiative designed to increase SCM throughput by as much as 100x.

The project’s vision is to enable platforms like Gerrit, GitLab, and GitHub Enterprise to handle the explosive repository growth and build traffic generated by modern AI workflows. By re-architecting key components for hyper-scalability, GHS represents a concrete path forward, ensuring that the industry’s most critical SCMs can meet the unprecedented demands of the AI-driven future.

The Road from Paris

The Gerrit User Summit 2025 was more than a look back at the “glorious days.” It was a statement. The Git and Gerrit communities are unified, energized, and actively building the next generation of SCM. The spirit of GItTogether 2011 is back, but this time it’s armed with 14 years of experience and a clear-eyed view of the challenges and opportunities ahead.


Antonio Barone – Gerrit Maintainer, Release Manager
Luca Milanesio – Gerrit Maintainer, Release Manager, Gerrit Engineering Steering Committee
Jacek Centkowski – Gerrit Maintainer

Gerrit Code Review future in 2025 and beyond

(TL;DR) The Gerrit Code Review project has an ambitious roadmap for 2025 and beyond. Gerrit 3.12 (H1 2025) will focus on JGit performance improvements, X.509 signed commit support, and enhanced Owners and Analytics plugins. Gerrit 3.13 (H2 2025) aims to further optimize push performance, UI usability, and plugin updates, including Kafka and Zookeeper integrations.

The k8s-Gerrit initiative will improve Gerrit’s deployment on Kubernetes, ensuring better scalability and resilience. Looking ahead, Gerrit 4.0 (2026/2027) plans to decouple the review UI from the JGit server, enabling alternative review interfaces and improved flexibility.

The February 2025 GerritMeets in Sunnyvale CA on the 19th of February 2025 will be all about roadmap and Q&A on what we would like to see coming in Gerrit. See below more details about Gerrit Code Review in 2025 and the future ahead.

Gerrit 3.12 (Target: H1 2025)

JGit Performance Improvements

The development team is prioritizing significant performance enhancements in JGit, the Java implementation of Git used by Gerrit. Key objectives include:

  • Speeding Up Conflicting Ref Names on Push: This aims to reduce delays during push operations when ref name conflicts occur.
  • Enhancing SearchForReuse Latency for Large Monorepos: The goal is to improve latency by at least an order of magnitude, facilitating more efficient operations in extensive monorepositories.
  • Improving Object Lookup Across Multiple Packfiles: Efforts are underway to accelerate object lookup times by at least tenfold, enhancing performance in repositories with numerous packfiles.
  • Parallelizing Bitmap Generation: By enabling bitmap generation across multiple cores, the team aims to expedite this process, contributing to overall performance gains.

Gerrit Core Experience Enhancements

A notable feature planned for this release is the support for X.509 signed commits, which will bolster security and authenticity in the code review process.

Owners Plugin Improvements

Enhancements to the Owners Plugin are set to provide clearer insights and streamlined interactions:

  • Action Requirements Display: Explicitly showing required actions by each owner at the file level.
  • Detailed Pending Reviews: Offering more comprehensive information on pending reviews by owners.
  • Easier Contact with File Owners: Facilitating more straightforward communication with file owners.

Analytics Plugin Optimization

The analytics plugin is slated for improvements to enhance speed and usability:

  • Repo Manifest Discovery: Native support for discovering repository manifests.
  • Faster Metrics Extraction: Accelerated extraction of metrics for branches, aiding in quicker data analysis.

Gerrit 3.13 (Target: H2 2025)

JGit Performance and Concurrency Enhancements

Building upon previous improvements, version 3.13 aims to further optimize performance:

  • Optional Connectivity and Collision Checks: Improving push performance by allowing the skipping of certain checks when appropriate.
  • Customizable Lock-Interval Retries: Providing flexibility in managing lock intervals to enhance concurrency handling.
  • Read-Only Multi-Pack Index Support: Introducing support for read-only multi-pack indexes to improve repository access efficiency.

Gerrit Core and UI Experience Enhancements

User experience remains a focal point with planned features such as:

  • File List Filtering: Allowing users to filter the file list in the change review screen for more efficient navigation.
  • Headless Gerrit Packaging: Offering a version of Gerrit that serves only read/write Git protocols, catering to users seeking a streamlined experience.

Plugin Updates

The roadmap includes updates to key plugins:

  • Kafka Events-Broker: Upgrading to support Kafka 3.9.0, enhancing event handling capabilities.
  • Zookeeper Global-Refdb: Updating to support Zookeeper 3.9.3, improving global reference database management.

Replication Plugin Enhancements

Efforts to simplify configuration and improve performance include:

  • Dynamic Endpoint Management: Introducing APIs for creating and updating replication endpoints dynamically.
  • UI Integration: Displaying replication status within the user interface for better visibility.
  • Reduced Latency on Force-Push: Improving replication latency during force-push operations by applying objects with prerequisites.

k8s-Gerrit

The k8s-Gerrit initiative focuses on deploying Gerrit within Kubernetes environments, aiming to enhance scalability, resilience, and ease of management. This approach leverages Kubernetes’ orchestration capabilities to provide automated deployment, scaling, and management of Gerrit instances, facilitating more efficient resource utilization and operational consistency.

Gerrit 4.0 (Target: 2026/2027)

Looking ahead, Gerrit 4.0 is set to introduce significant architectural changes:

  • Decoupling Gerrit Review UI and JGit Server: This separation will allow the Gerrit UI and JGit server to operate as independent services, providing greater flexibility in deployment and scaling.
  • Enabling Alternative Review UIs: By decoupling components, the platform will support the integration of other review interfaces, such as pull-request systems, offering users a broader range of tools tailored to their workflows.

The Gerrit community is encouraged to stay engaged with these developments, as the roadmap is subject to change. Contributors planning to work on features not listed are advised to inform the Engineering Steering Committee (ESC) to ensure alignment with the project’s goals.

The future of Gerrit Code Review in 2025 and beyond is truly remarkable, come and join GerritMeets on Wednesday, February 19, 2025 – 11:45 AM to 1:00 PM PDT

Luca Milanesio
Gerrit Code Review Maintainer
Gerrit Code Review Release Manager
Member of the Gerrit Code Review Engineering Steering Committee

GerritForge 2024 in review: Git, Gerrit Code Review and more

As 2024 draws to a close, it’s a great time to reflect on the milestones and achievements of the GerritForge team in innovating Git performance with the release of GHS, supporting and advancing the Gerrit Code Review project with 20 releases and over 1.5k commits. This year has been one of exciting developments and initiatives that have made Gerrit stronger, more accessible, and better equipped to meet the needs of modern software teams. Here’s what GerritForge accomplished for the Git and Gerrit Code Review ecosystem in 2024.

2024 in numbers

  • 1,500 changes merged
  • 40 repositories involved in the Gerrit and JGit projects
  • 20 releases
  • 5 conferences, 6 meetups
  • 11 contributors, 4 maintainers
  • 20k man/hours spent in open-source projects

Managing the Releases of Gerrit 3.10 and 3.11

As usual, we continue to ensure the regular releases of new Gerrit versions. This year was no exception with the releases of Gerrit 3.10 and 3.11. These releases introduced new features, performance optimisations, and bug fixes to further solidify Gerrit as the leading code review tool for large-scale projects.

Our team cooperated with the Gerrit Code Review community to ensure these releases were stable, well-documented, and aligned with community needs. From coordinating release schedules to ensuring compatibility with plugins and integrations, GerritForge played a key role in making these versions a reality.

Key Highlights

  • Gerrit 3.10 brought major improvements in user experience, including refined UI capabilities and enhanced search functionality.
  • Gerrit 3.11 introduced critical updates to multi-site support, security enhancements, and significant performance boosts.

GerritMeets success in both the USA and Europe

The GerritMeets series has become a cornerstone of community engagement, and 2024 was no exception. GerritForge organised six GerritMeets across several locations, including California, Germany, and the UK, bringing together contributors, users, and maintainers to share knowledge, discuss trends, and explore use cases.
Each session covered a diverse range of topics, including:

  • Best practices for multi-site setups.
  • Advanced plugin development.
  • Gerrit performance tuning for large repositories.
  • Innovations in CI/CD workflows with Gerrit.

These virtual meetups provided a platform for collaboration and learning, reinforcing Gerrit’s community spirit. All the recordings of the 2024 GerritMeets are available as a playlist on the GerritForge TV YouTube channel.

World-class Enterprise Gerrit-as-a-Service on Google Cloud Platform

In 2024, GerritForge expanded Gerrit’s capabilities by adding support for Gerrit-as-a-Service (GaaS) on the Google Cloud Platform (GCP). This initiative makes it easier than ever for organizations to adopt Gerrit without the operational overhead of managing infrastructure.

Benefits of Gerrit-as-a-Service on GCP

  • Scalability: Leveraging GCP’s powerful infrastructure to scale Gerrit deployments for enterprises of any size.
  • Simplicity: Reducing setup and maintenance complexity, allowing teams to focus on development and code reviews.
  • High Availability: Utilizing GCP’s advanced networking and storage capabilities for improved uptime and disaster recovery.

By enabling Gerrit on GCP, GerritForge is broadening the tool’s accessibility, particularly for teams looking for a cloud-native, fully managed solution.

Gerrit User Summit with Qualcomm

This year’s Gerrit User Summit, organized by Qualcomm in collaboration with GerritForge, was a highlight for the community. Held at the Qualcomm HQ in San Diego, CA (USA), the summit offered a chance for Gerrit enthusiasts worldwide to come together in person.
The agenda featured:

  • Keynotes by Gerrit maintainers and industry leaders.
  • Hands-on hackathon for contributors and users.
  • Insightful panels on the future of Gerrit.

The collaboration with Qualcomm not only expanded the summit’s reach but also highlighted Gerrit’s growing importance in enterprise environments.

Showcasing GHS and Gerrit Code Review worldwide

This year, GerritForge actively participated in numerous conferences to showcase GHS and expand Gerrit’s visibility, and demonstrate its unique capabilities. These events were a fantastic opportunity to uncover GHS and showcase Gerrit to hundreds of developers who had never encountered it.

At these conferences, we showcased:

  • GHS: It demonstrates its seamless integration with Git and showcases a whopping 100x performance improvement over a vanilla setup thanks to the power of AI and reinforcement learning.
  • K8s-Gerrit: Highlighting how Gerrit deployed on Kubernetes provides unparalleled flexibility, performance, and multi-site support.

Through live demos, presentations, and Q&A sessions, we highlighted GHS and Gerrit’s ability to scale, its unique review model, and its role in making software delivery pipelines up to 100x times faster.

K8s-Gerrit Hackathon with SAP

Collaboration was at the forefront of the k8s-Gerrit Hackathon, co-hosted by GerritForge and SAP. The hackathon brought together developers from both SAP and GerritForge teams to tackle the challenges of Kubernetes-based Gerrit deployments and multi-site support.

Outcomes of the Hackathon

  • Enhanced scalability for k8s-Gerrit deployments.
  • Breakthroughs in multi-site replication and disaster recovery.
  • Valuable contributions to the Gerrit codebase and documentation.

The event exemplified the power of open collaboration, pushing Gerrit further into cloud-native development.

Looking Ahead to 2025

As we celebrate the progress made in 2024, we remain focused on the road ahead. GerritForge is committed to:

  • Public offering of GHS
  • Gerrit-as-a-Service in Google Cloud
  • Gerrit Code Review v3.12 and v3.13
  • More GerritMeets and Gerrit User Summit events

Our gratitude goes out to the Git and Gerrit Code Review communities, contributors, and partners who have made this year a success. Together, we’re building a tool that empowers teams to deliver high-quality code faster and more efficiently.
Here’s to an even more impactful 2025!

Daniele Sassoli – Gerrit Code Review Community Manager
Luca Milanesio – JGit Committer / Gerrit Code Review Engineering Steering Committee / Release Manager and Maintainer

GHS Worldwide Roadshow

We’re thrilled to announce that our team will be speaking about our advancements with GerritForge AI Health Service (GHS) at several prestigious conferences in the coming months. These events provide an incredible opportunity to share our innovative AI solutions with a broader audience, engage with industry experts, and showcase how GHS is revolutionizing the way organizations maintain the health and stability of their Gerrit and Git systems.

Our journey begins at the Linux Open Source Summit in Vienna, from the 16th to the 18th of September. This summit is a cornerstone event for the open-source community, and we couldn’t be more excited to discuss how GHS leverages AI to ensure the seamless performance of Git and Gerrit systems, even in the most demanding environments.

Next, we’ll be in Berlin for Git Merge on the 19th and 20th of September. Git Merge is the go-to event for Git enthusiasts and professionals alike, and we’re eager to dive deep into the technical aspects of GHS, sharing insights on how our AI solution optimizes system performance, reduces downtime, and empowers development teams to focus on what they do best—creating great software.

In October, we’re particularly excited about the Gerrit User Summit in San Diego, on the 10th and 11th. This event is especially important to us as it brings together the Gerrit community to discuss the latest developments and best practices. We’ll be showcasing how GHS is enhancing Gerrit environments by providing intelligent and automated health monitoring and ensuring peak performance.

Following that, we’ll speak at the OCX conference in Mainz, from the 22nd to the 24th of October. OCX is known for bringing together top minds in DevOps and open-source technology, making it the perfect venue to highlight how GHS is transforming the management of code review and source control systems with intelligent, automated health monitoring and remediation.

Finally, we’re thrilled to wrap up our conference tour at KubeCon in Salt Lake City, from the 12th to the 15th of November. As one of the most anticipated events in the cloud-native ecosystem, KubeCon offers an unparalleled platform to demonstrate how GHS integrates with Kubernetes environments, ensuring that your SCM systems are always running at peak performance.

These conferences represent more than just speaking engagements for us—they are an opportunity to engage with the community, learn from our peers, and continue pushing the boundaries of what’s possible with AI in software development. We can’t wait to connect with you at these events and share how GHS can make a tangible difference in your organization’s success.

Stay tuned for more updates as we approach these dates, and be sure to catch our sessions if you’re attending any of these events!

Daniele Sassoli
GerritForge Engineering Manager
Gerrit Code Review Community Manager and Contributor

2023: New Year and opportunities for GerritForge and Gerrit Code Review

TL;DR: GerritForge has been dedicating its efforts to organising and managing the Gerrit User Summit in London back in November 2022, in conjunction with the release of Gerrit v3.7. The event has been a great success, with a significant presence on-site and record-breaking attendees on the GerritForge TV youtube channel. It has also committed to its promises to research and improve the JGit and Gerrit scalability to large mono-repos, with tens of millions of objects and refs. 2023 will see the finalisation of these efforts with an increase in development efforts and a new JGit Committer for pushing the platform to a new level of performance and scalability and a new innovating system for collecting and optimising the repository metrics automatically. Stay tuned.

Read the full story here below (9 mins read).


2022 has been a critical year for turning the Gerrit Code Review community and development back on track after the COVID-19 pandemic. At GerritForge, we’ve been working hard to make sure that the development, support, and innovation of Gerrit Code Review continue on its main objectives.

Gerrit Code Review v3.6 and v3.7

We have continued to deliver on the development and release of Gerrit Code Review and its plugins, helping the testing and releasing of versions v3.6.0 (May) and v3.7.0 (November).

Some numbers of the past 12 months’ development contributions by individual committers and companies:

  • 3,627 Changes have been merged on 76 projects related to the Gerrit Code Review platform, including JGit
  • 113 committers from 42 different organisations

A special mention to the top #10 contributors: Google (Ben Rohlfs, Edwin Kempin, Chris Pouchet, Dhruv Srivastava, Frank Borden, Milutin Kristofic), GerritForge (Luca Milanesio), Wikimedia (Paladox) and SAP (Matthias Sohn and Thomas Dräbing).

In comparison with 2021, we had 25% fewer changes merged but with more contributors coming from more companies, which is a symptom to a very healthy and thriving ecosystem of maintainers.

GerritForge has committed to resuming the face-to-face user summits, which were suspended since 2020.

The Gerrit User Summit 2022 took place in London, UK the 10-11 of November in a hybrid format, with people having the opportunity to participate either on-site or remotely on GerritForge’s YouTube TV channel.

It was a glorious success, with record-breaking attendance from all around the globe:

  • 50 people registered to attend on-site, 26 of them managed to arrive despite the London tube strike, whilst the others attended remotely
  • 235 people viewed the summit on YouTube with an average view time of 40 mins (one talk)

The summit survey had an outstanding report showing a huge acceptance and appreciation of the event:

  • 82% rated the remote video streaming as “good” or “outstanding”
  • 96% rated the quality of the summit as “good” or “outstanding.”
  • 100% would recommend the summit to a colleague, with 83% strongly recommending it

GerritHub.io SLA gets closer to five-nines.

We have been working hard to make Gerrit more stable and resilient throughout 2022, discovering and fixing many issues in the code base and on the multi-site software architecture.
In 2022, GerritHub.io had only six small hiccups for a total of 19 mins of downtime (SLA = 99.997%) over a 12-month period, a 75% reliability improvement compared to 2021.

We have run extensive RCAs on the causes of the downtime and identified two leading issues, which are explained in the details below.

The “anonymous unlimited query” hole in Gerrit
GerritHub.io has been subject to a 15 mins outage because of anonymous users being able to bring offline all the sites before the system could auto-recover.
Gerrit allows bypassing of all limits set in the ACLs for running queries by simply adding the “no-limit” parameter.
Returning an arbitrary payload without limits could allow a single user to generate a server-side workload for collecting and building a GBytes-sized JSON payload; unfortunately, that option was available to everyone, including anonymous users making any publicly faced Gerrit Code Review installation subject to deny-of-service attacks.
We have identified the issue, reported and fixed it in Gerrit with Change 333304, which has been included in Gerrit v3.3.10, v3.4.4, v3.5.1, and all v3.6.0 or later releases.

More granular monitoring and alerting
We have lowered the threshold of uptime checks on GerritHub.io to 1 minute, giving us the ability to detect and react immediately to 4 smaller hiccups. We have detected a lack of scalability for some specific higher-load projects. Those hiccups have been responsible for 2 mins of downtime over the 2nd part of 2022. Many more projects are also planning to be onboarded on GerritHub.io; hence we do need to address this project-specific capacity needs.

Scaling Gerrit Code Review and JGit beyond its limits

We have been investing a massive effort in building a test environment designed to stress Gerrit and JGit to its limits and identify all the limitations and bottlenecks that prevented us from scaling further.

Scaling the test repository
We have created over the months some test repositories that increased in every dimension:

  • Tens of millions of refs as both refs/changes and refs/heads
  • Millions of delta-chains
  • Tens of millions of Git objects
  • Packfiles of tens of Giga-bytes and packed refs of hundreds of megabytes

For generating a significant load on both client and server side, we have invested more into the aws-gerrit cloud setups and gatling-git performance loading tool.

There were some “well-known” issues and additional surprising ones.

SHA1 complexity and CPU utilization for large entities
JGit has been used SHA1 for identifying uniqueness not just for Git objects but also for other large entities. However, computing SHA1 has become increasingly CPU intensive because of the relatively recent findings about collisions on shattered.io.
We have highlighted two major potential improvements in cooperation with Matthias Sohn (SAP) on the raw SHA1 performance and its application for detecting packed-refs changes on the filesystem.

Commit priority queues
JGit has a custom implementation of priority queues which are intensively used in RevWalk, which has almost quadratic complexity. That isn’t a problem for small to medium chains of commits; however, when the number of commits reaches millions, the performance degradation becomes unbearable.
We have replaced the JGit’s custom implementation with the one provided by the Java JVM library, which has a logarithmic complexity that massively improves its performance with large commit chains.

Unwanted reachability checks
JGit needs to perform a full reachability check whenever a remote unknown client is advertising refs, which makes sense when serving a remote client. However, the cost of full reachability of millions of advertised refs can be a daunting task that may be alleviated if the remote end can be considered trusted.

Fixing JGit bitmaps
Since the introduction of Git bitmap, the whole community has learned how key they are in speeding up the counting and selection during the clone phase.
However, large and unoptimized bitmaps could be so unhelpful for Git that instead of speeding up, they could represent a massive overhead for the system, causing CPU spikes and, eventually, lowering the throughput of the server.
Git bitmaps are compressed using the JavaEWAH library, which is good for memory consumption but evil for CPU utilization: that is the reason why the smaller is best for performance.
We have discovered and fixed a critical issue with the JGit bitmap generation that was causing the inclusion of all commits and BLOBs pointed by annotated tags. Also, we have introduced the ability to inform JGit about the heads that can be excluded from the bitmap, allowing to shorten the creation tens of thousands times (5h generation time for a 2k refs to as little as 60s) and increase its effectiveness by 200%.

Millions of unneeded ref logs
When performing a clone of a repository with millions of heads, JGit created one local reflog file for every remote ref, including the ones there were not actually cloned but just fetched as remote references. This was creating a significant performance gap between JGit and Git, which would instead lazily create the reflog files once they are effectively checked out the first time. Cloning a single branch of a repository with millions of remote refs took around 1h, compared to a few minutes of Git.

All of the findings were included in multiple updates on the following components:

  • JGit changes: all fixes were also provided to stable-5.13, the last supported branch for Java 8, which allows benefiting from these improvements for older versions of Gerrit from v2.16 onwards.
  • pull-replication went through major performance improvements, achieving a 1000x times faster execution time compared to the traditional replication plugin
  • aws-gerrit is going through upgrades for making use of pull-replication plugin, including the support for the bearer token which allows to replicate virtually any repository, including All-Users.git
  • gatling-git: we have upgraded the Gatling version and JGit to the latest stable-5.13 to include the latest performance improvements.
  • git-repo-metrics: we have introduced a brand-new plugin that allows us to keep under control the major dimensions of a repository and therefore graph their increase over time.

GerritForge goals for 2023

We are definitely not done yet with the performance improvements on Gerrit and JGit: there are still significant improvements to be made, and JGit changes to get merged into the mainstream branches.
We believe we are on track to finalize the job and allow a stable and scalable platform for large Git repositories in 2023.

Finalise what we cooked in 2022 for JGit
JGit has a new maintainer, David Ostrovsky, awarded in 2022 as Git committer of the project. GerritForge’s devs are focused to get more reviews and attention to the JGit performance improvements. We are committed to finalising all the open changes related to large repositories.

JGit multi-pack indexes support
There is still a major gap between JGit and Git when dealing with very active repositories: multi-pack indexes. The proliferation of packfiles would eventually lead to a long and painful search-for-reuse phase for BLOBs which could be cut down 100s of times with a multi-pack index.

Git repository optimiser for Gerrit
We have been working on tracking the live information on the Git repository, thanks to the git-repo-metrics plugin. Wouldn’t it be nice to have a tool that can do something with it and automatically?
We would be doing R&D on how to correlate the repository metrics, the Git audit trail, and the performance data for making AI-based decisions on what needs to be improved on the repository.
This work stream is going to be useful for any Git repository, not just the ones powered by Gerrit Code Review. The ‘git-repo-metrics’ and the repository optimiser would also apply to other products, including GitHub and GitLab.

Gerrit v3.8 and projects-specific change numbers
We will finalise the design document for the transition to project-specific change numbers in Gerrit v3.8. That would allow the seamless migration of projects across Gerrit setups without having to worry about changes renumbering anymore.

Gerrit Code Review testing and GerritForge-certified binaries
GerritForge is spending a tremendous amount of time developing test environments and tools for serving the Gerrit community with more stable releases and improving the quality of its code. We want to intensify the effort and also offer our platinum support customers a unique service that includes the GerritForge digital signature and rubber stamp on the binaries of Gerrit Code Review and its plugins that have been successfully tested and validated for being production-ready.
Stay tuned; more details are coming soon …

GerritForge company forecast in 2023

GerritForge Inc. will finalise its roll-out to the USA, and all contracts and services will be run from Sunnyvale, CA and Europe. Over 2022, 60% of the customers and businesses have already been moved, and the operation will be completed over the course of 2023.

We are looking forward to doubling our revenue figures in 2023 and also our contributions to the open-source community, with a main focus on JGit as the driver of performance growth for Gerrit Code Review.


2023 is going to be an incredible year for GerritForge, Gerrit Code Review, and the JGit community altogether.

Happy New start of the Year 2023!

Luca Milanesio (GerritForge)
Gerrit Code Review Maintainer and Release Manager
Member of the Gerrit Engineering Steering Committee

New year, new JGit contributions coming

We have shared GerritForge’s goals for improving Gerrit in 2022: most of them will include significant contributions to JGit, the Java-based engine powering Gerrit Code Review’s support for Git data-format and protocol.

GerritForge will contribute many more changes to JGit during 2022, all focused on improving the functionality and performance of large mono-repos. All changes will go through the formal review through Eclipse Foundation’s JGit Gerrit Project.

Lack of knowledge and reviews

The JGit project has suffered from major losses in the past few years, which is clearly shown by the list of top-contributors vs. their recent 12 months activity. I have tried running a “git blame” against all the JGit code-base, which is a heuristic (therefore a rough approximation) of which part of the JGit code has been last written/edited.

  1. (49998 LOC) – Shawn Pearce
  2. (37854 LOC) – Thomas Wolf
  3. (31417 LOC) – Matthias Sohn
  4. (13593 LOC) – David Pursehouse
  5. (13200 LOC) – Christian Halstrick

See below the number of contributions (excluding merges and trivial changes) of the above 5 maintainers in the past 12 months:

  1. Shawn Pearce – 0 changes – He sadly passed away in 2018 
  2. Thomas Wolf – 86 changes
  3. Matthias Sohn – 128 changes
  4. David PurseHouse – 0 changes
  5. Christian Halstrick – 0 changes

The above stats show that the currently active maintainers (Thomas and Matthias) appear in the bit blame of 69k out of the total 390k LOCs.

Thomas and Matthias are doing a fantastic job in keeping up as much as possible with the incoming changes’ pace and reviewing them at their best. At times, though, the incoming change may touch parts of the code they are less familiar with and, therefore, would require more eyes or more time to review.

Breaking the vicious circle

The JGit project is in a dangerous vicious circle.

  1. Incoming changes would take longer to get reviewed and merged.
  2. The lengthy reviews cause detriment to contributors that may lose interest in following up contributions or upload new changes.
  3. The lack of contributions and merged changes would keep the pressure on current maintainers, which fuel point 1. again.

How can GerritForge help break this vicious circle, provide meaningful contributions, and get them merged fast and with proper and thorough reviews?

Keeping the pace of contributions is key to avoid detriment: GerritForge will therefore create a “dev branch” of JGit. All the GerritForge’s contributions to JGit master branch will be part of the dev branch and will go through a rigorous code-review and E2E validation cycle, including the Gatling tests for Gerrit.

Two-steps validation workflow

GerritForge’s workflow for validating JGit changes with Gerrit
  1. A new change is uploaded to the Eclipse Foundation JGit project.
  2. The normal Eclipse Foundation’s CI verification builds the change and, if passes all tests, provides a Verified +1
  3. One of the JGit maintainers, or members of the GerritForge’s contributors, can provide a Code-Review +1 score with the additional description “Approved for dev
  4. The special “Approved for dev” description triggers the cherry-pick of the Change onto the GerritForge’s JGit dev branch
  5. The new change for review on the JGit dev branch triggers the creation of a Change on gerrit-review.googlesource.com/gerrit with the update of the JGit submodule pointing to the open change.
  6. The JGit submodule update Change triggers the current E2E validation using the Gatling tests, developed and hosted by GerritForge. If all tests are passing, the Gerrit change receives a Verified +1.
  7. The cherry-picked Change on JGit dev branch receives a Verified +1
  8. The cherry-picked Change is merged to the JGit dev branch
  9. The merge of the cherry-picked Change is notified on the original Change with a Code-Review +1 score with description “Merged in dev”.
  10. One of the JGit maintainers can finalise the review and, if all is good, provides the final Code-Review +2 and merge the change on JGit master.

NOTE: The above workflow will only apply to the upcoming changes on the master branch, where we do need to innovate and implement new features at a faster pace. We have to plans to apply the workflow to any stable branches.

Plus and minuses

There are good things on the above workflow, however, there are also risks:

  • Complexity: the secondary dev branch will undergo an E2E validation process with Gerrit, which is obviously complex and it may break at times.
  • Danger of forking: if the JGit maintainers would veto the Change at step 10 the changes already merged in dev would make effectively dev and master branches diverge, which isn’t a good thing and it should be avoided as much as possible.

The augmented lifecycle, which also involves Gerrit E2E tests with Gatling, has also many advantages:

  • Additional E2E validation: incoming changes on JGit would involve an E2E validation with Gerrit against the suite of E2E Gatling tests, which is good feedback and gives more confidence in merging code also on less known parts of the JGit codebase.
  • Increased velocity: speedup validation of new incoming changes and getting them merged to the JGit dev branch, without impacting the pace and quality of reviews from the current JGit maintainers.
  • Gerrit edge release: allows to have a downloadable Gerrit change that includes the JGit dev branch, allowing canary deployments and see how Gerrit behaves with the latest and greatest of JGit code.

From a JGit project’s perspective, the flow of incoming changes will have an additional E2E validation, which is always a good thing. Additionally, it will bring more contributions and inspiration for new innovative changes on the project and attracting more and more talent.

Ready to gear-up contributions on JGit?

The workflow proposed is a starting point; however, we are committed to giving it a go and seeing how it would work in practice and if it will be enough to gear up the contributions to the JGit project.

How to enable Git v2 in Gerrit Code Review

git-2-26

(c) Shutterstock / spainter_vfx

Git protocol v2 landed in Gerrit 3.1 on the 11th of October 2019. This is the last email from David Ostrovsky concluding a thread of discussion about it:

It is done now. Git wire protocol v2 is a part of open source Gerrit and will be
shipped in upcoming Gerrit 3.1 release.

And, it is even enabled per default!

Huge thank to everyone who helped to make it a reality!

A big thanks to David and the whole community for the hard work in getting this done!

This was the 3rd attempt to get the feature in Gerrit after a couple of issues encountered along the path.

Why Git protocol v2?

The Git protocol v2 introduces a big optimization in the way client and server communicate during clones and fetches.

The big change has been the possibility of filtering server-side the refs not required by the client. In the previous version of the protocol, whenever a client was issuing a fetch, all the references were sent from the server to the client, even if the client was fetching a single ref!

In Gerrit this issue was even more evident, since, as you might know, Gerrit leverages a lot the refs for its internal functionality, even more with the introduction of NoteDb.

Whenever you are creating a Change in Gerrit you are updating/creating at least 3 refs:

  • refs/changes/NN/<change-num>/<patch-set>
  • refs/changes/NN/<change-num>/meta
  • refs/sequences/changes

In the Gerrit project itself, there are currently about 104K refs/change and 24K refs/change/*/meta. Imagine you are updating a repo which is behind just a couple of commits, you will get all those references which will take up most of your bandwidth.

Git protocol v2 will avoid this, just sending you back the references that the Git client requested.

Is it really faster?

Let’s see if it really does what is written on the tin. We have enabled Gerrit v2 at the end of 2019 on GerritHub.io, so let’s test it there. You will need a Git client from version 2.18 onwards.

> git clone "ssh://barbasa@review.gerrithub.io:29418/GerritCodeReview/gerrit"
> cd gerrit
> export GIT_TRACE_PACKET=1
> git -c protocol.version=2 fetch --no-tags origin master
19:16:34.583720 pkt-line.c:80           packet:        fetch< version 2
19:16:34.585050 pkt-line.c:80           packet:        fetch< ls-refs
19:16:34.585064 pkt-line.c:80           packet:        fetch< fetch=shallow
19:16:34.585076 pkt-line.c:80           packet:        fetch< server-option
19:16:34.585084 pkt-line.c:80           packet:        fetch< 0000
19:16:34.585094 pkt-line.c:80           packet:        fetch> command=ls-refs
19:16:34.585107 pkt-line.c:80           packet:        fetch> 0001
19:16:34.585116 pkt-line.c:80           packet:        fetch> peel
19:16:34.585124 pkt-line.c:80           packet:        fetch> symrefs
19:16:34.585133 pkt-line.c:80           packet:        fetch> ref-prefix master
19:16:34.585142 pkt-line.c:80           packet:        fetch> ref-prefix refs/master
19:16:34.585151 pkt-line.c:80           packet:        fetch> ref-prefix refs/tags/master
19:16:34.585160 pkt-line.c:80           packet:        fetch> ref-prefix refs/heads/master
19:16:34.585168 pkt-line.c:80           packet:        fetch> ref-prefix refs/remotes/master
19:16:34.585177 pkt-line.c:80           packet:        fetch> ref-prefix refs/remotes/master/HEAD
19:16:34.585186 pkt-line.c:80           packet:        fetch> 0000
19:16:35.052622 pkt-line.c:80           packet:        fetch< d21ee1980f6db7a0845e6f9732471909993a205c refs/heads/master
19:16:35.052687 pkt-line.c:80           packet:        fetch< 0000
From ssh://review.gerrithub.io:29418/GerritCodeReview/gerrit
 * branch                  master     -> FETCH_HEAD
19:16:35.175324 pkt-line.c:80           packet:        fetch> 0000

> git -c protocol.version=1 fetch --no-tags origin master
19:16:57.035135 pkt-line.c:80           packet:        fetch< d21ee1980f6db7a0845e6f9732471909993a205c HEAD\0 include-tag multi_ack_detailed multi_ack ofs-delta side-band side-band-64k thin-pack no-progress shallow agent=JGit/unknown symref=HEAD:refs/heads/master
19:16:57.037456 pkt-line.c:80           packet:        fetch< 07c8a169d6341c586a10163e895973f1bdccff92 refs/changes/00/100000/1
19:16:57.037489 pkt-line.c:80           packet:        fetch< 0014ca6443ac0af338e2677b45e538782bb7a12e refs/changes/00/100000/meta
19:16:57.037502 pkt-line.c:80           packet:        fetch< b4af8cad4d3982a0bba763a5e681d26078da5a0e refs/changes/00/100400/1
19:16:57.037513 pkt-line.c:80           packet:        fetch< 9ec6e507c493f4f1905cd090b47447e66b51b7e1 refs/changes/00/100400/meta
19:16:57.037523 pkt-line.c:80           packet:        fetch< a80359367529288eea3c283e7d542164bced1e2f refs/changes/00/100800/1
19:16:57.037533 pkt-line.c:80           packet:        fetch< 170cced6d81c25d1082d95e50b37883e113efd01 refs/changes/00/100800/meta
19:16:57.037544 pkt-line.c:80           packet:        fetch< 6cb616e0ad4b3274d4b728f8f7b641b6bd22dce4 refs/changes/00/100900/1
19:16:57.037554 pkt-line.c:80           packet:        fetch< 286d1ee1574127b76c4c1a6ef0f918ad4c61953a refs/changes/00/100900/meta
19:16:57.037606 pkt-line.c:80           packet:        fetch< 312ba566d2620b43fb90be3e7c406949edf6b6d9 refs/changes/00/10100/1
19:16:57.037619 pkt-line.c:80           packet:        fetch< dde4b73cb011178584aae4fb29a528018149d20b refs/changes/00/10100/meta

…. This will go on forever …. 

As you can see there is a massive difference in the data sent back on the wire!

How to enable it?

If you want to enable it, you just need to update you git config (etc/jgit.config in 3.1 and $HOME/.gitconfig in previous versions) with the protocol version to enable it and restart your server:

[protocol]
  version = 2

Enjoy your new blazing fast protocol!

If you are interested in more details about the Git v2 protocol you can find the specs here.

Fabio Ponciroli (GerritForge)
Gerrit Code Review Contributor

Gerrit User Summit LIVE!

Screenshot 2019-10-16 at 06.46.06

The Gerrit User Summit 2019 is going live and allows anyone to join and participate from across the world.

There are only 30 days left for the Gerrit User Summit 2019, the 12th annual event of the Gerrit Code Review community. It is the year of the records, with Gerrit reaching its largest audience ever in its 11 years of history:

  • Over 120 seats
  • People coming from 27 countries
  • 2 major dates and locations, in Sweden and in the USA
  • 20 talks and presentations
  • All seats sold out 2 months before the event

This is also a historical moment for the community because, for the first time since 2011, the JGit and Gerrit contributors will get together and talk to each other face to face, strengthening the cooperation between the two projects.

Do not miss the event, go live

We have received an enormous amount of requests to join the event on-site in Sunnyvale, much more than any previous year: the event was sold out on Eventbrite 2 months before the starting date.

GerritForge has then decided to invest further funding in sponsorship to organise a full live coverage of the event.

How to participate?

GerritForge has launched a new live event broadcasting site, https://live.gerritforge.com.

Watching the event will be FREE OF CHARGE and without adverts, thanks to the sponsorship by GerritForge. To assure the maximum quality of the video, there is a limit of on-line watchers and a pre-registration is needed.

  1. Go to https://live.gerritforge.com
  2. Click on the “Register to Watch” orange button
  3. Enter your full name, e-mail, company name and country of origin
  4. Click “Register to Watch” green button on the bottom of the page

The live event will allow remote attendants to ask questions and interact with the audience in Sunnyvale: it is going to be truly interactive and useful for the whole JGit and Gerrit community.

What to expect from the Sunnyvale event?

The Sunnyvale event includes a huge number of innovations on the JGit and Gerrit projects.

  • The introduction of the Git ref-table for repositories with huge number of refs
  • Support for Git protocol v2 in Gerrit
  • Git / Gerrit plugin for Gatling, for generating consistent end to end and load tests on Gerrit
  • Zuul support for the new Gerrit’s Checks CI integration
  • Introduction of Gerrit Code Review Analytics for the Android open-source project
  • Frictionless and zero downtime upgrades for Gerrit
  • and many more talks and presentations

Your last chance to attend, reserve your live spot now

There are brand-new ways this year to get in touch and be part of the Gerrit User Summit 2019.

Reserve your live spot today by registering at https://live.gerritforge.com and be part of this record event for the JGit and Gerrit Code Review community.

Luca Milanesio
Gerrit maintainer, release manager and ESC member

 

Gerrit v3.0 is here

GerritSprintHackathon2019.photo

Gerrit v3.0 has been released during the last Spring Hackathon at Google in Munich involving over 20+ developers for one week.

It can be downloaded from www.gerritcodereview.com/3.0.html and installed on top of any existing Gerrit v2.16/NoteDb installations. Native packages have been distributed through the standard channels and upgrading is as simple as shutting down the service, running the Rpm, Deb or Dnf upgrade command and starting again.

You can also try Gerrit v3.0 using Docker by simply running the following command:

docker run -ti -p 8080:8080 -p 29418:29418 gerritcodereview/gerrit:3.0.0

This article goes through the whole history of the Gerrit v3.0 development and highlights the differences between the previous releases.

Milestone for the Gerrit OpenSource Project

Finally, after 6 years, 18k commits and 1M lines of code written by 260+ contributors from 60+ different organizations, Gerrit v3.0 is finally out.

The event is a fundamental milestone for the project for two reasons:

  • The start of a new journey for Gerrit, without the legacy code of the old GUI based on Google Web Toolkit and without any relational database. Gerrit is now fully based on a Git repository and nothing else.
  • The definition of a clear community organization, with the foundation of a new Engineering Steering Committee and the role of Community Manager.

The new structure will drive the product forward for the years to come and will help to define a clear roadmap to bring back Gerrit at the center of the Software Development Pipeline.

Evolution vs. revolution

When a product release increments the first major number, it typically introduces a series of massive breaking changes and, unfortunately, a period of instability. Gerrit, however, is NOT a typical OpenSource product, because since the beginning it has been based on rigorous Code Review that brought stability and reliability from its initial inception back in 2008. Gerrit v3.0 was developed during the years by following a rigorous backward compatibility rule that has made Gerrit one of the most reliable and scalable Code Review systems on the planet.

For all the existing Gerrit v2.16 installations, the v3.0 will be much more similar to a rather minor upgrade and may not even require any downtime and interruption of the incoming read/write traffic, assuming that you have at least a high-availability setup. How is this possible? Magic? Basically, yes, it’s a “kind of magic” that made this happen, and it is all thanks to the new repository format for storing all the review meta-data: NoteDb.

Last but not least, all the feature that Gerrit v3.0 brings to the table, have been implemented iteratively over the last 6 years and released gradually from v2.13 onwards. Gerrit v3.0 is the “final step” of the implementation that fills the gaps left open in the past v2.16 release.

With regards to statistics of the changes from v2.16 to v3.0, it is clear that the code-base has been basically stabilized and cleaned up, as you can see from the official GerritForge Code Analytics extracted from analytics.gerrithub.io .

  • 1.5k commits from 63 contributors worldwide
  • 62k lines added and 72k lines removed
  • Google, CollabNet, and GerritForge are the top#3 organizations that invested in developing this release

In a nutshell, the Gerrit code-base has shrunk of 10k lines of code, compared to v2.16. So, instead of talking of what’s new in v3.0, we should instead describe what inside the 72k lines removed.

Removal of the GWT UI

The GWT UI, also referred to as “Old UI” has been around since the inception of the project back in 2008.

Gerrit.GWT-UI

Back in 2008, it seemed a good idea to build Gerrit UI on top of GWT, a Web Framework founded by Google two years earlier and aimed at reusing the same Java language for both backend and the Ajax front-end.

However, starting in 2012, things started to change. The interest of the overall community in GWT decreased, as clearly shown by the StackOverflow trends.

Screenshot 2019-05-18 at 23.34.42

In 2015, Andrew Bonventre from the Chromium Project, one of the major users of the Gerrit Code Review platform, apart from the Android Developers, presented the new prototype of the Gerrit Code Review UI, based on the Polymer project, with the code-name of PolyGerrit, and merged as change #72086.

commit ba698359647f565421880b0487d20df086e7f82a
Author: Andrew Bonventre <andybons@google.com>
Date: Wed Nov 4 11:14:54 2015 -0500

Add the skeleton of a new UI based on Polymer, PolyGerrit

This is the beginnings of an experimental new non-GWT web UI developed
using a modern JS web framework, http://www.polymer-project.org/. It
will coexist alongside the GWT UI until it is feature-complete.

The functionality of this change is light years from complete, with
a full laundry list of things that don't work. This change is simply
meant to get the starting work in and continue iteration afterward.

The contents of the polygerrit-ui directory started as the full tree of
https://github.com/andybons/polygerrit at 219f531, plus a few more
local changes since review started. In the future this directory will
be pruned, rearranged, and integrated with the Buck build.

Change-Id: Ifb6f5429e8031ee049225cdafa244ad1c21bf5b5

The PolyGerrit project introduced two major innovations:

  • Gerrit REST-API: for the first time the interaction of the code-review process has been formalized in stable and well-documented REST-API that can be used as “backend contract” for the design of the new GUI
  • The PolyGerrit front-end Team: for the first time, a specific experienced Team focused on user experience and UI workflow was dedicated to rethink and redesign iteratively all the components of the Gerrit Code Review interactions.

The GWT UI and PolyGerrit lived in the same “package” from v2.14 onwards for two years, with the users left with the option to switch between the two. Then in 2018 with v2.16 the PolyGerrit UI became the “default” interface and thus renamed just “Gerrit” UI.

With Gerrit v3.0, the entire GWT code-base in Gerrit has been completely removed with the epic change by David Ostrovsky “Remove GWT UI“, which deleted 33k lines of code in one single commit.

The new Polymer-based UI of Gerrit Code Review is not very different than the one seen in Gerrit v2.16, but includes more bug fixes and is 100% feature complete, including the projects administrations and ACLs configuration.

Screenshot 2019-05-18 at 22.58.13

Removal of ReviewDb

Gerrit v3.0 does not have a DBMS anymore, not even for storing its schema version as it happened in v2.16. This means that almost everything gets stored in the Git repositories.

The journey started back in October 2013, when Shawn Pearce gave to Dave Borowitz the task to convert all the review meta-data managed by Gerrit into a new format inside the Git repository, called NoteDb.

After two years of design and implementation, Dave Borowitz presented NoteDb at the Gerrit User Summit 2015 and called Gerrit v3.0 the release that will be fully working without the need of any other external DBMS (see the full description of the talk at https://storage.googleapis.com/gerrit-talks/summit/2015/NoteDB.pdf).

Google started adopting NoteDb in parallel with ReviewDb on their own internal setup and in June 2017, the old changes table was definitely removed. However, there was more in the todo-list: at the Gerrit User Summit 2017, Dave Borowitz presented the final roadmap to make ReviewDb finally disappear from everyone’s Gerrit server.

Screenshot 2019-05-18 at 23.18.28

In the initial plans, the first version with NoteDb fully working should have been v2.15. However, things went a bit differently and a new minor release was needed in 2018 to make the format really stable and reliable with v2.16.

Gerrit v2.16 is officially the last release that contains both code-bases and allows the migration from ReviewDb to NoteDb.

Dave Borowitz used the hashtag “RemoveReviewDb” to allow anyone to visualize the huge set of commits that removed 35k lines of code complexity from the Gerrit project.

Migrating to Gerrit v3.0, step-by-step

Gerrit v3.0 requires NoteDb as pre-requisite: if you are on v2.16 with NoteDb, the migration to v3.0 is straightforward and can be done with the following simple steps:

  1. Shutdown Gerrit
  2. Upgrade Gerrit war and plugins
  3. Run Gerrit init with the “batch” option
  4. Start Gerrit

If you are running Gerrit in a high-availability configuration, the above process can be executed on the two nodes individually, with a rolling restart and without interrupting the incoming traffic.

If you are running an earlier version of Gerrit and you are still on ReviewDb, then you should upgrade in three steps:

  1. Migrate from your version v2.x (x < v2.16) to v2.16 staying on ReviewDb. Make sure to upgrade through all the intermediate versions. (Example: migrate from v2.13 to v2.14, then from v2.14 to v2.15 and finally from v2.15 to v2.16)
  2. Convert v2.16 from ReviewDb to NoteDb
  3. Migrate v2.16 to v3.0

The leftover of a DBMS stored onto H2 files

Is Gerrit v3.0 completely running without any DBMS at all? Yes and no. There is some leftover that isn’t necessarily associated with the Code Review meta-data and thus did not make sense to be stored in NoteDb.

  • Persistent storage for in-memory caches.
    Some of the Gerrit caches store their status on the filesystem as H2 tables, so that Gerrit can save a lot of CPU time after a restart reusing the previous in-memory cache status.
  • Reviewed flag of changes.
    Represents the flag that enables the “bold” rendering of a change, storing the update status for every user. It is stored by default on the filesystem as H2 table, however, can be alternatively stored on a remote DBMS or potentially managed by a plugin.

New core plugins

Some of the plugins that have been initially distributed only with the Native Packages and Docker versions are now an integral part of the WAR distribution as well:

  • delete-project
    which allows removing a project from Gerrit and the associated changes.
  • gitiles
    a lightweight code-browser created by Dave Borowitz based on JGit
  • plugin-manager
    the interface to discover, download and install Gerrit plugins
  • webhooks
    the HTTP-based remote trigger to schedule remote builds on CI systems or active any other service from a Gerrit event

The above four plugins already existed before Gerrit v3.0, but they were not included in the gerrit.war.

Farewell to Dave Borowitz and the PolyGerrit Team

After having completed the feature parity between GWT and PolyGerrit, the original PolyGerrit Team members left the Gerrit Code Review project.

Their journey came to an end with the release of the new shiny Polymer-based Gerrit UI. The PolyGerrit Team contributed 45k lines of code on 5.3k commits in 4 years.

Then the last event unfolded during the release of Gerrit v3.0: Dave Borowitz announced that he was leaving the Gerrit Code Review project. I defined the event like “Linus Torvalds announcing he was abandoning the Linux Kernel project”.

Dave Borowitz contributed 316k lines of code on 3.6k commits over 36 repositories in 8 years. He helped also the development of the new Gerrit Multi-Site plugin by donating its Zookeeper-based implementation of a global ref-database.

On behalf of GerritForge and the Gerrit Code Review community, I would like to thank all the past contributors and maintainers that made PolyGerrit and NoteDb code-base into Gerrit: Dave, Logan, Kasper, Becky, Viktar, Andrew and Wyatt.

Luca Milanesio – GerritForge
Gerrit Code Review Maintainer, Release Manager
and member of the Engineering Steering Committee

Security: do not use Git v2 in Gerrit 2.16

Git protocol v2 was released as an experimental feature in Gerrit v2.16. However, the introduction came with a quite serious unnoticed bug: all refs are visible to all users,
regardless of the ACL configuration, giving to any registered users the complete access to
of all branches, tags and meta-data refs, their associated commit SHA1s and
the ability to fetch them locally … ouch!

What is the Security impact?

If you were using Gerrit v2.16 for OpenSource projects, not much: everything is visible to everyone anyway, so what’s the point?

For not-so-OpenSource projects, well, it cannot be used as-is in production for sure. It was flagged as experimental after all because it was intended more for an early adopter to “have a go with it” rather than using it at full scale in production for sensitive projects.

See the full details of the security advisory at:
https://groups.google.com/forum/#!topic/repo-discuss/z_QCw2QHbbc

How did we get there?

The problem is mainly located in the JGit implementation of the refs filtering for the Git protocol v2. Gerrit ACLs are enforced using the JGit’s AdvertiseRefsHook which calls
RefFilter, where a Gerrit-specific implementation of it processes the access permissions associated with a user.

The AdvertiseRefsHook is usually set by UploadPack.setAdvertiseRefsHook
but, if Gerrit has the protocol v2 enabled in the gerrit.config and the client
is leveraging the git protocol v2 feature, the hook is not invoked. The bug has been already fixed in JGit but, of course, before enabling the support again in v2.16 we need to be sure that no other vulnerabilities are exposed.

What if I have Gerrit v2.16 in production?

On Gerrit v2.16 and v2.16.1, the Git protocol v2 was disabled anyway by default.

If you had it enabled in production by mistake (or bravery?), then just set it as disabled explicitly.

[receive]
enableProtocolV2 = false

If you can, just upgrade to v2.16.2 whenever possible, where the Git protocol v2 is always
disabled.

What’s next?

Git protocol v2 is coming back to Gerrit v2.16 very soon, possibly as early as next week or the one after, ready for Christmas 🙂

The fix is ready, but the Gerrit and JGit teams are working hard to put more specific security testing in place so that the new reintroduction can be safer to be rolled out.

Merry Christmas to everyone … and hope that Git Protocol v2 will be back with us very soon, just in time for the end of year celebrations.

Luca Milanesio – Gerrit Code Review Maintainer.