From Luca Milanesio:
Allow GerritAccount Cookie authentication for Git/HTTP
Bug: Issue 14508
Change-Id: I2a56197ee0dad479f0973192157e5970d9deac25
Avoid multiple auth requests for Git/HTTP access
Bug: Issue 14497
Change-Id: Ibe41df0357b6be10bcdf0bd1f5a1b6160c34d4a4
Introduce LDAP metrics
Bug: Issue 14490
Change-Id: I18e5d5b797b272ca11a6745bc39dcd73cab68c34
Add unit-tests for ProjectBasicAuthFilter
Change-Id: Ib9abf133d2128b6a29751ecbeda26b0b43115bb3
From Han-Wen Nienhuys:
Documentation/user-review-ui: remove intraline diff mention
Change-Id: I15faa971d5cfe80ab82295f1af4e307da1d17f10
Documentation/user-review-ui: show just a single shortcut help screenshot
Change-Id: I2ad396a20767221f293d2ec59e6ac61a85e12629
Documentation/user-review-ui: remove mention of old UI
Change-Id: I88adb24c5883410bc62b9e290e4077e6b845474a
Documentation: rename all images for user-review.txt to gwt-*.png
Change-Id: I42735800679736ef277a053e042908677c4db5ef
Documentation: remove unused GWT image files
Change-Id: Ie470c983509310b4a6eb35eedbc03ffa5e8028cd
From Matthias Sohn:
Update jgit to a9579ba60cd2fd72179dfd8c2c37d389db5ec402
[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=552173
[2] https://bugs.eclipse.org/bugs/show_bug.cgi?id=573328
Change-Id: Id3a4adc0faf27e7bcd2017ab439aa2230cf92b33
Add new command convert-ref-storage to index
Change-Id: If3a93e65333d1a5f299273711162d81e1b653e1b
From Antoine Musso:
download_file: download to GERRIT_CACHE_HOME when set
Change-Id: Ie4fac83928527e0e71b159b9500983234c2261ac
From Prudhvi Akhil Alahari:
Fix EqualsLabelPredicate to not fail when calling match() from a plugin
Change-Id: Icd2541fe26c18a8e61ce855862e0c9814a91f5ef
From Thomas Dräbing:
Respect auth.userNameToLowerCase when creating accounts via REST or SSH
Bug: Issue 14246
Change-Id: If0f120f188e9f5bdf8008c4e66a55568180e7351
Issues Fixed
Issue 14246: Creation of internal account does not respect auth.userNameToLowerCase
Issue 14490: Missing LDAP metrics for authentication
Issue 14497: Git/HTTP traffic overloads LDAP with duplicate authentication requests
Issue 14508: Issue 14508: Gerrit authenticates more than once for a Git/HTTP high-level operation
Issues Raised
Issue 14388: Mobile diff context buttons do not work properly
From David Ostrovsky:
Tidy up dev-plugins.txt documentation
Change-Id: Ia3b09d5febcfe9ab1805bd2e55963aa6c6624f6a
Reftable: Add convert-ref-storage ssh command
Bug: Issue 14138
Change-Id: I0ddd2370619de287ec757597ba57e8b757611808
Bazel: Tidy up gerrit_js_bundle rule
Change-Id: I9b3e757717a64e8538d5d05bdb86b3b26c9f363d
Make srcs in gerrit_js_bundle optional
Change-Id: Ifaca618836098798dfbad10c2f5c2aee10f6097f
Fix link for change.enableAssignee configuration option
Change-Id: I2faef25ea62119184e1a837b647b9cccdc246354
ErrorJsonLogEntry: Use IP address as host value in logging event
Change-Id: Ie3a246ff277a0b8646dbe11acdb2b53aa659800e
Bazel: Add gerrit_js_bundle rule
Change-Id: Ia8421a9ef6f1ad91808bb698769da84022f63973
From Edwin Kempin:
Allow tests to check how much Counter0 metrics are increased by a call
Change-Id: I79da799c86325eb64922a92caec8a476e4e6a4ae
From Ben Rohlfs:
Improved and more sophisticated handling of check action results
Change-Id: I4f139bbe913c60fbdcea27dd9ea1e3f605bb19cb
Add documentation for the new Checks JavaScript Plugin API
Change-Id: I82cd3c9af9c918b1c26ea1cb8a70e3283138ac31
Add links to frontend plugin examples
Bug: Issue 14421
Change-Id: I123f1717ac691eda20709c20764db457c58b42e2
Remove draft warning from Checks API
Change-Id: I267b2792efee3baa27ebbe6915b0c7813efab566
Update all the plugin examples
Change-Id: Idc6473377bb5966f0bc199bfc68977297df4f802
Tiny documentation fix
Change-Id: I505419cdf03bfe5dd3a1dd28eeb84def36a3669a
Update the JavaScript Plugin API documentation
Bug: Issue 14421
Change-Id: I5869c8168fe27bea81226b0b8f1dc40b271c2454
From Youssef Elghareeb:
Rename SubmitRequirement to LegacySubmitRequirement
Change-Id: Ib55038627c95ece82a7eddd64e317183f1752487
From Patrick Hiesel:
SubmitRecord.Status: Reserve RULE_ERROR for user-caused errors
Change-Id: I018ce9e13fa40f2f24546df49ecf7fc3fbb72b1b
Issues Fixed
Issue 14138: expose JGit reftable conversion as Gerrit SSH command
Issue 14421: Update the Frontend Plugin Documentation
From David Ostrovsky:
Bazel: Remove assets parameter in polygerrit_plugin rule
Change-Id: I86b996a4e98c7ea8ea35bfda097ff60092078028
EventGsonProvider: Register generic event serializer adapter
Change-Id: I7dc6e6a859152ee3e30a90075ddd34814720eba1
EventJsonTest: Add test for custom event serialization
Change-Id: I7e45f77aa980c03896e3891c2d48680e31d2e0a6
Disable peer IP in reflog record per default
Bug: Issue 10810
Change-Id: Ia7a04cdce0425989059b92131afd8c2f6030d10b
Allow to suppress peer IP in reflog records
Bug: Issue 10810
Change-Id: I5645a4e68fb48ad155609573aea2518480074697
Remove enableReverseDnsLookup configuration option
Change-Id: If33c6e1901b70ca64d9b32b0ebb310f3a9daf017
Revert "Respect enableReverseDnsLookup option in ErrorLogJsonLayout"
Change-Id: I8f74ca039246753f775b7bfc96596e3f5159af01
Clean up polygerrit_plugin rule
Change-Id: Ie46a8f02a2b485a856916fe14002dc6d0e710c20
From Paladox:
Remove KnownExperimentId from gr-comment-thread
Change-Id: Ib13d07e6dc71e3133fdc249c72809082c0fa915b
Cleanup undefined check for access in getRepoAccess
Change-Id: Ieefab7e709a4124409ced7cb86ecdde37748d906
From Frank Borden:
Upgrade package.json dependencies
Change-Id: Ifb400606b6c420f732bb36e87bf43018253757ee
From Ben Rohlfs:
Add a SHOW ALL button to check result sections
Change-Id: I6be8585c285db6255e25060f61008cb57232f154
Add support for top-level links in the Checks Tab
Change-Id: Icca6e93b44b28d33607032e2a4d74327a684c2a8
Remove experiments for comment context and checks
Change-Id: I130b7846414e281062906def588151da71d72d96
Implement a detailed check run hovercard
Screenshot https://imgur.com/a/9grF5sm
Change-Id: I185264d0ce02726487facda19c116b9351fc9a7f
Fix target area for expanding/collapsing check result rows
Change-Id: Ibdd08e8732e047c62a453b02324408466817305a
Fix counting of selected and filtered check results
Change-Id: Iea4c1816dd23272678bcd5aa0f3ff549257606ed
Do not always show the checks error message
Change-Id: I61656729770322bb1795c44fd297fda4d1a0256d
Refined UI for checks error responses
https://imgur.com/a/baILcJl
Change-Id: I63f1cc5f128681f46cae3ab875744946b8537ea4
Add handling of not-logged-in and error states
Change-Id: I701a2fbdebca8d5b4adc437554b49597c8cc166a
Fix checks utility for link icons
https://imgur.com/a/ZMFEuGr
Change-Id: Ie242e578ff91def52c559a10124d82b836077b90
Change check tag/label font color from deemphasized to primary
Change-Id: Ie2de29d019d7ccab8075d212600a3da0267fc8e1
Add actions and links to check result rows
Change-Id: I2ff5a594bcc927910855d89c52484a5aef4581e0
Fix the color of the attention and status icons
Change-Id: I2a32e1f22848e122c0fd3016d8b91fbc16d029a6
Only render expanded checks row once the user has clicked the row
Change-Id: I4039953a63a84995577f309dcb2fbfee5680be86
From Milutin Kristofic:
Remove new-change-summary feature flag from gr-change-view
Change-Id: I971bc62bacc61b1985adb5de8d4fd36da68f2f46
Remove old implementation of gr-related-changes-list
Change-Id: Ic42f9af760042cd0e080a29b09c3d21d20bd161c
Move tests from old to new gr-related-changes-list
Change-Id: I631cde95d00a1e54b1820232733424b28f74bff4
Remove new-change-summary feature flag from gr-change-summary
Change-Id: Id27fb05cd42186cea5de068da791a73c7f0064cb
Remove new-change-summary feature flag from gr-change-metadata
Change-Id: Ie2116e83e258b46c546056b6a9b6867e1dadcd35
Remove new-change-summary feature flag from gr-thread-list
Change-Id: I8eaf790199c80956cf3c32e70d7e6fde133cabbd
Remove new-change-summary feature flag from gr-thread-list
Change-Id: I8eaf790199c80956cf3c32e70d7e6fde133cabbd
Remove new-change-summary feature flag from gr-editable-content
Change-Id: Ic324b6f210fc37ee5e52253dc0658c89e395c0ef
Remove new-change-summary feature flag from gr-reviewer-list
Change-Id: I6675f9e57244fd70b940d6a659e0dcd8d1533388
Remove 'shared-styles' from most frequent gr-elements
Change-Id: I7ab764397fb0bda92e7c784e5c0aac2d17c2e63d
Remove new-change-summary feature flag from gr-change-requirements
Change-Id: I77e6f174a73979b8c6e52f67953909e97964ae9e
New change summary UI - fix small A11y issues
Change-Id: I29310115583b7c5ac349923d6014dd49f66aaa1a
Always show add topic row in metadata
Change-Id: I794c028c8f9c81b43918511fac193b702c20f147
Pressing enter on focused button supress keyboard shortcuts
Change-Id: I6d28ebd022041a617be645fcd7f89ecb68d9cc54
From Dhruv Srivastava:
Only show Merged As info if change has been merged
Change-Id: I2405bf98dbb6542cb36cc0767dc79d61061bf3c8
Remove patchset level comments from patchset dropdown count
Change-Id: Ib4fc43e0c3797834b72a765ae2afce6f70871e9d
Show user first in list of reviewers in change view
Screenshot: https://imgur.com/a/FW60KC2
Issue: Bug 14366
Change-Id: I5ae9fc2e1b90f2dbbb7040d81be0e69f44ce9e3a
Clean up Add Patchset Description from File List header
Change-Id: Ia1ce16f776f81f734c0b03acce60a02e46190465
Move Merged as info from change header to change summary
Screenshot: https://imgur.com/a/MgZNe7G
Change-Id: Ie2a5715dc65a20447f296f5246962b6445836490
Show Code Review quick approve button with missing labels
Change-Id: I3f9179177d92a7b7932353da1a36c5decad599ec
(cherry picked from commit 25ca8ea4e9b9abb65ec6cc05af767a7e03b479da)
From Dmitrii Filippov:
Fixes for Typescript 4.2
Change-Id: I2387d5664c6748a1194ca52a0560d1d961ca64e4
Fix and update license generator
Bug: Issue 14175
Change-Id: Ifd38a8e3ae7f06354e00aee420f282773b64583a
From Edwin Kempin:
Export PluginPushOption for use in plugins
Bug: Issue 14362
Change-Id: Ie64c22bc4fed65950844dd4cd9a3e5328fe4eb8f
From Matthias Sohn:
Welcome administrator of freshly installed Gerrit site
Bug: Issue 12209
Change-Id: Ib26372a610ab2e9c8266a71c9b2c41cfda9a051a
From Luca Milanesio:
Avoid NPE when logging incoming HTTP request
Bug: Issue 14410
Change-Id: I4bdbe7418ccc4b223fa57b7fb49c2aa46f5d3982
Add initial stream-events acceptance tests
Bug: Issue 13799
Change-Id: I6987d413ba911039e56950425b501573b4555204
Issues Fixed
Issue 10810 : Gerrit leaks user IP address to project owners
Issue 12209: Remind users that install Gerrit newly to join the Gerrit community and register on repo-discuss
Starting from this week, GerritForge provides a weekly status update of how the Gerrit v3.4.0 release plan is progressing:
List of merged changes since the previous RC
List of issues fixed
Result of the Gatling E2E tests on AWS
We hope that having these regular updates would help you focus on what is changing during the release plan execution and do more research, learning or specific investigation on the areas that are more pertinent to your use case.
Also, because the Gerrit release candidates do not come with an associated set of release notes, the list below would help people understand the new functionalities or fixes coming through every week.
Both Gatling-Git and the AWS-Gerrit project with a complete production-ready setup are projects started by GerritForge and contributed as OpenSource to the Gerrit community.
Merged changes
From Matthias Sohn:
Log memory allocated per command in httpd_log
Change-Id: Ie0ce1382a8515e6dfb7d0d3fe10b3e64c0cf9aee
Log cpu usage per http request
Change-Id: I9e78bed5219f9baf57a2b76f0f947efff334ffe5
Log memory allocated per command in sshd_log
Change-Id: Ifc1d274bf42eb3cb9b2cf46271b6be0117aa8b18
Add metrics for monitoring Java memory pools
Change-Id: I60e5960899c0cff8c05983d299b414d7a646bb07
Log cpu usage in sshd_log
Change-Id: I1c53f64caf982c2f85195e6bda4c6d790f79a810
Encapsulate fields of SshScope.Context
Change-Id: If989630425ad40922aaf8958c4335aab0bb5c2c9
Log "-" for missing log fields in sshd_log
Change-Id: I90adc7618864f702b42029ab596c6014bd4c6cfe
From Ben Rohlfs:
Remove backend support for HTML UI plugins
Change-Id: I44cc0d15910937de7e1f9b9780a799d4b85b0673
Stop producing html version of plugins
Change-Id: I1036f06e385f2997f7bea849755729df2789acaa
From Milutin Kristofic:
A11y - show outline when focused on tab title in change view
Change-Id: Ie9456a5d886e70a77dae8f055a54a3a1a0045daf
A11y - show outline when focused on change subject in dashboard
Change-Id: I9c73e49de661a17928c6f96a290c2069503bdfb4
A11y - fix and improve label when navigating dashboard
Change-Id: Id01302aea38c783687443401290995bdd0764126
From Hermann Loose:
Allow setting image viewer max-width and max-weight externally
Change-Id: Ie151a85d8ede5cb7fa0899d9367ca0dccd887538
From Han-Wen Nienhuys:
Fix meta_diff documentation
Change-Id: I9c59a4857724cdfd59b995a6dc255f77d29b017e
From Paladox:
Update plugins/codemirror-editor and plugins/delete-project
Change-Id: Ieca7d2e36b9eefffe5c830962109e1fa62134b5c
gr-confirm-move-dialog: Fix _getProjectBranchesSuggestions
Change-Id: I8d4ead0bf3a8c30d0ecefdc190e4ba4ea7ede29d
Remove unused _handleDropdownTap
Change-Id: Ie4a8effd6d814985cafa6dea9239903f503dae33
Remove unused _handleDropdownTap
Change-Id: Ie4a8effd6d814985cafa6dea9239903f503dae33
From Dhruv Srivastava:
Clean up upload change help dialog
Change-Id: I0c72450c37326ec2c2922b74928e0b059df0043e
From Saša Živkov:
Fix binding of DELETE REST calls from plugins
Change-Id: I9b9632e8f719937e5f7c61466996be79e6f29c14
Gerrit Code Review is unstoppable: despite the recent COVID-19 pandemic and the cancellation of the Spring Hackathon 2020, the community has made an extraordinary effort to deliver remotely and on-time the Gerrit v3.2 release on the 1st of June.
GerritForge has already migrated GerritHub.io on the day-1 of the release and is happy to share with you the highlights of this new release. If you need help to assess your current setup and migrating, please get in touch with us at https://gerritforge.com/contact.
Get ready to migrate: get rid of zombie comments
The migration process performs the cleanup of the zombie draft comments in the All-Users.git repository that has been left behind since the introduction of NoteDb back in v2.16.
Every user commenting on any change was creating a series of commits on the All-Users.git repository, where the draft comments are stored. Once the comments were finalised and applied to the change, they were not fully removed from the All-Users.git. That created a backlog of zombie comments on All-Users.git that are now being completely removed during the Gerrit v3.2 migration process.
Since Gerrit v2.16.16, there is a standalone utility to remove the zombie draft comments. You may want to do that operation upfront to make sure that the migration to v3.2 does not have a lot of processing during the init step. Also, make sure that the All-Users.git resides on a fast access local filesystem for minimizing the migration time.
If you do nothing, the cleanup utility will be automatically executed when migrating to Gerrit v3.2, bearing in mind that it may take quite a long time to complete. In our tests, it took around 10 minutes for 10k zombie comments.
WARNING: the execution time is not linear and it may take up to 48h of processing time for a staggering number of 1M zombie comments.
Migrate with zero-downtime
If you have on Gerrit v3.1.x in a high-availability configuration, you can upgrade seamlessly to Gerrit v3.2, without having to suspend or degrading the service in any way. GerritForge has a record number of installations done in high-availability and multi-site: if you are running a single Gerrit master today, you should get in touch with the GerritForge Team to help moving to high-availability.
For the very first time, the whole Gerrit Community can benefit from the ability to perform a rolling upgrade without any downtime.
The zero-downtime upgrade consists of the following steps:
Have Gerrit masters upgraded to v3.1.6 (or later) in a high-availability configuration, healthy and able to handle the incoming traffic properly.
Set gerrit.experimentalRollingUpgrade to true in gerrit.config on both Gerrit masters.
Set the first Gerrit master unhealthy.
Shutdown the first Gerrit master and then upgrade to v3.2.
Startup the first Gerrit master and wait for the on-line reindex to complete.
Please verify that the first Gerrit master is working correctly and then make it healthy again.
Wait for the first Gerrit master to start serving traffic regularly.
Repeat steps 3. to 7. for the second Gerrit master.
Remove gerrit.experimentalRollingUpgrade from gerrit.config on both Gerrit masters.
NOTE: Gerrit v3.1.6 has not been released yet. However, if you want to perform a rolling upgrade today, you can download the latest build on the stable-3.1 branch from the GerritForge’s CI at https://gerrit-ci.gerritforge.com/job/Gerrit-bazel-stable-3.1/
GerritHub.io has been successfully upgraded on the 1st of June without any interruption of any kind using the above procedure.
Java 11 official support
Gerrit is now officially supported on Java 11, in addition to Java 8. Running on Java 11 was already possible from v2.16.13, v3.0.4 and v3.1.0, but not officially supported because of the lack of a CI validation on Java 11 for stable-2.16, stable-3.0 and stable-3.1 branches.
Gerrit v3.2 has been validated with Java 11, with the following known issues:
Issue 12639: WARNING: An illegal reflective access operation has occurred, when starting Gerrit.
After 24h of adoption of Gerrit v3.2 on GerritHub.io, we have seen two major benefits from the migration to Java 11: overall reduction of the “old generation” build up in the JVM heap and massive reduction of GC cycles times and full-GCs.
Before the 29th of May, all GerritHub.io nodes were on Gerrit v3.1 / Java8. The old-generation JVM heap keeps on building up constantly until it reaches the 60GB and triggers a full GC cycle. After the upgrade to Gerrit v3.2 / Java11, memory consumption is very much under control. There are still possibilities of peaks with associated full GCs (see the one on the 30th of May around 12:00 BST) but there isn’t build up of old-generation objects anymore.
Java11 brings a lot of benefits also in reducing the latency of the individual GC cycles, showing much better performance with large heaps.
After the migration on the 29th of May, the GC graph is pretty much flat. The only full GC peak that is noticeable on the 30th of May lasted for just 5 msecs while the normal GC cycles are well below 1 msec, barely noticeable.
Performance is a feature
Shawn Pearce, the Gerrit Code Review project founder, used to say “performance is a feature”, which is very true. Any software nowadays can provide some basic out of the box features, thanks to the plethora of open-source components available out of the box. However, designing architecture and making it scale and perform to the levels that an Enterprise Code Review system needs, it is not easy.
Gerrit v3.2 is yet another significant milestone in the continued effort of the Gerrit maintainers and contributors in making Gerrit Code Review faster, more stable and available than ever before.
Performance tuning isn’t a “one-off task” but is a continuous improvement on thousands of little details ranging from the front-end javascript tuning down to the backend of the platform.
New accounts cache
From the data collected on googlesource.com Patrick Hiesel (Google) has identified the accounts loading from NoteDb as a significant cause of the delay of backend calls. That is true for all Gerrit installations, but especially for distributed setups or setups that restart often.
Gerrit v3.2 introduces a brand-new AccountCache decomposed into smaller chunks that can be cached individually:
External IDs + user name (cached in ExternalIdCache)
CachedAccountDetails (newly cached)
Gerrit’s default settings CachedAccountDetails – a new class representing all information stored under the user’s ref (refs/users/<sharded-id>).
The new structure is cleverly designed to require a lot less I/O when an entry needs to be reloaded and lowering the ratio of cache-miss in case of user’s details updates.
The new structure has the following advantages:
CachedAccountDetails contains only details from refs/users/<sharded-id>. By that, we can use the SHA1 of that ref as cache key and start serializing the cache to eliminate cold start penalty as well as router assignment change penalty (for distributed setups). It also means that we don’t have to do invalidation ourselves anymore.
When the server’s default preferences change, we don’t have to invalidate all accounts anymore.
The projected speed improvements that come from persisting the cache makes it so that we can remove the logic to load accounts in parallel.
Migration to Polymer 3
PolyGerrit UX roadmap continues with yet another important milestone: the migration to Polymer 3. The result is visible with an improved polishing of the GUI and significant speedup of rendering and reduction of page loading times.
There are a significant amount of small refinements to the GUI as well, coming from a meticulous work of fixes included in this release.
Not by surprise, the number of issues fixed in v3.2 on the PolyGerrit UX outnumbers by far the overall changes in the release notes.
PolyGerrit is giving special attention to the classification of the feedback coming from robots rather than humans.
Most of the efforts made in the past 12 months target the improvement the support for robot-comments and giving some extra dedicated space for them.
In Gerrit v3.2 there is a special place for them in a brand-new “Findings” tab. It is currently empty on GerritHub.io as people did not start using them much. However, I do see a lot of space of adoption of this new feature, giving the ability for more integration of linters and automatic validation feedback in this tab.
A flooding of fixes and small improvements
The list of fixes and improvements in Gerrit v3.2 is really huge. Please check the release notes on the Gerrit Code Review release page for all the details.
There are a lot of reasons to migrate to Gerrit v3.2, the fastest, more stable and scalable release of Gerrit Code Review ever.
Thanks a lot to the whole Gerrit Code Review Community of maintainers and contributors for making this release happen. Thanks to Patrick Hiesel for the technical description of the account cache improvements and the replication clustering.
Git protocol v2 landed in Gerrit 3.1 on the 11th of October 2019. This is the last email from David Ostrovsky concluding a thread of discussion about it:
It is done now. Git wire protocol v2 is a part of open source Gerrit and will beshipped in upcoming Gerrit 3.1 release.
And, it is even enabled per default!
Huge thank to everyone who helped to make it a reality!
A big thanks to David and the whole community for the hard work in getting this done!
This was the 3rd attempt to get the feature in Gerrit after a couple of issues encountered along the path.
Why Git protocol v2?
The Git protocol v2 introduces a big optimization in the way client and server communicate during clones and fetches.
The big change has been the possibility of filtering server-side the refs not required by the client. In the previous version of the protocol, whenever a client was issuing a fetch, all the references were sent from the server to the client, even if the client was fetching a single ref!
In Gerrit this issue was even more evident, since, as you might know, Gerrit leverages a lot the refs for its internal functionality, even more with the introduction of NoteDb.
Whenever you are creating a Change in Gerrit you are updating/creating at least 3 refs:
refs/changes/NN/<change-num>/<patch-set>
refs/changes/NN/<change-num>/meta
refs/sequences/changes
In the Gerrit project itself, there are currently about 104K refs/change and 24K refs/change/*/meta. Imagine you are updating a repo which is behind just a couple of commits, you will get all those references which will take up most of your bandwidth.
Git protocol v2 will avoid this, just sending you back the references that the Git client requested.
Is it really faster?
Let’s see if it really does what is written on the tin. We have enabled Gerrit v2 at the end of 2019 on GerritHub.io, so let’s test it there. You will need a Git client from version 2.18 onwards.
As you can see there is a massive difference in the data sent back on the wire!
How to enable it?
If you want to enable it, you just need to update you git config (etc/jgit.config in 3.1 and $HOME/.gitconfig in previous versions) with the protocol version to enable it and restart your server:
[protocol] version = 2
Enjoy your new blazing fast protocol!
If you are interested in more details about the Git v2 protocol you can find the specs here.
We have already discussed in previous posts how important it is to speedup the feedback loop in your Software Development Lifecycle. Having early feedbacks gives you the chance of evaluating your hypothesis and eventually change direction if needed.
The more information you have, the smarter can be your decisions.
We recently added in our Gerrit DevOps Analytics the possibility of extracting data coming from Code Reviews’ metadata to extend the knowledge we can get out of Gerrit.
Furthermore, it is possible to extract meta-data from repositories not necessarily hosted on the Gerrit instance running the analytics processing. This is a big improvement since it allows to fully analyse repositories coming from any Gerrit server.
One important type of meta-data contained in the Code Reviews is the hashtag.
Hashtags are freeform strings associated with a change, like on social media platforms. In Gerrit, you explicitly associate hashtags with changes using a dedicated area of the UI; they are not parsed from commit messages or comments.
Similar to topics, hashtags can be used to group related changes together and to search using the hashtag: operator. Unlike topics, a change can have multiple hashtags, and they are only used for informational grouping; changes with the same hashtags are not necessarily submitted together.
You can use them, for example, to mark and easily search all the changes blocking a particular release:
Hashtags can also be used to aggregate all the changes people have been working on during a particular event, for example, the Gerrit User Summit 2019 hackathon:
The latest version of the Gerrit Analytics plugin exposes the hashtags attached to their respecting Git commit data. Let’s explore together some use cases:
The most popular Gerrit Code Review hashtags over the last 12 months
Throughput of changes created during an event
see for example the Palo alto hackathon (#palo-alto-2018). We can see at the end of the week the spike of changes to release Gerrit 2.16.
The extend of time for a feature
Removing GWT was an extensive effort which started in 2017 and ended in 2019. It took several hackathons to tackle the removal as shown by the hashtags distribution. Some changes were started in one hackathon and finalised in the next one.
Those were some example of useful information on how to leverage the power of GDA.
The above examples are taken from the GDA dashboardprovided and hosted by GerritForge on https://analytics.gerrithub.io which mirror commits and reviews on a regular basis from the Gerrit project and its plugin ecosystem.
How to setup GDA on Gerrit
Hashtag extraction is currently available from Gerrit 3.1 onwards. You can download the latest version released from the Gerrit CI.
To enable hashtag extraction you need to enable the feature extraction in the plugin config file as follow:
For more information on how to configure and run the plugin, look at the analytics plugin documentation.
Conclusion
Data is the goldmine of your company. You need more and more of it for making smarter decision. The latest version of the GDA allows you to leverage even more data produced during the code review process.
You can explore the potential of the information held in Gerrit on the analytics dashboard provided by GerritForge on analytics.gerrithub.io.
The 2019 has been an exceptional year, with the introduction of the next generation of Gerrit Code Review v3 releases and the largest ever Gerrit User Summit in the whole history of 11 years of the project.
As a community we want to improve even further and make the project and the community even better. Collecting metrics has been key for the improvements of the Gerrit product and its performance and, similarly, collecting feedback from the community events is the key to grow and increase the participation and sharing of the experiences about Gerrit Code Review.
Survey results
We have run a survey directed to all of those who have attended the two Gerrit User Summits this year, in Gothenburg and Sunnyvale. See below the executive summary of the results.
Did you achieve your objectives at the Summit?
All of the attendees achieved their objective, which were different for the people, depending on their position and role in the community.
Getting the latest news of what’s happening in the Gerrit community and open-source product
Meeting the existing members of the community and welcome new contributors
Networking with the other Gerrit admin and users around the world
Influencing with ideas the future Gerrit roadmap
Overall, how would you rate the event?
Over 76% of the people rated the event very good or excellent. However, as we strive for improvement, there is a substantial 24% of of people that are looking for a better event next year.
What did you like/dislike?
The positives of the event have been:
Presentation of the Gerrit roadmap and associated discussions
Successful mix of topics, including Zuul and JGit
People, atmosphere, friendship and networking
High quality of the talks and content
The not so positive sides where:
The summit covering the weekend
Too focused on Gerrit contributors and admins, no space for users
There was too much people for the chosen location
The talks and discussions went over the planned schedule
How organized was the event?
89% of the people considered the event very well organized, whilst 11% are looking for improvement, possibly with a bigger venue and better timing.
What topics would you like to see covered next year?
Evolution of the User-Interface, roundtable with developers, user-journeys
Migration talks and discussions
CI/CD integration
Monitoring
Load testing
GitHub integration and pull-requests
Gerrit with large clusters
User-stories on using Gerrit
Would you like to have a workshop next year?
The vast majority of people would like the next year event to be more informative, including a workshop for learning some of the features of Gerrit Code Review.
What would be the best time for the Summit next year?
For the majority of people (75%) the best time for next year event would be two days during the week, rather than having it again over the weekend.
Thanks everyone again for attending the Gerrit User Summit 2019 in Gothenburg and Sunnyvale, and thanks to GerritForge, Volvo Cars and Google for sponsoring it. We are looking forward to seeing you next year.
As a Gerrit administrator, making sure there is no performance impact while upgrading from one version to another can be difficult.
It is essential to:
have a smooth and maintainable way to reproduce traffic profiles to stress your server
easily interpret the results of your tests
Tools like wrk and ab are simple and good to run simple benchmarking tests, but when it comes to more complex scenarios and collection of client-side metrics, they are not the best tools to use.
Furthermore, they only support Close Workload Models, which might not always fit the behaviour of your system.
For those reasons in GerritForge we started to look at more sophisticated tools, and we started adopting Gatling, an open-source load testing framework.
The tool
The Gatling homepage describes it this way:
“Gatling is a highly capable load testing tool. It is designed for ease of use, maintainability and high performance…
Out of the box, Gatling comes with excellent support of the HTTP protocol…..
As the core engine is actually protocol-agnostic, it is perfectly possible to implement support for other protocols…
Based on an expressive DSL, the scenarios are self-explanatory. They are easy to maintain and can be kept in a version control system…” In this article, we focus on the maintainability and protocol agnosticism of the tool.
What about Git?
Gatling natively supports HTTP protocol, but since the core engine is protocol-agnostic, it was easy to write an extension to implement the Git protocol. I started working on the Gatling git extension in August during a Gerrit hackathon in Sweden, and I am happy to see that is starting to get traction in the community.
This way we ended up among the official Gatling extension on the official Gatling homepage:
The code of the git extension if opensource and free to use. It can be found here, and the library can be downloaded from Maven central. In case you want to raise a bug, you can do it here.
Maintainability
Gatling is written in Scala, and it expects the load tests scenarios to be written in Scala. Don’t be scared; there is no need to learn crazy functional programming paradigms, the Gatling DSL does a good job in abstracting the underneath framework. To write a scenario you just have to learn the building blocks made available by the DSL.
Here a couple of snippets extracted from a scenario to understand the DSL is:
class ReplayRecordsFromFeederScenario extends Simulation {
// Boireplate to select the protocol and import the configuration
val gitProtocol = GitProtocol()
implicit val conf = GatlingGitConfiguration()
// Feeder definition: the data used for the scenario will be loaded from "data/requests.json"
val feeder = jsonFile("data/requests.json").circular
// Scenario definition:
val replayCallsScenario: ScenarioBuilder =
scenario("Git commands") // What's the scenario's name?
.forever { // How many time do I need to run though the feed?
feed(feeder) // Where shall I get my data?
.exec(new GitRequestBuilder(GitRequestSession("${cmd}", "${url}"))) // Build a Git request
}
setUp(
replayCallsScenario.inject(
// Traffic shape definition....pretty self explanatory
nothingFor(4 seconds),
atOnceUsers(10),
rampUsers(10) during (5 seconds),
constantUsersPerSec(20) during (15 seconds),
constantUsersPerSec(20) during (15 seconds) randomized
))
.protocols(gitProtocol) // Which protocol should I use?
.maxDuration(60 seconds) // How long should I run the scenario for?
}
Here another example reproducing the creation of a WIP change using the REST API:
class WIPWorkflow extends Simulation {
// Configuration bolierplate
implicit val conf: GatlingGitConfiguration = GatlingGitConfiguration()
val baseUrl = "https://review.gerrithub.io"
val username: String = conf.httpConfiguration.userName
val password: String = conf.httpConfiguration.password
val httpProtocol: HttpProtocolBuilder = http
.baseUrl(baseUrl)
.userAgentHeader("Gatling test")
val request_headers: Map[String, String] = Map(
"Content-Type" -> "application/json"
)
val scn = scenario("WIP Workflow") // What's the name of my scenario?
.exec(
http("Create WIP change")
.post("/a/changes/") // Which url and which HTTP verb should I use?
.headers(request_headers)
.basicAuth(username, password) // How do I authenticate?
.body( // What's the body of my request?
StringBody("""{
"project" : "GerritForge/sandbox/e2e-tests",
"subject" : "Let's test this Gerrit! Create WIP changes!",
"branch" : "master",
"work_in_progress": "true",
"status" : "NEW"
}""")
)
.check(status.is(201)) // What's the response code I expect?
)
setUp(scn.inject(
atOnceUsers(1) // Traffic profile
)
).protocols(httpProtocol)
}
Jenkins integration
Running load tests can be tedious and time-consuming, but yet essential to spot any possible performance regression in your application.
Providing the least possible friction is essential to incentivize people in running them. If you are already using Jenkins in your company, you can leverage the Gatling plugin to scale your load quickly and provide easy access to metrics.
A real use case: Gerrit v3.0 Vs Gerrit v3.1 load test…in production!
Let’s go through a real case scenario to show how useful and easy to read are the metrics provided by Gatling.
The closest environment to your production one is…production!
gerrithub.io runs Gerrit in a multi-site configuration, and this gives us the luxury of doing canary releases, only upgrading a subset of the master nodes running Gerrit. One of the significant advantages is that we can run A/B tests in production.
That allows us also to run meaningful load tests against the production environment. See below a simplified picture of our Gerrit setup where it is possible to see the canary server with a higher Gerrit version.
We ran against 2 servers in Germany the same load tests which:
Create 100 chained Change Sets via REST API
Submit all the changes together via REST API
We then compared the server-side and client-side metrics to see if a good job has been done with the latest Gerrit version.
Server-side metrics
Server-side metrics come from the Prometheus exporter plugin. The image is showing the HTTP requests mean time:
We can see the improvement in the latest Gerrit version. The mean requests time is almost halved and, of course, the overall duration is decreased.
Client-side metrics
Let’s see what is going on on the client-side using the metrics provided by Gatling.
Among all the metrics we are going to focus on one step of the test, since we have more data points about it, the creation of the change:
We can already see from the overall report the reduction of the response time in the latest Gerrit version:
If we look in-depth to all the response times, we can see that the distribution of the response times is pretty much the same, but the scale is different….again we confirmed the result we previously encountered.
What can we say…Good job Gerrit community!
Wrapping up
(You can see my presentation about Gatling in the last Gerrit User summit in Sunnyvale here <- add this when the talk will be sharable)
I have touched superficially several topics in this blog posts:
simplicity and maintainability provided by the Gatling DSL
Integration with Jenkins
Gatling extensions and reuse of the statistic engine
Example scenarios
I would like to write more in-depth about all these topics in some follow-up blog posts. Feel free to vote for the topic you are more interested in or suggest new ones in the comments section of this post.
If you need any help in setting up your scenario or understand how to run load tests against your Gerrit installation effectively, GerritForge can help you.
The Gerrit User Summit 2019 is going live and allows anyone to join and participate from across the world.
There are only 30 days left for the Gerrit User Summit 2019, the 12th annual event of the Gerrit Code Review community. It is the year of the records, with Gerrit reaching its largest audience ever in its 11 years of history:
Over 120 seats
People coming from 27 countries
2 major dates and locations, in Sweden and in the USA
20 talks and presentations
All seats sold out 2 months before the event
This is also a historical moment for the community because, for the first time since 2011, the JGit and Gerrit contributors will get together and talk to each other face to face, strengthening the cooperation between the two projects.
Do not miss the event, go live
We have received an enormous amount of requests to join the event on-site in Sunnyvale, much more than any previous year: the event was sold out on Eventbrite 2 months before the starting date.
GerritForge has then decided to invest further funding in sponsorship to organise a full live coverage of the event.
Watching the event will be FREE OF CHARGE and without adverts, thanks to the sponsorship by GerritForge. To assure the maximum quality of the video, there is a limit of on-line watchers and a pre-registration is needed.
Enter your full name, e-mail, company name and country of origin
Click “Register to Watch” green button on the bottom of the page
The live event will allow remote attendants to ask questions and interact with the audience in Sunnyvale: it is going to be truly interactive and useful for the whole JGit and Gerrit community.
What to expect from the Sunnyvale event?
The Sunnyvale event includes a huge number of innovations on the JGit and Gerrit projects.
The introduction of the Git ref-table for repositories with huge number of refs
Support for Git protocol v2 in Gerrit
Git / Gerrit plugin for Gatling, for generating consistent end to end and load tests on Gerrit
Zuul support for the new Gerrit’s Checks CI integration
Introduction of Gerrit Code Review Analytics for the Android open-source project
Frictionless and zero downtime upgrades for Gerrit
We have already discussed in 
GerritForge Blog 