Skip to content
This repository was archived by the owner on Jul 22, 2026. It is now read-only.

Update zero-allocation-hashing and xx hash method name - #959

Closed
shivamgupta1 wants to merge 1 commit into
masterfrom
sgupta8/xx
Closed

Update zero-allocation-hashing and xx hash method name#959
shivamgupta1 wants to merge 1 commit into
masterfrom
sgupta8/xx

Conversation

@shivamgupta1

@shivamgupta1 shivamgupta1 commented Dec 20, 2023

Copy link
Copy Markdown
Contributor

Summary

The zero-allocation-hashing library versions > 0.7 are missing the method LongHashFunction.xx_r39 required by pegasus, thus causing failures. However, it appears that the newer versions are being increasingly used by OSS libraries and thus with LI too.

It looks like they just renamed xx_r39 (xxHash algo release 39) to xx to stop tying it to a particular release: OpenHFT/Zero-Allocation-Hashing@658079a

Thus, it should be safe for us to move over to xx.

@zackthehuman

Copy link
Copy Markdown
Contributor

I think this is going to be a breaking change, or at least require a lot of spot fixing when it re-enters the internal LinkedIn ecosystem. If you check internal code search, we use LongHashFunction.xx_r39 all over the place. I think this was a mistake on behalf of the library owner to remove LongHashFunction.xx_r39 from the public API on a minor release. Since this is a minor version bump for pegasus, it's going to result in broken builds for any code base that pulls in pegasus with this change. We are not currently planning on releasing the next major version of pegasus, so a change like this is worrisome. Can you help em understand what the rollout plan is? (We can discuss internally). cc @mchen07

Side note: I don't think we should link to our internal systems (JIRA/Slack/etc.) from OSS PRs.

@bohhyang

Copy link
Copy Markdown
Contributor

can we roll it out with a minor version bump, like 29.49.0?

@zackthehuman

Copy link
Copy Markdown
Contributor

can we roll it out with a minor version bump, like 29.49.0?

Not without introducing runtime breaking changes and a lot of toil to clean up. Instead, we should try to migrate our internal usage of LongHashFunction.xx_r39 to LongHashFunction.xx. We are trying to upstream as patch to make that possible: OpenHFT/Zero-Allocation-Hashing#84. If we can leverage that patch then we will have a smooth path forward that doesn't involve breaking pegasus users' runtimes.

Comment thread build.gradle
'testng': 'org.testng:testng:6.13.1',
'velocity': 'org.apache.velocity:velocity-engine-core:2.2',
'zero_allocation_hashing': 'net.openhft:zero-allocation-hashing:0.7',
'zero_allocation_hashing': 'net.openhft:zero-allocation-hashing:0.16',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can use the custom fork now, right?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not yet; the custom fork is only released internally to LinkedIn. We need to update internal users to use our fork, and then change their code to use xx() instead of xx_r39(). After that, we are free to use net.openhft:zero-allocation-hashing:0.16 in Pegasus. We won't need to fork anymore at that point.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it. Thanks!

@ChrisCarini

Copy link
Copy Markdown
Member

The linkedin/rest.li repository is being archived - see #1178 for details.

Ahead of it's archiving, we are closing all issues and PRs.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants