Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow protobuf v5 as dependency #8627

Merged
merged 23 commits into from
Jun 4, 2024
Merged

Conversation

LukasMasuch
Copy link
Collaborator

@LukasMasuch LukasMasuch commented May 7, 2024

Describe your changes

This PR updates the pin in setup.py to allow protobuf 5.x as dependency. It also removes the vendored protoc dependency and installs 26.1 from the official Github release page.

Related to #8624 and #8701


Contribution License Agreement

By submitting this pull request you agree that all contributions to this project are made under the Apache 2.0 license.

@LukasMasuch LukasMasuch added the dev:upgrade-dependencies Forces updating of the latest PyPi dependencies label May 7, 2024
@LukasMasuch LukasMasuch changed the title Remove upper pin for protobuf dev requirement [WIP] Remove upper pin for protobuf dev requirement Jun 3, 2024
@LukasMasuch LukasMasuch changed the title [WIP] Remove upper pin for protobuf dev requirement Allow protobuf 5 as dependency Jun 4, 2024
@LukasMasuch LukasMasuch marked this pull request as ready for review June 4, 2024 11:10
@LukasMasuch LukasMasuch requested review from mayagbarnes and a team as code owners June 4, 2024 11:10
@@ -50,7 +50,7 @@ runs:
# Actions moves to a newer version of Ubunutu.
- name: Add vendored `protoc` to $PATH
run: |
echo "./vendor/protoc-3.20.3-linux-x86_64/bin" >> $GITHUB_PATH
echo "./vendor/protoc-26.1-linux-x86_64/bin" >> $GITHUB_PATH
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was the decision to include protoc in our repo made to make the GitHub CI more stable and don't depend on too many external factors? I think that might make sense although we have to install a lot of external dependencies, so I am curious

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the decision to vendor it in was made because the apt-get version you can install on the ubuntu version used by our CI is quite old (see the comment above). I double checked this and the CI would install a protoc < 3.20 version. I think the alternative would be to download the protoc version from a remote location as part of our CI workflow.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have recently updated our docs and included the commands used to install protoc on ubuntu, see the "On Linux (ARM) section":

curl -OL https://github.com/protocolbuffers/protobuf/releases/download/v3.20.0/protoc-3.20.0-linux-aarch_64.zip
sudo unzip -o protoc-3.20.0-linux-aarch_64.zip -d /usr/local bin/protoc
sudo unzip -o protoc-3.20.0-linux-aarch_64.zip -d /usr/local 'include/*'

# (optional) remove old version
rm /usr/bin/protoc
ln -s /usr/local/bin/protoc /usr/bin/protoc

# Print out your System's Protoc version
protoc --version

I think for non-ARM Linux, the aarch_64 would need to be replaced with x86_64, but then it should work. It might be nicer than vendor it in our own repo as updating in the future would be just a single line (if we extract the version to a variable). What do you think?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, I will try to use this to install the protoc version

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@raethlein, I updated it to install it from the GitHub release page 👍

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmhm good question, if everything works then probably not needed 🙂 I just checked real quick the official installation instructions but did not find anything, so probably a legacy artifact

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The includes where needed. Something did fail in the CI so I added it back again

@LukasMasuch LukasMasuch changed the title Allow protobuf 5 as dependency Allow protobuf v5 as dependency Jun 4, 2024
@LukasMasuch
Copy link
Collaborator Author

@raethlein ready for a second review iteration.

Copy link
Collaborator

@raethlein raethlein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@LukasMasuch LukasMasuch merged commit b8b4219 into develop Jun 4, 2024
39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants