Skip to content

fix(dataflow): update pipeline_with_dependencies deps for dependabot and fix sample bugs - #14648

Draft
XrossFox wants to merge 1 commit into
GoogleCloudPlatform:mainfrom
XrossFox:dependabot-alerts-pymongo
Draft

XrossFox wants to merge 1 commit into
GoogleCloudPlatform:mainfrom
XrossFox:dependabot-alerts-pymongo

Conversation

@XrossFox

@XrossFox XrossFox commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Description

  • Upgrade dependencies and Python runtime (3.13 / Apache Beam 2.77.0) to resolve Dependabot alerts.
  • Fix sample build and test issues discovered during verification (update Dockerfile/packaging, add .gcloudignore, and normalize bucket URI in conftest.py).
image

Fixes b/573677639

Checklist

Testing

  • I have tested this change on a live environment and verified it works as intended.

Compliance & Style


Post-Approval Actions

  • Please merge this PR for me once it is approved

…and fix sample bugs

- Upgrade dependencies and Python runtime (3.13 / Apache Beam 2.77.0) to resolve Dependabot alerts.
- Fix sample build and test issues discovered during verification (update Dockerfile/packaging, add .gcloudignore, and normalize bucket URI in conftest.py).
@product-auto-label product-auto-label Bot added api: dataflow Issues related to the Dataflow API. samples Issues that are directly related to samples. labels Oct 15, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the Python version from 3.14 to 3.13 and upgrades the Apache Beam SDK to version 2.77.0 across the pipeline_with_dependencies flex template files, including the Dockerfile, pyproject.toml, and requirements files. It also introduces a .gcloudignore file and modifies conftest.py to strip the 'gs://' prefix from the GOOGLE_CLOUD_BUCKET environment variable. Regarding the feedback, it is recommended to use .removeprefix('gs://').rstrip('/') in conftest.py to make the bucket name parsing cleaner and more robust against trailing slashes.

Comment thread dataflow/conftest.py
Comment on lines +337 to +338
if bucket_name.startswith("gs://"):
bucket_name = bucket_name.replace("gs://", "", 1)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using replace("gs://", "", 1) to strip the prefix is less robust and can be simplified. Since the project is using Python 3.13, you can use removeprefix("gs://") which is cleaner and more idiomatic. Additionally, if the environment variable contains a trailing slash (e.g., gs://my-bucket/), it can result in double slashes when constructing GCS paths (e.g., gs://my-bucket//template.json). It is safer to also strip any trailing slashes using .rstrip("/").

            bucket_name = bucket_name.removeprefix("gs://").rstrip("/")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: dataflow Issues related to the Dataflow API. samples Issues that are directly related to samples.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant