Skip to content

NGLASS-2330 feat: Upgrade official Argo chart, runtime and Python client - #887

Closed
afonsobspinto wants to merge 6 commits into
developfrom
codex/NGLASS-2330-argo-4.1-emissary
Closed

afonsobspinto wants to merge 6 commits into
developfrom
codex/NGLASS-2330-argo-4.1-emissary

Conversation

@afonsobspinto

@afonsobspinto afonsobspinto commented Oct 28, 2026 •

Copy link
Copy Markdown
Member

Superseded by #888, using source branch feature/CH-300. The replacement preserves the same code and commit history.

Use the official ArgoProj Helm chart 2.0.14, deploying Argo Workflows v4.1.6 with Emissary, and upgrade the Python client to argo-workflows 6.7.18. Native progress updates reach Workflow status without a patched controller.

This is the alternative to #886, based on develop. That PR and downstream application repositories remain unchanged.

Changes

  • Replace the adapted chart with the unmodified official 2.0.14 release, the latest published release checked on 28 October 2026. A regression verifies every vendored file against the released chart's tree digest.
  • Keep CloudHarness integration outside that chart: the argo-server address, argo-workflows workflow service account, namespace scope, server authentication mode and legacy pod-naming default. Explicit environment overrides still take precedence.
  • Retain controller Secret get access for private-image entrypoint discovery without adding Secret list/watch. Existing CloudHarness workflow service-account bindings remain unchanged.
  • Use the official chart's pre-install/pre-upgrade CRD installer. Normal CloudHarness Helm deployment now handles CRD updates; no separate kubectl apply is needed with the default settings.
  • Keep the Python SDK upgrade and compatibility adapters for raw workflow submission, template overrides, optional fields, partial responses, pagination and existing exceptions. Public CloudHarness method signatures and wrapper attributes remain stable; generated SDK model classes change.
  • Serialize the internal pod-GC delay as a duration while callers continue supplying TTL seconds. Remove obsolete PNS configuration and --pod-workers.
  • Add chart-generation regressions and an opt-in live pipeline test covering shared storage, scripts and retries.
  • Add samples API integration tests for asynchronous shared-file workflows, synchronous completion and queue-backed sums with two input pairs. The existing Codefresh samples API stage discovers these tests automatically.

The PR also contains a separate commit correcting a pre-existing deployment CLI syntax error that blocked checks. A separate sample-task fix changes its command to python /main.py $a $b: the sum script is copied to /main.py, while the inherited image working directory is /usr/src/app. The old command failed without Argo too.

Deployment

Let existing PNS workflows finish before switching to Emissary. Generate and deploy through the normal CloudHarness flow, keeping Helm hooks enabled unless CRDs are managed externally.

Review consuming deployment values for old component image pins and removed executor settings. Update controller, server and executor together. In particular, MNP #1452's old image pins need a separate consuming-repository change before adopting this upgrade.

Self-reported progress

Self-reporting is optional. A task can publish a completed/total ratio, for example:

echo 3/10 > "$ARGO_PROGRESS_FILE"

When using this feature, the file must contain the ratio format. Argo normally reports updates about once a minute; the result appears in status.nodes[...].progress through the existing Workflow API. Reconciliation and application caching can add latency.

How to test

  1. Generate and install the CloudHarness chart in a disposable test namespace using the normal deployment flow.
  2. Submit an existing CloudHarness workflow. Verify completion, file outputs and log retrieval.
  3. For live progress, write a ratio to ARGO_PROGRESS_FILE, keep the task running for at least a minute, and inspect its Workflow status.
  4. For the repeatable checks, run:
python -m unittest discover -s applications/argo/tests -v
python -m pytest -q libraries/cloudharness-common/tests libraries/models/test
python -m pytest -q tools/deployment-cli-tools/tests

To run the samples application tests against a dedicated deployment generated from this branch, use harness-test . -i samples -a and harness-test . -i samples -e -hl. Deploy its normal authentication, events, workflows and storage dependencies first. The new tests require APP_URL and use the configured test user; they skip when no deployment URL is provided.

The live Kubernetes tests are opt-in. Setup and environment variables are in the test guide.

Validation

Tested with a disposable kind Kubernetes 1.31.9 cluster using the generated CloudHarness parent chart:

  • Fresh installation and an actual Helm upgrade from this branch's previous chart both succeeded, including the official CRD hook. A workflow submitted before the upgrade completed, and its status and logs remained readable through CloudHarness afterwards.
  • The live SDK test passed: create/list/get/delete, running progress 1/10 and 7/10 at default reporting intervals, file output, success/failure exit handlers, logs and implicit image entrypoints.
  • A CloudHarness pipeline passed with a script task writing to a shared PVC and a second task deliberately failing once before succeeding on retry.
  • All 6 chart regressions, 73 common/model tests and 166 deployment CLI tests passed.

These checks exercise the existing CloudHarness contract. They do not exhaust every downstream custom Workflow or deployment-value override. Existing application deployments were not modified for this chart test.

Full local samples deployment

Built and deployed this branch's actual sample application, workflow service, task images, accounts, PostgreSQL, Kafka and common service in a disposable kind cluster. The deployed sample reported SDK 6.7.18 and used the official Argo 4.1.6 components.

  • All 9 samples API pytest cases passed: five existing cases plus the new asynchronous, synchronous and two queue-result cases. Results were 2 + 3 = 5 and -4 + 1 = -3.
  • All 11 generated samples API operations, 4 workflows API operations and 2 common-service API operations passed.
  • Both samples browser tests, the events UI smoke test and both sample application unit tests passed.
  • Chart integrity/regression tests, all 73 common/model tests and repository formatting checks passed after adding the tests.

Local setup used single-node storage, a bounded Kafka CPU/memory limit and preloaded Docker images. Their commands were read from the built images and supplied through the official chart's emissary.images setting; production registry access was not exercised by this run. These overrides remain outside the repository. The pre-existing NFS failover/multi-node suite was excluded, and no NFS code was changed.

created.append(result.name)
return result.name

def wait(name):
@afonsobspinto afonsobspinto changed the title NGLASS-2330 feat: Upgrade to official Argo 4.1.6 and Emissary NGLASS-2330 feat: Upgrade official Argo chart, runtime and Python client Oct 28, 2026
assert deleted.status_code in (200, 204, 404), deleted.text


def wait_for_success(api, name):
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant