CH-300 feat: Upgrade official Argo chart, runtime and Python client by afonsobspinto · Pull Request #888 · MetaCell/cloud-harness

afonsobspinto · 2026-10-28T14:49:29Z

Disclaimer: Done by Codex

Closes https://metacell.atlassian.net/browse/CH-300
Closes https://metacell.atlassian.net/browse/CH-283

Changes

Replace the adapted chart with the unmodified official 2.0.14 release, the latest published release checked on 28 October 2026. A regression verifies every vendored file against the released chart's tree digest.
Keep CloudHarness integration outside that chart: the argo-server address, argo-workflows workflow service account, namespace scope, server authentication mode and legacy pod-naming default. Explicit environment overrides still take precedence.
Retain controller Secret get access for private-image entrypoint discovery without adding Secret list/watch. Existing CloudHarness workflow service-account bindings remain unchanged.
Use the official chart's pre-install/pre-upgrade CRD installer. Normal CloudHarness Helm deployment now handles CRD updates; no separate kubectl apply is needed with the default settings.
Keep the Python SDK upgrade and compatibility adapters for raw workflow submission, template overrides, optional fields, partial responses, pagination and existing exceptions. Public CloudHarness method signatures and wrapper attributes remain stable; generated SDK model classes change.
Serialize the internal pod-GC delay as a duration while callers continue supplying TTL seconds. Remove obsolete PNS configuration and --pod-workers.
Add chart-generation regressions and an opt-in live pipeline test covering shared storage, scripts and retries.
Add samples API integration tests for asynchronous shared-file workflows, synchronous completion and queue-backed sums with two input pairs. The existing Codefresh samples API stage discovers these tests automatically.

The PR also contains a separate commit correcting a pre-existing deployment CLI syntax error that blocked checks. A separate sample-task fix changes its command to python /main.py $a $b: the sum script is copied to /main.py, while the inherited image working directory is /usr/src/app. The old command failed without Argo too.

Deployment

Let existing PNS workflows finish before switching to Emissary. Generate and deploy through the normal CloudHarness flow, keeping Helm hooks enabled unless CRDs are managed externally.

Review consuming deployment values for old component image pins and removed executor settings. Update controller, server and executor together. In particular, MNP #1452's old image pins need a separate consuming-repository change before adopting this upgrade.

Self-reported progress

Self-reporting is optional. A task can publish a completed/total ratio, for example:

echo 3/10 > "$ARGO_PROGRESS_FILE"

When using this feature, the file must contain the ratio format. Argo normally reports updates about once a minute; the result appears in status.nodes[...].progress through the existing Workflow API. Reconciliation and application caching can add latency.

How to test

Generate and install the CloudHarness chart in a disposable test namespace using the normal deployment flow.
Submit an existing CloudHarness workflow. Verify completion, file outputs and log retrieval.
For live progress, write a ratio to ARGO_PROGRESS_FILE, keep the task running for at least a minute, and inspect its Workflow status.
For the repeatable checks, run:

python -m unittest discover -s applications/argo/tests -v
python -m pytest -q libraries/cloudharness-common/tests libraries/models/test
python -m pytest -q tools/deployment-cli-tools/tests

To run the samples application tests against a dedicated deployment generated from this branch, use harness-test . -i samples -a and harness-test . -i samples -e -hl. Deploy its normal authentication, events, workflows and storage dependencies first. The new tests require APP_URL and use the configured test user; they skip when no deployment URL is provided.

The live Kubernetes tests are opt-in. Setup and environment variables are in the test guide.

Validation

Tested with a disposable kind Kubernetes 1.31.9 cluster using the generated CloudHarness parent chart:

Fresh installation and an actual Helm upgrade from this branch's previous chart both succeeded, including the official CRD hook. A workflow submitted before the upgrade completed, and its status and logs remained readable through CloudHarness afterwards.
The live SDK test passed: create/list/get/delete, running progress 1/10 and 7/10 at default reporting intervals, file output, success/failure exit handlers, logs and implicit image entrypoints.
A CloudHarness pipeline passed with a script task writing to a shared PVC and a second task deliberately failing once before succeeding on retry.
All 6 chart regressions, 73 common/model tests and 166 deployment CLI tests passed.

These checks exercise the existing CloudHarness contract. They do not exhaust every downstream custom Workflow or deployment-value override. Existing application deployments were not modified for this chart test.

Full local samples deployment

Built and deployed this branch's actual sample application, workflow service, task images, accounts, PostgreSQL, Kafka and common service in a disposable kind cluster. The deployed sample reported SDK 6.7.18 and used the official Argo 4.1.6 components.

All 9 samples API pytest cases passed: five existing cases plus the new asynchronous, synchronous and two queue-result cases. Results were 2 + 3 = 5 and -4 + 1 = -3.
All 11 generated samples API operations, 4 workflows API operations and 2 common-service API operations passed.
Both samples browser tests, the events UI smoke test and both sample application unit tests passed.
Chart integrity/regression tests, all 73 common/model tests and repository formatting checks passed after adding the tests.

Local setup used single-node storage, a bounded Kafka CPU/memory limit and preloaded Docker images. Their commands were read from the built images and supplied through the official chart's emissary.images setting; production registry access was not exercised by this run. These overrides remain outside the repository. The pre-existing NFS failover/multi-node suite was excluded, and no NFS code was changed.

+            created.append(result.name)
+            return result.name
+
+        def wait(name):


+        assert deleted.status_code in (200, 204, 404), deleted.text
+
+
+def wait_for_success(api, name):


filippomc · 2026-11-06T15:32:15Z

@copilot resolve the merge conflicts in this pull request

# Conflicts: # tools/deployment-cli-tools/tests/test_helm.py Co-authored-by: filippomc <46561561+filippomc@users.noreply.github.com>

Copilot · 2026-11-06T15:37:12Z

@copilot resolve the merge conflicts in this pull request

Resolved the merge conflict and merged develop in commit 17c59a6.

+from test import typing_utils
+
+
+def _deserialize(data, klass):


@@ -0,0 +1,41 @@
+# coding: utf-8
+
+import sys


@@ -0,0 +1,28 @@
+import unittest
+
+from flask import json


@@ -0,0 +1,28 @@
+import unittest
+
+from flask import json


+import os
+from pprint import pprint
+import schemathesis as st
+from schemathesis.checks import response_schema_conformance, not_a_server_error


+import schemathesis as st
+from schemathesis.checks import response_schema_conformance, not_a_server_error
+
+from cloudharness_test import apitest_init  # include to perform default authorization


@@ -0,0 +1,28 @@
+import unittest
+
+from flask import json


@@ -0,0 +1,147 @@
+import datetime
+
+import typing


afonsobspinto added 6 commits October 28, 2026 02:15

NGLASS-2330 feat: Migrate to official Argo 4.1.6 and Emissary

773b984

NGLASS-2330 fix: Correct CLI syntax and unblock upgrade validation

38b5e9b

NGLASS-2330 feat: Upgrade Argo Python client to 6.7.18

5416d8f

NGLASS-2330 feat: Adopt official Argo Helm chart 2.0.14

73e580f

NGLASS-2330 fix: Correct sample sum task script path

d680679

NGLASS-2330 feat: Add samples workflow integration tests

23392ce

afonsobspinto mentioned this pull request Oct 28, 2026

NGLASS-2330 feat: Upgrade official Argo chart, runtime and Python client #887

Closed

github-code-quality Bot found potential problems Oct 28, 2026

View reviewed changes

Comment thread applications/argo/tests/test_emissary_integration.py

created.append(result.name)

return result.name

def wait(name):

Comment thread applications/samples/test/api/test_workflows.py

assert deleted.status_code in (200, 204, 404), deleted.text

def wait_for_success(api, name):

afonsobspinto marked this pull request as ready for review October 28, 2026 14:51

CH-300 refactor: Merge develop into Argo upgrade

a1182e5

afonsobspinto requested review from alxbrd and filippomc October 28, 2026 17:11

MetaCell deleted a comment from afonsobspinto Nov 6, 2026

filippomc changed the title ~~NGLASS-2330 feat: Upgrade official Argo chart, runtime and Python client~~ CH-300 feat: Upgrade official Argo chart, runtime and Python client Nov 6, 2026

filippomc approved these changes Nov 6, 2026

View reviewed changes

filippomc assigned Copilot Nov 6, 2026

Copilot started work on behalf of filippomc November 6, 2026 15:32

View session

Merge remote-tracking branch 'origin/develop' into feature/CH-300

17c59a6

# Conflicts: # tools/deployment-cli-tools/tests/test_helm.py Co-authored-by: filippomc <46561561+filippomc@users.noreply.github.com>

Copilot finished work on behalf of filippomc November 6, 2026 15:37

Copilot AI requested a review from filippomc November 6, 2026 15:37

Fix dependencies on Flask apps

fe6c402

github-code-quality Bot found potential problems Nov 6, 2026

View reviewed changes

Linting fix

9272369

filippomc merged commit 32b367b into develop Nov 6, 2026
8 of 11 checks passed

Navigation Menu

Repository navigation

CH-300 feat: Upgrade official Argo chart, runtime and Python client - #888

CH-300 feat: Upgrade official Argo chart, runtime and Python client#888
filippomc merged 10 commits into
developfrom
feature/CH-300

afonsobspinto commented Oct 28, 2026 •

edited

Loading

Uh oh!

filippomc commented Nov 6, 2026

Uh oh!

Copilot AI commented Nov 6, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		assert deleted.status_code in (200, 204, 404), deleted.text


		def wait_for_success(api, name):

Conversation

afonsobspinto commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Deployment

Self-reported progress

How to test

Validation

Full local samples deployment

Uh oh!

filippomc commented Nov 6, 2026

Uh oh!

Copilot AI commented Nov 6, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

afonsobspinto commented Oct 28, 2026 •

edited

Loading