[mwiebe]

Fixes: no GitHub issue filed.

Related PRs:

• fix: Test format strings in task parameter range lists openjd-specifications#207: conformance tests for this behavior, and a spec clarification it relies on.
• fix(model)!: accept format strings in INT range lists and type range list elements as list items openjd-rs#486: the same change in the Rust implementation.

What was the problem/requirement? (What/Why)

A job template can give a task parameter a list of values to run, called its range. Each value in that list can be a format string, such as "{{Param.LastFrame}}", that is filled in from a job parameter when the job is created. This package fills in every such value as text, which causes two problems.

• Values that aren't numbers get through. The 2023-09 Template Schemas say a format string in an INT, CHUNK[INT] or FLOAT range must fill in to a number. This package kept whatever text it got. An INT task parameter could end up with the value abc, and the task would then fail trying to use it. For CHUNK[INT], an uncaught ValueError was raised later, while the tasks were being listed.
• EXPR lists aren't handled. With the EXPR extension, the Expression Language says a value in a list can evaluate to null, which removes it, or to a list, which is spliced in. openjd-specifications#207 makes clear this covers range lists of every parameter type, and openjd-rs#486 implements it. This package did neither: {{ null }} became an empty value, and {{ Param.Names }} with a LIST[STRING] parameter became a single value whose text was ["b", "c"]. So the two implementations made different jobs from the same template.

What was the solution? (How)

• A new method, FormatString.resolve_list_item, resolves one value of a list. If the value is exactly one EXPR {{ ... }} expression, it is evaluated as the parameter's type: null gives no values, a list gives one value per member, and a result of the wrong type is an error. Any other value, and every value in a template without EXPR, is filled in as text, as before.
• Job creation uses it for the range field of all five task parameter types, through a new JobCreationMetadata.list_item_fields setting.
• Each resolved value is then checked for its type. An INT or CHUNK[INT] value must be an integer, and a FLOAT value must be a finite number. Errors name the value's position in the list as written, for example value 'abc' is not an integer.
• A range that ends up with no values is rejected. The 1024-value limit is checked as the list is built, so a value that expands to millions of entries is rejected right away. Before, one took 8.4 s and 1.8 GB before failing; now it takes 0.04 s.
• openjd.expr is still only imported when an EXPR expression is evaluated, so import openjd.model doesn't load the native extension.

What is the impact of this change?

This package and openjd-rs now make the same job from the same template:

• With EXPR, range values that evaluate to null are removed and lists are spliced in.
• INT, CHUNK[INT] and FLOAT values that aren't numbers are rejected when the job is created, not later when a task runs.

Templates without EXPR behave as before, except that a format string that fills in to something other than a number is now rejected.

How was this change tested?

See DEVELOPMENT.md for information on running tests.

• Have you run the unit tests? Yes. The full suite passes (6366 passed, 26 skipped, 7 xfailed), with 94.45% coverage against the 94% gate. black --check, ruff check and mypy src test pass.
• test/openjd/model_v0/test_range_list_items.py covers every parameter type, with and without EXPR, and checks the full error messages.
• test/openjd/model_v1/test_known_gaps.py gets two strict xfails. They record that the Rust-backed openjd.model._v1, which is built against openjd-model 0.13.0, still has the old behavior. They will pass once the bindings move to an openjd-model release that includes openjd-rs#486.
• With openjd-cli and openjd-sessions built from source, all 24 new conformance tests in openjd-specifications#207 pass. Across the whole suite, failures drop from 30 to 21, and mainline fails those same 21.
• The tests of openjd-cli (318 passed) and openjd-sessions-for-python pass against this change. Two sessions tests fail because psutil isn't installed, and they fail on mainline too.

Was this change documented?

• Are relevant docstrings in the code base updated? Yes, for each new and changed function, class and field.

Is this a breaking change?

Yes. Most of it only affects templates that use the EXPR extension:

• A STRING or PATH range value that evaluates to a list now gives one task per member, instead of one task whose value is the list's text.
• A range value that evaluates to null is removed, instead of becoming an empty string.
• When a whole INT or FLOAT range value is one expression, its result is converted to the parameter's type instead of being read as text. So a STRING value " 7 " used as the whole value is now an error, and a STRING "+5" in a FLOAT range becomes 5.

With or without EXPR, an INT, CHUNK[INT] or FLOAT range value that fills in to something other than a number now fails job creation.

Does this change impact security?

No files or permissions are involved. The change limits the work done for a range value that expands to a very long list.

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[github-actions]

Claude review · advisory

Reviewed 84615d9 (changes since dcf51a8): 0 new, 0 resolved.

Open: 0 blocking · 0 should-fix · 0 nit. ✅ Nothing blocking.

_{Fix or reply to each thread; the next revision's review re-checks open threads and resolves those it agrees are handled. Resolving a thread yourself also closes it. Later revisions review only what changed.}