Repository navigation
test: run test modules headless unless they need a workbench - #1622
Draft
joaodinissf wants to merge 6 commits into
Draft
joaodinissf wants to merge 6 commits into
joaodinissf wants to merge 6 commits into
Conversation
…ests The generated CheckInjectorProvider restores the global EMF registries to a copy taken on its first use, not to the state before each test. When that first use precedes the activation of the Check UI plug-in, every later standalone Check test wipes the Check validator the UI plug-in has registered since. Tests using the UI injector, such as CheckApiAccessValidationsTest and CheckValidationTest, then validate without any Check validator and miss the expected issues. The hand-maintained suites only avoided this by their order. Add RegistryIsolatingCheckInjectorProvider, which restores the state from before each test, and use it in the standalone Check tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
These tests were never part of the aggregated test run and had rotted: - ErrorLogListenerTest ignored a location named after the old class name of the test, and checked the log before the job manager had logged the failure. Wait for the logged exception, and also assert that it is ignored. - DeChKeyboardLayoutTest referenced the layout under a package which no longer exists. SWTBot fixes its keyboard layout globally on first use, so typing with DE_CH would also break all later SWTBot tests; check the layout's keystrokes directly instead. - SwtBotRadioTest opened the preferences via the Window menu, which does not exist on macOS, and changed a workbench preference. Use a shell of its own with a radio group and look the buttons up within it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BugAig1084, BugAig830, BugAig1314 and BugDsl27 are the only test classes whose names do not end in Test or Tests, which automatic test discovery by class name relies on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
All tests ran in the single OSGi runtime of com.avaloq.tools.ddk.xtext.test, through AllTests and one hand-maintained @suite per test bundle. A test class not listed in a suite never ran, and the tests of com.avaloq.tools.ddk.test.ui.test were not referenced at all. Each eclipse-test-plugin bundle now runs its own tests in its own runtime. tycho-surefire-plugin is configured once in ddk-parent and finds the test classes by name, so a new test class needs no registration. Classes run in alphabetical order within a module, and the timeout applies per module. AllTests, the suites, their launch configurations and the JUnit suite dependencies are removed. The UI test bundles keep a launch configuration each, which runs all tests of the project via JUnit discovery in Eclipse. These launches and the Maven test runs use the org.eclipse.platform.ide product, as the target platform has no org.eclipse.sdk bundle. com.avaloq.tools.ddk.checkcfg.ui.test only held an empty suite, so the bundle is removed. With -T, test runtimes run one at a time through reactorConcurrencyLevel, as they never overlapped with the single runtime before, and CheckQuickfixTest failed intermittently on macOS when they did. Everything else stays parallel, and a build without -T is unaffected. Two bundles relied on the shared runtime. The packaging of com.avaloq.tools.ddk.check.test.runtime.tests is now eclipse-test-plugin, as its tests would not run otherwise. com.avaloq.tools.ddk.checkcfg.core.test now declares its runtime dependencies on com.avaloq.tools.ddk.check.test.runtime.ui and com.avaloq.tools.ddk.check.ui. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t logs With one test run per module, a single surefire report no longer shows that the tests ran. The check now requires a report for every concrete class declaring tests, which also catches a test bundle not packaged as eclipse-test-plugin, a class name Surefire does not include and a module the reactor skipped. Failed builds now archive the workspace log of every test module. The log of xtext.test was never archived: upload-artifact ignores hidden files by default, so the step found no files for .metadata/.log. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Test modules use Tycho's default headless harness. A module that needs a workbench opts in by setting tycho.surefire.useUIHarness=true, and fails loudly otherwise, so a new module does not start a workbench by accident. Seven modules need one and opt in: check.core.test, check.ui.test, test.ui.test, checkcfg.core.test, check.test.runtime.tests, sample.helloworld.ui.test and xtext.ui.test. Only these take the Surefire reactor lock (test.reactorConcurrencyLevel=1), since a workbench could take focus from a SWTBot test in another one. The other seven have no workbench and run in parallel under -T. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test modules now run under Tycho's default headless harness, so a module only starts a workbench if it asks for one. Based on #1599, which should merge first.
tycho.surefire.useUIHarness=trueandtest.reactorConcurrencyLevel=1, so they take turns under-T:check.core.test,check.ui.test,test.ui.test,checkcfg.core.test,check.test.runtime.tests,sample.helloworld.ui.test,xtext.ui.test.test,typesystem.test,xtext.export.test,check.runtime.core.test,xtext.test,xtext.generator.test,xtext.format.test.PlatformUI.getWorkbench()throws, and the Check injector can't be created without anIWorkbench.AGENTS.mdexplains how to opt in.Performance
Local builds on macOS with Temurin 21. The
masterand #1599 figures come from earlier runs on older bases, and other work was running on the machine, so small differences are noise.master-T 3C-T 3Cmaster. That comes from one runtime per module in test: run each test bundle in its own runtime with JUnit discovery #1599; this PR recovers part of it.Validation
./mvnw -U -T 3C clean verify checkstyle:check pmd:pmd pmd:cpd pmd:check pmd:cpd-check spotbugs:check -f ddk-parent/pom.xml --batch-mode --fail-at-endpasses.Known limits
checkcfg.core.testhit aConcurrentModificationExceptionin validation once when tried headless. It's unexplained, and it hasn't recurred with that module on the UI harness.🤖 Generated with Claude Code