Skip to content

[tests] Re-enable tests and clean up stale TODOs - #13680

Merged
jonathanpeppers merged 6 commits into
mainfrom
simonrozsival-todo-comments-audit
Nov 20, 2026
Merged

jonathanpeppers merged 6 commits into
mainfrom
simonrozsival-todo-comments-audit

Conversation

@simonrozsival

Copy link
Copy Markdown
Member

Context: #13679

The TODO audit found several test skips and comments that no longer describe the current CoreCLR and NativeAOT implementations. Re-enable the cases that passed locally, adjust their fixtures where runtime-specific paths and targets had changed, and keep skips for scenarios that still fail.

Restore NativeAOT coverage for resource processing, trimmed AndroidX assemblies, spaced clang paths, Java peers, incremental builds, publishing, and resource-designer generation. Restore the CoreCLR AppDomain warning test using its current XA2000 diagnostic. On device, restore the NativeAOT launcher and styleable tests and the SdkOnly linker-descriptor tests for both runtimes. Use a netstandard2.0-compatible lock in the shared test-server source and avoid applying Android runtime properties to the netstandard test library.

Remove or clarify stale TODOs in tests, binding-generator comments, native build inputs, and documentation. Update the remaining device-test skip explanations with the failures observed during retesting; the Full-trimming SQLite, Xamarin.Forms, test-only APK-set, Unicode library-loading, and NativeAOT remapping skips remain. The umbrella issue tracks these failures and other audit findings; it is not resolved by this change.

Validation

The host and device tests ran in the audit's local child worktrees using the same changed test sources as this branch, except for comments and the text of one retained skip. Local result-directory paths are shown as <results> below.

  • dotnet test tests/Java.Interop.Tools.JavaSource-Tests/Java.Interop.Tools.JavaSource-Tests.csproj --filter 'FullyQualifiedName~SourceJavadocToXmldocGrammar' — passed 26/26; these existing grammar tests still verify the current unsupported Javadoc behavior.
  • ./dotnet-local.sh build src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Xamarin.Android.Build.Tests.csproj -c Debug --no-restore -p:BuildProjectReferences=false -v:q — passed in the Debug audit worktree after building the required outputs and registering its local workload.
  • ./dotnet-local.sh test bin/TestDebug/net11.0/Xamarin.Android.Build.Tests.dll --filter '((Name~CheckXmlResourcesFilesAreProcessed|Name~CreateMultiDexWithSpacesInConfig|Name~Plugin_Maui_Audio|Name~CheckNothingIsDeletedByIncrementalClean|Name~DotNetPublishDefaultValues|Name~SimilarAndroidXAssemblyNames)&Name~NativeAOT)|Name~WarnAboutAppDomains|Name~GenerateDesignerFileFromRtxt|Name~GenerateJavaStubsAndAssembly|FullyQualifiedName=Xamarin.Android.Build.Tests.IncrementalBuildTest.LinkAssembliesNoShrink|Name~AndroidResourceChange' --logger 'trx;LogFileName=todo-final.trx' --results-directory <results> — 19 passed, one intentionally skipped NativeAOT AppDomain case, zero failed.
  • make leeroy CONFIGURATION=Release MSBUILD_ARGS='-m:4' — passed in the device-audit worktree.
  • ./dotnet-local.sh build tests/MSBuildDeviceIntegration/MSBuildDeviceIntegration.csproj -c Release --no-restore -m:4 -nr:false --verbosity:quiet — passed.
  • ./dotnet-local.sh build tests/Mono.Android-Tests/Mono.Android-Tests/Mono.Android.NET-Tests.csproj -c Release -m:4 -nr:false --verbosity:quiet — passed after restore, including the modern-target test-server source.
  • ADB_TARGET='-s emulator-5554' NUNIT_MSBUILD_ARGS='-m:4' ./dotnet-local.sh test bin/TestRelease/MSBuildDeviceIntegration/net11.0/MSBuildDeviceIntegration.dll --filter 'Name~NativeAOT&(Name~ClassLibraryMainLauncherRuns|Name~SingleProject_ApplicationId|Name~AppWithStyleableUsageRuns)' --logger 'trx;LogFileName=core-nativeaot.trx' --results-directory <results> — four passed, including the launcher and both styleable cases; the intentionally unskipped test-only install case failed with INSTALL_FAILED_TEST_ONLY and remains skipped in this branch.
  • ADB_TARGET='-s emulator-5554' NUNIT_MSBUILD_ARGS='-m:4' ./dotnet-local.sh test bin/TestRelease/MSBuildDeviceIntegration/net11.0/MSBuildDeviceIntegration.dll --filter 'Name~CustomLinkDescriptionPreserve' --logger 'trx;LogFileName=link-descriptor-final.trx' --results-directory <results> — two SdkOnly cases passed, two Full cases intentionally skipped.
  • git diff --check 78ef747b64..HEAD — passed.

The device runs used an API 35 arm64 emulator. The first Debug make leeroy attempts failed due to concurrent shared Android SDK staging interference; the host-test worktree subsequently built the necessary outputs and registered its workload using the existing targets. The full Debug make leeroy was not rerun there. An earlier CoreCLR library test run also encountered unrelated missing lib/x86_64/libfoo.so APK entries and was not investigated. The preview API device trials required staging a missing analysis assembly locally and did not exercise the API-37 runtime branch; they are not treated as clean SDK validation.

simonrozsival and others added 6 commits November 20, 2026 14:58
#10069 (CoreCLR on-device tests) is closed. SSL was restored for
CoreCLR in #10495 and is now only excluded for NativeAOT, which
#10079 tracks, so point the SSL TODOs there. RuntimeConfig was fixed
by #10669 and is no longer excluded anywhere, so drop that TODO.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The TryCreateInstanceOfSomeClass check stopped calling
Activator.CreateInstance in 2020 and now only uses
typeof(Library1.SomeClass).Assembly, so the ILLink .ctor-removal
TODO no longer applies. Rename the check and fix the failure
message to describe what is actually exercised.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Re-enable NativeAOT coverage whose TODO-documented failures no longer
reproduce against a locally built SDK: XML custom views, trimmed
SimilarAndroidXAssemblyNames, spaced clang configuration,
Plugin.Maui.Audio peers, FileListAbsolute incremental checks, publish
defaults, resource-parser cases, and Java stub/bootstrap incremental
assertions.

Adapt fixtures to current behaviour: NativeAOT library references use
Release output, the bootstrap target is runtime-specific, CoreCLR now
reports AppDomain usage as XA2000 (dotnet/linker#2982 is closed), and
LinkAssembliesNoShrink uses an explicit single RID. Remove the obsolete
AAB/APK CodeBehind question.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
An audit of remaining unlinked TODOs found completed work, obsolete release targets, and compatibility decisions described as temporary implementation gaps. Document minor API entries, existing enum merging, module visibility compatibility, generator formatting, bundled property lookup, and empty-array coverage; remove superseded exporter code and the Xamarin.Android 5.0 note.

Keep unresolved behavior unchanged. Make the NativeAOT source-tracking reminder actionable and replace the obsolete managed-typemap prerequisite on the CoreCLR exclusion with a request for revalidation.

Validation: git diff --check passed. Source comparison ignoring line comments and XML comparison ignoring comments confirmed unchanged code and build/metadata semantics. No SDK build or device tests were needed for comment/documentation-only edits.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
ClassLibraryMainLauncherRuns and both NativeAOT Release
AppWithStyleableUsageRuns variants pass on a device against a locally
built SDK, so remove their stale skips.

Clarify the two NativeAOT skips that still fail:

  * SingleProject_ApplicationId (testOnly): InstallApkSet does not pass
    bundletool's --allow-test-only, so INSTALL_FAILED_TEST_ONLY.
  * TypeAndMemberRemapping: R8 drops example.ViewHelper from the DEX,
    causing ClassNotFoundException; the CoreCLR Release control passes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Device retesting shows CustomLinkDescriptionPreserve passes with SdkOnly on both CoreCLR and NativeAOT. Keep Full skipped for the reproduced SQLite dynamic-provider constructor failure on CoreCLR and missing TodoTask columns on NativeAOT. Avoid applying Android runtime properties to the netstandard2.0 library and keep its shared server source compatible with C# 8 while preserving System.Threading.Lock on modern targets.

Replace historical Xamarin.Forms cast traces with the currently reproduced ResourcesProvider constructor failure, retain the Unicode library-loading skip, clarify the missing NativeAOT performance coverage, and remove the minor-API TODO whose permission constants are already exercised.

Validation in the device-audit child worktree on API 35 arm64:

  * make leeroy CONFIGURATION=Release MSBUILD_ARGS='-m:4' passed.
  * ./dotnet-local.sh build tests/MSBuildDeviceIntegration/MSBuildDeviceIntegration.csproj -c Release --no-restore -m:4 -nr:false --verbosity:quiet passed.
  * ./dotnet-local.sh build tests/Mono.Android-Tests/Mono.Android-Tests/Mono.Android.NET-Tests.csproj -c Release -m:4 -nr:false --verbosity:quiet passed after restore.
  * ADB_TARGET='-s emulator-5554' NUNIT_MSBUILD_ARGS='-m:4' ./dotnet-local.sh test bin/TestRelease/MSBuildDeviceIntegration/net11.0/MSBuildDeviceIntegration.dll --filter 'Name~CustomLinkDescriptionPreserve' passed 2 cases with 2 intentional Full skips; verified the final TRX.
  * Preview 37.1/37.2 Debug device cases passed only after staging the missing analysis assembly from the local stable API 37 pack. API-37 runtime branches were not exercised on API 35; no SDK packaging fix is included.
  * Integrated executable test sources match the validated child sources except for a pre-existing skip message. git diff --check passed.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 20, 2026 15:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues block the restored test coverage and fixture updates.

No open issues 🎉

What changed in this PR

Restores .NET for Android test coverage identified by the TODO audit in #13679, updates fixtures for current runtime behavior, and removes stale comments. Known failures remain skipped; this PR does not resolve the umbrella issue.

Changes:

  • Re-enables NativeAOT build and device tests, CoreCLR AppDomain warning coverage, and SdkOnly linker-descriptor cases.
  • Corrects runtime-specific paths and targets, and makes shared test-server locking compatible with netstandard2.0.
  • Refreshes retained skip explanations, implementation comments, and API-level documentation.
File Description
tests/​MSBuildDeviceIntegration/​Tests/​PerformanceTest.cs Clarifies the remaining NativeAOT coverage TODO.
tests/​MSBuildDeviceIntegration/​Tests/​InstallAndRunTests.cs Restores device cases, fixes library configuration, and updates skips.
tests/​MSBuildDeviceIntegration/​Tests/​DebuggingTest.cs Restores NativeAOT launcher coverage and refreshes a skip explanation.
tests/​MSBuildDeviceIntegration/​Tests/​BundleToolNoAbiSplitTests.cs Updates the Xamarin.Forms startup skip explanation.
tests/​MSBuildDeviceIntegration/​Resources/​LinkDescTest/​MainActivityReplacement.cs Aligns the test comment and failure message with the assembly lookup.
tests/​Mono.Android-Tests/​Mono.Android-Tests/​Xamarin.Android.Net/​LocalTestServers.cs Uses compatible locking syntax and type for netstandard2.0.
tests/​Mono.Android-Tests/​Mono.Android-Tests/​Xamarin.Android.Net/​AndroidMessageHandlerTests.cs Corrects the SSL issue link in a comment.
tests/​Mono.Android-Tests/​Mono.Android-Tests/​Xamarin.Android.Net/​AndroidHandlerTestBase.cs Corrects the SSL issue link in a comment.
tests/​Mono.Android-Tests/​Mono.Android-Tests/​System/​AppContextTests.cs Removes a stale issue comment without changing categorization.
tests/​Mono.Android-Tests/​Mono.Android-Tests/​System.Net/​SslTest.cs Corrects the SSL issue link in a comment.
tests/​Mono.Android-Tests/​Mono.Android-Tests/​Java.Lang/​ObjectTest.cs Clarifies the historical CoreCLR exclusion.
tests/​Mono.Android-Tests/​Java.Interop-Tests/​Java.Interop/​JavaPrimitiveArrayContract.cs Clarifies empty-array pointer cleanup in a comment.
src/​Xamarin.SourceWriter/​Models/​MethodWriter.cs Explains existing modifier-order compatibility.
src/​Xamarin.Android.Tools.Bytecode/​XmlClassDeclarationBuilder.cs Documents existing visibility compatibility behavior.
src/​Xamarin.Android.Build.Tasks/​Tests/​Xamarin.Android.Build.Tests/​XASdkTests.cs Re-enables NativeAOT publishing coverage.
src/​Xamarin.Android.Build.Tasks/​Tests/​Xamarin.Android.Build.Tests/​Tasks/​ManagedResourceParserTests.cs Restores NativeAOT designer tests with the correct library path.
src/​Xamarin.Android.Build.Tasks/​Tests/​Xamarin.Android.Build.Tests/​Tasks/​LinkerTests.cs Restores CoreCLR Release coverage using XA2000.
src/​Xamarin.Android.Build.Tasks/​Tests/​Xamarin.Android.Build.Tests/​IncrementalBuildTest.cs Restores assertions and updates runtime-specific test setup.
src/​Xamarin.Android.Build.Tasks/​Tests/​Xamarin.Android.Build.Tests/​CodeBehindTests.cs Removes stale comments and a commented-out APK expectation.
src/​Xamarin.Android.Build.Tasks/​Tests/​Xamarin.Android.Build.Tests/​BuildTest2.cs Re-enables NativeAOT spaced-path and Java-peer cases.
src/​Xamarin.Android.Build.Tasks/​Tests/​Xamarin.Android.Build.Tests/​BuildTest.cs Adds the trimmed NativeAOT AndroidX case.
src/​Xamarin.Android.Build.Tasks/​Tests/​Xamarin.Android.Build.Tests/​AndroidUpdateResourcesTest.cs Re-enables NativeAOT XML resource processing coverage.
src/​native/​native.targets Clarifies the TODO for missing native build inputs; no target changes.
src/​native/​clr/​include/​host/​host-environment.hh Documents bundled-property lookup behavior.
src/​Mono.Android/​metadata Removes an obsolete color-binding TODO comment.
src/​Java.Interop.Tools.JavaTypeSystem/​Adapters/​JavaXmlApiExporter.cs Removes obsolete commented-out filtering code.
src/​generator/​SourceWriters/​Attributes/​RegisterAttr.cs Explains existing generated-attribute formatting compatibility.
Documentation/​workflow/​HowToAddNewApiLevel.md Documents minor API entries and removes an obsolete checklist step.
build-tools/​enumification-helpers/​README Describes the existing merge attribute without a stale TODO.

Review effort: Balanced

@simonrozsival simonrozsival added Area: Unit Tests Issues with repo unit tests. Area: NativeAOT Issues that only occur when using NativeAOT. labels Nov 20, 2026
@jonathanpeppers

Copy link
Copy Markdown
Member

CI triage for build 1692330: the build is red (31 jobs passed, 5 failed), but the failures do not appear related to this PR.

All six failed test results are from AndroidDependenciesTest.InstallAndroidDependenciesTest on Linux, macOS, and Windows. The Android CLI could not install platform-tools@37.0.1. The same failure occurred on this PR's exact base commit (build 1691566), so this is not evidence of a regression from this change. PR #13666 updates the platform-tools pin to 37.0.3.

I also checked the tests this PR re-enables: all 16 distinct cases passed across the 29 host and emulator results that ran them. The cases intentionally left disabled by the PR were not included in that count.

@jonathanpeppers
jonathanpeppers merged commit 8287b17 into main Nov 20, 2026
33 of 39 checks passed
@jonathanpeppers
jonathanpeppers deleted the simonrozsival-todo-comments-audit branch November 20, 2026 20:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: NativeAOT Issues that only occur when using NativeAOT. Area: Unit Tests Issues with repo unit tests.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants