Repository navigation
[tests] Re-enable tests and clean up stale TODOs - #13680
Conversation
#10069 (CoreCLR on-device tests) is closed. SSL was restored for CoreCLR in #10495 and is now only excluded for NativeAOT, which #10079 tracks, so point the SSL TODOs there. RuntimeConfig was fixed by #10669 and is no longer excluded anywhere, so drop that TODO. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The TryCreateInstanceOfSomeClass check stopped calling Activator.CreateInstance in 2020 and now only uses typeof(Library1.SomeClass).Assembly, so the ILLink .ctor-removal TODO no longer applies. Rename the check and fix the failure message to describe what is actually exercised. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Re-enable NativeAOT coverage whose TODO-documented failures no longer reproduce against a locally built SDK: XML custom views, trimmed SimilarAndroidXAssemblyNames, spaced clang configuration, Plugin.Maui.Audio peers, FileListAbsolute incremental checks, publish defaults, resource-parser cases, and Java stub/bootstrap incremental assertions. Adapt fixtures to current behaviour: NativeAOT library references use Release output, the bootstrap target is runtime-specific, CoreCLR now reports AppDomain usage as XA2000 (dotnet/linker#2982 is closed), and LinkAssembliesNoShrink uses an explicit single RID. Remove the obsolete AAB/APK CodeBehind question. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
An audit of remaining unlinked TODOs found completed work, obsolete release targets, and compatibility decisions described as temporary implementation gaps. Document minor API entries, existing enum merging, module visibility compatibility, generator formatting, bundled property lookup, and empty-array coverage; remove superseded exporter code and the Xamarin.Android 5.0 note. Keep unresolved behavior unchanged. Make the NativeAOT source-tracking reminder actionable and replace the obsolete managed-typemap prerequisite on the CoreCLR exclusion with a request for revalidation. Validation: git diff --check passed. Source comparison ignoring line comments and XML comparison ignoring comments confirmed unchanged code and build/metadata semantics. No SDK build or device tests were needed for comment/documentation-only edits. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
ClassLibraryMainLauncherRuns and both NativeAOT Release
AppWithStyleableUsageRuns variants pass on a device against a locally
built SDK, so remove their stale skips.
Clarify the two NativeAOT skips that still fail:
* SingleProject_ApplicationId (testOnly): InstallApkSet does not pass
bundletool's --allow-test-only, so INSTALL_FAILED_TEST_ONLY.
* TypeAndMemberRemapping: R8 drops example.ViewHelper from the DEX,
causing ClassNotFoundException; the CoreCLR Release control passes.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Device retesting shows CustomLinkDescriptionPreserve passes with SdkOnly on both CoreCLR and NativeAOT. Keep Full skipped for the reproduced SQLite dynamic-provider constructor failure on CoreCLR and missing TodoTask columns on NativeAOT. Avoid applying Android runtime properties to the netstandard2.0 library and keep its shared server source compatible with C# 8 while preserving System.Threading.Lock on modern targets. Replace historical Xamarin.Forms cast traces with the currently reproduced ResourcesProvider constructor failure, retain the Unicode library-loading skip, clarify the missing NativeAOT performance coverage, and remove the minor-API TODO whose permission constants are already exercised. Validation in the device-audit child worktree on API 35 arm64: * make leeroy CONFIGURATION=Release MSBUILD_ARGS='-m:4' passed. * ./dotnet-local.sh build tests/MSBuildDeviceIntegration/MSBuildDeviceIntegration.csproj -c Release --no-restore -m:4 -nr:false --verbosity:quiet passed. * ./dotnet-local.sh build tests/Mono.Android-Tests/Mono.Android-Tests/Mono.Android.NET-Tests.csproj -c Release -m:4 -nr:false --verbosity:quiet passed after restore. * ADB_TARGET='-s emulator-5554' NUNIT_MSBUILD_ARGS='-m:4' ./dotnet-local.sh test bin/TestRelease/MSBuildDeviceIntegration/net11.0/MSBuildDeviceIntegration.dll --filter 'Name~CustomLinkDescriptionPreserve' passed 2 cases with 2 intentional Full skips; verified the final TRX. * Preview 37.1/37.2 Debug device cases passed only after staging the missing analysis assembly from the local stable API 37 pack. API-37 runtime branches were not exercised on API 35; no SDK packaging fix is included. * Integrated executable test sources match the validated child sources except for a pre-existing skip message. git diff --check passed. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues block the restored test coverage and fixture updates.
No open issues 🎉
What changed in this PR
Restores .NET for Android test coverage identified by the TODO audit in #13679, updates fixtures for current runtime behavior, and removes stale comments. Known failures remain skipped; this PR does not resolve the umbrella issue.
Changes:
- Re-enables NativeAOT build and device tests, CoreCLR AppDomain warning coverage, and
SdkOnlylinker-descriptor cases. - Corrects runtime-specific paths and targets, and makes shared test-server locking compatible with netstandard2.0.
- Refreshes retained skip explanations, implementation comments, and API-level documentation.
| File | Description |
|---|---|
| tests/MSBuildDeviceIntegration/Tests/PerformanceTest.cs | Clarifies the remaining NativeAOT coverage TODO. |
| tests/MSBuildDeviceIntegration/Tests/InstallAndRunTests.cs | Restores device cases, fixes library configuration, and updates skips. |
| tests/MSBuildDeviceIntegration/Tests/DebuggingTest.cs | Restores NativeAOT launcher coverage and refreshes a skip explanation. |
| tests/MSBuildDeviceIntegration/Tests/BundleToolNoAbiSplitTests.cs | Updates the Xamarin.Forms startup skip explanation. |
| tests/MSBuildDeviceIntegration/Resources/LinkDescTest/MainActivityReplacement.cs | Aligns the test comment and failure message with the assembly lookup. |
| tests/Mono.Android-Tests/Mono.Android-Tests/Xamarin.Android.Net/LocalTestServers.cs | Uses compatible locking syntax and type for netstandard2.0. |
| tests/Mono.Android-Tests/Mono.Android-Tests/Xamarin.Android.Net/AndroidMessageHandlerTests.cs | Corrects the SSL issue link in a comment. |
| tests/Mono.Android-Tests/Mono.Android-Tests/Xamarin.Android.Net/AndroidHandlerTestBase.cs | Corrects the SSL issue link in a comment. |
| tests/Mono.Android-Tests/Mono.Android-Tests/System/AppContextTests.cs | Removes a stale issue comment without changing categorization. |
| tests/Mono.Android-Tests/Mono.Android-Tests/System.Net/SslTest.cs | Corrects the SSL issue link in a comment. |
| tests/Mono.Android-Tests/Mono.Android-Tests/Java.Lang/ObjectTest.cs | Clarifies the historical CoreCLR exclusion. |
| tests/Mono.Android-Tests/Java.Interop-Tests/Java.Interop/JavaPrimitiveArrayContract.cs | Clarifies empty-array pointer cleanup in a comment. |
| src/Xamarin.SourceWriter/Models/MethodWriter.cs | Explains existing modifier-order compatibility. |
| src/Xamarin.Android.Tools.Bytecode/XmlClassDeclarationBuilder.cs | Documents existing visibility compatibility behavior. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/XASdkTests.cs | Re-enables NativeAOT publishing coverage. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Tasks/ManagedResourceParserTests.cs | Restores NativeAOT designer tests with the correct library path. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Tasks/LinkerTests.cs | Restores CoreCLR Release coverage using XA2000. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/IncrementalBuildTest.cs | Restores assertions and updates runtime-specific test setup. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/CodeBehindTests.cs | Removes stale comments and a commented-out APK expectation. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BuildTest2.cs | Re-enables NativeAOT spaced-path and Java-peer cases. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BuildTest.cs | Adds the trimmed NativeAOT AndroidX case. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AndroidUpdateResourcesTest.cs | Re-enables NativeAOT XML resource processing coverage. |
| src/native/native.targets | Clarifies the TODO for missing native build inputs; no target changes. |
| src/native/clr/include/host/host-environment.hh | Documents bundled-property lookup behavior. |
| src/Mono.Android/metadata | Removes an obsolete color-binding TODO comment. |
| src/Java.Interop.Tools.JavaTypeSystem/Adapters/JavaXmlApiExporter.cs | Removes obsolete commented-out filtering code. |
| src/generator/SourceWriters/Attributes/RegisterAttr.cs | Explains existing generated-attribute formatting compatibility. |
| Documentation/workflow/HowToAddNewApiLevel.md | Documents minor API entries and removes an obsolete checklist step. |
| build-tools/enumification-helpers/README | Describes the existing merge attribute without a stale TODO. |
Review effort: Balanced
|
CI triage for build 1692330: the build is red (31 jobs passed, 5 failed), but the failures do not appear related to this PR. All six failed test results are from I also checked the tests this PR re-enables: all 16 distinct cases passed across the 29 host and emulator results that ran them. The cases intentionally left disabled by the PR were not included in that count. |
Context: #13679
The TODO audit found several test skips and comments that no longer describe the current CoreCLR and NativeAOT implementations. Re-enable the cases that passed locally, adjust their fixtures where runtime-specific paths and targets had changed, and keep skips for scenarios that still fail.
Restore NativeAOT coverage for resource processing, trimmed AndroidX assemblies, spaced clang paths, Java peers, incremental builds, publishing, and resource-designer generation. Restore the CoreCLR AppDomain warning test using its current XA2000 diagnostic. On device, restore the NativeAOT launcher and styleable tests and the
SdkOnlylinker-descriptor tests for both runtimes. Use a netstandard2.0-compatible lock in the shared test-server source and avoid applying Android runtime properties to the netstandard test library.Remove or clarify stale TODOs in tests, binding-generator comments, native build inputs, and documentation. Update the remaining device-test skip explanations with the failures observed during retesting; the Full-trimming SQLite, Xamarin.Forms, test-only APK-set, Unicode library-loading, and NativeAOT remapping skips remain. The umbrella issue tracks these failures and other audit findings; it is not resolved by this change.
Validation
The host and device tests ran in the audit's local child worktrees using the same changed test sources as this branch, except for comments and the text of one retained skip. Local result-directory paths are shown as
<results>below.dotnet test tests/Java.Interop.Tools.JavaSource-Tests/Java.Interop.Tools.JavaSource-Tests.csproj --filter 'FullyQualifiedName~SourceJavadocToXmldocGrammar'— passed 26/26; these existing grammar tests still verify the current unsupported Javadoc behavior../dotnet-local.sh build src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Xamarin.Android.Build.Tests.csproj -c Debug --no-restore -p:BuildProjectReferences=false -v:q— passed in the Debug audit worktree after building the required outputs and registering its local workload../dotnet-local.sh test bin/TestDebug/net11.0/Xamarin.Android.Build.Tests.dll --filter '((Name~CheckXmlResourcesFilesAreProcessed|Name~CreateMultiDexWithSpacesInConfig|Name~Plugin_Maui_Audio|Name~CheckNothingIsDeletedByIncrementalClean|Name~DotNetPublishDefaultValues|Name~SimilarAndroidXAssemblyNames)&Name~NativeAOT)|Name~WarnAboutAppDomains|Name~GenerateDesignerFileFromRtxt|Name~GenerateJavaStubsAndAssembly|FullyQualifiedName=Xamarin.Android.Build.Tests.IncrementalBuildTest.LinkAssembliesNoShrink|Name~AndroidResourceChange' --logger 'trx;LogFileName=todo-final.trx' --results-directory <results>— 19 passed, one intentionally skipped NativeAOT AppDomain case, zero failed.make leeroy CONFIGURATION=Release MSBUILD_ARGS='-m:4'— passed in the device-audit worktree../dotnet-local.sh build tests/MSBuildDeviceIntegration/MSBuildDeviceIntegration.csproj -c Release --no-restore -m:4 -nr:false --verbosity:quiet— passed../dotnet-local.sh build tests/Mono.Android-Tests/Mono.Android-Tests/Mono.Android.NET-Tests.csproj -c Release -m:4 -nr:false --verbosity:quiet— passed after restore, including the modern-target test-server source.ADB_TARGET='-s emulator-5554' NUNIT_MSBUILD_ARGS='-m:4' ./dotnet-local.sh test bin/TestRelease/MSBuildDeviceIntegration/net11.0/MSBuildDeviceIntegration.dll --filter 'Name~NativeAOT&(Name~ClassLibraryMainLauncherRuns|Name~SingleProject_ApplicationId|Name~AppWithStyleableUsageRuns)' --logger 'trx;LogFileName=core-nativeaot.trx' --results-directory <results>— four passed, including the launcher and both styleable cases; the intentionally unskipped test-only install case failed withINSTALL_FAILED_TEST_ONLYand remains skipped in this branch.ADB_TARGET='-s emulator-5554' NUNIT_MSBUILD_ARGS='-m:4' ./dotnet-local.sh test bin/TestRelease/MSBuildDeviceIntegration/net11.0/MSBuildDeviceIntegration.dll --filter 'Name~CustomLinkDescriptionPreserve' --logger 'trx;LogFileName=link-descriptor-final.trx' --results-directory <results>— twoSdkOnlycases passed, two Full cases intentionally skipped.git diff --check 78ef747b64..HEAD— passed.The device runs used an API 35 arm64 emulator. The first Debug
make leeroyattempts failed due to concurrent shared Android SDK staging interference; the host-test worktree subsequently built the necessary outputs and registered its workload using the existing targets. The full Debugmake leeroywas not rerun there. An earlier CoreCLR library test run also encountered unrelated missinglib/x86_64/libfoo.soAPK entries and was not investigated. The preview API device trials required staging a missing analysis assembly locally and did not exercise the API-37 runtime branch; they are not treated as clean SDK validation.