Repository navigation
[tests] Migrate remaining test suites to MSTest/MTP - #13487
Open
simonrozsival wants to merge 9 commits into
Open
simonrozsival wants to merge 9 commits into
simonrozsival wants to merge 9 commits into
Conversation
simonrozsival
force-pushed
the
simonrozsival-mstest-migrate-remaining-suites
branch
from
November 11, 2026 12:20
567ebc4 to
d6fe575
Compare
simonrozsival
force-pushed
the
simonrozsival-mstest-migrate-remaining-suites
branch
from
November 11, 2026 17:13
d6fe575 to
64f0dfd
Compare
Base automatically changed from
simonrozsival-mstest-delivery-host-ci
to
main
November 17, 2026 14:35
simonrozsival
force-pushed
the
simonrozsival-mstest-migrate-remaining-suites
branch
from
November 17, 2026 15:10
c727bc5 to
9ee1bd8
Compare
simonrozsival
marked this pull request as ready for review
November 17, 2026 15:10
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The APK size scripts can copy stale .apkdesc files because positional DLL arguments no longer run BuildReleaseArm64, and the same invocation needs correction in the test documentation.
1 open issue
What changed in this PR
Completes the test-framework migration started in #13434, unifying .NET for Android’s remaining test suites on MSTest and Microsoft.Testing.Platform (MTP).
Changes:
- Converts test attributes, assertions, data sources, lifecycle hooks, and parallelism settings.
- Centralizes MTP configuration and updates CI test execution, filtering, and retry settings.
- Removes NUnit infrastructure and refreshes testing documentation.
| File | Description |
|---|---|
| tests/Xamarin.SourceWriter-Tests/Xamarin.SourceWriter-Tests.csproj | Uses shared MSTest references. |
| tests/Xamarin.SourceWriter-Tests/TypeReferenceWriterTests.cs | Converts test attributes and null assertion. |
| tests/Xamarin.SourceWriter-Tests/README.md | Documents MTP host-test execution. |
| tests/Xamarin.SourceWriter-Tests/PropertyWriterTests.cs | Converts tests to MSTest. |
| tests/Xamarin.SourceWriter-Tests/MethodWriterTests.cs | Converts tests to MSTest. |
| tests/Xamarin.SourceWriter-Tests/InterfaceWriterTests.cs | Converts tests to MSTest. |
| tests/Xamarin.SourceWriter-Tests/FieldWriterTests.cs | Converts tests to MSTest. |
| tests/Xamarin.SourceWriter-Tests/EventWriterTests.cs | Converts tests to MSTest. |
| tests/Xamarin.SourceWriter-Tests/DelegateWriterTests.cs | Converts tests to MSTest. |
| tests/Xamarin.SourceWriter-Tests/ConstructorWriterTests.cs | Converts tests to MSTest. |
| tests/Xamarin.SourceWriter-Tests/ClassWriterTests.cs | Converts tests and disables assembly parallelism. |
| tests/Xamarin.Android.Tools.Bytecode-Tests/Xamarin.Android.Tools.Bytecode-Tests.csproj | Uses shared MSTest references. |
| tests/Xamarin.Android.Tools.Bytecode-Tests/NullableAnnotationTests.cs | Converts attributes and null assertions. |
| tests/Xamarin.Android.Tools.Bytecode-Tests/NonGenericGlobalTypeTests.cs | Converts tests to MSTest. |
| tests/Xamarin.Android.Tools.Bytecode-Tests/ModuleInfoTests.cs | Converts attributes and assertions. |
| tests/Xamarin.Android.Tools.Bytecode-Tests/JvmOverloadsConstructorTests.cs | Converts tests to MSTest. |
| tests/Xamarin.Android.Tools.Bytecode-Tests/JavaTypeTests.cs | Converts tests to MSTest. |
| tests/Xamarin.Android.Tools.Bytecode-Tests/JavaTypeNoParametersTests.cs | Converts tests to MSTest. |
| tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.RNCTests.cs | Converts tests to MSTest. |
| tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.RNC.RPNCTests.cs | Converts tests to MSTest. |
| tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.PSCTests.cs | Converts tests to MSTest. |
| tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.ASCTests.cs | Converts tests to MSTest. |
| tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.1Tests.cs | Converts tests to MSTest. |
| tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.1MyStringListTests.cs | Converts tests to MSTest. |
| tests/Xamarin.Android.Tools.Bytecode-Tests/JavaEnumTests.cs | Converts tests to MSTest. |
| tests/Xamarin.Android.Tools.Bytecode-Tests/JavaAnnotationTests.cs | Converts tests to MSTest. |
| tests/Xamarin.Android.Tools.Bytecode-Tests/IJavaInterfaceTests.cs | Converts tests to MSTest. |
| tests/Xamarin.Android.Tools.Bytecode-Tests/ExpectedTypeDeclaration.cs | Switches assertion alias to MSTest. |
| tests/Xamarin.Android.Tools.Bytecode-Tests/ExpectedMethodDeclaration.cs | Switches assertion alias to MSTest. |
| tests/Xamarin.Android.Tools.Bytecode-Tests/ExpectedInnerClassInfo.cs | Switches assertion alias to MSTest. |
| tests/Xamarin.Android.Tools.Bytecode-Tests/ExpectedFieldDeclaration.cs | Switches assertion alias to MSTest. |
| tests/Xamarin.Android.Tools.Bytecode-Tests/ClassFileTests.cs | Preserves exact exception assertions in MSTest. |
| tests/Xamarin.Android.Tools.Bytecode-Tests/ClassFileFixture.cs | Converts setup, cleanup, and assertions. |
| tests/Xamarin.Android.Tools.ApiXmlAdjuster-Tests/Xamarin.Android.Tools.ApiXmlAdjuster-Tests.csproj | Uses shared MSTest references. |
| tests/Xamarin.Android.Tools.ApiXmlAdjuster-Tests/TypeResolverTest.cs | Converts tests and class initialization. |
| tests/Xamarin.Android.Tools.ApiXmlAdjuster-Tests/OverrideMarkerTest.cs | Converts tests and class initialization. |
| tests/Xamarin.Android.Tools.ApiXmlAdjuster-Tests/JavaApiTest.cs | Converts tests and class initialization. |
| tests/Xamarin.Android.Tools.ApiXmlAdjuster-Tests/GenericInheritanceMappingTest.cs | Converts tests and class initialization. |
| tests/Xamarin.Android.Tools.AndroidSdk-Tests/Xamarin.Android.Tools.AndroidSdk-Tests.csproj | Adopts MSTest and references the streaming-test tool. |
| tests/Xamarin.Android.Tools.AndroidSdk-Tests/JdkVersionInfoTests.cs | Converts tests to MSTest. |
| tests/Xamarin.Android.Tools.AndroidSdk-Tests/AndroidSdkWindowsTests.cs | Converts the test-class attribute. |
| tests/Xamarin.Android.Tools.Aidl-Tests/Xamarin.Android.Tools.Aidl-Tests.csproj | Uses shared MSTest references. |
| tests/Xamarin.Android.Tools.Aidl-Tests/AidlCompilerTests.cs | Converts data rows and assertions. |
| tests/Xamarin.Android.Tools.Aidl-Tests/AidlCompilerTestBase.cs | Converts shared assertions and skip reporting. |
| tests/TestRunner.Core/TestRunner.Core.NET.csproj | Removes the NUnit runner project. |
| tests/MSBuildDeviceIntegration/Tests/WebViewJavascriptInterfaceTests.cs | Converts runtime cases, assertions, and cleanup logging. |
| tests/MSBuildDeviceIntegration/Tests/SystemApplicationTests.cs | Converts categories and combinatorial runtime cases. |
| tests/MSBuildDeviceIntegration/Tests/ScopedTypeMapR8Tests.cs | Converts attributes and string assertions. |
| tests/MSBuildDeviceIntegration/Tests/R8ObfuscationTests.cs | Converts runtime cases and cleanup logging. |
| tests/MSBuildDeviceIntegration/Tests/InterfaceCollectionTests.cs | Converts attributes and assertions. |
| tests/MSBuildDeviceIntegration/MSBuildDeviceIntegration.csproj | Configures reflection discovery and MTP retries. |
| tests/MSBuildDeviceIntegration/AssemblyInfo.cs | Preserves assembly-wide sequential execution. |
| tests/Mono.Android-Tests/Mono.Android-Tests/Xamarin.Android.Net/AndroidMessageHandlerCancellationTests.cs | Removes an outdated NUnit teardown comment. |
| tests/Mono.Android-Tests/Mono.Android-Tests/Java.Lang/ObjectTest.cs | Removes NUnit frames from a comment. |
| tests/Mono.Android-Tests/Mono.Android-Tests/Java.Interop/JnienvTest.cs | Makes diagnostic comments framework-neutral. |
| tests/Mono.Android-Tests/Mono.Android-Tests/Java.Interop/JavaPeerProxyTests.cs | Removes the NUnit test fixture. |
| tests/Mono.Android-Tests/Mono.Android-Tests/global.json | Removes redundant project-local runner selection. |
| tests/Mono.Android-Tests/Mono.Android-Tests/Android.Widget/AdapterTests.cs | Removes NUnit frames from a comment. |
| tests/Mono.Android-Tests/Mono.Android-Tests/Android.Runtime/PInvokeOverrideTests.cs | Removes the NUnit test fixture. |
| tests/Mono.Android-Tests/Mono.Android-Tests/Android.OS/AsyncTaskTests.cs | Removes the NUnit test fixture. |
| tests/Mono.Android-Tests/Mono.Android-Tests/Android.Content/ContentValuesTests.cs | Removes the NUnit test fixture. |
| tests/Mono.Android-Tests/Java.Interop-Tests/Java.Interop/JniNativeInterfaceTests.cs | Removes the excluded NUnit fixture. |
| tests/Mono.Android-Tests/Java.Interop-Tests/Java.Interop-Tests.NET.csproj | Removes the deleted fixture’s compile exclusion. |
| tests/Microsoft.Android.Build.BaseTasks-Tests/Microsoft.Android.Build.BaseTasks-Tests.csproj | Uses shared MSTest references. |
| tests/Microsoft.Android.Build.BaseTasks-Tests/MemoryStreamPoolTests.cs | Converts lifecycle hooks and typed assertions. |
| tests/Microsoft.Android.Build.BaseTasks-Tests/AsyncTaskTests.cs | Converts attributes and test-output handling. |
| tests/Microsoft.Android.Build.BaseTasks-Tests/AsyncTaskExtensionsTests.cs | Converts tests to MSTest. |
| tests/Java.Interop.Tools.Maven-Tests/PropertySubstitutionTests.cs | Converts tests to MSTest. |
| tests/Java.Interop.Tools.Maven-Tests/ProjectResolverTests.cs | Converts tests and exact exception assertions. |
| tests/Java.Interop.Tools.Maven-Tests/MavenVersionTests.cs | Converts tests to MSTest. |
| tests/Java.Interop.Tools.Maven-Tests/MavenVersionRangeTests.cs | Converts tests to MSTest. |
| tests/Java.Interop.Tools.Maven-Tests/Java.Interop.Tools.Maven-Tests.csproj | Shares MSTest packages and global imports. |
| tests/Java.Interop.Tools.JavaTypeSystem-Tests/JavaTypeReferenceTests.cs | Converts tests to MSTest. |
| tests/Java.Interop.Tools.JavaTypeSystem-Tests/JavaTypeNameTests.cs | Converts tests to MSTest. |
| tests/Java.Interop.Tools.JavaTypeSystem-Tests/JavaTypeModelsTests.cs | Converts tests and class initialization. |
| tests/Java.Interop.Tools.JavaTypeSystem-Tests/Java.Interop.Tools.JavaTypeSystem-Tests.csproj | Uses shared MSTest references. |
| tests/Java.Interop.Tools.JavaTypeSystem-Tests/GenericInheritanceMappingTests.cs | Converts tests and class initialization. |
| tests/Java.Interop.Tools.JavaTypeSystem-Tests/BaseMethodTests.cs | Converts tests and class initialization. |
| tests/Java.Interop.Tools.JavaSource-Tests/SourceJavadocToXmldocParserTests.cs | Converts parser cases to dynamic data. |
| tests/Java.Interop.Tools.JavaSource-Tests/SourceJavadocToXmldocGrammarFixture.cs | Converts the shared fixture to MSTest. |
| tests/Java.Interop.Tools.JavaSource-Tests/SourceJavadocToXmldocGrammar.HtmlBnfTermsTests.cs | Converts grammar tests to MSTest. |
| tests/Java.Interop.Tools.JavaSource-Tests/JavaStubParserTests.cs | Converts attributes and null assertions. |
| tests/Java.Interop.Tools.JavaSource-Tests/Java.Interop.Tools.JavaSource-Tests.csproj | Uses shared MSTest references. |
| tests/Java.Interop.Tools.JavaCallableWrappers-Tests/Java.Interop.Tools.JavaCallableWrappers/InstrumentationTests.cs | Converts attributes and materializes collection assertions. |
| tests/Java.Interop.Tools.JavaCallableWrappers-Tests/Java.Interop.Tools.JavaCallableWrappers/IdentifierValidatorTests.cs | Converts tests to MSTest. |
| tests/Java.Interop.Tools.JavaCallableWrappers-Tests/Java.Interop.Tools.JavaCallableWrappers/Crc64Tests.cs | Converts tests to MSTest. |
| tests/Java.Interop.Tools.JavaCallableWrappers-Tests/Java.Interop.Tools.JavaCallableWrappers/ActivityTests.cs | Converts attributes and materializes collection assertions. |
| tests/Java.Interop.Tools.JavaCallableWrappers-Tests/Java.Interop.Tools.JavaCallableWrappers-Tests.csproj | Uses shared MSTest references. |
| tests/Java.Interop.Tools.Generator-Tests/Utilities/NamingConverterTests.cs | Converts tests and exact exception assertions. |
| tests/Java.Interop.Tools.Generator-Tests/Utilities/AndroidSdkVersionTests.cs | Converts attributes and renames a test method. |
| tests/Java.Interop.Tools.Generator-Tests/Metadata/NamespaceTransformTests.cs | Converts tests to MSTest. |
| tests/Java.Interop.Tools.Generator-Tests/Java.Interop.Tools.Generator-Tests.csproj | Uses shared MSTest references. |
| tests/generator-Tests/Unit-Tests/TypeNameUtilitiesTests.cs | Converts dynamic data and preserves culture restoration. |
| tests/generator-Tests/Unit-Tests/SymbolTableTests.cs | Converts tests to MSTest. |
| tests/generator-Tests/Unit-Tests/SourceWriters/WeakImplementorFieldTests.cs | Converts tests to MSTest. |
| tests/generator-Tests/Unit-Tests/SourceWriters/SourceWritersTestBase.cs | Converts the shared test-class attribute. |
| tests/generator-Tests/Unit-Tests/SourceWriters/PeerMembersFieldTests.cs | Converts tests to MSTest. |
| tests/generator-Tests/Unit-Tests/SourceWriters/MethodExtensionAsyncWrapperTests.cs | Converts tests to MSTest. |
| tests/generator-Tests/Unit-Tests/SourceWriters/MethodExplicitInterfaceImplementationTests.cs | Converts tests to MSTest. |
| tests/generator-Tests/Unit-Tests/SealedProtectedFixupsTests.cs | Converts tests to MSTest. |
| tests/generator-Tests/Unit-Tests/ReportTests.cs | Converts tests to MSTest. |
| tests/generator-Tests/Unit-Tests/NRTJavaInteropCodeGeneratorTests.cs | Converts the inherited test-class attribute. |
| tests/generator-Tests/Unit-Tests/ManagedExtensionsTests.cs | Converts tests to MSTest. |
| tests/generator-Tests/Unit-Tests/InterfaceConstantsTests.cs | Converts tests to MSTest. |
| tests/generator-Tests/Unit-Tests/CodeGeneratorTestBase.cs | Converts shared initialization and cleanup hooks. |
| tests/generator-Tests/Unit-Tests/CodeGeneratorContextTests.cs | Converts tests to MSTest. |
| tests/generator-Tests/README.md | Updates test-framework terminology. |
| tests/generator-Tests/Integration-Tests/Streams.cs | Converts tests to MSTest. |
| tests/generator-Tests/Integration-Tests/StaticProperties.cs | Converts tests to MSTest. |
| tests/generator-Tests/Integration-Tests/StaticMethods.cs | Converts tests to MSTest. |
| tests/generator-Tests/Integration-Tests/StaticFields.cs | Converts tests to MSTest. |
| tests/generator-Tests/Integration-Tests/PamareterXPath.cs | Converts tests to MSTest. |
| tests/generator-Tests/Integration-Tests/NormalProperties.cs | Converts attributes and string assertions. |
| tests/generator-Tests/Integration-Tests/NormalMethods.cs | Converts attributes and string assertions. |
| tests/generator-Tests/Integration-Tests/NonStaticFields.cs | Converts tests to MSTest. |
| tests/generator-Tests/Integration-Tests/NestedTypes.cs | Converts tests to MSTest. |
| tests/generator-Tests/Integration-Tests/Java_Util_List.cs | Converts tests to MSTest. |
| tests/generator-Tests/Integration-Tests/Java_Lang_Object.cs | Converts tests to MSTest. |
| tests/generator-Tests/Integration-Tests/Java_Lang_Enum.cs | Converts tests to MSTest. |
| tests/generator-Tests/Integration-Tests/Interfaces.cs | Converts tests to MSTest. |
| tests/generator-Tests/Integration-Tests/InterfaceMethodsConflict.cs | Converts tests to MSTest. |
| tests/generator-Tests/Integration-Tests/GenericArguments.cs | Converts tests to MSTest. |
| tests/generator-Tests/Integration-Tests/CSharpKeywords.cs | Converts tests to MSTest. |
| tests/generator-Tests/Integration-Tests/Core_Jar2Xml.cs | Converts tests to MSTest. |
| tests/generator-Tests/Integration-Tests/Core_ClassParse.cs | Converts tests to MSTest. |
| tests/generator-Tests/Integration-Tests/Constructors.cs | Converts tests to MSTest. |
| tests/generator-Tests/Integration-Tests/Compiler.cs | Switches compiler helpers to MSTest assertions. |
| tests/generator-Tests/Integration-Tests/BaseGeneratorTest.cs | Converts shared test initialization. |
| tests/generator-Tests/Integration-Tests/Arrays.cs | Converts tests to MSTest. |
| tests/generator-Tests/Integration-Tests/Android_Graphics_Color.cs | Converts tests to MSTest. |
| tests/generator-Tests/Integration-Tests/Adapters.cs | Converts tests to MSTest. |
| tests/generator-Tests/Integration-Tests/AccessModifiers.cs | Converts tests to MSTest. |
| tests/generator-Tests/generator-Tests.csproj | Uses shared MSTest references. |
| tests/generator-Tests/AssemblyInfo.cs | Enables discovery of internal test classes. |
| tests/CodeGen-Binding/Xamarin.Android.JcwGen-Tests/Xamarin.Android.JcwGen-Tests.csproj | Drops timing exclusion and updates a runtime comment. |
| tests/CodeGen-Binding/Xamarin.Android.JcwGen-Tests/global.json | Removes redundant project-local runner selection. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.ProjectTools/Xamarin.ProjectTools.csproj | Replaces NUnit dependencies with MSTest assertions. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.ProjectTools/Utilities/ProcessExtensions.cs | Switches process helpers to MSTest assertions. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.ProjectTools/Common/DownloadedCache.cs | Converts framework import and diagnostic output. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.ProjectTools/Common/Builder.cs | Removes NUnit-specific MSBuild argument handling. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Xamarin.Android.Build.Tests.csproj | Adopts shared MSTest packages with reflection discovery. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/WearTests.cs | Converts runtime combinations and parallelism settings. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Utilities/ValidRuntimeConfigurationsAttribute.cs | Supplies runtime configurations to combinatorial tests. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Utilities/BuildHelper.cs | Switches build helpers to MSTest assertions. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Utilities/AssertionExtensions.cs | Switches assertion helpers to MSTest. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Tasks/ZipAlignTests.cs | Converts lifecycle hooks, assertions, and parallelism. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Tasks/ProcessNativeLibrariesTests.cs | Converts attributes and test-output handling. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Tasks/JsonExtensionsTests.cs | Converts parameterized cases to data rows. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Tasks/JavaToolTaskTests.cs | Converts parameterized cases to data rows. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Tasks/GenerateResourceCaseMapTests.cs | Converts attributes, assertions, and output handling. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Tasks/BundleToolTaskTests.cs | Converts attributes and preserves sequential execution. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/PackagingUtilsTests.cs | Converts attributes and preserves sequential execution. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/ManifestTest.TestCaseSource.cs | Switches the test-framework import. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/DeferredBuildTest.cs | Converts runtime parameters to combinatorial data. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BuildTest.TestCaseSource.cs | Types test-source data as argument rows. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AssemblyInfo.cs | Enables method-level test parallelism. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AidlTest.cs | Converts runtime parameters to combinatorial data. |
| src/Microsoft.Android.Sdk.Analysis/Tests/Microsoft.Android.Sdk.Analysis.Tests.csproj | Adopts shared MSTest packages; disables packing. |
| src/Microsoft.Android.Sdk.Analysis/Tests/DNAS0001Tests.cs | Converts attributes while retaining the ignored test. |
| src/Microsoft.Android.Build.Tasks/Tests/Microsoft.Android.Build.Tasks.Tests/TypeMapProguardNativeObject.cs | Converts unavailable-tool reporting to inconclusive. |
| src/Microsoft.Android.Build.Tasks/Tests/Microsoft.Android.Build.Tasks.Tests/TrimmableTypeMapRidCallbackTests.cs | Converts attributes and exception assertions. |
| src/Microsoft.Android.Build.Tasks/Tests/Microsoft.Android.Build.Tasks.Tests/TrimmableTypeMapIncrementalTests.cs | Converts attributes and parameterized cases. |
| src/Microsoft.Android.Build.Tasks/Tests/Microsoft.Android.Build.Tasks.Tests/TransferredReferenceActivationTests.cs | Converts parameterized cases to data rows. |
| src/Microsoft.Android.Build.Tasks/Tests/Microsoft.Android.Build.Tasks.Tests/TemporaryDirectoryTargetsTests.cs | Converts data rows and string assertions. |
| src/Microsoft.Android.Build.Tasks/Tests/Microsoft.Android.Build.Tasks.Tests/Tasks/EnvironmentBuilderTests.cs | Converts attributes and direct no-throw execution. |
| src/Microsoft.Android.Build.Tasks/Tests/Microsoft.Android.Build.Tasks.Tests/ResourcesTests.cs | Converts localization cases to data rows. |
| src/Microsoft.Android.Build.Tasks/Tests/Microsoft.Android.Build.Tasks.Tests/NativeToolTestHelper.cs | Converts filesystem assertions to MSTest. |
| src/Microsoft.Android.Build.Tasks/Tests/Microsoft.Android.Build.Tasks.Tests/NativeAotBootstrapTestTools.cs | Uses the application directory for repository discovery. |
| src/Microsoft.Android.Build.Tasks/Tests/Microsoft.Android.Build.Tasks.Tests/Microsoft.Android.Build.Tasks.Tests.csproj | Uses shared MSTest references. |
| src/Microsoft.Android.Build.Tasks/Tests/Microsoft.Android.Build.Tasks.Tests/GenerateCoreClrRuntimeProviderSourcesTests.cs | Converts attributes, assertions, and output handling. |
| src/Microsoft.Android.Build.Tasks/Tests/Microsoft.Android.Build.Tasks.Tests/FastDeployTypeMapTests.cs | Converts parameterized cases to data rows. |
| src/Microsoft.Android.Build.Tasks/Tests/Microsoft.Android.Build.Tasks.Tests/DesignTimeResourceTargetsTests.cs | Converts combinations to MSTest combinatorial data. |
| src/Microsoft.Android.Build.Tasks/Tests/Microsoft.Android.Build.Tasks.Tests/CreateAssemblyStoreTests.cs | Converts data rows, assertions, and output handling. |
| src/Microsoft.Android.Build.Tasks/Tests/Microsoft.Android.Build.Tasks.Tests/BaseTest.cs | Uses MSTest context, display names, and cleanup. |
| src/Microsoft.Android.Build.Tasks/Tests/Microsoft.Android.Build.Tasks.Tests/AssemblyInfo.cs | Enables method-level test parallelism. |
| src/Microsoft.Android.Build.Tasks/Tests/Microsoft.Android.Build.Tasks.Tests/AssemblyCompressionTargetsTests.cs | Converts parameterized cases to data rows. |
| Microsoft.Android-Tests.slnx | Removes the NUnit runner project. |
| Makefile | Removes obsolete NUnit test targets. |
| global.json | Selects MTP repository-wide. |
| eng/Versions.props | Adds the MTP retry-extension version. |
| eng/Version.Details.xml | Tracks the retry-extension dependency. |
| Documentation/workflow/DevelopmentTips.md | Updates device-test framework terminology. |
| Documentation/guides/OneDotNet.md | Removes NUnit entries from sample template output. |
| Documentation/building/windows/instructions.md | Updates host and device testing instructions. |
| Directory.Build.props | Updates a comment and removes the runner warning exemption. |
| Configuration.props | Removes NUnit versions and legacy analyzer exclusion. |
| build-tools/scripts/UpdateApkSizeReference.sh | Updates the test assembly path and filter syntax. |
| build-tools/scripts/UpdateApkSizeReference.ps1 | Replaces legacy execution with the built test assembly. |
| build-tools/scripts/TimingDefinitions.txt | Removes the NUnit result marker. |
| build-tools/scripts/NUnitReferences.projitems | Removes shared NUnit dependencies. |
| build-tools/scripts/nunit3-console.cmd | Removes the Windows NUnit console launcher. |
| build-tools/scripts/nunit3-console | Removes the Unix NUnit console launcher. |
| build-tools/scripts/MSTestReferences.projitems | Centralizes MSTest packages and MTP configuration. |
| build-tools/automation/yaml-templates/variables.yaml | Removes NUnit variables and converts category filters. |
| build-tools/automation/yaml-templates/stage-package-tests.yaml | Uses the MSTest execution template. |
| build-tools/automation/yaml-templates/stage-msbuild-emulator-tests.yaml | Switches to MTP slicing and retries. |
| build-tools/automation/yaml-templates/setup-test-environment.yaml | Removes test-slicer configuration forwarding. |
| build-tools/automation/yaml-templates/setup-test-environment-steps.yaml | Removes NUnit test-slicer installation. |
| build-tools/automation/yaml-templates/setup-test-environment-public.yaml | Removes test-slicer configuration forwarding. |
| build-tools/automation/yaml-templates/run-nunit-tests.yaml | Removes the NUnit execution template. |
| build-tools/automation/yaml-templates/run-msbuild-tests.yaml | Uses MSTest execution and slicing. |
| build-tools/automation/yaml-templates/kill-processes.yaml | Removes NUnit process cleanup. |
| build-tools/automation/yaml-templates/diagnose-hanging-processes.yaml | Removes NUnit-specific process diagnostics. |
| build-tools/automation/yaml-templates/build-windows-steps.yaml | Converts test-template calls and TRX arguments. |
| .github/skills/tests/SKILL.md | Updates the skill description to MTP terminology. |
| .github/skills/code-review/references/testing-rules.md | Replaces NUnit guidance with MSTest guidance. |
| .github/copilot-instructions.md | Updates the test-framework description. |
Review effort: Balanced
simonrozsival
force-pushed
the
simonrozsival-mstest-migrate-remaining-suites
branch
2 times, most recently
from
November 18, 2026 09:57
f49b6f5 to
8452e3f
Compare
Context: #13434 MSTest on Microsoft.Testing.Platform (MTP). Every other test project was still NUnit, so the repo carried two frameworks, two runners, TestRunner.Core, the NUnit console scripts, and dotnet-test-slicer. Migrate all remaining NUnit test projects to MSTest/MTP: * Java tooling: generator-Tests, Java.Interop.Tools.{Generator,JavaCallableWrappers,JavaSource,JavaTypeSystem,Maven}-Tests, Xamarin.Android.Tools.{Bytecode,ApiXmlAdjuster}-Tests * Android tooling: Xamarin.Android.Tools.{Aidl,AndroidSdk}-Tests, Microsoft.Android.Build.BaseTasks-Tests, Microsoft.Android.Sdk.Analysis.Tests * Build tasks: Microsoft.Android.Build.Tasks.Tests, Xamarin.Android.Build.Tests, Xamarin.ProjectTools * Device integration: MSBuildDeviceIntegration `build-tools/scripts/NUnitReferences.projitems` is replaced by `MSTestReferences.projitems`; versions come from `eng/Versions.props`. The root `global.json` selects the MTP runner. CI: `run-nunit-tests.yaml` and `run-sliced-nunit-tests.yaml` are replaced by `run-mstest-tests.yaml`, which runs the test app directly with `--report-trx`. Slicing uses `--list-tests` and `--filter-uid` with the same count-based round-robin split as dotnet-test-slicer, retries use `Microsoft.Testing.Extensions.Retry`, and NUnit filters are translated to MSTest filter syntax. StreamingTests now run the built Microsoft.Android.Run out of process under `dotnet test --test-modules`: with MTP 2.6, an in-process `TestApplication` using `--report-trx` relaunches the MSTest test app itself and recurses. LocalizationTests now restore `persist.sys.locale` to `en-US`. Remove the now unused TestRunner.Core, nunit3-console scripts, NUnit version properties, dead NUnit test sources and stale references. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…erts Restore the project-local global.json files for Mono.Android.NET-Tests and Xamarin.Android.JcwGen-Tests. Without them, the `_MTPBuild` MSBuild process that `dotnet test` spawns in apk-instrumentation.yaml resolved the 11.0.100-rc.1 Android workload manifest and failed with NETSDK1139. InstallWithoutSharedRuntime: NUnit's Assert.IsNotNull on the RuntimeInfo struct always passed. The MSTest conversion asserted on struct fields instead, which turned the no-op into a real check that fails on CI. Remove both always-true asserts to keep main's behavior. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CallerCancellationDuringVerificationPropagates: NUnit's CatchAsync accepts derived exceptions, but the conversion used ThrowsExactlyAsync, which rejects the TaskCanceledException that HttpClient throws. Use ThrowsAsync to keep the original semantics. DebuggerAttribute: the build directory did not include the runtime, so the CoreCLR and NativeAOT rows shared it. NUnit ran the rows one at a time; MSTest's method-level parallelism ran them concurrently and the builds collided. Add the runtime to the directory name. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Main added or moved NUnit tests in Aapt2Tests, LinkerTests, ManagedResourceParserTests, CollectNonEmptyDirectoriesTests, AndroidCreateDebugKeyTests, FilesTests and KotlinFixupsTests. Convert them to MSTest: add [TestClass]/[DoNotParallelize] to fixtures that had no attributes, use TestId/AppContext.BaseDirectory instead of NUnit's TestContext members, AddResultFile for attachments, and .ToArray () for CollectionAssert. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
ExtractTypeMapKeysFromNativeAotObject: #13515 changed RunObjdumpAsync to Process.RunAndCaptureTextAsync, which buffers stdout until the process exits. ObjdumpStreamsOutputBeforeProcessExit (added in #12988) checks that llvm-objdump output is streamed, so it failed on macOS and Linux. CI did not notice because Microsoft.Android.Build.Tasks.Tests only runs on Windows, where the test is skipped. Restore the ProcessUtils.StartProcess call that #12988 used. BindingBuildTest: the AndroidMavenLibrary_* tests all download androidx.core:core 1.9.0 into the same Maven cache. When they ran concurrently, one failed with "The process cannot access the file". Mark them [DoNotParallelize], like AndroidMavenLibrary. JavaMarshalGCBridgeTests: ManagedPeerCycleSurvivesJavaRootAndIsCollectedAfterRelease returned from WaitForBridgeRound on the first completed bridge round. A round that started before the cycle existed could complete first, leaving no GlobalToWeak events for the cycle (tracked in #13429). Wait for a round that emitted them. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Locally, Xamarin.ProjectTools creates test projects under bin/Test*, inside the git checkout, so SourceLink adds /sourcelink and embedded untracked files. With DebugType="" no PDB is emitted and csc fails with CS2044/CS2045 (ManifestTest.DebuggerAttribute). On CI the projects live outside the repo and SourceLink is inactive. Set EnableSourceControlManagerQueries=false in the generated Directory.Build.props so local and CI builds behave the same. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
simonrozsival
force-pushed
the
simonrozsival-mstest-migrate-remaining-suites
branch
from
November 21, 2026 13:15
630256a to
71d859e
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Context: #13434
#13434 (merged) moved the on-device suites (
Mono.Android.NET-Tests,Java.Interop-Tests,Xamarin.Android.JcwGen-Tests) andXamarin.SourceWriter-Teststo MSTest on Microsoft.Testing.Platform (MTP). Every other test project was still NUnit, so the repo had to carry two runners,TestRunner.Core, the NUnit console scripts,dotnet-test-slicer, and per-projectglobal.jsonfiles to select MTP.This PR migrates all remaining NUnit test projects, which means one framework and one runner for the whole repository.
Projects
generator-Tests,Java.Interop.Tools.{Generator,JavaCallableWrappers,JavaSource,JavaTypeSystem,Maven}-Tests,Xamarin.Android.Tools.{Bytecode,ApiXmlAdjuster}-TestsXamarin.Android.Tools.{Aidl,AndroidSdk}-Tests,Microsoft.Android.Build.BaseTasks-Tests,Microsoft.Android.Sdk.Analysis.TestsMicrosoft.Android.Build.Tasks.Tests,Xamarin.Android.Build.Testsand its sharedXamarin.ProjectToolslibraryMSBuildDeviceIntegrationbuild-tools/scripts/NUnitReferences.projitemsis replaced byMSTestReferences.projitems. It makes each project an MTP executable (EnableMSTestRunner) and referencesMSTest.*andMicrosoft.Testing.Extensions.TrxReportwith versions fromeng/Versions.props.MSTest.SourceGenerationis used everywhere exceptXamarin.Android.Build.TestsandMSBuildDeviceIntegration. Those two seeXamarin.Android.Build.Tasks' internalDynamicallyAccessedMemberTypespolyfill throughInternalsVisibleTo, which conflicts with the generated code (CS0433), so they use reflection discovery (UseMSTestSourceGeneration=false).The root
global.jsonnow selects MTP ("test": { "runner": "Microsoft.Testing.Platform" }). TheDOTNET_TEST_RUNNERenvironment variable is removed. The per-projectglobal.jsonfiles for the on-device suites (Mono.Android.NET-Tests,Xamarin.Android.JcwGen-Tests) are kept: without them, the_MTPBuildMSBuild process thatdotnet testspawns inapk-instrumentation.yamlresolved the 11.0.100-rc.1 Android workload manifest and failed with NETSDK1139.Removed:
tests/TestRunner.Core,build-tools/scripts/nunit3-console*, theNUnitVersion/NUnitConsoleVersionproperties, and the deadrun-nunit-tests/list-nunit-testsmake targets (RunTests.targetsno longer exists).Conversion notes
[TestCase]/[TestCaseSource]→[DataRow]/[DynamicData].TestCaseData.SetName()→TestDataRow<T>.DisplayName.[Values]→[CombinatorialData]/[CombinatorialValues].[ValueSource (…ValidRuntimeConfigurations)]→ a small[ValidRuntimeConfigurations]values provider.[Category]→[TestCategory],[Platform]→[OSCondition],[Explicit]→[Ignore]. NUnit[Retry (n)](n total attempts) → MSTest[Retry (n - 1)](n - 1 retries).Xamarin.Android.Build.TestsandMicrosoft.Android.Build.Tasks.Testsrun method-level parallel. Classes that were not[Parallelizable (Children/All)]are[DoNotParallelize].MSBuildDeviceIntegrationstays assembly-wide[DoNotParallelize].BaseTestuses the MSTestTestContext.TestNamecomes fromTestDisplayName, so every data row still gets its own directory. Outcome checks useCurrentTestOutcome, and attachments useAddResultFile().[OneTimeSetUp]/[OneTimeTearDown]→[ClassInitialize]/[ClassCleanup]withInheritanceBehavior.BeforeEachDerivedClass.TimeZoneInfoTestsandLocalizationTestsbecome abstract bases with…CoreCLRTests/…NativeAOTTestssubclasses. Each subclass installs its app lazily, once per runtime.BundleToolTestsandBundleToolNoAbiSplitTeststake the runtime as a[DataRow].CI
run-nunit-tests.yamlandrun-sliced-nunit-tests.yamlare replaced byrun-mstest-tests.yaml. It runs the test app directly (dotnet <dll> --report-trx) and publishes its TRX. A failing run marks the step SucceededWithIssues, andfail-on-issue.yamlthen fails the job, the same as before.--list-tests jsonlists the selected tests. The job takes every$(System.TotalJobsInPhase)th test, sorted by UID, and passes those UIDs in a--filter-uidresponse file. This replacesdotnet-test-slicer, which only understands NUnit. It is the same count-based round-robin split the slicer did: our pipelines never passed it a--balance-file, so it weighed every test equally, and neither approach balances by test duration, so slice sizes are unchanged.MSBuildDeviceIntegrationreferencesMicrosoft.Testing.Extensions.Retry, new ineng/Versions.propsandeng/Version.Details.xml. The lanes that used the slicer's auto-retry now pass--retry-failed-tests 1and publish every attempt fromRetries/.Name = BuildTest→ClassName=Xamarin.Android.Build.Tests.BuildTest(also forPackagingTest,XASdkTests,AndroidDependenciesTestsandSystemApplicationTests).method == X→Name~X.cat != A & …→TestCategory!=A&…(ExcludedNightlyNUnitCategoriesis renamed toExcludedNightlyTestCategories).dotnet test --project … --report-trx, becauseDotNetCoreCLI@2injects--logger trx, which MTP rejects. Therun-dotnet-preview.yamlcallers switch to--report-trx.Results
Rebased onto main (bfda00d). All 17 migrated projects build with
-p:_AndroidTreatWarningsAsErrors=true.Standalone suites, MTP run on this branch (passed/failed/skipped):
The four
AndroidSdk-Testsfailures (Constructor_SetValuesFromPath,CreateProcessStartInfo_*) are environmental on my machine and fail the same way under NUnit.Comparison with main's NUnit suites:
BaseTasks-TestsandAndroidSdk-Testswere run in a clean worktree. Totals are 170 and 459 (NUnit--list-tests), the same as MTP, with identical per-method case counts.[Values]that become rows (AndroidUpdateResourcesTest+3,TrimmableTypeMapBuildTests+1,ExtractTypeMapKeysFromNativeAotObjectCoverageTests+2,BundleTool*Tests+1 each,DebuggingTest+1).Xamarin.Android.Build.Tests1853,MSBuildDeviceIntegration4533.Device runs before the rebase, with a local Release SDK on an API 35 arm64 emulator:
Xamarin.Android.Build.TestsTestCategory=SmokeTestsAssert.Inconclusive)UncaughtExceptionTests,BundleToolTests,WebViewJavascriptInterfaceTests(with--retry-failed-tests 1)LocalizationCoreCLRTests.CheckLocalizationIsCorrectNode15(thenpersist.sys.localeis back toen-US)ar-KM: the app didn't start within the logcat window after the reboot, all 3 attempts)InstallTests(UID slice)Discovery differs from NUnit only in naming. Rows that NUnit named with
TestName =/SetName()now appear under their method name with aDisplayName, and fixture-parameterized classes appear under their new subclass names.The
run-mstest-tests.yamlscript was exercised locally withpwsh: slicingXamarin.Android.Build.Tests3 ways (8/8/8 of 24), plus a 4-way dry run over the 1182 TimeZone tests. All pipeline YAML parses.Notable conversions
AndroidSdk-Tests'StreamingTests(added on main) built an in-process MTPTestApplicationwith--report-trx. Under the MTP 2.6 that MSTest brings in,--report-trxalways makes the app a test-host controller, which relaunches the entry assembly (the MSTest test app itself) and recurses. They now run the builtMicrosoft.Android.Rununderdotnet test --test-modules, which is how Android test projects run it, with a fake adb replaying anam instrumenttranscript. This also tests the MTP 2.2.1 thatMicrosoft.Android.Runships with. Cancellation is driven by the fake adb sending SIGINT toMicrosoft.Android.Run. The missing-adb case uses a non-executable adb, becauseMicrosoft.Android.Runrejects a missing path before MTP starts. The two parser/state tests stay in process.LocalizationTestsrestorepersist.sys.localetoen-USin a[ClassCleanup]. The NUnit version never restored it, so later suites on the same device ran in the last tested locale.AndroidComputeResPathsTestsandResolveSdksTaskTestsdropped main's no-opAfterAllTestsoverrides, becauseAfterAllTestsis now a static[ClassCleanup]. The only effect is an extraadb kill-serveron Windows after those classes.UncaughtExceptionTests'Assert.Multiple(no MSTest equivalent) collects the property-restore failures and asserts once.Known issues
Xamarin.Android.Build.TestsandMSBuildDeviceIntegrationsuites were not run locally; they take many hours. CI will be the first full run.AndroidUpdateResourcesTest.LightlyModifyLayoutdrops NUnit's[Repeat (3)]; it runs once.ZipArchiveExTests'[Repeat (100)]becomes an explicit loop.[Explicit]tests become[Ignore]. MSTest has no opt-in-only equivalent, so they no longer run when selected explicitly by name.