Support Xamarin.UITest

[PureWeen]

Archive.zip

The attached project is just a basic sample created from the VS MAC Xamarin.Forms Templates. I've added a cake script so you can run the tests from the command line

Command line

Make sure to startup a single device

dotnet tool install --global Cake.Tool
dotnet cake

Visual Studio

You can also open up the SLN file and run the test via the test runner in VS but just make sure to deploy the app first.

The attached zip currently only works on Android because Xamarin.UITest is broken for Xcode 12.

Once they fix Xcode 12 support I'll update the zip file
microsoft/appcenter#2050

[premun]

@PureWeen can you please show me where is the code for the Cake tasks in build.cake? Things like ListAppleSimulators or LaunchiOSApplication

[mandel-macaque]

After taking a quick look, these are the possible steps that we need to take to be able to enable the Xamarin.Forms team to move foward:

iOS

• Verify that the UITestsApplications are correctly launched.
• Identify the communication channel used by calabash-ios.
• Modify calabash-ios to use the TCP tunnel.
• Orchestrate the execution of the nunit console runner once the application has launched and connected to the TCP tunnel.

Android

• Verify that the UITestsApplications are correctly launched.
• Identify the communication channel used by calabash-android.
• Add support for a tcp tunnel in android to perform the communication. This feature is not present in android, but should be easy to implement via adb: adb forward tcp:6100 tcp:7100 or adb reverse tcp:3000 tcp:3000 Either one works, the smart things to do is to copy the behaviour on iOS to simplify any calabash change and ensure both platforms connect in the same way.
• Modify calabash-android to use the TCP tunnel.
• Orchestrate the execution of the nunit console runner once the application has launched and connected to the TCP

There have been ideas of integration the execution of the nunit runner within xharness vias the NUnit library, there are some thoughts about it:

• Will simplify the build story.
• Will simplify the deployment story.
• Process management is hard to do right.

The problem with this approach is that ATM we do not believe that calabash + XamarinUI tests will be the solution, and others are in the radar, like apium. If that is the case, we should not invest any effort on integrating the runner within xharness because we will have to move to the apium runner and that will be another process. Adding a solution that just calls a process once the app is running might be the best approach knowing the possible future move.

[PureWeen]

@premun

https://github.com/cake-contrib/Cake.AppleSimulator

It looks like most of the commands are built here
https://github.com/cake-contrib/Cake.AppleSimulator/blob/beb5e9b60b4d8ea5bbc706d90f67979c634386f0/src/Cake.AppleSimulator/SimCtl/SimCtlRunner.cs#L107

And then they are fed to xcrun
https://github.com/cake-contrib/Cake.AppleSimulator/blob/beb5e9b60b4d8ea5bbc706d90f67979c634386f0/src/Cake.AppleSimulator/XCRun/XCRunTool.cs#L82

Pinging @Redth in case he might have additional nuggets of info

The Cake.AppleSimulator bits are purely for spinning up simulators and deploying the app and have nothing to do directly with running the tests. If xHarness can already do all of that then we don't really need to use the Cake.AppleSimulator tools.