søndag 8. juli 2012

Overview of how the Click Test DSL is implemented

One of the main goals, while implementing this framework, was to make it as readable as possible, also for non-developers. Therefore most objects and methods have names which try to describe the GUI and the operations from a testers viewpoint, instead of using terms more common to developers.
Our DSL/framework has been developed as a set of Gui-object classes. They represent objects like: Button, Label, TextBox, DataGrid, DataGridCell, etc. All of these classes contain the commonly used operations and validations performed on these objects. Some of the objects also has a class whose name is pluralized; these classes return collections of the different types, like Labels, TextBoxes, etc. The pluralized classes mainly contain validations like AllShouldBeEmpty/HaveContent or AllShouldBeEnabled/Disabled.
All the object classes were created to hide away as much as possible of the MS UI Automation code and code which tries to simulate that it is a user doing these operations.
An example of an operation that simulates users waiting for the applicaion to be ready is the button click method: After actually clicking the button, the method will call Process.WaitForInputIdle() which waits for the application process to enter an idle state.
The actual MS UI Automation code is found in extension methods under AutomationCode, and it is these methods the object classes use. The MS UI Automation code is more or less just different ways to search for the correct AutomationElement, i.e. the actual element in the current application, or the correct pattern, like InvokePattern, ValuePattern or SelectionItemPattern.

Some more concrete samples:

GuiButton

The method Button("") in the superclass hides a call to a static method in the GuiButton class. That method runs the correct UI Automation searches and returns an object of the GuiButton class. This class then contains the AutomationElement for the button. The buttons have different action/validations methods:
  • The Click method will do a search for the stored AutomationElements InvokePattern, invoke the pattern and wait for the process to become idle.
  • The ShouldBeEnabled method will assert that its AutomaionElement.Current.IsEnabled property is true.
  • The ShouldBeVisible method will assert that its AutomaionElement.Current.IsOffscreen property is false.

 GuiCheckBox

The method CheckBox("") works in the same way as the Button("") method. The GuiCheckBox class contains the AutomationElement of the checkbox and the TogglePattern for it.
  • IsChecked / IsUnchecked / IsIndeterminate returns the
    AutomationElement.Current.ToggleState == ToggleState.On / Off / Indetermindate.
  • ShouldBeChecked / ShouldNotBeChecked asserts that the checkbox is in the correct state
  • Check and UnCheck executes the TogglePattern.Toggle method until the checkbox is in the correct state. So in practice, they cycle through all the possible states of the checkbox.

GuiComboBoxes

The pluralized version of the class for comboboxes inherits from List, and its static call will return all comboboxes whose identifier starts with a certain prefix. This class contains the following methods:
  • ShouldShow(params string[] values) which will assert that all the supplied values is the selected item of at least one combobox on the screen.
  • CountShouldBe(int expectedcount) asserts that the number of comboboxes is as expected.
  • SetValues(string[] selectValues) loops through the selectValues and sets them as the selected item in the different comboboxes. Note, there must be at least as many comboboxes as there are values to select.

Overview of how to use the Click Test DSL

The click-test framework uses inheritance to make all methods available to the actual tests.
When starting to use the framework we recommend creating a super-class, for your project, that inherits from UiTestDslCoreCommon. This class should have property shortcuts for often used objects, and methods for UserControls created specially for your project. In our project this class has properties for the most commonly used datagrids, buttons without identifying text, some specialized usercontrols, and common operations like changing to a specific tab. This class also holds the StartApplicationAndLogin-method which is set to be the [TestInitialize] method.
The actual test classes inherit from the super class. This ensures that, when the test is about to start, the application is started and the user is logged in.
After the user is logged in, our tests then follow this pattern:
  1. Change to the tab they are going to test
  2. Setup of the specific test context
  3. Perform the actions to test
  4. Optional validation (failed validations will trigger a screenshot)
  5. Close the application (Framework)
There is some validation built into the framework, so the tests won't always need explicit validation. These validations will detect unexpected exceptions by checking for open dialogs when the application closes. If any such dialogs are found, the framework will take a screenshot of the application and fail the test. We have set the DispatcherUnhandledException in our applications App.xaml to log the error and show a dialog with "Fatal error" for uncaught exceptions.
Another useful pattern is to start with a call to the method CreateNewUniqueIdentifier which assigns a new Guid to the property UniqueIdentifier. This text-string is then unique to the specific test run. We use this to test our automatching features, and to validate saving and retrieving of strings to our persistent data layer.
The actions and validations typically take the form <GuiObject>("<Identifier>").<Action>(...); where GuiObject is Button, Label, TextBox, etc; Identifier is the caption of the button, or name of the component; Action can be actions like Click or Type, or a validation call like ShouldBeEnabled, RowCountShouldBe or ShouldRead.
The Identifier of an object (other than captioned buttons) can be somewhat tricky to find. If you look in the code, it is most often the Name or AutomationId property of the object. Although this is not our prefered method, we prefer using the Inspect Objects-program from the Windows SDK download. Snoop will also show the AutomationProperties.AutomationId. Both of these applications allow us to find the identifier of a component in a running application.
Note: some action calls contain WaitWhileBusy and Thread.Sleep calls to allow the GUI time to respond to the simulated user interaction. This is something we quickly learned that was something we had to implement. Computer interaction is many orders of magnitude faster than human interaction, in addition to the fact that humans will normally wait for the screen to update after pushing a button. A computer will only do that if you tell it to.
Although we have implemented the most common actions and validations in the framework, others will surely miss some functionality. Since it now is a open source, anyone is invited to expand it with methods they feel should be there.
Please see my previous blog post for a code sample showing a complete test.

The framework also contains some control classes, which are used to start/stop the application, to find and run all tests if started as a command line executable, and a class that takes screenshots. Some of these classes require some setup, but after that you most likely will not look at them again. This setup mainly consists of setting path and filename for the executable, any code you would like to run before all tests, before or after each test, and adding naming options. The naming options we have added mostly consist of mappings between button captions to WPF Command Binding texts, or localized to English versions of the captions. The SampleProgram.cs in the github repository shows most of these.