iPhone test development with UISpec

iPhone apps are becoming more complex, which means rigorous testing is needed throughout the development cycle. You can get away with manual exploratory testing, but we all know this is error prone. Therefore, you need to rely on automated tests that validates your core functionality executed regularly, in additions to performing exploratory testing.

The iPhone SDK already comes with UIRecorder, which is part of the Instruments app. However, from my brief experience with it I found it not very flexible, and does not offer scripting capability.

This is where UISpec comes in play.

UISpec seems to offer scripting capability using Objective-C, and it's open source. Check it out link. I will write up a more detailed review in the upcoming days.

Also checkout this short video which demonstrate an automated test using UISpec.

Design Automation system with states for debugging purpose

It is very important to design your automation system with debugging in mind. For example, you have a an automated suite that runs in sequence, and you needed to make changes in the last test or even add a new test; if you don't design your system correctly, you will end up running through your entire test in order to debug your test under development.

Thus, the solution is to identify states in your system, which makes each of the test of test suite be dependent on any of these states.

For example, we could have the following states in our system:

• StartUpState
• LoggedInState
• FileOpenState

This way, each test is independent, and you no longer are required to run the entire suite in sequence. This will make debugging and maintenance a breeze in comparison to create a sequential test suite. This concept is used heavily in SILK, with the Base State concept.

The interesting part is how to implement this concept taking advantage of object oriented language such as C#? I will cover this in my next post, stay tuned!

Agile Testing and Test Automation

Agile Testing and Test Automation goes hand in hand; without automated tests, agile testing becomes almost impossible.

Agile testing is all about evaluating software changes as soon as possible and as often as possible during the software release cycle. The key success to agile testing is to automate the tests that are executed most often, and in most cases the smoke test and the priority-1 tests are executed most often.

The way to introduce agile testing in an organization the first step is to put together a plan to move toward the agile testing approach. The plan will need to identify specific tasks and goals in meeting this goal, which is to test early and often.

A plan may look like this:-

1. Identify system smoke test, and system priority-1 tests.
2. Automate System smoke test and system priority-1 tests.
3. Identify area specific smoke test, and area specific priority-1 tests.
4. Automate area specific smoke test, and area specific priority-1 tests.

In large systems, critical areas within the system gets to be automated first.

The result will be a repository of automated set of tests that will provide evidence that the software is OK from the testing side. Of course, prior to running this tester level automated tests, a set of developer level automated tests is run to provide evidence that the software is OK from the developer side. Both automated tests, tester tests and developer tests are executed together. This what makes agile testing possible.

Using The Microsoft UI Automation Library to Drive UI Automation

The UI Automation Library is included in the .NET Framework 3.0. The concept seems to be straightforward. Every UI control is an "AutomationElement" which can be found through searching from the Parent "AutomationElement". The top AutomationElement is defined as the "RootElement", which is the Desktop. Another way to get the AutomationElement Object to a given Window is by starting a "Process", and then using the Process MainWindowHandle to get the AutomationElement object.

The code shown below is basically a test that Invoke Notepad, Enters an input text, and then it compares with the text which is read from Text property of the Notepad Text Field, and then closes Notepad without saving.

using System;
using System.Windows.Automation;
using System.Diagnostics;
using System.Threading;
using System.Collections.Generic;
using System.Text;

namespace UIAuto_01
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Starting Notepad!!!");

            // #1 Start Notepad
            //
            Process notepadProcess = Process.Start("notepad.exe");
            Thread.Sleep(500);
            AutomationElement aeDesktop = AutomationElement.RootElement;

            // #2 Find Notepad
            //
            AutomationElement aeNotepad = AutomationElement.FromHandle(notepadProcess.MainWindowHandle);

            if (null == aeNotepad)
            {
                throw new Exception("Notepad not found!");
            }
            Console.WriteLine("Found Notepad !!!!");

            // #3 Find Notepad Text Field Control
            //
            AutomationElementCollection aeAllTextBoxes =
                                    aeNotepad.FindAll(TreeScope.Children,
                                    new PropertyCondition(AutomationElement.ControlTypeProperty,
                                    ControlType.Document));

            if (0 == aeAllTextBoxes.Count)
            {
                throw new Exception("Notepad Edit field not found!!");
            }

            Console.WriteLine("Found Notepad Edit Field!!!!");

            AutomationElement aeNotepadDoc = aeAllTextBoxes[0];

            // #4 Type the input value
            //
            aeNotepadDoc.SetFocus();
            string InputValue = "Hello World!";
            System.Windows.Forms.SendKeys.SendWait(InputValue);

            // #5 Get the Text from the Text Field
            //
            TextPattern tpNotepadDoc = (TextPattern)aeNotepadDoc.GetCurrentPattern(TextPattern.Pattern);
            string ActualValue = tpNotepadDoc.DocumentRange.GetText(-1).Trim();

            // #6 Compare input and actual
            //
            if (ActualValue == InputValue)
            {
                Console.WriteLine("Test Passed!");
            }
            else
            {
                Console.WriteLine("Test Failed: Expected={0} Actual={1}", InputValue, ActualValue);
            }

            // #7 Now close Notepad
            //
            WindowPattern wpNotepad = (WindowPattern)aeNotepad.GetCurrentPattern(WindowPattern.Pattern);
            wpNotepad.Close();

            // #8 Find The Save, Don't Save, Cancel Dailog
            //
            AutomationElement aeSaveDailog = findElement(aeNotepad, "Notepad", 10);
            if (null == aeSaveDailog)
            {
                throw new Exception("Notepad Save dailog was not found!");
            }

            // #9 Find the "Don't Save" button and Click it
            //
            AutomationElement aeDontSaveBtn = findElement(aeSaveDailog, "Don't Save", 10);
            if (null == aeDontSaveBtn)
            {
                throw new Exception("Notepad Don't Save Button was not found!");
            }
            InvokePattern ipDontSaveBtn = (InvokePattern)aeDontSaveBtn.GetCurrentPattern(InvokePattern.Pattern);
            ipDontSaveBtn.Invoke();

            // #10 Done
            //
            Console.ReadLine();
        }

        // Returns a Automation Element based on the parent AutomationElement, and caption.
        //
        private static AutomationElement findElement(AutomationElement parent, string caption, int timeout)
        {
            AutomationElement aeNew = null;
            int numWaits = 0;
            do
            {
                aeNew = parent.FindFirst(TreeScope.Children,
                                         new PropertyCondition(AutomationElement.NameProperty, caption)
                                         );
                ++numWaits;
                Thread.Sleep(100);
            } while (aeNew == null && numWaits < timeout);
            return (aeNew);
        }
    }
}

Scripting with C# Script

I have been playing cscript recently and I have found it a better alternative to Jscript. With C# Script you have the power of C# Language with the scripting capability. Hence, you don't have to compile your code when making modification to it.

You also have the option to create an executable out of the script.

Here is an example of accessing COM from C# Script. Notice the include tag on the first line

//css_prescript com(SYSINFO.SysInfo.1, SisInfoLib);
using System;
using SisInfoLib;
class Script
{
[STAThread]
static public void Main(string[] args)
{
SysInfoClass sysInfo = new SysInfoClass();
switch (sysInfo.ACStatus)
{
case 0:
Console.WriteLine("Not using AC power");
break;
case 1:
Console.WriteLine("Using AC power");
break;
default:
Console.WriteLine("Unknown AC power status");
break;
}
if (sysInfo.BatteryLifePercent != 255)
Console.WriteLine("Battery life " + sysInfo.BatteryLifePercent + "%");
else
Console.WriteLine("Battery charge status not known");
}
}