TestDriven.Me

I hope to spend some time discussing testing with VB.NET. I don't think of myself as an expert, but I have had some experience and certainly some opinions. I will try to share some of those opinions along with some information and hopefully generate some discussion so that we all can improve our skills a bit.

This Blog

Syndication

Search

Navigation

March 2008 - Posts

How to Use WatiN to Test Web Pages - Part 1

As I have mentioned elsewhere, you can drive WatiN tests with either an application or a test framework. Examples of each will be given in this section, even though most developers seem to prefer a test framework. The TestDriven.NET add-in for the Visual Studio IDE makes using a test framework even easier.

Testing with an Application

With either a console or a Windows Form application the developer will have to provide certain utilities that are built into test frameworks such as NUnit for determining the success or failure of tests and also for displaying the test results. The MyAssert and Logger classes have been written to provide these services. Full source code for these classes is provided at the end of this section. You are welcome to use them or modify them as needed if you choose to control your tests with an application.

Create a new Application

To begin, create a new project, selecting the Console Application template. You will need to select a name for the application and a place to create it as shown next:

Clicking the OK button will create a bare project like that shown next:

I like to immediately rename Module1.vb to MainModule.vb so it is clear which the controlling module is.

Add References and Utilities

Right click on the project name (WatiNConsoleApplication) and select AddReference from the menu. Add the WatiNCore.DLL from the Bin folder that you installed WatiN into.

The next step is to copy the source of MyAssert.vb and Logger.vb from wherever you have it saved on your drive into the project folder. Then right click on the project name in Solution Explorer and select Add – Existing Item as shown next:

NOTE: If you have not yet saved copies of the utilities locally, you can just use the Add – Class context menu on your solution and then copy and paste the code into the classes as you create them in this solution.

At this point you should be able to get a clean build (Alt-Shift-B). Not very useful yet, but it helps to build and test often so you know that something you just did is either the cause of any problems you encounter, or it caused an earlier created problem to become visible.

Create Modules to be Tested

Let’s start the testing process by creating a class that will contain math functions. Everyone knows what math functions should do, so they will be easy to create tests for. Normally, you would want to write the unit test prior to creating the class, but since this section is focused on how to use the tools, it will be easier to understand if we have a class to test already completed.

Right click on the project name in your solution and select Add – Class. Create a class called MathFunctions.vb.

In the new class enter the GetPercentage function we examined in the Introduction section:

    Public Function GetPercentage(ByVal whatNumber As Double, _
ByVal whatDivisor As Double) As Decimal
        Dim answer As Decimal
 
        answer = (whatNumber / whatDivisor) * 100
         Return answer
    End Function
 

Create Unit Tests

Create a new class that will contain the unit tests for the math functions. Right click on the project name and select Add – Class. Create a class called MathFunctionsTest.vb.

In the new class enter the following test, which is very similar to the NUnit test we used earlier.

    Public Sub TestGetPercentage()
        Dim answer As Decimal
        Dim numberOne As Double
        Dim numberTwo As Double
        Dim myMathFunctions As New MathFunctions
         numberOne = 2.0
        numberTwo = 4.0
        answer = myMathFunctions.GetPercentage(numberOne, numberTwo)
  MyAssert.IsTrue("TestGetPercentage", answer = 50D, _
"GetPercentage returned " & _
            answer.ToString & " when provided " & _
            numberOne.ToString & " and " & numberTwo.ToString & _
            " when expected 50", False)
    End Sub

Once again Alt-Shift-B should get you a clean build, but the test still will not run, because it has not been called from the main module. Modify Sub Main() of the Main Module as shown next:

    Sub Main()
        Dim myMathFunctionsTest As New MathFunctionsTest
 
        myMathFunctionsTest.TestGetPercentage()
    End Sub

Now the program runs just fine, but you have no idea if the test failed or not (unless you remembered from looking at the code in MyAssert.vb and Logger.vb that an error log called ErrorLog.txt would be written to the C:\AppData\ folder if an assert fails). You could look for that, but hopefully it is not there from which you can infer that there were no errors. The missing file still does not give a warm fuzzy feeling that everything is all right though.

This points out another of the drawbacks of the console app. You have to put in some extra logic just to make your logging efforts more useful and more visible. Once we add a few lines to both TestGetPercentage and Sub Main, they look like the following:

    Public Sub TestGetPercentage()
        Dim answer As Decimal
        Dim numberOne As Double
        Dim numberTwo As Double
        Dim myMathFunctions As New MathFunctions
 
        m_logger.LogResultToTextFile("TestGetPercentage Test Start" & _
            StrDup(32, "-"), True, True)
 
        numberOne = 2.0
        numberTwo = 4.0
        answer = myMathFunctions.GetPercentage(numberOne, numberTwo)
        MyAssert.IsTrue("TestGetPercentage", answer = 50D,
           "GetPercentage returned " & _
            answer.ToString & " when provided " & _
            numberOne.ToString & " and " & numberTwo.ToString & _
            " when expected 50", False)
 
        m_logger.LogResultToTextFile("TestGetPercentage Test End " & _
      StrDup(32, "-"), True, True)
 
    End Sub
 
    Sub Main()
        Dim myMathFunctionsTest As New MathFunctionsTest
 
        'Open test log at start of test
        m_logger.LogResultToTextFile("Test Run Begins ", False, True)
 
        myMathFunctionsTest.TestGetPercentage()
 
        m_logger.LogResultToTextFile("Test Run Ends ", True, True)
 
        m_logger.ShowLogs() 
    End Sub

Now when you run the test, you are rewarded with the following file displayed in NotePad:

================================

Wednesday, February 27, 2008 2:36 PM

Test Run Begins 

================================

Wednesday, February 27, 2008 2:36 PM

TestGetPercentage Tests Start --------------------------------

================================

Wednesday, February 27, 2008 2:36 PM

TestGetPercentage Tests End --------------------------------

================================

Wednesday, February 27, 2008 2:36 PM

Test Run Ends

You can see that the test run began, it called the TestGetPercentage module and everything closed up with no error message given. This is rather satisfactory, but if we refactor the code just a little bit, it will be much more extensible for adding more tests:

In the MathFunctionsTest class, add the following functions at the top of the class prior to the current TestGetPercentage subroutine.

    Public Sub RunTests()
 
        TestSetup()
 
        'Call each test that will be run within the class here
        TestGetPercentage()
 
        TestTeardown()
 
    End Sub
 
    Sub TestSetup()
 
        'Open test log at start of test
        m_logger.LogResultToTextFile("MathFunctionsTest Tests Begin", _
True, True)
 
    End Sub
 
    Sub TestTeardown()
 
        m_logger.LogResultToTextFile("MathFunctionsTest Tests End ", _ 
True, True)
 
    End Sub

In the Main Module modify the Sub Main and add the following methods as shown next:

    Sub Main()
        Dim myMathFunctionsTest As New MathFunctionsTest
 
        TestSetup()
 
        myMathFunctionsTest.RunTests()
 
        TestTeardown()
 
    End Sub
 
    Sub TestSetup()
 
        'Open test log at start of test
        m_logger.LogResultToTextFile("Test Run Begins ", False, True)
 
    End Sub
 
    Sub TestTeardown()
 
        m_logger.LogResultToTextFile("Test Run Ends ", True, True)
 
        m_logger.ShowLogs() 
    End Sub

By adding the TestSetup and TestTeardown methods to each class, you can segregate code that needs to be run for all of the tests in the class. The RunTests method in the MathFunctionsTest and future test classes gives you an entry point to call within the class from Sub Main without having to reference every test in the class. That is what RunTests will do as you add further tests to the class.

Running the application now should provide you with the same display of tests started and ended with no errors. Now for the fun. Let’s create a test that will fail, highlighting one of the limitations of our GetPercentage function.

Add the following subroutine to MathFunctionsTest:

    Public Sub TestGetPercentageBadData()
        Dim answer As Decimal
        Dim numberOne As Double
        Dim numberTwo As Double
        Dim myMathFunctions As New MathFunctions
         m_logger.LogResultToTextFile("TestGetPercentage Test Start" & _ 
StrDup(32, "-"), True, True)
 
        numberOne = 2.0
        numberTwo = 0.0
        answer = myMathFunctions.GetPercentage(numberOne, numberTwo)
        MyAssert.IsTrue("TestGetPercentage", answer = 50D,
"GetPercentage returned " & _
            answer.ToString & " when provided " & _
            numberOne.ToString & " and " & numberTwo.ToString & _
            " when expected 50", False)
 
        m_logger.LogResultToTextFile("TestGetPercentage Tests End" & _
StrDup(32, "-"), True, True)
 
    End Sub

And then modify the TestStart sub in MathFunctionsTest to call this new sub after the original test:

    Public Sub RunTests()
 
        TestSetup()
 
        'Call each test that will be run within the class here
        TestGetPercentage()
        TestGetPercentageBadData()
 
        TestTeardown()
 
    End Sub

This will build just fine, but you will get a run-time error like that shown next when you try to run it.

Our tests have shown that the GetPercentage function is not protected against a divide by zero error. Also, the console application cannot log the error and continue with the tests once it reaches a run-time error. Time to break out the requirements for the project. Should the GetPercentage function trap a divide by zero and other errors that might occur or should it just throw an error when the developer tries something that should not be done.

You will find a good number of developers on each side of this question. One side will argue for Try…Catch block in the GetPercentage function to ensure that it does cause a system crash when it receives bad data. The other side will argue that the crash is good, because it shows that there really is a bug somewhere else that is sending the zero in the first place. Adding the Try…Catch block will mask the real error and it may never be corrected!

Our tests have shown that either the GetPercentage function needs to be refactored to have a Try…Catch block, or the tests themselves need to have a Try…Catch block added around each call to the actual method being tested. We are going to suggest not masking the real error and force it to be caught by a later test, so our change to the TestGetPercentageBadData is shown next:

    Public Sub TestGetPercentageBadData()
        Dim answer As Decimal
        Dim numberOne As Double
        Dim numberTwo As Double
        Dim myMathFunctions As New MathFunctions
 
        m_logger.LogResultToTextFile("TestGetPercentage Test Start" & _
StrDup(32, "-"), True, True)
 
        numberOne = 2.0
        numberTwo = 0.0
        Try
            answer = myMathFunctions.GetPercentage(numberOne, _
numberTwo)
        Catch ex As Exception
            m_logger.LogResultToTextFile("TestGetPercentageBadData" & _
" threw a divide by zero error as expected ", & _
True, False)
        End Try
        MyAssert.IsTrue("TestGetPercentage", answer = 0D, _
"GetPercentage returned " & _
            answer.ToString & " when provided " & _
            numberOne.ToString & " and " & numberTwo.ToString & _
            " when expected 0", False)
 
        m_logger.LogResultToTextFile("TestGetPercentage Tests End " & _
StrDup(32, "-"), True, True)
 
    End Sub

Now running the application provides a nice clear report of no errors.

This test is quite a change from the original NUnit test shown earlier, however. This points out the other drawback of using a console application to drive unit tests; the tests are harder to write and not as clear. In that entire sub there are really only two lines important to the test itself, the call to GetPercentage and the assert. The rest is there to make the test run well and to be nice about showing us if it passed or failed.

As an exercise, the reader is invited to complete the coverage of the unit tests for the GetPercentage function by adding tests in the MathFunctionsTest class for sending one or both parameters as negative numbers and also as Nothing to see how the function will behave.

At this point, it seems safe to recommend using NUnit as the framework to run unit tests.

Source Code for Console Application Utilities

The following source is provided for the utilities required to make tests run correctly and report their results from console applications. They are provided, as is, and you are free to include them in your own applications, or to modify them as you see fit:

Logger.vb

This utility is provided to assist in logging test results to two different files. One file will contain detailed information about the tests, which is frequently desired by Quality Assurance testers to ensure that all required test steps are run and to keep track of information about data and the internal states of each test. The second file is for errors and minimal navigational information about which tests are being run.

''' <summary>

''' Create and maintain logs of what happens during a test run.  

''' Two logs will be created, one containing only errors with test start/stop messages,

''' and the other more detailed with indications of all actions taken.

''' </summary>

''' <remarks></remarks>

Public Class Logger

    Private ErrorLogPath As String = "C:/TestData/"

    Private TestLogName As String = ErrorLogPath & "TestLog.txt"

    Private ErrorLogName As String = ErrorLogPath & "ErrorLog.txt"

    ''' <summary>

    ''' Save the error message to the currently defined text file.

    ''' </summary>

    ''' <param name="errorMessage"></param>

    ''' <param name="appendFile"></param>

    ''' <remarks>ErrorLogName contains the name of the currently defined error log file.

    ''' </remarks>

    Public Sub LogResultToTextFile(ByVal errorMessage As String, ByVal appendFile As Boolean, ByVal isError As Boolean)

        Try

            If Dir(ErrorLogPath).Length = 0 Then

                System.IO.Directory.CreateDirectory(ErrorLogPath)

            End If

            If Dir(TestLogName).Length = 0 Then

                Dim textWriter2 As New System.IO.StreamWriter(TestLogName)

                With textWriter2

                    .Write(" ")

                    .Flush()

                    .Close()

                End With

            End If

            If Dir(ErrorLogName).Length = 0 Then

                Dim textWriter2 As New System.IO.StreamWriter(ErrorLogName)

                With textWriter2

                    .Write(" ")

                    .Flush()

                    .Close()

                End With

            End If

            Dim textWriter As New System.IO.StreamWriter(TestLogName, appendFile)

            With textWriter

                .Write(StrDup(32, "=") & vbCrLf)

                .Write(Now.ToLongDateString & " " & Now.ToShortTimeString & vbCrLf)

                .Write(errorMessage & vbCrLf)

                .Close()

            End With

            If isError Then

                Dim errorWriter As New System.IO.StreamWriter(ErrorLogName, appendFile)

                With errorWriter

                    .Write(StrDup(32, "=") & vbCrLf)

                    .Write(Now.ToLongDateString & " " & Now.ToShortTimeString & vbCrLf)

                    .Write(errorMessage & vbCrLf)

                    .Close()

                End With

            End If

        Catch ex As Exception

            MsgBox("There was a system error writing to one of the logs." & vbCrLf & vbCrLf & ex.Message & vbCrLf & vbCrLf & ex.StackTrace, MsgBoxStyle.OkOnly, "Error writing to Log")

        End Try

    End Sub

    ''' <summary>

    ''' Display the error log using the standard NotePad application 

    '''  that should be found in the path of all Windows computers.

    ''' </summary>

    ''' <remarks>

    ''' Usually called at the end of a test run to display any failed test messages.

    ''' </remarks>

    Public Sub ShowLogs()

        Dim dblWordID As Integer

        dblWordID = Shell("notepad " & ErrorLogName, vbNormalFocus)

        AppActivate(dblWordID)

    End Sub

End Class

MyAssert.vb

Asserts are used to determine if a specific test step has passed or failed. These methods are built into test frameworks, but must be provided by the developer if an application is used to drive tests.

Imports WatiN.Core

Public Class MyAssert

    Public Shared Function AreEqual(ByVal whatData As String, ByVal valueOne As String, ByVal valueTwo As String, Optional ByVal whatMessage As String = "", Optional ByVal expectException As Boolean = False) As Boolean

        Dim aReturnValue As Boolean = True

        Dim m_logger As New Logger

        Try

            If IsNothing(valueOne) = True Then

                valueOne = ""

            End If

            If IsNothing(valueTwo) = True Then

                valueTwo = ""

            End If

            If valueOne.ToUpper.Equals(valueTwo.ToUpper) Then

                m_logger.LogResultToTextFile(whatData & " matched.", True, False)

                aReturnValue = True

            Else

                m_logger.LogResultToTextFile(whatData & ": [" & valueOne & "] did not match [" & valueTwo & "]" & whatMessage & StrDup(64, "<"), True, True)

                aReturnValue = False

            End If

        Catch ex As Exception

            If expectException = True Then

                aReturnValue = True

            Else

                m_logger.LogResultToTextFile(whatData & ": [threw an exception " & ex.Message & "]" & whatMessage & StrDup(64, "<"), True, True)

                aReturnValue = False

            End If

        End Try

        Return aReturnValue

    End Function

    Public Shared Function Contains(ByVal whatData As String, ByVal valueOne As String, ByVal valueTwo As String, Optional ByVal whatMessage As String = "", Optional ByVal expectException As Boolean = False) As Boolean

        Dim aReturnValue As Boolean = True

        Dim m_logger As New Logger

        Try

            If IsNothing(valueOne) = True Then

                valueOne = ""

            End If

            If IsNothing(valueTwo) = True Then

                valueTwo = ""

            End If

            If valueOne.ToUpper.IndexOf(valueTwo.ToUpper) > 0 Then

                m_logger.LogResultToTextFile(whatData & " contained value.", True, False)

                aReturnValue = True

            Else

                m_logger.LogResultToTextFile(whatData & ": [" & valueTwo & "] is not contained in [" & valueOne & "]" & whatMessage & StrDup(64, "<"), True, True)

                aReturnValue = False

            End If

        Catch ex As Exception

            If expectException = True Then

                aReturnValue = True

            Else

                m_logger.LogResultToTextFile(whatData & ": [threw an exception " & ex.Message & "]" & whatMessage & StrDup(64, "<"), True, True)

                aReturnValue = False

            End If

        End Try

        Return aReturnValue

    End Function
 
    Public Shared Function IsBlank(ByVal whatData As String, ByVal valueOne As String, Optional ByVal whatMessage As String = "", Optional ByVal expectException As Boolean = False) As Boolean

        Dim aReturnValue As Boolean = True
size=3>        Dim m_logger As New Logger

        Try

            If IsNothing(valueOne) = True Then

                valueOne = ""

            End If

            If valueOne.Length = 0 Then

                m_logger.LogResultToTextFile(whatData & " correctly found to be blank", True, False)

                aReturnValue = True

            Else

                m_logger.LogResultToTextFile(whatData & ": contained data when expected to be blank [" & valueOne & "]" & whatMessage & StrDup(64, "<"), True, True)

                aReturnValue = False

            End If

        Catch ex As Exception

            If expectException = True Then

                aReturnValue = True

            Else

                m_logger.LogResultToTextFile(whatData & ": [threw an exception " & ex.Message & "]" & whatMessage & StrDup(64, "<"), True, True)

                aReturnValue = False

            End If

        End Try

        Return aReturnValue

    End Function

    Public Shared Function IsFalse(ByVal whatData As String, ByVal valueOne As Boolean, Optional ByVal whatMessage As String = "", Optional ByVal expectException As Boolean = False) As Boolean

        Dim aReturnValue As Boolean = True

        Dim m_logger As New Logger

        Try

            If IsNothing(valueOne) = False AndAlso valueOne = False Then

                m_logger.LogResultToTextFile(whatData & " was false.", True, False)

                aReturnValue = True

            Else

                m_logger.LogResultToTextFile(whatData & " was not false." & whatMessage & StrDup(64, "<"), True, True)

                aReturnValue = False

            End If

        Catch ex As Exception

            If expectException = True Then

                aReturnValue = True

            Else

                m_logger.LogResultToTextFile(whatData & ": [threw an exception " & ex.Message & "]" & whatMessage & StrDup(64, "<"), True, True)

                aReturnValue = False

            End If

        End Try

        Return aReturnValue

    End Function

    Public Shared Function IsTrue(ByVal whatData As String, ByVal valueOne As Boolean, Optional ByVal whatMessage As String = "", Optional ByVal expectException As Boolean = False) As Boolean

        Dim aReturnValue As Boolean = True

        Dim m_logger As New Logger

        Try

            If IsNothing(valueOne) = False AndAlso valueOne = True Then

                m_logger.LogResultToTextFile(whatData & " was true.", True, False)

                aReturnValue = True

            Else

                m_logger.LogResultToTextFile(whatData & " was not true." & whatMessage & StrDup(64, "<"), True, True)

                aReturnValue = False

            End If

        Catch ex As Exception

            If expectException = True Then

                aReturnValue = True

            Else

                m_logger.LogResultToTextFile(whatData & ": [threw an exception " & ex.Message & "]" & whatMessage & StrDup(64, "<"), True, True)

                aReturnValue = False

            End If

        End Try

        Return aReturnValue

    End Function

    Public Shared Function NotBlank(ByVal whatData As String, ByVal valueOne As String, Optional ByVal whatMessage As String = "", Optional ByVal expectException As Boolean = False) As Boolean

        Dim aReturnValue As Boolean = True

        Dim m_logger As New Logger

        Try

            If IsNothing(valueOne) = False AndAlso valueOne.Length > 0 Then

                m_logger.LogResultToTextFile(whatData & " contains data [" & valueOne & "]", True, False)

                aReturnValue = True

            Else

                If IsNothing(valueOne) Then

                    m_logger.LogResultToTextFile(whatData & ": Value to compare is null." & whatMessage & StrDup(64, "<"), True, True)

                    m_logger.LogResultToTextFile(whatData & ": Increase pause length." & StrDup(64, "<"), True, True)

                Else

                    m_logger.LogResultToTextFile(whatData & ": did not contain data when expected to do so." & whatMessage & StrDup(64, "<"), True, True)

                End If

                aReturnValue = False

            End If

        Catch ex As Exception

            If expectException = True Then

                aReturnValue = True

            Else

                m_logger.LogResultToTextFile(whatData & ": [threw an exception " & ex.Message & "]" & whatMessage & StrDup(64, "<"), True, True)

                aReturnValue = False

            End If

        End Try

        Return aReturnValue

    End Function

    Public Shared Function NotContains(ByVal whatData As String, ByVal valueOne As String, ByVal valueTwo As String, Optional ByVal whatMessage As String = "", Optional ByVal expectException As Boolean = False) As Boolean

        Dim aReturnValue As Boolean = True

  &nb