Pex, Unit Test but Better, Parameterized Unit Testing

Finally I had a chance to play with the Pex (Program EXploration) , I heard it on lang.net for the very first time and I was quite impressed, but it took a while to explore it.

I downloaded the latest commercial evaluation package 0.22.50128.1 from Pex web site.

It is about 12 MB, it comes with different installation options with the support of VS 2008 and 2010. It installed without any problems on my VS2010 beta 2.

Here is what you get from the installation :

  • Pex command line (Visual Studio not required)

  • Pex Visual Studio Add-in, for Visual Studio 2008 and Visual Studio 2010 Beta2

  • Moles (which includes Stubs), lightweight test stubs and detours for .NET

  • API reference, tutorials and documentation

  • Samples

The documentation is really detailed, and the samples are worth investigating. I quite liked the Pex Tutorial document, which explains in detail what Pex is all about.

The good thing is it supports many different testing frameworks, including .NET’s own, nunit, xunit etc. By the way, I have to say VS2010 is awesome!

Pex is pretty much like a unit test but better in many ways. I’m quite surprised by the dynamic coverage window of a test method that you execute. This way, it is clear to see if the code is fully covered by the Pex test.

Pex is more like test method that takes inputs. In general, unit tests are not taking any inputs, they are isolated from each other and from external access, other than calling them parameterless.

Input generation in Pex based on the static analysis of the methods. It generates inputs such that, the code will be covered in full, however for some special types, the inputs might need to be explicitely told to test framework.  The tools it brings to Visual Studio is extremely simple and easy to use. It does a very good job on incorporating those results.

Here is a very simple function that I generated my first Pex tests from

1
2
3
4
5
6
7
8
9
10
11
12
 public static int Compare(int i, string s)
        {
            if (i > 500)
            {
                i *= 2;
            }
            else
            {
                return Compare2(i);
            }
            return i * i;
        }

When you run Pex, it generates potential inputs to cover all the code. According to the documentation, it uses static analysis to figure the potential values. So it is not a random number generated test, or a brute force test, which is quite interesting. Pex can cover some obscure side of the code, which fully supports many Unicode string inputs or regular expressions.

However, when I converted this very simple code, from System.Int32 to System.Numeric.BigInteger (a new type in .NET 4), the test only generated for a null value. Unless I’m missing something, it doesn’t support all the types as yet, but it looks like a promising and valuable product to be added to your toolset