Article on agile development

I read an article on agile development by Martin Fowler. I thought it was an excellent read. Just to get you excited I have extracted some of the points I found most interesting.

Traditional engineering and software development differ in a significant way. E.g., in building a bridge the design is about 10% of the effort, but in software development it is about 50% of the effort. This changes the nature of the work since design is a much more unpredictable process, requiring gifted (non replaceable) individuals, but construction is rather automated and less demanding on particular skills. Additionally, the requirements for software projects are more liquid making the software design process even harder to predict.

Since the individual developer plays such a big role in a software development the agile processes are focuses on how to mange them and their interactions, as opposed to the traditional approach based on the assumption that the individuals are replaceable parts. Finding a good measure of progress for these processes is difficult, and Martin quotes Robert Austin's conclusion that measurement-based management has to be abandoned for delegatory management.

The unpredictability of software development makes it hard to fix a budged up-front: it is impossible to fix time, price and scope. However, using agile methods, it is possible to allow the scope to vary while keeping the price and time fixed. The success of the project should therefore be measured by how much business-value it provides to the customer, rather than how well it meets its plan.

Martin concludes his article by describing some of the numerous agile methods in existence, this I found of less interest.

Using database transactions in unit tests

I started using the advice of Roy Osherove regarding how to use transactions in unit tests (see previous blog entry).

To summarize the method:

        using System.EnterpriseServices;

        ServiceConfig config = new ServiceConfig();
        config.Transaction= TransactionOption.RequiresNew;
        ServiceDomain.Enter(config);

        [database CRUD code]

        if(ContextUtil.IsInTransaction)
        {
            ContextUtil.SetAbort();
        }
        ServiceDomain.Leave();

This code worked fine after having sorted out some configuration problems. I am accessing the SQL Server 2000 on a Windows 2003 server from a Windows XP on a different domain. First I got "The partner transaction manager has disabled its support for remote/network transactions.", this changed when I allowed for MSDTC on the server. Then I got "The transaction manager has disabled its support for remote/network transactions" which was fixed by allowing MSDTC on the client machine. The final error was because the firewall was blocking the connection.

Therefore in order to make this work (for this scenario) it is necessary to:

• Allow MSDCT on the client and server: Administration tools -> Component Services -> Computers, right-click on My Computer, click on the "MSDTC" tab, click on "Security Configuration" and allow everything :) In particular: "Allow Outbound" on client, "Allow Inbound" on server (the SQL Server machine), set "No Authentication Required" on both server and client, and enable TIP and XA Transactions on both server and client.
  
• In the firewall open up for the msdtc.exe: %root%\WINDOWS\system32\msdtc.exe on both client and server.
  

Now I am able to control the initial conditions of the database programmatically in a simple manner :)

P.s. The above settings work for me, but further instructions can be had here and here.

Interesting figures

On Tomshardware there is a interesting short comparison on present and past computer capabilities.

Unit testing reality

I has been a pleasure reading Roy Osherove's articles on unit testing. In "Write Maintainable Unit Tests That will Save You Time And Tears" he talks about some of the pitfalls of unit testing, the list is probably not complete, but I good read.

On a side-note: I feel that articles written with the intention of explaining shortcomings (scope) and pitfalls are much more educational then the ones simply explaining the usage. I felt this strongly when reading about Fitness where I had problems in figuring out its scope.

The other article, discussed unit testing database access layer code. It had the excellent suggestion of using COM+ 1.5 SWC to encapsulate the unit tests in transactions, thus ensuring independence between unit test. I am looking forward to following this suggestion.

Polymorphism and Interfaces

The following code did not behave as I expected. It returns "A::f" twice, but I expected it to return first "A::f", then "B::f".

There are two ways of getting the desired behavior, either declaring A.f() virtual or explicitly stating that B implements I. The latter is not desirable since I don't think B should need to know that A implements I, and the former I don't think is intuitive.

Any thoughts?

   public interface I
   {
       void f();
   }
   public class A : I
   {
       public void f() { Console.WriteLine("A::f") ;}
   }
   public class B : A
   {
       public void f() { Console.WriteLine("B::f"); }
   }
   public class C
   {
       static void Main()
       {
           I i1 = new A();
           I i2 = new B();

           i1.f();
           i2.f();
       }
   }

Completing the design

The simple concept of refactoring has helped me allot. I have the tendency of not being able to start implementing before I think I have the design all figured out. This is something I think I got imprinted in school through all the "bugs-found vs. development-phase" graphs, and perhaps it is in my character as well :) Now I just tell myself: "this might not be perfect, but I will just refactor it later", and it does wonders for my throughput :)

I have in fact turned quite against completing the design before coding. In particular, I think that one should not try to guess how a particular class might be used in the future, e.g., by adding numerous accessor functions no one uses but need to be maintained and unit tested.

Delegate smell cont.

How come that delegates can call private functions, as in (inspired by MSDN):

ElapsedEventHandler eh = new ElapsedEventHandler(OnTimedEvent);
...
private void OnTimedEvent(object source, ElapsedEventArgs e) { }

Smelly.

== does not equal Equals()

I discovered that for boxed System.ValueType the == operator behaves differently from the Equals() function. When using == with two boxed ValueType-s it will compare their references, thus "always" returning false. Equals(), on the other hand, has been overloaded by Microsoft for the ValuType-s, so that it does a value comparison of the boxed objects, giving the expected result.

I wonder why Microsoft did not also overload the == operator?

Deep copy of Hashtable

In order to solve my problem of returning a hashtable by reference (see previous post), I first tried to use Hashtable.Clone() to give me a copy of the hashtable. However since Clone() only does a shallow copy of the hashtable, this did not work since the hashtable contained Objects and changing them in the calling code caused side-effects in the originating class. So what was needed was a deep copy of the hashtable and this proved to be really simple:

       private Hashtable mMeasurements = new Hashtable();
       ...
       MemoryStream ms = new MemoryStream();
       BinaryFormatter bf = new BinaryFormatter();
       bf.Serialize(ms,mMeasurements);

       ms.Seek(0,SeekOrigin.Begin);
       clonedMeasurements = (Hashtable)bf.Deserialize(ms);

Are there any concerns regarding this code?

Mocking SAO

I have been unit testing a class that interacted via Remoting with a Singleton. I wished to eliminate the remoting part and use a mock object for the singleton. There was one hinge to that: One of the singleton's public methods returns a Hashtable. The Remoting automatically took care of serializing the hashtable and recreating it at the client. However, when I started using the singleton directly (in the same assembly) as the class under test, the hashtable was returned by reference causing undesired side-effects when the class under test started manipulating the hashtable!

More on the solution shortly...


P.s.
Same problem arrises for SingleCall objects and CAOs.

UdpClient.Receive()

I have been exposed to System.Net.Sockets.UdpClient recently. One of its member functions is Receive(ref IPEndPoint). I was wondering why IPEndPoint was passed to the function as a reference (pointer to a pointer actually). So I downloaded the source code for the Mono project, thinking that it would give me a clue. There the object passed in is never used, but a new IPEndPoint created and it passed back to the calling function. This actually makes sense since the returned IPEndPoint contains the IP address of the party sending the UDP package. But why then not define Receive() as Receive(out IPEndPoint)? Thus making the intention clear.

In fact setting IPEndPoint as null before passing it into Receive() works fine.

P.s. It was not necessary for me to pull out the Mono source, the Reflector gave me the code just fine :)

public byte[] Receive(ref IPEndPoint remoteEP)
{
      EndPoint point1;
      if (this.m_CleanedUp)
      {
            throw new ObjectDisposedException(base.GetType().FullName);
      }
      if (this.m_Family == AddressFamily.InterNetwork)
      {
            point1 = IPEndPoint.Any;
      }
      else
      {
            point1 = IPEndPoint.IPv6Any;
      }
      int num1 = this.Client.ReceiveFrom(this.m_Buffer, 0x10000, SocketFlags.None, ref point1);
      remoteEP = (IPEndPoint) point1;
      if (num1 < 0x10000)
      {
            byte[] buffer1 = new byte[num1];
            Buffer.BlockCopy(this.m_Buffer, 0, buffer1, 0, num1);
            return buffer1;
      }
      return this.m_Buffer;
}

Listening to UDP broadcasts

I found the following statement on MSDN:

"The UdpClient class can broadcast to any network broadcast address, but it cannot listen for broadcasts sent to the network. You must use the Socket class to listen for network broadcasts."

However this simple UdpClient:

    client = new UdpClient( 1234 );
    point = new IPEndPoint( IPAddress.Any, 0 );
    while (true)
    {
        Console.WriteLine(Encoding.ASCII.GetString( client.Receive( ref point ) ));
    }

works equally well for broadcasted (*.255) as well as unicasted (*.131) udp packages.

I am confused.

Small VS2003 tid bit

I found out an interesting fact about adding references to 3rd party dlls in VS2003 projects (have not tried this in VS2005). If, when browsing to the dll, you go through "My Documents" the dll will be placed in the the project file (*.csproj) with an absolute path. If, however, you store the dll in some 'normal' folder on the hard disk (C-drive typically), then the added reference will placed in the project file with a relative path.

The reason for this behaviour is probably that "My Documents" is treated as a mapped network drive.

This tid bit is not going to be of much interest to you unless you share your project files with other developers, then it will cause you pain if your paths to the project files are different. The lesson is not to store your code under "My Documents" :)

Introducing mock objects in unit testing

The problem is: During unit testing I wish to replace (mock) a certain class (A) that is being used by the class (B) being tested.
The question is: how is this best done?

I can think of three ways:

1. Inherit B (with class C) and override the (protected) method that loads up A. In C, A-mock would be loaded instead of A. During testing run the unit tests on C instead of B.
2. Add public or internal methods to B that allow A being set. During testing give B A-mock instead of A using these new methods.
   
3. Use reflection to load A dynamically based on the configuration (in app.config, e.g.). During testing replace A with A-mock in the configuration.
   

All these ways impose some change to the class, B, under test, but what change is most acceptable?

Unfortunately I don't have the answer to that :)

Fitnesse

I took a look at Fitnesse "a software development collaboration tool", I had had some experience with it earlier, but had forgotten how it functioned :)

It helps me to understand what Fitnesse is by breaking it up into two parts:

• It is an extension to xUnit which allows you to configure the unit tests (without changing code).
• It is an UI which facilitates the configuration of the tests.
  

In fact the UI part is presently implemented using a Wiki, but could "easily" be replaced by another technology.

One thing bothered me while reading about Fitnesse, i.e., that the Fitnesse tests are called "acceptance tests". I think it is misleading to say that the customer writes the test, what he is actually doing is configuring the test that have been written by the developer. For me "acceptance test" is where the customer actually defines how and what to test, he should not rely on the developer to write the correct tests. The customer gets the final product and plays with it.

This, however, does not diminish the usefulness of Fitness tests, they do add value to the unit tests, especially if they get the customer more involved in the project. Maybe I am going overboard with this discussion, not having even installed or tested Fitnesse :)

O/R libraries

htomasso suggested in a comment to my last blog entry that I should take a look at NHibernate for my object/relational mapping needs. I remember learning about object oriented databases, but this seems solve the problem as well, i.e., persisting objects to a database (for retrieval).

My current problem is storing data from a hierarchal DTO model to a database. This data is used to calculate summary statistics, doing trend analysis, profiling, and debugging. But the DTOs are never actually retrieved back as DTOs from the database. So, although it looked promising initially (simplified the storing part), I think that O/R libraries are not suited for this scenario.

I did some web-research to find out what was being said as to when O/R libraries apply, but could not find any definite guidelines :(

StrokeIt

I use the mouse-gestures in Firefox, it is one of the key reasons for why I favor Firefox over IE (the two others being adblocking and tabed-browsing). Recently I found out that there is an application that allows you to add mouse-gestures to all applications, it comes with the thought-provoking name "StrokeIt". I immediately found one good use for it, namely to navigate back and forth in VisualStudio. It has not caused me any problems yet (2 days of use), so I feel that I can recommend it.
Happy "stroking"!

P.s. I finally gave up on StrokeIt, it was causing some spurious mouse movements that in time became too irritating to tolerate.

Other nice tools

I found another nice tool from the same people as did the Snippet compiler: SmartSniff. It sniffs the network for activity, very useful for analyzing program behavior. While on the subject, I also frequently use FileMon to locate problems with programs I use, it is a life-saver.

Other nice tools I use are:

• ScreenHunter, for screen-capture. Often handy when reporting some failure.
• Notepad2, a slightly improved version of Notepad. There are several Notepad replacements, I also have installed EditPadLite.
• Reflector gave me a slight shock, it showed me that the .NET dlls are very easily decompiled. We will be using an obfuscator to amend that.
  
• I have been using Baretail to tail my log4net-logs, but I am still looking for another tool I once used and was better :(
• FireFox needs mention since it is probably the tool I use the most (with the Adblock and All-in-one Gestures extensions).
  

This list is not complete, and I don't think I will keep it up-to-date, but we'll see.

CruiseControl

I recently installed CruiseControl.NET, and now it is building 3 of my projects. Included in a build is checking out the newest code from Subversion, compiling it, running the unit-tests and generating the documentation (NDoc). It is on the agenda adding NCover and FxCop to this build process.

Actually, I first started installing Draco because I had read it somewhere that it was preferable for smaller projects/teams. I was rather disappointed with that. First, the schema for the config file was too strict, it did not allow me to specify the mailserver without a domain ending or the email senders/recipients without a host ending, both of which are valid for my setup. This forced me to add an alias domain to the SMTP server of IIS (an inconvenience). Second, I am currently not using a Subversion server (Apache nor Svnserve), but simply a network share (FSFS), and I have not figured out to make this work with Draco. Anyhow, CruiseControl is working fine and was not hard to set-up, so I will stick with that.

Nice tool

I found this nice tool: Snippet compiler, which is just a mini environment to program in. Next simplest thing to using Notepad and CSC.