2010-01-18

ASP.NET MVC Faking Web Services

At the most recent Northern Michigan .NET User's Group meeting, Derek Smith gave a great introduction to ASP.NET MVC. One thing that he touched on in his presentation was the URL routing feature of MVC (which can be used independently of the rest of the MVC framework). In response to my question, he went on to explain that the MVC controllers (which are routed to by the URL routing) may return data instead of just a view (in particular, XML and JSON).

This means that ASP.NET MVC may be used to directly expose the controllers as web services, without all the configuration necessary to set up a web-friendly WCF interface hosted in IIS. In going the MVC route, one would lose the automatic metadata export for the WCF service, but one would gain ease of implementation and an easy-to-use "friendly URL" router.

This is an interesting concept (and Google reveals that it's one that's been kicking around a few years). Manning has a free chapter covering URL routing from their "ASP.NET MVC in Action" book (which I just bought; it is 50% off today only).

Super-Agile or Cowboy-Coding?

This is not one of my normal "hey, I found an obscure .NET thing" kind of posts. This is a brain-dump of something I've been thinking about for a while.

A few months ago, there was a bit of discussion on the 'ol blog-o-sphere. Somebody said something about "shipping is a feature" and somebody else said something about "cowboy coding". If you missed it, no big deal; it's the old how much design is the right amount debate. Earlier this week at lunch, I was reading a magazine article where the author made the implicit assumption that coupling is bad, and that prompted this blog post.

To summarize, there are two extreme positions in the argument. Some people push towards perfect designs: there's no coupling, everything is interface-based, every class has high cohesion, and it's documented completely. Other people push towards a working product: something that actually ships, works, has all the expected features, and generates income for the company.

It's an interesting read, but I take a moderate position. I've been pushed both ways in my career: one time I was told to remove the dependency on the Dependency Injection component (just in case we decided to change to a different DI component); another time I was told to just "make it work, and fast" (due to an unethical coworker, the project was dumped on me with no code two weeks after the contracted ship date). I can say from experience: neither extreme is pretty.

Coding with Engineers

I must say, I really enjoy my new job. I'm back at a small engineering company, where everyone you meet is an engineer of some kind. [Even the guy in charge of the network is an ME; and I don't mean an IT guy with an ME degree - I mean an ME who just does the IT stuff in his spare time]. I'm now in my 8th year working at an engineering shop, and let me say that it takes a special kind of programmer to work in this kind of environment.

Engineering companies (at least the small ones in Northern Michigan) don't really know how to do software. After all, that's what programmers are for, right? If they don't know how to do software, who would? Programming positions in engineering shops end up being of the "sink or swim" variety; you either know what you're doing and excel, or don't and are let go.

There is a distinction between good programmers and great programmers; based on my experience, great programmers are between 5 and 12 times more productive than good programmers. Engineering shops cannot afford to hire an entire division of programmers and let them all "balance each other out;" they can only afford to hire the great programmers.

Aside - What Makes a Naturally Great Programmer?

I remember reading the Hacker HOWTO years ago and realizing how true it was; there are a few things mentioned in that article as being common among naturally great programmers. I repeat the ones I've found most accurate:

  • "Learn to write your native language well." It is almost always the case that people who spell correctly, capitalize correctly, and use proper grammar also are great programmers. Similarly, it is almost always true that bad spellers, inconsistent capitalizers, and abusers of sentence structure produce worse code.
  • "Train in a martial-arts form." The great programmer seems drawn to martial arts of some kind, especially "those which emphasize mental discipline, relaxed awareness, and control". It's been a few years since I've done Tae Kwon Do, but I still enjoy running through my exercises from time to time.
  • "Develop an analytical ear for music... Learn to play some musical instrument well, or how to sing." Again, there is an eerie correlation between good programmers and musical abilities; I myself play the violin and sing (getting into barbershop recently).
  • "Develop your appreciation of puns and wordplay." Great programmers do seem to enjoy these types of amusements. This is the reason why a lot of programmers have a dry sense of humor; they prefer jokes that require some thought. One of my favorite books is Pride and Prejudice, simply because of Mr. Bennet's statements.

It is rather strange that all these things show up so commonly among the greatest programmers. The Hacker HOWTO postulates a reason, with which I agree wholeheartedly: "they're connected with a mix of left- and right-brain skills that seems to be important; hackers need to be able to both reason logically and step outside the apparent logic of a problem at a moment's notice." I've taken several "measure your brain" tests, and every single one shows a perfectly equal balance; I am neither right-brained nor left-brained. Furthermore, I believe that this is important in being a naturally great programmer: it's possible to break down larger problems into small steps and be very detail-oriented, and it's possible to make the intuitive leaps into new algorithms - at the same time.

For the record, I do believe that a good programmer may learn to become a great programmer. The previous few paragraphs are only discussing common traits of naturally great programmers.

Back to the Engineer Shop

As mentioned, engineer shops cannot afford a software division. If the truth were told, a lot of software methodologies are designed around "balancing" the good and great programmers; there's a constant training of the good programmers and a "holding back" of the great programmers (e.g., by putting off changes until everyone understands them, or just rejecting such changes).

Engineer shops follow a different principle: only hire the great programmers. Usually, there is a probation period that lasts several months, during which time the other engineers determine whether you are a good programmer or a great programmer. At the end of that time, if you don't make the cut, you are shown the door. That might sound harsh, but we are talking about small engineer shops in Northern Michigan here; it's a simple matter of the survival of the company.

In order to make the cut at an engineer shop, each and every prospective programmer must be:

  • Motivated - you've got to be self-starting. There is always more work to do, and sometimes you won't have any work assigned. It is up to you to find the work that needs doing and "task it to yourself".
  • Able to communicate - part of self-starting is proactively gathering requirements. In an engineering shop, there are no market analysts, designers, or architects. Alternatively, one could say that every programmer is a market analyst, designer, and architect. It's normal for a single programmer to be responsible for gathering requirements, designing, implementing, testing, and supporting (in the field) the software.
  • Always learning - no one trains you; there is no programmer with enough time to teach you to be a programmer. Engineering shops generally do not pay for software conferences or seminars. They will provide you the Internet; it is up to you to keep your skills up-to-date.
  • Conservative - the decision of when to jump on new technology falls on you; if you jump too soon, you can seriously impact the success of the entire company. In most engineering shops, there's a kind of informal consensus for big decisions; but if everyone knows you championed the MegaFrizzle Software Development Paradigm that just impacted everyone's bottom line, you don't last long.
  • Experienced - to make the correct design and architectural decisions, you must be experienced both deeply and widely. "Deep" experience is more than writing some test programs; it's knowing the limitations of an approach by having felt them. "Wide" experience is necessary as well, so that you may choose the best path. I've utilized object-oriented, generic, procedural, and functional designs in various projects, and none of them is inherently better than another.

There are a couple of advantages of being a programmer in an engineer shop:

  • Empowerment - we know the true meaning of the word. We have true empowerment.
  • Left alone (mostly) - the only people we really have to talk to are all really smart. All the people we work on a day-to-day basis are engineers (even our customer contacts). The data-to-noise ratio of conversations has a high average. :)

However, there are a few disadvantages of being a programmer in an engineer shop, especially in Northern Michigan:

  • Difficult to use social programming techniques - in my current job I'm the only .NET programmer. What's even more scary, I've never met anyone in my life who has ever done pair programming or a design or code review. It can be difficult to get others to participate in user "stories". There's a lot of work being done now exploring this kind of "social programming", which is near impossible for us.
  • Pressure - we have to stay the best, not just be the best once or twice. This is not so much a risk of losing our jobs, but it's a weight to have that much of the company's success riding on our decisions.

Conclusion

Each engineering programmer stands or falls on their own, with a tremendous amount of autonomy. We almost have too much empowerment...

This brings up the question that started this blog post: Am I "Super-Agile" or just a "Cowboy Coder"?

I don't know the answer, but I'm interested in knowing. If I ever find out, I'll post the results. :)

2010-01-05

Extension-Based Types

There's a new paradign rising in .NET: extension-based types (EBTs).

Introduction

It's an exciting time to be a .NET developer. I'm reminded of the time when Boost was young, and programmers were first starting to realize the capabilities of templates. Generic programming became a powerful mainstream paradign, producing techniques now described in C++ Templates and C++ Template Metaprogramming.

Currently, we're seeing a similar transformation taking place with C# generics. The adoption of generics is just beginning to reach a critical mass, where new and inventive approches are discovered. The thrust towards generic programming is driven by a desire for higher levels of abstraction and extensibility.

The EBT approach allows writing .NET libraries that permit an unprecedented level of end-user extensibility while discouraging leaky abstractions. This approach combines the clean abstraction of "programming to an interface" with the convenience of additional, more complex operations (some of which may be contributed by end-user code).

Extension-Based Types

An EBT is a type that is primarily defined through extension methods on interfaces. Only a few core methods are defined on the interfaces themselves. Properly-designed EBTs have the minimal set of methods defined in their interface (often only one, and sometimes none).

EBTs are useful for end-user extensibility (and raising the level of abstraction), but they also have a few "gotchas" since they are dependent on the compile-time nature of method overload resolution.

Here's an example of an EBT that defines a single method, MethodA:

public interface IMyInterface { }
public sealed class MyImplementation : IMyInterface { }

public static class MyMethods
{
    public static void MethodA(this IMyInterface @this) { Console.WriteLine("IMyInterface.MethodA()"); }
}

class Program
{
    static void Main()
    {
        var obj = new MyImplementation();
        obj.MethodA(); // Prints: "IMyInterface.MethodA()"

        Console.ReadKey();
    }
}

Simple Inheritance

EBT inheritance is performed using interface inheritance. Here's an example of a base interface that defines MethodA and a derived interface defining MethodB; the derived interface ends up supporting both methods:

public interface IBase { }
public interface IDerived : IBase { }
public sealed class Derived : IDerived { }

public static class MyMethods
{
    public static void MethodA(this IBase @this) { Console.WriteLine("IBase.MethodA()"); }
    public static void MethodB(this IDerived @this) { Console.WriteLine("IDerived.MethodB()"); }
}

class Program
{
    static void Main()
    {
        var obj = new Derived();
        obj.MethodA(); // Prints: "IBase.MethodA()"
        obj.MethodB(); // Prints: "IDerived.MethodB()"

        Console.ReadKey();
    }
}

Overriding Inherited Methods: Simple Overriding

A derived EBT may override a base EBT method by defining its own method with an identical signature. Here's a derived type that overrides the MethodA defined by its base type:

public interface IBase { }
public sealed class Base : IBase { }
public interface IDerived : IBase { }
public sealed class Derived : IDerived { }

public static class MyMethods
{
    public static void MethodA(this IBase @this) { Console.WriteLine("IBase.MethodA()"); }
    public static void MethodA(this IDerived @this) { Console.WriteLine("IDerived.MethodA()"); }
}

class Program
{
    static void Main()
    {
        var obj1 = new Base();
        obj1.MethodA(); // Prints: "IBase.MethodA()"

        var obj2 = new Derived();
        obj2.MethodA(); // Prints: "IDerived.MethodA()"

        Console.ReadKey();
    }
}

Overriding Inherited Methods: Invoking the Base Method

In order to invoke the base method when the derived EBT overrides it, the compile-time type of the variable must explicitly be the base type. Here's an example that invokes the derived and base MethodA implementations on the same object:

public interface IBase { }
public interface IDerived : IBase { }
public sealed class Derived : IDerived { }

public static class MyMethods
{
    public static void MethodA(this IBase @this) { Console.WriteLine("IBase.MethodA()"); }
    public static void MethodA(this IDerived @this) { Console.WriteLine("IDerived.MethodA()"); }
}

class Program
{
    static void Main()
    {
        var d = new Derived();
        d.MethodA(); // Prints: "IDerived.MethodA()"

        IBase b = d;
        b.MethodA(); // Prints: "IBase.MethodA()"

        Console.ReadKey();
    }
}

For convenience, an "identity transformation method" is usually provided that restricts the type of a subexpression; this way, a separate variable is not necessary. By convention, the identity transformation method is named "As{I}". The following example shows how an "AsBase" method removes the need for the IBase variable:

public interface IBase { }
public interface IDerived : IBase { }
public sealed class Derived : IDerived { }

public static class MyMethods
{
    public static IBase AsBase(this IBase @this) { return @this; }
    public static void MethodA(this IBase @this) { Console.WriteLine("IBase.MethodA()"); }
    public static void MethodA(this IDerived @this) { Console.WriteLine("IDerived.MethodA()"); }
}

class Program
{
    static void Main()
    {
        var obj = new Derived().AsBase();
        obj.MethodA(); // Prints: "IBase.MethodA()"

        Console.ReadKey();
    }
}

Overriding Inherited Methods: The Importance of Compile-Time Types

It's important to note that the compile-time type of the expression is what's used for method overloading, so the EBT style of overriding inherited methods is not like object-oriented virtual function overriding:

public interface IBase { }
public interface IDerived : IBase { }
public sealed class Derived : IDerived { }

public static class MyMethods
{
    public static IBase AsBase(this IBase @this) { return @this; }
    public static void MethodA(this IBase @this) { Console.WriteLine("IBase.MethodA()"); }
    public static void MethodA(this IDerived @this) { Console.WriteLine("IDerived.MethodA()"); }
}

class Program
{
    static void Main()
    {
        IBase obj = new Derived();
        obj.MethodA(); // Prints: "IBase.MethodA()", NOT "IDervied.MethodA()"

        Console.ReadKey();
    }
}

In fact, even something as minor as missing a "using" statement could cause the wrong method to be called. Consider the case where "MethodA(this IDerived)" is defined in a class in a different namespace. It must be brought into scope via a "using" statement before it could be considered by method resolution.

Multiple Inheritance

Multiple inheritance is supported for EBTs; any ambiguity causes a compiler error:

public interface IBaseA { }
public interface IBaseB { }
public interface IDerived : IBaseA, IBaseB { }
public sealed class Derived : IDerived { }

public static class MyMethods
{
    public static void MethodA(this IBaseA @this) { Console.WriteLine("IBaseA.MethodA()"); }
    public static void MethodA(this IBaseB @this) { Console.WriteLine("IBaseB.MethodA()"); }
}

class Program
{
    static void Main()
    {
        var obj = new Derived();
        obj.MethodA(); // Compiler error: ambiguous

        Console.ReadKey();
    }
}

Ambiguity may be resolved by overriding the method in the derived EBT, or by constraining the compile-time type using the identity transformation method. The second approach is more flexible, since it allows any user-defined extensions. This example uses the second approach:

public interface IBaseA { }
public interface IBaseB { }
public interface IDerived : IBaseA, IBaseB { }
public sealed class Derived : IDerived { }

public static class MyMethods
{
    public static IBaseA AsBaseA(this IBaseA @this) { return @this; }
    public static void MethodA(this IBaseA @this) { Console.WriteLine("IBaseA.MethodA()"); }
    public static IBaseB AsBaseB(this IBaseB @this) { return @this; }
    public static void MethodA(this IBaseB @this) { Console.WriteLine("IBaseB.MethodA()"); }
}

class Program
{
    static void Main()
    {
        var obj = new Derived();
        obj.AsBaseA().MethodA(); // Prints: "IBaseA.MethodA()"
        obj.AsBaseB().MethodA(); // Prints: "IBaseB.MethodA()"

        Console.ReadKey();
    }
}

Properties

Due to .NET limitations, properties may only be defined on interfaces (there's no such thing as an "extension property"). However, they may be simulated:

public interface IBase { int Property { get; } }
public interface IDerived : IBase { }
public sealed class Derived : IDerived
{
    int IBase.Property
    {
        get { return this.GetProperty(); }
    }
}

public static class MyMethods
{
    public static IBase AsBase(this IBase @this) { return @this; }
    public static int GetProperty(this IDerived @this) { Console.WriteLine("IDerived.GetProperty()"); return 13; }
}

class Program
{
    static void Main()
    {
        var obj = new Derived();
        obj.GetProperty(); // Prints: "IDerived.GetProperty()"
        int test = obj.AsBase().Property; // Prints: "IDerived.GetProperty()"

        Console.ReadKey();
    }
}

One may think of this as the interface holding the property declaration while the extension methods (derived EBT methods) hold the property definition.

Note that the property getter will always call the same derived method, regardless of whether it is accessed through an IBase interface. However, the extension method "GetProperty" (if it were defined on IBase) would call either the IBase or IDerived implementation, depending on the compile-time type.

Limitations: Inability to Override Interface Methods

If a method is defined in the interface instead of as an extension method, then that method may never be overridden by a derived EBT type:

public interface IBase { }
public interface IDerived : IBase { void MethodA(); }
public sealed class Derived : IDerived
{
    public void MethodA() { Console.WriteLine("Derived.MethodA()"); }
}

public static class MyMethods
{
    public static IBase AsBase(this IBase @this) { return @this; }
    public static void MethodA(this IBase @this) { Console.WriteLine("IBase.MethodA()"); }
    public static IDerived AsDerived(this IDerived @this) { return @this; }
    public static void MethodA(this IDerived @this) { Console.WriteLine("IDerived.MethodA()"); }
}

class Program
{
    static void Main()
    {
        var obj = new Derived();
        obj.MethodA(); // Prints: "Derived.MethodA()"
        obj.AsBase().MethodA(); // Prints: "IBase.MethodA()"
        obj.AsDerived().MethodA(); // Prints: "Derived.MethodA()" (NOT "IDerived.MethodA")

        Console.ReadKey();
    }
}

For this reason, it is important to distill as many methods as possible out of the interface that defines the EBT.

Limitations: Dependency on Compile-Time Types

Code that uses EBTs must "see" all the associated methods in order for overridden derived methods to work properly. The parallel problem for C++ templates eventually resulted in "header file libraries", where libraries transitioned from dlls to source code that was included in the program using the library.

A similar transition will probably occur if EBTs are embraced. Currently, if EBTs are compiled into a library (e.g., System.Core.dll), then end-user code may supplement but not replace existing behavior.

Real-World Examples from LINQ

As LINQ continues to evolve, it approaches EBTs. In particular, the IEnumerable<T> interface is extended by the Enumerable class, including an AsEnumerable identity transformation method. IOrderedEnumerable<T> is one derived type; the interface adds no methods, but the Enumerable class does. The rarely-used IQueryable<T> derived type is extended by Queryable. Grouping and ordering follow the same pattern.

The new PLINQ has a similar structure (though the derived ParallelQuery<T> is a class instead of an interface).

Rx (a.k.a. "LINQ to Events") continues the tradition. IObservable<T>, IObserver<T>, and IScheduler<T> provide basic definitions of types, with extension methods in the Observable, Observer, and Scheduler classes. Rx follows the EBT pattern in defining minimal interfaces; for example, IObservable<T> only defines one method (Subscribe), and extension methods are used to provide 5 overloads for Subscribe. The Rx team has a video in which they describe some of the design behind IScheduler/Scheduler.

Future Blog Posts

In the (hopefully near) future, I'll be showing how to use EBTs to do compile-time generic specialization (including partial specialization, generic method specialization, and inheritance specialization). I also intend to cover "namespaces" and wrapper objects. Lots of fun! :)

2010-01-02

Best Computer Programming Book Ever

I had heard rave reviews of Code Complete for years. Supposedly, it's required reading for every Microsoft employee. Programmers I respect glowed about it. Rumors abounded about its ability to cure warts... OK, maybe the rumors didn't quite say that, but they were pretty close.

Perhaps I'm a bit of a skeptic (which is funny, considering that I'm a Christian as well - but even in church, where things must be taken on faith, I do view everything in a highly skeptical light unless it is grounded on the Bible). "Surely," I thought, "no programming book is that good!"

I now stand corrected. I have nearly completed my first reading of this book (and I really took my time with this one). It is, without doubt, the best programming book I have ever read.

It reads as the collective wisdom of many independent programmers brought together from years of experience, which is the only true teacher in our profession. The book does an excellent job of identifying common pitfalls in a language-agnostic fashion.

There are only a handful of problems with it; the most major of these is that it could use some updating. For example, the C# language sidesteps entire sections of problems due to its design. There are also a couple of very minor points where I disagree with the author, and one tiny omission (I hope to blog more about these in detail later).

I say again: this is the best book on programming I have ever read. I fully intend to read it continuously (starting again each time I finish). It should be read by anyone entering the field as well as anyone working in the field.

The name is Code Complete, and it is a "must-read." No programmer's bookshelf is complete without a copy.

View of 2010

Last year ended in a very busy fashion. I ended up not getting my MCPD/Enterprise, though I did take the WCF test. Furthermore, with my new job and other responsibilities, getting the MCPD/Enterprise is no longer of a high enough priority to warrant the time.

Regarding the new year (2010), expect some of my posts to take a web-based turn. Up until this point, I've almost entirely done desktop applications with a little dabbling in Silverlight. Starting in the next few days, however, I'll be transitioning a web site to ASP.NET; and I'll be kept busy in the foreseeable future maintaining it.

Of course, my desktop days are not over. The web site is only an additional responsibility; my day job is entirely desktop-based, so there will be plenty of multithreading and asynchronous posts (especially as VS2010 is released), as well as adding to the TCP/IP FAQ.