Anatomy of a Windows Service - Part 4

Deciding how to trigger the work of the service usually comes down to some sort of threading solution. And threading in Windows services is, as everywhere else, a thorny issue. The most basic need we have is to be able to respond to orders from the Service Control Manager (SCM) without blocking processing at an inconvenient spot. Beyond that, we may want to have a bit of work kicking off periodically, with a short period, or we may want to respond to some environmental event.

Responding to environmental events is fairly easy when the framework provides a callback mechanism for it, such as the FileSystemWatcher. But if there's no such facility, then you're stuck with polling, which is in the same boat as the guy who needs to do work on a quick periodic schedule. This leads us to worker threads, and all the headaches that go along with that.

I have no intention of getting deep into threading concepts and strategies here. Not least because I am woefully uninformed and inexperienced with them. If you need to be highly responsive, highly predictable, or highly parallel, that's an entirely new problem space. So it will have to suffice to say here that you should not just hack something together quickly. Do your research and do it right.

What I have done in the past is to use a Thread.Timer to fire a callback on my desired period. But I use a flag to make sure multiple work units do not process in parallel, and I don't worry about missing a period by a nanosecond simply because the work completed just after I checked the flag. Instead I just wait for the next one. This limits the responsiveness of my service, but it's simple and reliable because it sidesteps the issues that arise out of mutually writeable shared state and truly parallel work.

For our sample project here, I introduced this mechanism by creating a worker class that manages the timer. Let's take a look at that.


Note the flag properties, which are used as a guard on the timer event to make sure that processing doesn't re-enter before the previous iteration completes. The timer interval is set to start things rolling, and cleared to stop them. The added complexity in the Stop method is crucial to understand as well. Timers are tricky to get rid of without blocking the thread. We use a wait handle to do it. Wait handles are worth reading up on if you deal with threading at all. Another nice feature of this worker class is that an object of this type can be reused: Start can be called to spin things up again after Stop has finished.

The worker presents essentially the same interface as the service itself, which may seem a bit redundant. All I am really trying to achieve by this is to respect the separate responsibilities of management of service as a whole, as opposed to management of the work timer and triggers. The service itself is just processing events from the SCM. It need not know, when it tells the worker to start, whether the worker is firing up a new thread to get started, waiting for the moment the current one finishes, or just taking a pass on this period.

The service class will now delegate to the worker. This is very straightforward. I just had to replace the IGreeter property with a property for the worker. Now in OnStart, we spin up the worker, and in OnStop we finally have some code as well, to spin down the worker.


You can find the source code for our service project at this point tagged in the repo for this post on GitHub.

Now that we've got the service doing everything it's supposed to do, we may decide that we'd like to be able to change exactly how it does it's job. Essentially, we want to make it configurable. The only real parameters of our service at this point are the work interval and the file name, and those are hard-coded. We could easily make the service more flexible by taking these hardcoded values and making them configurable.

One way to do that would be to use the service argument array, which operates just like the regular app argument array. We don't have much to configure here, so that would probably work for us. But to imitate a real-world scenario where there may be lots to configure, let's pretend that won't be sufficient.

Another option would be to use a config file. Services, being normal .NET executables, can have app.config files just like a command line or Windows app does. I'm personally not fond of the default .NET config mechanism though, so I think I'll roll my own, because that can be quick and easy too, especially when our needs are as simple as they are here. Let's throw together a little XML format for our two settings.


We can load this pretty easily without relying on the magic of the .NET configuration mechanism, by creating a serializable XML data object. We can put it behind a general settings interface, and have a loader class to populate it. The interface will be enough to shield the rest of the app from the persistence details.

Here is the data class, and the settings interface.


The loader class is nearly as simple. Standard XML deserialization.


We can ensure that the code that uses these classes is kept simple, by registering these properly with the IoC container.


This registration will ensure that any IConfigSettings-typed dependency within a given Autofac lifetime context will receive the same ConfigSettings instance. The first resolution of that ConfigSettings instance will be created using the ConfigLoader service.

After identifying what we want to be configurable, we can go about defining some settings providers. I'll remind you that the value of settings providers is that they allow you to define dependencies on a given configuration value, while also decoupling the dependent class from the other unrelated settings that happen to cohabit the configuration source.

Below the work interval settings provider. I'll spare you the very similar code found in the file name settings provider. The IoC registration is likewise straightforward.


With the service now configurable, we can perform one final trick. When a service stops, the process may not necessarily stop. So we need some way to establish when the configuration is reloaded. The natural place for this to happen is when the OnStart event is triggered. Since the config file is loaded when a new IConfigSettings dependency is resolved for the first time in a context, we just need a new context to be spun up with each call to OnStart.

We can make this happen by taking advantage of Autofac's relationship types again. A Func<T> dependency will re-resolve each time the function is executed. And an Owned<T> dependency will cause a new lifetime scope to be created each time the dependency is resolved. So all we need to do is combine the two.

Here is the new service class, with the Func<Owned<T>> dependency.


When OnStart is called, assuming the service isn't already running, we create a new worker, from the delegate dependency. With this new worker comes a new scope, so it's dependency on IConfigSettings will be resolved with a new object, and the config file will be loaded on the spot. Conversely, OnStop allows the worker to go out of scope, resulting in the lifetime scope being cleaned up.

With that, we have finally completed our Hello World Windows service. The guts are all there, and it's cleanly designed around interfaces for maximum testability should we desire to have a unit test sweet. And finally, we use Autofac to conveniently manage a relevant object lifetime scope, making sure our settings are loaded and unloaded at the appropriate time. The actual core behavior of the service itself can be as simple or as complex as it needs to be, while remaining easily accessible and manageable via the worker object and it's lifetime scope.

The full source for the service in its final incarnation can, like all the waypoints, be found in this post's GitHub repo. Thanks for bearing with me through this month of Windows service architecture! I'll see if I can't find something new and different to tackle for next week.

Anatomy of a Windows Service - Part 3

After going through the remaining work I had planned for our service specimen, I realized that the resulting post would be very long, and so I have decided to extend the series by one more post. This week we give the app a job, and add logging.

We've spent enough time working on this service so far that we should probably give it a job at this point.  You may have guessed from the name by this point that I've been intending to make this a "hello world" for services. The types of things that services typically do are either to respond to system events or network communications, or to process units of work as they become available. A nice easy job we can give to our service is to touch a file on a very short interval--an interval short enough that a scheduled task isn't appropriate. Let's say 5 seconds.

We'll worry about handling the interval later. For now, let's just add the behavior so that it happens once, when the service starts.

First we create a straightforward little task-oriented class and interface. This should require no explaining. And of course we'll register this with the IoC container as well. You should be familiar with that code by now, so from here on out, I won't mention the IoC container unless something out of the ordinary is required for the registration or resolution.


With this done, we add a dependency to the interface in our GreetService constructor, and then a call to the task in OnStart.


This will cause the C:\Hello\World.txt file to be created when the service starts. The service will then be running, according to the Service Control Manager (SCM), and can be stopped. But while running it will do nothing more. If you want to see the full sample project source at this point of progress, I have tagged it in the GitHub repo.

This is a good time to add some logging. If we have problems later on, logging will help us to troubleshoot what's going on.  Right now, if the service were to fail somehow, we wouldn't know why. If AutoLog were set to true in the service contructor, then the service would log to the Application event log when it crashed, but not with much useful information. But we don't want that anyway, if our log is an important one. Rather it should have it's own log, so we can always be certain we're looking at the right info and we don't have to sift through hundreds of other entries to find it.

Adding a custom event log is actually quite simple. There are two parts to the task. One is to make sure it gets created when the service is installed. The other is to use it. Creating an event log on install is trivial, mostly because it's already being done, and we can just hook our own data onto that. That's right, the ServiceInstaller class by default contains an EventLogInstaller that is configured to install an event source for our service in the Application event log. All we need to do is override that with our own log and source name.

Since both the installer and the logger class we'll write will need to know those two pieces of text, we'll create a settings provider for them. This will allow us to control the values for both usages in a single place.


Now that we've got a source from which to obtain the proper names, we can add a dependency to our service installer, and use it to override the default event log settings. All we need to do is pull the default EventLogInstaller out of the ServiceInstaller.Installers collection, and change the properties.


When we go to install the service using InstallUtil we will see it output that it is creating a new event source and event log. And if you go to the Windows Event Viewer in the Administrative Tools, you'll find a log called GreeterService.

Of course in order for anything to show up in there, we'll need to log some things to it. The safest way to do that is to throw in a logger object that can try to log to the event logs, but which won't crash out if it can't.  The last thing we want is for the service to crash due to a log issue if the work itself is humming along. (Unless, of course, we need an audit trail or something, but that's a different story.) Note the explicit exception suppression in the logger class below for this purpose.


This logger could be enhanced in a few ways, for more serious services. One good thing to do is to support different "levels" of logging. I have four standard levels that I use. In increasingly voluminous order they are: fatal errors, exception cases, progress, and verbose. An enum parameter added to the logging methods would allow the logger to check to see if it's in a mode that should allow an entry for the specified level. A step further, we might replace the String parameters with Func callbacks. The callers can place the actual work of assembling the message into these callbacks, and the logger would only call them if it had determined that the logging would be performed. By only doing the string manipulation if it is absolutely necessary this keeps performance up and avoids unnecessary failure points.

With the logger object available it would be a good idea to add some logging when the service is starting and stopping. Here are the updated OnStart and OnStop methods from our GreetService class:


Now you can install the service using InstallUtil.exe, and when you start and stop via the SCM, you'll see entries show up in the GreeterService event log.

We'll stop here for now, as this post is getting long and there is much more to do. You can see the full service code to this point so far where I have tagged it in the GitHub repo. Next time will be our final (I promise) post in this series. We'll bring it home by figuring out the best way to make sure the service does its work when we want it to, and then making it easily configurable, and reconfigurable.

Anatomy of a Windows Service - Part 2

This post contains gist code samples. My experiments indicate that these do not display in Google Reader, so if you are reading this there, you may want to click through to the post.

Last time, we dissected a Windows service template project to get a handle on the different moving pieces.  We identified an installer class, as well as two run-time configured installer objects, a service class, and the framework injection point for spinning up a service.

The template is a very simple project, and it gets the job done.  There are some deficiencies though.  From an architectural standpoint, the design doesn't lend itself well to testability, or composability.  It also doesn't provide any good place to bootstrap an IoC container for the installer portion, which would be beneficial to have if we want to have any custom behavior happening at install.  From a feature standpoint, there's no logging, and no worker thread to separate the service control events from the actual behavior of the app.

Let's start out by improving the composability, which will put us on a good footing for testability as well.  The biggest obstacle to that in the template project was all of the direct instantiation and configuration of objects.  For example, in the template project, the ProjectInstaller class's constructor, which is an entry point that is triggered directly by msiexec or installutil, was responsible for creating and initializing the sub-installers.  In our application, we'll defer to an IoC container, rather than construct these objects directly.

Let's add a reference to my personal favorite IoC container, Autofac, and then create a bootstrapper that will do the registration necessary for the installer portion of the app. We'll leave it empty to start, because we haven't established what we'll need from it yet.


Now lets create the ProjectInstaller.  The constructor, since it's an application entry point, will have a direct reference to the IoC container.  We'll leave out the needless code we identified last time, but don't forget the crucial RunInstaller attribute.  The constructor is left responsible for setting up the Installers collection.  If we assume that we'll be registering all our installers with the IoC container, then Autofac makes this easy for us with the IEnumerable relationship type, which will resolve all implementors of the specified service that are available at the resolution location.  Let's put all that together and see what we get.


Compared to the template project, we've eliminated some explicit object instantiation, but we've also lost some explicit configuration.  That has to be exported to somewhere else.  There are a couple of options for re-creating this functionality with our IoC container in place.  One is to give these exact configuration responsibilities, i.e. the setting of properties after construction, to the IoC container.  This is not a strong-point of most containers--even Autofac.  It can be done, but it's not really straightforward.  It results in complex registration code that doesn't communicate well what's going on.

Alternatively, we could move these responsibilities into factories.  I've used this strategy before, and it can be advantageous if there are classes that will have dependencies on the factories themselves.  But in this case, the only thing that would need to know about the factories is the IoC.  So instead, I'd recommend deriving special-purpose sub-classes of the desired installer base classes, and putting the configuration in those constructors instead.  Here's what that looks like.



Simple, straightforward, self-contained, and single responsibility.  Now we just need to update the IoC registration to include them, and they'll get picked up automatically by the ProjectInstaller constructor.  We could take advantage of Autofac's fluent registration API and auto-wiring to do this without referencing the individual types.  But I have these two classes in separate namespaces (the service installer is in the same place as the service, because these must come in pairs), and the ProjectInstaller derives from the same base, it would actually take more code to deal with these special circumstances.  So explicit registration it is.


Note also here that we've registered these as InstancePerLifetimeScope, because there's really no reason more than one of either of these should be floating around.

That covers the install portion of the service.  We now have separated the three installation components into their own separate places.  Changes to the configuration of any one of them are isolated in their own locations in the code.  And as a bonus we've made the ProjectInstaller constructor trivially simple.

Let's shift gears and take a look at the service portion of the application.  We'll set up a blank bootstrapper for this portion as well. I'll refrain from posting the code as it's essentially identical to what I showed you for the other one.

The remaining two significant pieces that need to be implemented to replicate the functionality we saw in the template project are the service class itself, and the program mainline.  The mainline is going to be simple just like the ProjectInstaller.  In fact it's structure mirrors the installer very closely, just with different type names in a few strategic places.


I've chosen to request an IEnumerable, even though we only have one implementation.  Maybe we'll add more services to this assembly some day, and maybe we won't.  To make a decision on that basis triggers my YAGNI reflex.  I used an IEnumerable resolver just for my own aesthetic sense, because the framework runner takes a collection anyway.

Next let's look at the service class itself.  This is another class that, in the template project, had a lot of noise that we identified as unnecessary.  So in this iteration, it's going to be comparatively short and sweet.


There's very little to see here, and that's very much as it should be, since our service doesn't actually do anything yet.  We just cut out the noise and were left with something trivially simple.  So let's move on to the last piece left to put in place: the service bootstrapper.


We're just blasting through code files here, as this one is another very simple one, with just one registration.  We'll add more, but for right now we have completely matched the functionality of the template project.

Before we put the project to bed for another week, though, there's a small refactoring we can make that will make a nice little illustration of the benefits of how we've organized the project.  You may have noticed that we have some hard-coded strings in the constructors of the service and the service installer.  And particularly, there's one that's intentionally repeated, since it has to be the same in both places.  We can eliminate the hard-coded string and make sure that the values in both locations always match, by using the setting injection pattern I described a few months ago.

With just a few lines of code we can set up the necessary setting dependency.



With this class and interface in place, we can add a dependency to the constructors of the installer and the service.



And finally, in order to support the dependency injection, we just add one new registration to each of the bootstrappers.


That's the end of our work for this week.  We have replicated the template project's functionality, such as it is, and run through a lot of simple little snippets of code.  Structurally, the Solution Explorer doesn't look a ton different, but I think the code is simpler and more straightforward.  Plus we have built a good foundation on which to add the functionality that people expect of a service.  All the code for the project we worked on in this post can be found on GitHub at https://github.com/cammerman/turbulent-intellect/tree/master/HelloSvc.

Next week, we'll add logging to an event log, and we'll make it actually do some work.  Finally, we'll figure out how that work needs to get done, inline, or in one or more worker threads.

Anatomy of a Windows Service - Part 1

div.aside { color: #A0A0A0; margin-left: 2em; margin-right: 2em; }

This post contains gist code samples. My experiments indicate that these do not display in Google Reader, so if you are reading this there, you may want to click through to the post.

I recently had to estimate the effort for creating a one-off Windows Service for massaging some document indexing data as it passed between a capture system and an ECM system. I was quite confident in my ability to code up the "massaging" part, but the service part was unfamiliar territory. So I decided to do some investigation and figure out just what it takes to get a service set up, what special considerations need to be made for logging and injecting configuration, and how to preserve good design sensibilities through it all.

The easiest way to create a Windows service is to use the associated project template in Visual Studio. I specifically wanted to work in .NET 4.0 for the investigation, as I would be on the actual project, but unfortunately I don't personally have a license for the full version of Visual Studio 2010, and VS Express 2010 doesn't have a project template for Windows services. Fortunately, I do have VS 2008, and the BCL infrastructure for services hasn't changed between 3.5 and 4. So I decided to generate what I could in VS 2008, add some googling, and use the combined information to create a service from scratch in VS Express.

This week, we'll analyze what VS will give us for free with the project template and special code-generating actions. We'll identify the purpose of the different pieces of code, and make note of how we can use this information to build our own service, complete with installers, from scratch.

So to start, let's take a look at the structure that the project template creates for us. Here's a shot of what the initial solution looks like in a brand new Windows Service project.

Note the two structural differences between this template project and a blank or console project. There's an extra reference to System.ServiceProcess.dll, which we'll need to remember, and there is an extra component class called Service1. There's also a .Designer.cs file for Service1, but this is really just because Visual Studio automatically adds these any time an item that is a subclass of

Component

is added to your project, either via a template, or the "Add New Item" command.

Before we take a look inside the service class itself, we should see how the Program class differs from other types of project templates. Here's the Program class.

There are a few differences to note here. The biggest difference is obviously that the main method contains some generated code. It actually looks similar to what is generated with the new Windows Forms project template. We have the service class, representing the actual core of the program, being instantiated and then passed to a runner method provided by the framework that handles spinning up the appropriate context.

Now lets take a look at the service class. It comes in two parts: the main class file, and the designer file. First the main class file.

The most prominent features here is the base class,

ServiceBase

, and the two override methods, OnStart and OnStop. ServiceBase is, of course, the common base class that all .NET windows service classes must derive from, in order to take advantage of the framework support. The overrides are two of the possible entry points into the service. They correspond directly to the actions that you can take on a service in the Services control panel. OnStart is where you would place the code that processes the arguments, spins up workers, etc. Naturally OnStop is the other end of things, where you'd put your cleanup code to prepare for termination. There's also a call to InitializeComponent in the constructor, which is typical of generated Component-derived classes. We'll see if there's anything interesting happening there next, in the designer file.

There's a bunch of generated code here, but there's actually less going on. So what is all this doing? We can easily see an implementation of a Dispose idiom, which is disposing of any components contained inside this one. But when we look back at the InitializeComponent method, the Container is constructed and then left empty. The only significant code here is the assignment of the ServiceName property. We'll need to remember this for when we create our own service class from scratch. Everything else in this file could be dropped. The ServiceName initialization could just as easily be put directly into the constructor.

What we've seen so far is the absolute minimum service. If you put some behavior into the OnStart method, you could run from the command-line, and it would operate. But it wouldn't show up in the Services control panel, because it doesn't have an installer to register it. And that means you couldn't Stop it once it was started.

What does it take to register a service with the OS? VS offers a shortcut to generate the code for this as well. If you right-click the service class Service1.cs in the solution explorer, the context menu has an "Add Installer" option, which adds a new class to the solution, called ProjectInstaller. This is the last piece of our example puzzle, and it's another two-parter. First the base class file.

There's not much code here either. Like the service, we see this is a Component-derived class, by way of

Installer

) with an InitializeComponent method called in the constructor. We'll take al ook inside there in a moment. First, make note of the attribute on the class. The RunInstaller attribute is a signpost for the InstallUtil.exe installer tool that comes with the .NET Framework, or for the VS Setup Package Custom Action Installer. The Boolean true argument tells the installers that that they should indeed invoke this installer class when they find it in the assembly.

The attribute is important, but it's not the meat of the class. That's found in the designer file.

The form of this file is very similar to that of the service's designer file (and any designer file, really). Once again we see the disposal idiom, and once again, the container that it disposes is never populated with anything. The body of the InitializeComponent method is concerned with creating and initializing two more installer objects. One is a

ServiceInstaller

, which is a class that knows how to install a particular service in the registry and such, given a ServiceBase representing it. The

ServiceProcessInstaller

handles install tasks common to all services, and so there should be only one inside a given project installer.

The MSDN documentation  doesn't go into any more detail about what the responsibility differences are between these two types of installer, or why they are peers, as far as the main Installer is concerned. But it is clear about

how

they should be used. The installer objects need to have properties initialized to indicate how the install should proceed, and then they must be placed in the Installers collection. This also could just as easily be done in the constructor, eliminating the need for the designer file.

One important thing the documentation highlights is that the ServiceName property on the ServiceInstaller must match the same property on the service which it is associated with, since this is the only thing that links the service registration with the actual implementation. We'll examine the other installation-guiding properties, on both objects, in detail when we put together our own service from scratch.

For now, we've covered everything in the project template. What we have here is a complete, spartan service that does nothing. It will install, simply by compiling the executable, and then running InstallUtil.exe from the .NET framework folder against it. And it will run from the service control panel... but it has no visible side-effects.

Next time, we'll get to work replicating the moving pieces of this solution in a spanking new, blank C# 4 project. It will look a little different, structurally, but all the prominent features will still be there.

P.S. All the code that appears in this series will be available on my github page. For now, I have it in this repo:

https://github.com/cammerman/turbulent-intellect

. The code for this post in particular is found here:

https://github.com/cammerman/turbulent-intellect/tree/master/ServiceExample35

. I expect to reorganize this a bit in the future, but I'll update this link when I do so.

The First Rule of Extension Methods

Extension methods are a tremendously useful feature of C# 3. Briefly, they allow you to bundle new behavior into an existing class that wasn't included by the class's original author, but without opening up the implementation internals of the class. In a very general sense, this is useful if you come up with useful behavior related to a class and the best place for it is IN that class, but you don't want to bend or break encapsulation by inheriting. I won't spend any more words on an introduction, but rather offer a caution, for those of you who have seen their usefulness and would like to start taking advantage.

Before you even begin to think about whether a particular behavior belongs on an existing class, or as a service, or what have you, you should internalize one cardinal rule of dealing with extension methods. Thankfully this First Rule of Extension Methods is nothing like the First Rule of Fight Club, because if it were, I wouldn't be able to help you at all. No, the First Rule of Extension Methods is: DO NOT export extension methods in the same namespace as your other public classes.

The reason for this is very very simple: odds are very good that if you have come up with an idea of including a particular new behavior on an existing class, someone else has or will as well. There will only be a finite number of relevant names to give this new method, which means that if it's a simple method, especially one without parameters, there's a decent chance that the signatures of your method and this other programmer's will be identical. And if both you and this other programmer happen to include the method on public static classes in a namespace with other functionality the user needs, then someone trying to use both libraries could very likely run up against identifier collisions with the two methods.

Note that the rule begins with "do not export..." This is important, because you can be 100% certain to save your users any collision headaches if you just don't export your extension method. Why wouldn't you export your extension method? Well, there's a very good chance that just because you found your extension method to be useful and, dare I say, clever (Tyler Durden: "how's that working out for you?"), that doesn't mean the consumer of your library will. So consider carefully whether you should even make the method public at all, or rather just keep it internal.

If you decide that your method is just so darned handy that to keep it locked up would just be sadistic, then make a separate namespace for each batch of related extension methods you want to export. If the user decides they'd like to use them too, they can import the namespace as necessary. This will minimize the chance that they will need to fully qualify the namespaces of the other library with the identically named method whose author wasn't as responsible as you were and polluted their namespaces with extension methods.

This tactic won't guarantee anything, because there's always a chance some other library could use the same namespace identifiers as well. But with each additional namespace, you dramatically increase possible naming permutiations, and proportionally decrease the chance of collisions.

Useful Extension Methods 1 through 3 of N

Quite often when I'm writing code I'll notice a very small bit of logic that keeps popping up all over the place. It's usually something so trivial that most people barely notice it. But it's also usually something that shows up so often that, despite being small and ignorable, it constitutes a fairly constant level of noise in the code. By noise I simply mean something that takes up more characters than it needs to, obscuring the real meat of the logic of your code. Common functionality like this that everyone knows and understands should just get out of the way, fade into the background, and let the unique logic stand out.

You may have also heard of this noise idea by another name: accidental complexity.

As Reg Braithwaite has pointed out, looping syntax is one of these things. With the ubiquity of IEnumerable and the advent of extension methods in C#, there is almost never a good reason to write an explicit for loop anymore. Looping is a ubiquitous bit of logic that nonetheless takes up quite a lot of characters. Even the vaunted foreach loop is now officially more verbose than it very often needs to be.

Microsoft made it easy to get rid of the explicit loop when what you are doing is essentially a mapping operation, with the inclusion of the IEnumerable.Select extension method.

Say your Foo class has a static function taking a Bar and returning a Foo, and you want to use this function to take a collection of Bars and create a collection of Foos.

You could do this:

IEnumerable<Bar> bars = GetAllBars();
List<Foo> foos = new List<Foo>();

foreach (var bar in bars)
foos.Add(Foo.FromBar(bar));

Or you could do this, which is obviously much more concise:

IEnumerable<bar> bars = GetAllBars();
IEnumerable<foo> foos = bars.Select(Foo.FromBar).ToList();

Note that the Select function completely takes care of the looping logic. Once you know that, this code reveals itself as being extremely elegant. But what if the action you're taking doesn't return anything? You're "stuck" writing an explicit loop, right? Not at all.

Take this code:

IEnumerable<Foo> foos = bar.GetAllItems();

foreach (var foo in foos)
PrintToScreen(foo);

To start removing the noise, first define an extension method for IEnumerable called ForEach. This is extension method #1.

public static IEnumerable<X> ForEach<X>(this IEnumerable<X> lhs, Action<X> func)
{
foreach (X x in lhs)
{
   func(x);
   yield return x;
}
}

Then rewrite:

bar.GetAllItems()
.ForEach(PrintToScreen)
.ToList();

Now there's just the issue of that nasty little ToList call. Right now, we need that in order to force the collection to be iterated. The yield return syntax essentially causes a function's execution to be deferred until an element is actually requested. This is actually potentially useful even if none of the things you need to do will return values. You can chain together a bunch of actions on the collection by chain-calling ForEach with different delegates. But it's still silly to create and throw away a List just to do this.

So we create an Evaluate function that does a simple explicit iteration and nothing more, to force the iterator to be evaluated. This is extension method #2.

public static void Evaluate<X>(this IEnumerable<X> lhs)
{
 foreach (X x in lhs) ;
}

And now you can replace ToList with Evaluate, which will iterate the collection without allocating a new List.

bar.GetAllItems()
.ForEach(PrintToScreen)
.Evaluate();

This is nice, and is going to be useful if we need to chain ForEach calls. But when we don't need that there's still that Evaluate call at the end that's going to be repeated every time we want this functionality, which could be an awful lot. So, let's get rid of that too.

To do that we define a Visit function (named for the Visitor Pattern), that will call ForEach with the given delegate, and then Evaluate as well. This is extension method #3.

public static void Visit<X>(this IEnumerable<X> lhs, Action<X> func)
{
  lhs
      .ForEach(func)
      .Evaluate();
}

Now we can finally get all this done in a single function call:

bar.GetAllItems()
  .Visit(PrintToScreen);

This takes some getting used to. But it really has the potential to condense your code. It isn't readily apparent looking at one bit of code, but once you start talking about nested loops or consecutive loops, you'll see the difference. Not to mention the total effect it will have across your codebase. Loops are everywhere. Shave off 50 characters from each one and your talking about a lot of characters in aggregate.

For my part, after I determined to avoid explicit loops whenever possible, the comparatively verbose explicit looping syntax became almost painfully extraneous to my eyes. I feel like function calls are much more elegant.

LINQ: Not just for queries anymore

Take a look at this LINQ raytracer, discovered via Scott Hanselman's latest source code exploration post.

For those of you who don't know what the heck LINQ is, it was created as a language extension for .NET to allow querying of datasets using an in-place SQL-inspired notation. This was a huge step up from maintaining separate bodies of stored queries, or hard-coding and dynamically assembling queries as strings in your source code.

Lately LINQ has been enhanced and expanded upon even further to become a more broadly usable miniature functional/declarative language within .NET. This is impressively illustrated by the raytracer code. The author disclaims that it's probably not the best way to write it, which is probably true. But this in no way detracts from its illustration of the power, expressiveness, and flexibility of LINQ.

I love it! It's a great example of taking a deceptively simple language and showing its power by doing things with it that aren't strictly within the purview of its design. It reminds me of some SQL code that I've written for my current employer, to get functionality out of inefficient PL/SQL and into blazing-fast pure SQL. Stuff that it was said couldn't be done in pure SQL. Of course, this is usually said by people who think that SQL is a second-class citizen among languages, not realizing the power of its declarative foundations. This is something that I hope to write about more extensively in the future, either here or in a new SQL blog I'm considering starting with a friend and coworker.

Anyway, I'm tempted to say that there's a bit of a down side in that this LINQ raytracer is mostly comprised of LETs, which feel more imperative than declarative. However, I've long claimed that just such functionality in actual SQL would make it a tremendously more compact and elegant language, so I won't complain too much about that. =)

My gut reaction upon seeing this code is that it feels like a crazy hybrid of LISP and SQL, but it's written in C#. Which all makes my head spin, but in a good way. I love that C# is becoming such a great hybrid of procedural and functional programming paradigms.