New Open Source Projects

I’ve recently released several new open source projects on GitHub:


Kamsar.WebConsole

A library that makes it easy to build progress-reporting "console" pages in websites. These are useful for pages that execute long-running processes like search index rebuilds, CMS publishing, etc.
An example WebConsole page using the default styling

https://github.com/kamsar/Kamsar.WebConsole

Beaver

Beaver is a flexible build/configure/deploy system for ASP.NET projects based in PowerShell. It is designed to simplify the task of maintaining builds that may span many target environments (such as dev, QA, production) where each has its own slightly different configuration requirements.

Everything in Beaver is based on a consistent, simple idea: the "pipeline." Pipelines are simply a folder that contain "buildlet" files that perform tasks in alphabetical order. Their structure is quite similar to POSIX System V-style init scripts, only instead of a pipeline per runlevel there's a pipeline per build stage. Buildlets can be written in several domain-specific languages (PowerShell, XDT Transform, MSBuild, or even sub-pipelines), and you can implement your own buildlet provider if you wish. Using this architecture allows you to implement small, easily digestible  single-responsibility units of action - as well as gain an immediate understanding of how the build process works by simply looking in a few folders and seeing visually the order things run in and their descriptions.

To manage multiple target environments, Beaver implements a powerful inheritance chain of pipelines. Archetypes are a way to extend the global pipelines to perform a specific sort of configuration that might apply to multiple environments - for example, configuring live error handling or hardening a CMS site's content delivery-type servers. Environments then declare themselves as using 0-n archetypes (in addition to being able to extend the global pipelines themselves), allowing them to inherit cross-environment configurations.

Beaver was originally designed to support build and deploy operations for the Sitecore CMS. However, Sitecore is merely a set of archetypes - the system can be used for any sort of ASP.NET project.

Unicorn

Unicorn is a utility for Sitecore that solves the issue of moving templates, renderings, and other database items between Sitecore instances. This becomes problematic when developers have their own local instances - packages are error-prone and tend to be forgotten on the way to production. Unicorn solves this issue by using the Serialization APIs to keep copies of Sitecore items on disk along with the code - this way, a copy of the necessary database items for a given codebase accompanies it in source control.

Unicorn avoids the need to manually select changes to merge unlike some other serialization-based solutions because the disk is always kept up to date by the event handlers. This means that if you pull changes in from someone else's Sitecore instance you will have to immediately merge and/or conflict resolve the serialized files in your source control system - leaving the disk still the master copy of everything. Then if you execute the sync page, the merged changes are synced into your Sitecore database.

Installing Unicorn is very simple and using it is nearly automatic as long as you follow the conventions that allow it to work (local databases required).

Selecting a Sitecore Rendering Technology


Sitecore has a dizzying array of ways you can write renderings. While there is documentation that explains what they are, there’s a lot less about where each kind should be used. This will attempt to explain when you should use each kind of rendering.

So what kinds of renderings are there?
  • Sublayouts (UserControls/.ascx)
  • Web controls (WebControl/.cs)
  • Razor renderings (.cshtml) - Sitecore MVC-style
  • Razor renderings (.cshtml) - WebForms-style
  • Method renderings (.cs)
  • URL renderings
  • XSLT renderings (.xslt)
  • Custom - yes, you can write your own rendering provider
That's a lot of ways to emit HTML to a page. Let's take a look at each kind and examine their strengths and weaknesses.

Sublayouts (User Controls)

These are probably the kind of rendering you should be using most of the time unless you have Razor at your disposal. Sublayouts are confusingly named because most of the time they are simply a rendering and not an actual subdivision of a layout. These are basically identical to a traditional .NET User Control - they have a page lifecycle, code behinds, events and all the other traditional Web Forms trappings. They have a relatively HTML-like appearance that makes them sensible to edit if you have HTML/CSS folks collaborating with you, unlike the C#-based renderings.

However they also have the same issues as their User Control cousins. Web Forms' at times utterly verbose syntax and confusing event/postback model can introduce bugs. Highly branched markup emission is also very hard in User Controls because the markup is all encoded in the .ascx file, and you have to resort to MultiViews or PlaceHolders and setting a ton of Visible properties to make it work.

Verdict: Use these for relatively static markup emission or places where the Web Forms event model will help you - like custom form renderings.

Web Controls

Web controls are simply C# classes that derive from the Sitecore WebControl base class. Web controls are perfect if you have to do a rendering whose markup has a lot of branching, for example a list that might have two or three different kinds of elements in the list because you can modularize the rendering into C# methods.

On the other hand WebControls can be extremely hard to read if not written in a disciplined manner. There is no obvious HTML emission, so you'll have trouble with HTML/CSS folks when they need to change markup or add a class - it's all in C#. You can also write spaghetti renderings that are very hard to follow how the code runs. You also have to remember that unless you override the GetCachingID() method, your WebControl cannot be output cached.

Verdict: Use these for places where you need tight control over HTML emitted, or have a lot of branches in your markup emission.

Razor Renderings (MVC-style)

Razor is a templating language traditionally associated with ASP.NET MVC. It dispenses with a lot of the usually unnecessary page lifecycle of a Web Form for a more simple template that is both readable as HTML and allows a decent amount of flexibility in terms of composing branch-heavy renderings. If you're using Sitecore's MVC support it's a no-brainer to use Razor renderings for nearly all purposes.

However to use the built in Razor support you must use Sitecore's MVC mode - which means you have to do everything in Razor. You also have to register controllers and views as items, and lose a lot of module support - for example, Web Forms for Marketers cannot presently run in MVC mode. At present this makes it nearly untenable to implement most real world sites using Sitecore's MVC mode.

Verdict: If you've got a Sitecore MVC site, use it.

Razor Renderings (custom)

There are a couple of third party shared source modules (such as Glass) that have implemented a custom rendering type that invokes the Razor template engine outside of a MVC context. This means you could reap the benefits of a Razor template's syntax, without needing to resort to Sitecore's shaky configuration-over-convention MVC implementation. These are dependent on how you feed the view data to the Razor rendering however, and each implementation works slightly differently.

Verdict: If you're using one of these modules, you'll probably implement most of your renderings in Razor without needing Sitecore MVC

Method Renderings, URL Renderings

These render output from a C# method that returns a string, and a given URL respectively.

Verdict: There's almost no good use case for either of these rendering types.

XSLT Renderings

These were once the main type of rendering in use. They use extended XSLT syntax to transform an item "XML" into a rendering. While they can be useful for EXTREMELY simple renderings, they do not scale well to any level of complexity. Most very simple renderings may at some point gain a measure of complexity, which would then mean either very ugly and slow XSLT or a complete rewrite in a different rendering technology. Do yourself a favor and save the rewrite - use something other than XSLT in the first place.

Verdict: Don't use.

Feel free to take these recommendations with a grain of salt. These are my opinions, based on the projects I've worked on. Your project may come across good reasons why I'm absolutely wrong about one of these options. Keep your eyes open :)

C#4: Defining optional parameters in interfaces is very unreliable

See this example

In short, when the object is treated as the interface the default value from the interface applies. If it’s treated as the implementation, the implementation’s default applies. But the implementation isn’t required to have a default for it at all - in which case the call will fail, unless it’s treated as the interface.

It’s all very unreliable, and while I’m sure it works this way for good technical reasons under the hood, realistically it doesn’t present a very coherent interface.

Making asp:TextBox do HTML5 input types

I was watching a presentation on HTML5 from one of my coworkers a few weeks ago and we were talking about the new input types in HTML5, like numbers and dates. They’re backward compatible with non-HTML5 browsers (which render them as text boxes), but provide very useful UI features particularly to smartphones where the on screen keyboard can change to be more appropriate.


Anyway, let’s just say they’re a good idea. A good idea that, much as I like ASP.NET, are not likely to make an appearance any time soon in the core framework. So, I looked around to see if I could figure out a way to patch a standard asp:TextBox to render these new HTML5 types. Yes, you MVC types already can do this easily. For the rest of us that are using third party CMS tools, WebForms is still the way of life :)


I came across a post by Phil Haack about a sneaky method of overriding HtmlTextWriter to change the attributes output by WebControls, and repurposed it to accomplish the HTML5-ization. Basically you override the AddAttribute() method and make it ignore the attribute you want to manually handle - in this case, the type attribute of the TextBox. Then I made a new TextMode-style enum that encompasses the HTML5 types and wrote a bit of logic that manually creates the right type attribute. The new enum is exposed in the Html5TextMode property, and the value is proxied back into the default TextMode property if it’s compatible with it.


So how do you use it? It’s pretty simple:

<prefix:Html5TextBox runat="server" Html5TextMode="Tel" ID="telephone" />

And the code for the control itself:

public class Html5TextBox : TextBox
{
 /// <summary>
 /// When using non-HTML5 constructs this mode will be accurate. If using HTML5, it will return SingleLine.
 /// </summary>
 public override TextBoxMode TextMode
 {
  get
  {
   try
   {
    return (TextBoxMode)Enum.Parse(typeof(TextBoxMode), Html5TextMode.ToString());
   }
   catch(ArgumentException) { return TextBoxMode.SingleLine; }
  }
  set
  {
   Html5TextMode = (Html5TextBoxMode)Enum.Parse(typeof(Html5TextBoxMode), value.ToString());
  }
 }

 /// <summary>
 /// Sets the text mode of the control including HTML5 text modes such as Num and DateTime
 /// </summary>
 public Html5TextBoxMode Html5TextMode
 {
  get
  {
   object textMode = this.ViewState["5Mode"];
   if (textMode != null)
   {
    return (Html5TextBoxMode)textMode;
   }
   return Html5TextBoxMode.SingleLine;
  }
  set
  {
   this.ViewState["5Mode"] = value;
  }
 }

 /// <remarks>
 /// Adds the normal attributes (since the writer is actually a PatchedHtmlTextWriter from Render() the type attribute won't be rendered),
 /// then explicitly adds the appropriate type attribute including the HTML5 extensions
 /// </remarks>
 protected override void AddAttributesToRender(HtmlTextWriter writer)
 {
  base.AddAttributesToRender(writer);

  string type = null;
  if (Html5TextMode == Html5TextBoxMode.SingleLine)
   type = "text";
  else if (Html5TextMode == Html5TextBoxMode.DateTimeLocal)
   type = "datetime-local";
  else type = Html5TextMode.ToString().ToLowerInvariant();

  if (type != null)
   (writer as PatchedHtmlTextWriter).AddTypeAttribute(type);
 }

 /// <remarks>
 /// Patches the type of HtmlTextWriter that the control renders to
 /// </remarks>
 protected override void Render(HtmlTextWriter writer)
 {
  base.Render(new PatchedHtmlTextWriter(writer));
 }

 /// <summary>
 /// A version of HtmlTextWriter that intentionally ignores the "type" attribute when it is added.
 /// </summary>
 /// <remarks>
 /// Technique courtesy of Phil Haack
 /// http://haacked.com/archive/2006/01/18/UsingaDecoratortoHookIntoAWebControlsRenderingforBetterXHTMLCompliance.aspx
 /// </remarks>
 private class PatchedHtmlTextWriter : HtmlTextWriter
 {
  internal PatchedHtmlTextWriter(HtmlTextWriter basis) : base(basis) { }

  public override void AddAttribute(HtmlTextWriterAttribute key, string value)
  {
   if(key != HtmlTextWriterAttribute.Type)
    base.AddAttribute(key, value);
  }

  public override void AddAttribute(string name, string value)
  {
   if(name != "type")
    base.AddAttribute(name, value);
  }

  /// <summary>
  /// Manually adds a type attribute
  /// </summary>
  public void AddTypeAttribute(string value)
  {
   base.AddAttribute(HtmlTextWriterAttribute.Type, value);
  }
 }
}

/// <summary>
/// Extends the TextMode enum with additional HTML5 types
/// </summary>
public enum Html5TextBoxMode { MultiLine, SingleLine, Password, DateTime, DateTimeLocal, Date, Month, Time, Week, Number, Range, Email, Url, Search, Tel, Color }

Sitecore multi-site installations and output caching

When running Sitecore in a multi-site configuration you may run into an odd issue: output caching may seem to get too greedy and not clear when you’d expect it to.

There’s a simple culprit: the default Sitecore setup includes an event handler, Sitecore.Publishing.HtmlCacheClearer, that is invoked on the publish:end event. This event handler has a list of sites assigned to it, and the default is “website” - great, until you need to have more than one site and publishing doesn’t clear your site’s output cache. Fortunately it’s easy to configure more sites: just add more site nodes to the XML. You cannot however use config includes to allow each site to individually add itself to the list from its own config file.

There’s also a nuclear option: you can implement your own event handler that clears all sites’ caches. I’m not sure if this would have a detrimental effect on any of the system sites (i.e. shell), but you could exclude it. An example of doing that:

string[] siteNames = Factory.GetSiteNames();
for (int i = 0; i < siteNames.Length; i++)
{
   SiteInfo siteInfo = Factory.GetSiteInfo(siteNames[i]);
   if (siteInfo != null)
   {
        siteInfo.HtmlCache.Clear();
   }
}

Making Visual Studio 2010 Web.config Transformations Apply on Every Build

Disclaimer: This was written about Visual Studio 2010 RC. I suspect the same approach will continue to work with the final release but there are no guarantees.

When I first heard about Web.config transformations in Visual Studio 2010 I was excited. Our build process involves keeping a working build in SVN such that our dev servers may merely check out a working copy and go, and the problem of people committing their own, or the dev server's, web.configs had been a problem. Someone else's settings would come to your machine, causing explosions and other hijinks. Wouldn't it be great, I thought, to have each developer keep a delta for their own web.config settings and apply them only on their box, so not only would the commit problem be solved, we could also keep everyone's personal configs safe in SVN.

My problem was that I assumed the transformations would apply more or less at runtime. Their dirty secret is that they're only applied when you publish the site. We don't use VS publishing ever, as our release process is entirely based around SVN tags. This also kills its utility for the situation of having multiple developers with disparate web.configs being able to maintain individual transformations that apply to their computer only.

Feeling disappointed, I set out to remedy the situation. As I knew this was all handled by MSBuild thanks to Hanselman's talk at MIX10, I went spelunking in the global targets files. I found the definition of the TransformWebConfig target, which I learned about from this blog post, in the C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.targets file. With a little fooling around, I began to understand how it works in its essential nature: the Microsoft.Web.Publishing.Tasks.TransformXml task (defined in an assembly in the same directory as the targets file).

Armed with the knowledge of how the task works by example and Reflector, I set out to try and make it do what I wanted: after a build, apply the correct transformation file and overwrite the Web.config. Since there had to be a source file other than the Web.config to transform on, I settled on "Web.generic.config" as a name for the untransformed config.

It turned out to be quite simple to implement. In my project's .csproj file, there lies at the bottom a commented out section with BeforeBuild and AfterBuild tasks already skeletoned out.

I uncommented the AfterBuild task and added the TransformXml task to it, like so:

<target name="AfterBuild">
    <TransformXml Source="Web.generic.config"
             Transform="$(ProjectConfigTransformFileName)"
             Destination="Web.Config" />
</target>

Voila - after each build, publishing or no, the Web.generic.config has the active solution configuration's transformation applied to it and copied to the Web.config. Note that unlike the version run during publishing, this one only checks the root Web.config. Subdirectories' config files are ignored. Someone better than I with MSBuild could probably remedy that shortcoming :)

I suspect that with the right importation of tasks into your project file, this solution could be made to work against regular old App.config files as well as Web projects, but I don't do enough app development to dive particularly deeply into it.

Visual Studio 2010 New Stuff

Visual Studio 2010 brings about a bunch of new "coder-centric" features, some of which I'm practically drooling over. Post based on a session with Jeff King, Program Manager of Visual Studio Web Tools. He's a cool guy.

  • Improved CSS2.1 support, particularly in the designer view for attribute selectors and the like.
  • Multiple monitor enhancements: Grab a file's tab in the editor and drag it somewhere else to have a mini window that can be moved to a different monitor
  • There's already support for C# snippets, but now there's also HTML/JS snippets. Default ones included for standard things eliminates typing runat="server"s and <style type="text/javascript"> or `` from being so boilerplate. Type tag ("a", "requiredfieldvalidator") or name of snippet and two tabs to insert. Supports sneaky things like when inserting a RequiredFieldValidator it will set the ControlToValidate to the last validatable control it finds automatically. Jeff literally wrote up a form with 2 fields, validation, and a radio button list in less than 1 minute and about 80 keystrokes including whitespace.
  • Neat trick that I think may exist in 2008 as well: make selection, right click, surround with... (update panel, div, etc)
  • Pretty darn good js intellisense based on a virtual evaluation engine - including intellisense for jquery with no vsdoc file, and intellisense on jquery extensions added with extend() that aren't even obviously declared as a function even. Very impressive virtual DOM implementation.
  • Ability to "consume first" by referencing a nonexistant object, and have the object created based on the context of your mentions of it. Could prove quite useful to write tests prior to some code's existence as the consume first model would create only the minimal set of required fields to work.
  • Ctrl-, opens a 'quick find' to find files in large projects - very much like jump to file in winamp - and set focus to them. I like this idea, even if you organize well the file lists just get too long in larger projects.
  • "call hierarchy," a reflector-style analysis of who calls function x and what functions x calls
  • Application packaging, which includes files, databases (including merging schema changes), ssl certs, web.config transforms, etc into a single .zip package for deployment. Uses the MSDEPLOY tool that can be implemented on the server using a web service to eliminate the need for non-HTTP access as well as provide rsync-style synchronization. If you'll excuse me being 12 years old for a minute, the rather amusing out of context phrase "you can take your package and give it to the community" was said while discussing this feature.
  • Web.config transformations allow automatic transformation of a web.config file based on project configurations (e.g. debug/release), transformations are diffs from the main web.config and use a XSLT-like (but less verbose) language. This looks like it would be a wonderful tool for development groups like mine (and @u2elan's as well) who have multiple developers doing local development, but each requiring a slightly different configuration per machine. The only problem is that it would require creating a build profile for every developer, and them to remember to switch to it before building on their machine. I suggested to Jeff King that some sort of provider model to programmatically select which transformation(s?) to load might be a good idea, so we could implement something like a switch based on machine name, or (@tvancil's idea) logged in username. He said he'd have some people call me to discuss it.

New Features in ASP.NET 4.0

ASP.NET 4.0 is going to introduce a lot of pain-point reducing features, particularly around the efficient delivery of content and standards compliance. Here are the main points as I saw them in Stephen Walther's talk this morning.

  • FormView, as well as several other table-based controls: RenderTable="false" disables table wrappers
  • ListView: no longer requires a , only an item template. Be careful with empty data sets though, as your wrap tags might be left exposed.
  • ViewState can be globally disabled and then selectively enabled on controls using the Control.ViewStateMode="Disabled" (Enabled, Inherit) - it defaults to 'inherit'. This is different than EnableViewState=false in that (1) it inherits and (2) you can re-enable it as needed on specific controls
  • Control.ClientIdMode property allows you greater control over the ID attribute emitted. Options include "Legacy," [how it is now] "Static," [use what you said] "Predictable," [wasn't defined] or "Inherit" allows overriding the ID value on a control. Can be set in the web.config element as well but that's probably a bad idea to change except on an as-needed basis to avoid ID collisions in repeating controls. It was noted that the "Legacy" option would probably have its name changed before release.
  • New Response.RedirectPermanent() creates a 301 permanent redirect as opposed to Response.Redirect()'s 302 temporary redirect
  • ASP.NET "velocity" distributed caching, allows creating custom cache providers as well as caching on multiple machines
  • Web.config transforms allow multiple iterations of a web.config to be stored with the application. More about these later.

MIX09: Keynote Announcements

Bill Buxton of MS Research led off, looking quite like the professor in Back to the Future. The key to UX, he says, is to balance budget with multiples - multiple ways of accomplishing the same task, that can be chosen between. "The transitions are just as important as the states," he says. Better transition documentation would end up with a better UX implementation of said process in the final product. "Ideation," basically brainstorming, actually lowers usability unless followed by a stage of reflection/iteration on the ideas created.

Expression web 3 is coming. SuperPreview feature can load previews using different browser engines, including Firefox and Safari from within the Expression interface, as well as a side-by-side view with the original comp or overlaying the comp as a transparency. It can also get previews for things like Safari Mac by using a MS cloud service to provide the rendering. It can also firebug-style outline elements while n side-by-side view to compare browsers (ftw!). Free version of SuperPreview only available now, which enables side by side display of multiple versions of IE with no VMs, etc. More on SuperPreview later, after I condense my notes from discussing it with one of the developers on the project.

ASP.NET MVC 1.0 RTM is released.

Some features of Visual Studio 2010 were discussed but I will discuss those in my post about the VS session I was at later in the day.

IIS 7's FTP module getting an update including SFTP support. Much needed, IIS FTP has been pretty terrible for ages now.

Web Platform Installer 2.0 coming out, available from http://microsoft.com/web, which extends on the 1.0 featureset of being able to configure IIS and install Microsoft development tools (SQL/VS express, IIS 7 extensions like URL Rewrite) by adding an "app store for your web server" as ScottGu put it. I think that sounds like a great idea. The Web PI app gallery has an open interface and many common products are already on it - even not traditionally Microsoft apps like PHP and Drupal, as well as open source .NET apps like DotNetNuke and Umbraco.

Silverlight 3 beta is released today. Many many new features to compete with Flash and AIR.

  • New Codecs: supports H.264/AAC/MP4
  • GPU acceleration including pixel shaders and 3d perspective transforms on images and video
  • improved media analytics
  • deep linking support
  • better text quality (ClearType support, including on the Mac)
  • multi-touch support
  • Silverlight library caching support (i.e. download external code libraries once)
  • Can run outside a browser similar to AIR, with a solid security sandbox limiting the need for "are you sure xxx?" dialogs
  • Download size 40k less than Silverlight 2 thanks to code optimizations
  • "Smooth" streaming using adaptive bitrates depending on network conditions enables several interesting behaviors including lowering the bitrate temporarily to avoid rebuffering a stream, and instant seek in streams by restarting the stream instantly with a low bitrate and kicking it back up as the buffer refills.
  • DVR-style stop and rewind abilities for live streams

That's all folks. Some exciting stuff coming down the pipe here, with more details in some entries I need to write from my sessions today.

Going to MIX09, or ASP.NET MVC vs the mighty CMS

Well I'll be getting on the plane to go to MIX09 in a few hours. I'm really looking forward to it, since there are a lot of ASP.NET MVC talks this year by many .NET people who I have a lot of respect for.

Since I do a lot of CMS development ASP.NET MVC has always a bit of an odd proposition technically. CMSes pretty much always want to enforce their own idea of how to program, which tends to be technically about as advanced as putting SQL statements right on the page so their marketing department can make claims about how easy it is to use if you require practically no advanced functionality. It's a constant battle to figure out ways of writing maintainable, "good" code within the confines of a CMS's API, and ASP.NET MVC is definitely one of the tools I'm looking at.

The main problem with integrating ASP.NET MVC into a CMS installation is that almost all CMS like to have a fully hierarchical URL setup; i.e. http://mysite.com/some/long/path/to/rewritten/content.aspx that would match /some/long/path... in the admin interface. The default routing system from what I've been looking at won't really support this setup since what we're really doing is defining what item in the (dynamically created and modifiable) hierarchy to look at as opposed to a (static, defined in code) controller action to invoke. So I think it's likely I'd need to implement my own version of routing, but that brings with it it's own issue that the custom routing then needs to know a lot about the data before passing control off to a controller action to render a page using a specific template renderer (based on the type of page it finds at the location in the hierarchy). I'm still looking around and hopefully can corner one of the MVC guys at MIX and see if they have better ideas :)