A Multiple Root Treelist Field

Recently I came across a need to allow selecting items for a TreeList field from several locations - specifically, we needed to be able to choose images OR videos to assign, and those were in different places in the content tree.

This has been done before, but there was altogether too much copied private method in that solution for my taste - and that it appears to not work with Sitecore 7.5 sealed the deal. To the hackmobile!

Turned out to not be too hard to reduce it to one file: Sitecore itself provides a MultiRootTreeList control that is used for rendering the insert data source item dialog (which supports multiple roots). With a little creative manipulation of the control tree, here we go:

And, the code (also as a Gist) - tested on Sitecore 7.2 Update-3 and Sitecore 8 Update-2:

using System;
using System.Linq;
using Sitecore.Diagnostics;
using Sitecore.Globalization;
using Sitecore.Shell.Applications.ContentEditor;
using Sitecore.Text;
using Sitecore.Web;
using Sitecore.Web.UI.HtmlControls;
using Sitecore.Web.UI.WebControls;

namespace Kamsar.FieldTypes
{
    /// <summary>
    /// This field type is like a tree list, but you can specify more than one root item to select from (for example, videos or photos)
    /// The data source roots are specified using pipe delimiting just like regular Sitecore Query language
    /// </summary>
    public class MultiRootTreeList : TreeList
    {
        protected override void OnLoad(EventArgs args)
        {
            Assert.ArgumentNotNull(args, "args");
            base.OnLoad(args);

            if (!Sitecore.Context.ClientPage.IsEvent)
            {
                // find the existing TreeviewEx that the base OnLoad added, get a ref to its parent, and remove it from controls
                var existingTreeView = (TreeviewEx)WebUtil.FindControlOfType(this, typeof(TreeviewEx));
                var treeviewParent = existingTreeView.Parent;

                existingTreeView.Parent.Controls.Clear(); // remove stock treeviewex, we replace with multiroot

                // find the existing DataContext that the base OnLoad added, get a ref to its parent, and remove it from controls
                var dataContext = (DataContext)WebUtil.FindControlOfType(this, typeof(DataContext));
                var dataContextParent = dataContext.Parent;

                dataContextParent.Controls.Remove(dataContext); // remove stock datacontext, we parse our own

                // create our MultiRootTreeview to replace the TreeviewEx
                var impostor = new Sitecore.Web.UI.WebControls.MultiRootTreeview();
                impostor.ID = existingTreeView.ID;
                impostor.DblClick = existingTreeView.DblClick;
                impostor.Enabled = existingTreeView.Enabled;
                impostor.DisplayFieldName = existingTreeView.DisplayFieldName;

                // parse the data source and create appropriate data contexts out of it
                var dataContexts = ParseDataContexts(dataContext);

                impostor.DataContext = string.Join("|", dataContexts.Select(x => x.ID));
                foreach(var context in dataContexts) dataContextParent.Controls.Add(context);

                // inject our replaced control where the TreeviewEx originally was
                treeviewParent.Controls.Add(impostor);
            }
        }

        /// <summary>
        /// Parses multiple source roots into discrete data context controls (e.g. 'dataSource=/sitecore/content|/sitecore/media library')
        /// </summary>
        /// <param name="originalDataContext">The original data context the base control generated. We reuse some of its property values.</param>
        /// <returns></returns>
        protected virtual DataContext[] ParseDataContexts(DataContext originalDataContext)
        {
            return new ListString(DataSource).Select(x => CreateDataContext(originalDataContext, x)).ToArray();
        }

        /// <summary>
        /// Creates a DataContext control for a given Sitecore path data source
        /// </summary>
        protected virtual DataContext CreateDataContext(DataContext baseDataContext, string dataSource)
        {
            DataContext dataContext = new DataContext();
            dataContext.ID = GetUniqueID("D");
            dataContext.Filter = baseDataContext.Filter;
            dataContext.DataViewName = "Master";
            if (!string.IsNullOrEmpty(DatabaseName))
            {
                dataContext.Parameters = "databasename=" + DatabaseName;
            }
            dataContext.Root = dataSource;
            dataContext.Language = Language.Parse(ItemLanguage);

            return dataContext;
        }
    }
}

Have fun!

Sitecore 8 Experience Editor Performance Optimization

Sitecore 8 is pretty nice, but not without its share of rough edges. One persistently annoying issue I came across is that after you compile your solution in Visual Studio, it is glacially slow to start up in Experience Editor (formerly Page Editor) mode. Given that most sites built in modern Sitecore fashion are using component based architecture that is Experience Editor centric, this is a big drag on development time.

In this post I’ll outline the story so far, and some tweaks you can perform to improve the performance of Sitecore 8 after a compilation.

As a baseline, on my i7-3770k, 32GB RAM, and all-SSD system EE 8.0 takes 1:30 to startup after a compile. Yeah, a minute and a half. Sitecore 6.5 will start in the same situation in 0:21.

Option 1: Precompilation

John West and Kevin Obee noted that I should try disabling the SPEAK precompilation features. These are in the <initialize> pipeline and registered in Sitecore.Speak.config and ContentTesting\Sitecore.ContentTesting.config. If you comment these out, you trade having the SPEAK interfaces come up slower the first time (because they are not precompiled) for faster initial startup time (because you skip precompiling).

With those two precompilers commented out, my startup time dropped massively to 0:41. Still twice as slow as Sitecore 7.2, but oh so worth it when you’re compiling all day.

Option 2: Optimize Compilation

Still unsatisfied with spending 20 seconds compiling unchanged .cshtml SPEAK views every time I compile my application, I started looking around into the actual compilation process. An interesting facet of this issue is that it’s not triggered by simply restarting the app pool. You must touch an assembly in the bin folder to trigger the slow startup.

A clue came in the form of this StackOverflow answer from @pbering. He mentioned enabling the optimizeCompilations setting in the web.config. What does this do? Well, off to MSDN. If you read this document, it makes it clear that when you change a top-level file (which includes anything in bin) it invalidates all dynamically compiled assets because a dependency of their compilation has changed. Razor files, such as what SPEAK uses, are dynamically compiled - .NET parses them, compiles them to a C# class, and stores them off in the Temporary ASP.NET Files folder.

So essentially what happens is that Sitecore 8 depends on a WHOLE LOT of dynamically compiled SPEAK renderings. When you start it up the first time, ASP.NET compiles them and stores them in its compilation cache. As long as you don’t change any dependencies - e.g. dlls, global.asax - .NET can update the cache selectively when the cshtml file is changed, and it’s fast. But as soon as a dependency is changed - e.g. your site dll - .NET throws out the whole site’s cache and compiles it all again, just to be safe. With Sitecore 8, this takes 20-60 seconds at least.

You can instruct .NET to ignore dependency changes and only recompile dynamically compiled files when the file itself is modified. This has some potential dangers if you modify a dependency without changing the dynamically compiled file and make a breaking change; see the MSDN article for details of the things to be aware of if you enable optimization.

The good news is, if you turn on <compilation optimizeCompilations="true"> in your web.config, there is no longer any SPEAK recompiling on startup! Even better, it starts up in 0:13, beating 7.2 by 8 seconds!

That said, be aware that there are significant issues to be aware of when enabling optimizeCompilations. Read the MSDN article. I have not deeply tested using this setting, other than it removes the startup speed issue. If you run into problems, stopping IIS and clearing the Temporary ASP.NET files folder should clear it up. Rebuild may not help.

Option 3: Disable the SPEAK Experience Editor

A separate ticket on Sitecore support (relating to EE in Sitecore MVC) rustled up this other possible solution: disable the SPEAK-based Experience Editor, and go back to the Sheer one from previous Sitecore versions. This removes all cshtml compilation issues from the equation, and feels slightly faster in the browser to me. It has a less-polished UI appearance however, likely due to it not being a priority to reskin :)

To disable the SPEAK EE, open App_Config/Include/Sitecore.MvcExperienceEditor.config. Locate the comment around like 33 regarding ‘uncomment the page extenders below to switch to sheer’ and follow the directions in the comment.

Hope that helps!

Quick note: Sitecore MVC partials

Here’s a bit of an obtuse trick that lets you use @Html.Partial() effectively on Sitecore MVC View renderings without specifying the full path to the partial you want to render. You might think that regular MVC syntax would work here:

@Html.Partial("_MyPartial")

But you’d be wrong. Instead, you’ll get an error that the partial could not be found, and the perplexing message that it tried to search [path-to-views]/Sitecore/_MyPartial.cshtml. Where’d that “/sitecore” come from? To understand this, a bit of background on how ASP.NET MVC resolves partials is in order.

The view resolution convention is based on the controller name that executes the view, not the view’s physical location. This makes sense in a pure ASP.NET MVC environment where every view is the result of a controller. However in a View Rendering, there’s just the cshtml. Except that in the background, Sitecore executes the View Rendering using a shim controller - a shim that goes by SitecoreController. This is where the /Views/Sitecore search path comes from.

So what’s the trick? Well, as long as all your views are in a MVC-like hierarchy, you can refer to the partial using the parent path:

@Html.Partial("../ViewParentFolder/_MyPartial")

This jumps out of the controller rendering folder (always Sitecore) and lets you specify the parent folder directly instead, without resorting to a full absolute path which - especially if you use areas or moved views around later - could break.

In case you’re wondering why I’m using native partials instead of @Html.Sitecore().Rendering() or @Html.Sitecore().ViewRendering() and such, these are static components of a MVC layout that are shared between multiple layouts - so it doesn’t make much sense to also register them as dynamic renderings and take the performance hit of that lookup.

Automatic ASP.NET connection string encryption

Encrypting your connection strings is a good practice as it obfuscates your database connection information from an attacker who has gained filesystem access to your web server. It’s a bit of a pain to implement if you’re wanting to keep all of your build happening on a CI server however, as encrypting it requires a executing aspnet_regiis on the target server because the encryption is keyed by the machine key of the destination server.

The other day I was reading Thomas Stern’s article about encrypting Sitecore connection strings and it got me to thinking of ways to make connection string encryption fully automatic with minimal configuration. Turns out it’s actually quite easy.

There are actually quite nice C# APIs to encrypt and decrypt so I set to try and make the application self-encrypt on startup using these APIs. That way you can deploy an unencrypted configuration file from CI, and as soon as the app starts up (which is immediately in my case since we run some HTTP calls to it as part of the deploy) it will encrypt itself using the local machine key.

To use this code example you will need to install the WebActivator NuGet package that allows it to register itself as an application startup task. You can also just call the AutoEncrypt method in Global.asax for the same effect, but WebActivator makes it completely self-contained.

This example only runs encryption if <compilation debug="false"> is in the web.config (so for local debugging you keep yourself decrypted), and the section has not already been encrypted. It is also fully compatible with the default App_Config\ConnectionStrings.config that Sitecore uses as the config reader is smart enough to save the right file.

Hope that’s helpful.

Indexing Subcontent

Often in Sitecore solutions we will utilize the page component architecture, where a page is composed of a base item and one or more “subcontent” items that are data sources for renderings that we have added to the page to customize its layout. This grants editors extreme flexibility to customize what is shown on their pages, but causes a major headache if you’re trying to have your search index contain the full content of a page because now the content is spread across potentially many backend items.

Note: While the code presented in this article is mine, the original idea was my coworker Erik Brown’s. Unfortunately he has no Twitter or blog that I know of, but if he ever gets one you should follow it :)

The solution we came up with utilizes Sitecore 7’s computed index fields (blog posts: John West, Martina Welander), and a handy but less than obvious trick.

The trick is that the Sitecore indexes can contain more than one value for the same index field. This means that if you want to augment the value of the system _content field (which contains all the indexable text fields on the item), all you have to do is define a computed field that is also named _content. Now the _content index field stores two values: the system value, and your computed value - and it will search on both values when you query it.

Armed with this knowledge I set out to use the layout field on the base item to find all rendering datasources it points to and use the same code that Sitecore uses to create the _content value for the main item. This turned out to not be all that hard (though I had the advantage of having done a lot of layout field messing around before):

Registering the computed field to augment the main _content index field requires a config patch file (note that this targets Lucene; your patch for SOLR or Coveo would be different):

Another handy tweak for indexing this kind of item is to use Nick Wesselman‘s handy code to cause the main item to be updated in the index whenever any of its datasource items are modified. Nice work!

Hope you find this helpful :)

Using Web API 2 Attribute Routing with Sitecore

There are a lot of times when developing a Sitecore site that you need to introduce dynamic, API-driven content - such as search autocompletes, single page apps, and other dynamic client-side content. Sitecore itself provides the Item Web API, but sometimes you want a more targeted solution designed for end users to acquire data in a fashion that exposes as little as possible.

Microsoft provides a technology that does this pretty nicely: the aptly named ASP.NET Web API. Web API uses a ASP.NET MVC-like controller to easily expose REST services over JSON or XML. I set out to make it work with Sitecore. Of course, some other folks had already got this to work. Unfortunately their work is also in some cases out of date - so let’s untangle things.

How to get Web API working on Sitecore, by version

Sitecore 6.x or 7.0 (without Sitecore MVC)

If you’re using Sitecore 6 or 7.0 without Sitecore MVC enabled, you can use the directions in the previously referenced blog posts to get Web API configured. By default, Sitecore will not respect ASP.NET routing unless Sitecore MVC is enabled so you have to use the solutions these blogs present to make Sitecore ignore routed URLs.

Sitecore with Sitecore MVC enabled

When Sitecore MVC is enabled (and you cannot turn it off on Sitecore 7.1 and later), you do not need the pipeline handler presented in the existing blog posts about Sitecore and Web API - Sitecore will automatically hand off routed requests to the route handler out of the box.

Web API versioning

At this time there are two versions of Web API:

  • v1, which released concurrently with ASP.NET MVC 4.
  • v2, which released concurrently with ASP.NET MVC 5.

If you’re on Sitecore 6 you’ll be using v1, because Sitecore 6 runs on .NET 4.0, and Web API 2 requires .NET 4.5. On Sitecore 7, you have more options. With 7.0 and MVC disabled, you can run v1 or v2. However on Sitecore 7.0 with MVC or 7.1, this adds a dependency on ASP.NET MVC 4 so you are limited to Web API v1 unless you follow the instructions in this post to add binding redirects and config modifications to enable MVC 5. The landscape thankfully improves with Sitecore 7.2, as that uses ASP.NET MVC 5.1 which easily adapts to Web API 2. Note that attribute routing, the point of this post, requires Web API 2

Attribute routing?

About time we started talking about the core of this post, eh? Attribute routing is a snazzy new way to register routes for Web API and ASP.NET MVC that was introduced in Web API 2/MVC 5 (and it was adapted from a community developed module for MVC 4). In earlier versions, you had to register routes in a global route table that was generally constructed all in one global class. This made developing relatively ad-hoc data services that provide what amount to an “AJAX backend for a CMS page” somewhat problematic as your dependency of a small part of the site now has to stick its fingers in everyone else’s pie by adding a global route.

Attribute routing solves this issue by allowing you to use standard .NET attributes (you know, like [HttpPost] on a MVC action method) to define the route to the code. This makes a lot more sense to my way of thinking. Here’s an example attribute-routed Web API 2 controller:

Microsoft has written pretty detailed documentation about Web API 2 attribute routing, which I suggest you review before getting serious about using this stuff.

Enabling attribute routing in Sitecore

If you read the aforementioned documentation, you’re probably already well on your way to enabling attribute routing in Sitecore since it’s the same as any other site - all you have to do is register the attribute routes.

The question then becomes how to do that in Sitecore. I’d say the official route is to add a pipeline handler to the initialize pipeline that does the registration. I decided to go with a more pure .NET approach, and register it using WebActivator - look ma, no config file! I created App_Start\WebApiConfig.cs - a pattern which may seem familiar to folks who’ve recently worked on ASP.NET MVC sites - and registered my attribute routes:

So now all I have to do is:

  • Install WebActivatorEx from NuGet
  • Copy WebApiConfig.cs into my project
  • Create an attribute-routed controller to do stuff

Pretty simple, and - if you’re using Sitecore 7.2 - easy to get set up on as well. For Sitecore before 7.2 there are some potential hoops to jump through to get attribute routing.

Anyhow, I hope this was helpful to someone.

Have two web databases? Don't forget the IndexUpdateStrategy

If you have two web databases in a Sitecore 7 site (e.g. for a pre-production preview or staging site) and you’re using Lucene here’s something to be aware of:

Simply copying the Sitecore.ContentSearch.Lucene.Index.Web.config file, renaming the index and changing the <locations> entr(ies) is not sufficient to make your new index work correctly! You’ll notice that if you publish items to that database that the updates don’t make it into the index. Why not?

Simply put, the default IndexUpdateStrategies are confusingly named. The web index defaults to the onPublishEndAsync strategy, which makes sense to use for our staging database as well - we want the index to get updated by the event queue after a publish completes. But there’s one minor detail: the onPublishEndAsync strategy should really be called onPublishEndAsyncWebdb or something similar. Let’s look at how the strategy is defined:

<onPublishEndAsync type="Sitecore.ContentSearch.Maintenance.Strategies.OnPublishEndAsynchronousStrategy, Sitecore.ContentSearch">
    <param desc="database">web</param>
    <CheckForThreshold>true</CheckForThreshold>
</onPublishEndAsync>

Derp - that <param desc="database">web</param>! The strategy actually is only watching the event queue for updates from a single database - and that database is not our staging database. The solution is to define a copy of the onPublishEndAsync strategy for your staging index and change the database parameter, then change your index declaration to reference that strategy instead:

In app_config\include\sitecore.contentsearch.lucene.defaultindexconfiguration.config (or a patch thereof):

<onPublishEndAsyncStaging type="Sitecore.ContentSearch.Maintenance.Strategies.OnPublishEndAsynchronousStrategy, Sitecore.ContentSearch">
  <param desc="database">web_staging</param>
  <CheckForThreshold>true</CheckForThreshold>
</onPublishEndAsyncStaging>

In App_Config\Include\Sitecore.ContentSearch.Lucene.Index.Web_Staging.config (or wherever you’re defining your staging index), reference the new strategy instead of the existing strategy:

<strategy ref="contentSearch/indexUpdateStrategies/onPublishEndAsyncStaging" />

Note that if you defined an index that spanned multiple databases you’d need to define multiple strategies for its updates, and reference all of them from the index config - an index can have multiple update strategies.

Hope this is helpful :)

Using MVC 5 with Sitecore 7.1

If you’re using something that is already on ASP.NET MVC 5, for example the latest release of Blade, and want to upgrade to Sitecore 7.1 you may be worried that as of 7.1 Sitecore takes a dependency on ASP.NET MVC 4. Thankfully that’s actually pretty simple to work around in such a way that both your MVC5 code and Sitecore’s MVC4 code can work together.

Tell Sitecore to use MVC5

First you need to redirect bindings to MVC4 to MVC5. You accomplish this by adding a binding redirect to your Web.config file like so:

<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
  <dependentAssembly>
    <assemblyIdentity name="System.Web.Mvc" publicKeyToken="31bf3856ad364e35" />
    <bindingRedirect oldVersion="1.0.0.0-4.0.0.0" newVersion="5.0.0.0" />
  </dependentAssembly>
 </assemblyBinding>
 </runtime>

Once you’ve done this, the new SPEAK backend will actually be making calls to the MVC5 System.Web.Mvc library - which it appears to be quite happy doing.

Comment out the extra Razor section definition

In Sitecore 7.1, there is a Web.config added to sitecore/shell/client that contains a definition of the Razor config section. If you’re defining the Razor config section yourself on the Web.config - like Blade does - then this is a duplicate declaration and will throw an exception. All you need to do is comment out this duplicate definition and it will fix the issue:

<!--   <configSections>       
    <sectionGroup name="system.web.webPages.razor" type="System.Web.WebPages.Razor.Configuration.RazorWebSectionGroup, System.Web.WebPages.Razor, Version=2.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35">
        <section name="host" type="System.Web.WebPages.Razor.Configuration.HostSection, System.Web.WebPages.Razor, Version=2.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" requirePermission="false" />
        <section name="pages" type="System.Web.WebPages.Razor.Configuration.RazorPagesSection, System.Web.WebPages.Razor, Version=2.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" requirePermission="false" />       
    </sectionGroup>        
</configSections> -->

That’s all you need to do to get MVC5 running with Sitecore 7.1. The Web.config in the shell will automatically override whatever Razor settings you may have in your Web.config file and keep SPEAK running when it needs to.

Sitecore Data Architecture

Introduction

There are many times where you need to extend the Sitecore data architecture in some way. One of the most obvious, and most used, is attaching an event handler to one of the item events (e.g. item:saved, item:renamed) but this is not by any means the only way to manipulate items before they get to the database.

This blog post came out of a discussion I had with Alex Shyba about the DataEngine, a few hours on a plane, and a lot of decompilation. I’m going to attempt to show what happens when an item gets saved and where you can plug into that.

The entry point

The most high level API is of course the Item class. Saving an item using this API is very simple:

using(new EditContext(item))
{
    item["MyField"] = "newValue";
}

When the EditContext goes out of scope, this causes item.Editing.AcceptChanges() to be invoked, and our journey down the rabbit-hole begins there.

The Item Manager

Our next step is the highest level item data API in Sitecore: the ItemManager. At first blush ItemManager appears to be a rather ugly giant static class, with tons of manipulaton methods: SaveItem, AddVersion, etc. However, it’s not as bad as it looks. ItemManager is basically a static facade around the current ItemProvider.

The Item Provider

Here’s where things get interesting. You can register multiple item providers in the config under itemManager/providers. While this appears to provide a lot of power as an extension point - being able to plugin your own high level provider - it unfortunately seems to be rather ungainly. If multiple providers are registered you can access them directly using ItemManager.Providers, but the other ItemManager methods only execute against the default provider. You could replace the default provider with your own, but this is less than ideal as it can lead to contention for whose item provider gets used if more than one module wants to extend using it.

Extension points aside, the ItemProvider is the most broad data API Sitecore provides. It deals in high level objects and abstracts lower level concepts like databases and item definitions away. Most all methods in the ItemProvider result in invocations of the next lower level construct, the DataEngine.

UPDATE: Nick Wesselman pointed out that some queries will bypass the Item Provider and go directly to the Data Engine. This gist has some examples of the types of things that bypass the Item Provider. In light of this I’d say the item provider is generally not a good candidate for extending things.

The Data Engine

DataEngine (generally accessed by databaseObject.DataManager.DataEngine) is a very interesting component for extension. Unlike the item provider, the DataEngine is database-specific. This is also the layer at which item event handlers (e.g. item:saved) are processed, but more interestingly is also a place where you can hook events in an uninterruptible fashion. Regular event handlers, such as item:saved, are vulnerable to being disabled when someone scopes an EventDisabler. Generally this is good, as events are disabled for performance reasons during things like serialization bulk load operations. But occasionally, such as in Unicorn, you need an event handler that cannot be disabled. Having a Unicorn event not fire would likely mean data being lost.

The internal architecture of DataEngine works something like this:

  • An engine method gets invoked
  • The engine creates and initializes an DataEngineCommand<> object, using a stored prototype of the command
  • The DataEngineCommand has its Execute method invoked.

The EngineCommand is a generic type, and each action that the engine can take has its own implementation - for example, there is a Sitecore.Data.Engines.DataCommands.SaveItemCommand class. Each of these command types is bootstrapped by a prototype system on the DataEngine itself. The engine, by way of its Commands property, stores copies of each command type. When a new command is needed, the existing prototype command is cloned (Clone()), initialized (Initialize()), and then used to execute the action. These prototypes are settable, including via configuration, so this allows you to inject your own derived implementation of each command and thus inject ‘event handler like’ functionality into the DataEngine.

Item Buckets utilizes this type of functionality injection, replacing the AddFromTemplatePrototype with its own extended implementation. This is the relevant section of Sitecore.Buckets.config:

  <database id="master" singleInstance="true" type="Sitecore.Data.Database, Sitecore.Kernel">
    <Engines.DataEngine.Commands.AddFromTemplatePrototype>
      <obj type="Sitecore.Buckets.Commands.AddFromTemplateCommand, Sitecore.Buckets" />
    </Engines.DataEngine.Commands.AddFromTemplatePrototype>
  </database>

This same functionality could be used to inject non-disableable event handler type functionality into other events, for example by replacing the SaveItemPrototype. Unfortunately this method also suffers from possible contention issues if multiple modules wished to patch these commands, so be careful when using them.

The DataEngineCommand

Once the DataEngine creates and executes a command, the command generally does two things:

  • Pass its task down to the Nexus DataApi to be handled
  • Fire off any normal item event handlers (item:saved), as long as events are not disabled

Most of the methods on DataEngineCommand are virtual so you could extend them. The most obvious candidate, of course, is the DoExecute method that performs the basic action of the command.

Further down the rabbit-hole we arrive to the Nexus APIs.

The Nexus DataApi

The Nexus assembly, due to it containing licensing code I believe, is the only obfuscated assembly in Sitecore. This makes following what the data API is doing rather difficult, but I’m pretty sure I traced it back out as a call to the databaseItem.DataManager.DataSource APIs, which thankfully are not obfuscated. It’s worth noting that the Nexus APIs appear to also be using a separate command-class based architecture internally. Given the sensitive nature of the Nexus assembly however, I would look elsewhere for extension points.

The DataSource

The Sitecore.Data.DataSource appears to largely be a translation layer between slightly higher level APIs, where things like Item are used, and the low level API of the data providers (which use constructs like IDs and ItemDefinition instead). The data source is very generic and does not appear to be designed for extension, which is fine because we can extend both above and below it.

Methods within DataSource eventually make calls down to static methods on the DataProvider class.

The Data Provider

There are two faces to the DataProvider class. The internal static methods that the DataSource is invoking are helper methods that invoke non-static methods on all of the data providers attached to the database. (Yes, you can have multiple data providers within the same database, which may not even look at the same backend database at all…hehe) The actual data provider instances are the lowest level data APIs in Sitecore. They deal with primitive objects and have a lot of unspoken rules about them that make them tougher to implement than most other extension points. However they are also the most powerful extension point there is. Using a data provider you can manipulate the content tree in nearly any fashion for example Rhino, a data provider that makes serialized item files on disk appear to be real content items in Sitecore.

You’re still reading this?

Hopefully this is a useful post for some crazy nuts like myself who like to bend the guts of Sitecore. It’s definitely a long and byzantine road from saving an item to it getting to the database. Rather amazing that Sitecore is as fast as it is with all of these layers. Part of me suspects that some are baggage from Sitecore 4 for backwards compatibility ;)

Make Web Forms for Marketers use a regular connection string

One of my pet peeves about the Sitecore Web Forms for Marketers module is that it manages the forms database connection string in its own config file, instead of the regular ConnectionStrings.config file.

Fortunately this is easy to fix if you want to manage all your connection strings in one place. First, you need to override the default WFMDataProvider like so:

Then, you modify the data provider declaration in App_Config\Include\forms.config (replace the namespace and assembly with wherever you place the class above in your project):

Finally, simply add a connection string named forms (or whatever name you configured in forms.config above) to your ConnectionStrings.config and you’re good to go!