Configuring domains from patch files

Sitecore domains are logical security groupings, for example the product ships with “sitecore” (backend users) and “extranet” (frontend users). But you do not have to stick with only the domains the product ships with, as usual for Sitecore you can extend and add your own.

Normally adding a domain means editing App_Config\Security\Domains.config, but we don’t want to do this. Why? Because editing standard Sitecore config files makes it difficult to upgrade Sitecore and introduces error prone file merging.

What we want to do instead is use config patch files. These allow us to add our config completely separately from Sitecore’s standard configuration files. But there’s a problem when it comes to domains: Domains.config does not live in the requisite <sitecore> config section so we cannot patch it.

Fortunately there’s a little known way around this. Someone at Sitecore implemented a config-based domain manager, but it’s not the default - presumably for backwards compatibility. And you can activate that, and add domains to it, using patch files.

So next time you want to add a domain to Sitecore, like say adding a domain for each site to provide logical author role groupings, switch over to the config domain manager. You know you want MySite\Editor instead of sitecore\MySite Editor. It’s easy to do, too.

This patch will activate the config domain manager (the defaults for the config manager already are the same as Domains.config so nothing else needs to be registered):

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
    <sitecore>
        <domainManager defaultProvider="file">
            <patch:attribute name="defaultProvider">config</patch:attribute>
        </domainManager>
    </sitecore>
</configuration>

And this patch is an example of adding a domain to the config domain manager:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
    <sitecore>
        <domainManager>
            <domains>
                <domain id="MyNewDomain" type="Sitecore.Security.Domains.Domain, Sitecore.Kernel">
                    <param desc="name">$(id)</param>
                    <ensureAnonymousUser>false</ensureAnonymousUser>
                </domain>
            </domains>
        </domainManager>
    </sitecore>
</configuration>

And there you have it. Enjoy!

Synthesis 8.2.1 Released

I am happy to announce that Synthesis 8.2.1 is available on NuGet. This release primarily adds additional features.

What’s New?

Improved Multiple Configuration Support

Previously registering multiple configurations in Synthesis was possible but way too hard. Configurations may now register themselves using code, similar to MVC area registrations.

To register a new configuration with Synthesis 8.2.1:

  • Add a class to the assembly you want the configuration’s model to live in that derives from SynthesisConfigurationRegistration
  • Implement the abstract members of SynthesisConfigurationRegistration. Much deeper customization is also available by overriding other optional members.
  • Add a SynthesisConfigRegistrar processor to the initialize pipeline that is set to scan your assembly, e.g.

    1
    <pipelines>
    	<initialize>
    		<!-- IMPORTANT: Each registrar instance must have a unique hint value for the patch to work correctly. -->
    		<processor type="Synthesis.Pipelines.Initialize.SynthesisConfigRegistrar, Synthesis" hint="Hinty McHintface">
    			<assemblies hint="list:AddAssembly">
    				<meAssembly>My.Assembly.WithModel</meAssembly>
    			</assemblies>
    		</processor>
    	</initialize>
    </pipelines>
  • That’s it! Your configuration will now be activated.

Auto Friending

WHAT DID I EVER DO TO YOU?

Along with simpler configuration registration, allowing your configurations to reference each other’s generated classes is also far easier with “Auto-Friending.”

To illustrate this, suppose you were using Habitat and in your Project layer you had templates that inherited Feature templates. Without auto-friending (or previously manual friending), the Project would generate duplicate interfaces for the Feature templates. This is a bad thing. But with friending, the Project generated template will simply inherit from the already existing Feature model, extending implicit dependency in the database into explicit dependency at a code level (also a good thing!).

With auto-friending, configurations automatically friend each other in the order they are registered. So for the example above, as long as the Feature’s model configuration was registered before the Project’s model, everything would Just Work. You can control the order of registration by the order in the SynthesisConfigRegistrar assemblies.

IRenderingContext and improved IoC support (Synthesis.Mvc)

A pattern that I’ve been using for a while (as have others) to improve testability is to make a facade over the RenderingContext that turns it into a Synthesis API and removes all Item dependencies. This IRenderingContext can be registered with your IoC container (to SitecoreRenderingContext) and constructor-injected into controller renderings to make Item-free controllers that are easy to test without any hacks. Even awesome hacks like FakeDb.

For example:

1
public class FooController : Controller
{
    private readonly IRenderingContext _renderingContext;

    public FooController(IRenderingContext renderingContext) 
    {
        _renderingContext = renderingContext;
    }

    public ActionResult Foo() 
    {
        var dataSource = _renderingContext.GetRenderingDatasource<IExpectedTypeItem>();

        if(dataSource == null) 
        {
            // no datasource set, or datasource is wrong template type (or context item, if no datasource set)
            return Content("Derp.");
        }

        var model = new FooViewModel(dataSource);

        // set other model props here

        // Note that none of this controller directly used Sitecore APIs and thus does not require FakeDb nor HTTP context
        // to have unit tests written against it.

        return View(model);
    }
}

Additional, more specific interfaces are also available for more specific use cases: IContextItem, IContextDatabase, and IContextSite. These can all be bound to SitecoreRenderingContext and provide smaller interfaces for more specific tasks.

Config Patching by Default

Synthesis now ships with Synthesis.LocalConfig.config.example, which is designed to be duplicated to form your own configuration patch. This encourages leaving the default configurations alone, which in turn greatly simplifies upgrading. The documentation and README has also been updated to reflect this.

Improvements

  • The default settings have been improved:
    • Creating model backup files is now disabled by default because it is of questionable utility when using source control
    • The InterfaceOutputPath and ItemOutputPath settings have been deprecated and merged into a single ModelOutputPath because it’s silly to emit more than one model file. The separate settings will still operate if you wish to use them.
    • Uncommonly used settings have been removed from Synthesis.config (SitecoreKernelAssemblyPath, SynthesisAssemblyPath, and InterfaceSuffix). They continue to work if set, but are removed for brevity as they are generally not used other than at default values.
  • The SynthesisEditContext class has been marked obsolete because the pattern is a bad idea
  • WebForms related classes have been marked obsolete because don’t use WebForms
  • The ability to attempt to automatically rebuild the project containing the model on startup has been removed due to being a generally bad idea
  • The ModelOutputBasePath setting has been added. This path is prepended to the ModelOutputPath for all configurations. The advantage of using this is that for people who work out of webroot, they can use <sc.variable> values in a setting (e.g. the out of webroot project location) whereas Sitecore does not expand variables in the ModelOutputPath. #27

Bug Fixes

  • Setting max backups to 0 no longer results in an infinite loop and now does not make backup files #28
  • Registering the same assembly twice in the type list provider will no longer scan the assembly twice
  • Wildcards that do not end in (e.g. Foo..Web) will not operate correctly when adding assemblies to the type list provider

Upgrading

Upgrading should be as simple as a NuGet upgrade*. If you have customized your Synthesis.config you may need to merge it with the default (or even better make it a patch file).

* as long as you are using Sitecore 8.1. As with the 8.2.0 release, it is designed only for Sitecore 8.1 due to breaking Sitecore API changes in 8.1. Sorry about that :(

Thanks!

Thank you to the community members who contributed to this release:

As always, happy coding!

Unicorn 3.1.5 Released

Unicorn 3.1.5 and Rainbow 1.3 are released to NuGet right now. This is a maintenance release. Upgrading is recommended, particularly if you are using Transparent Sync.

New Features

The new <syncConfiguration> dependency (see Unicorn.config) enables configurations to opt in to having synced items be updated into the link database and/or the search indexes after being changed by a sync.

This improves Unicorn’s compatibility with content syncing scenarios that are less developer focused, at the expense of reduced sync performance.

Transparent Sync also supports these flags, and items changed while Sitecore is alive to see them will have their links and indexes taken care of if set to do so. However if the IIS App Pool is not active during the file changes, nothing is updated.

Per-configuration concurrency

Unicorn 3 has always supported multithreaded sync and reserialize, but it has been disabled since release due to a concurrency bug in the Sitecore Template Engine that results in sync deadlocks when it is used to sync template item changes.

3.1.5 brings the ability to set the max thread count per-configuration (using the same <syncConfiguration> as indexing support) so that for configurations that are not syncing templates (or on Sitecore prior to 8.0U2) you may again enable concurrency. Threaded syncing can be 30-50% faster than non threaded.

Control panel UX improvements

Unicorn 3.1.5 brings a ‘Sync all’ button to the control panel, and a few minor UI tweaks such as relegating the log verbosity to an options modal.

Bug Fixes

  • Checkboxes inherited through several layers of standard values will no longer result in the derived template being reset to base standard values
  • Auto-publish no longer will miss publishing unversioned fields in certain cases
  • Renaming or moving items with transparent sync enabled and data cache turned on no longer results in a corrupted data cache
  • Change Template now works correctly with transparent sync
  • Unicorn now understands content items that are using Sitecore 8.1 language fallback without creating extra versions on the serialized copy
  • The log verbosity feature has had several issues fixed that would result in logs incorrectly not being shown at the selected level
  • Unicorn’s Micro IoC container now understands int constructor parameters
  • Adding or changing serialized template items when using transparent sync will now correctly clear the template engine, preventing an error
  • Automated build powershell examples are now more flexible when invoked in unusual path locations
  • In the control panel, the batch action buttons will no longer ‘overlay’ configuration checkboxes in certain situations
  • Logging of unversioned field updates will no longer incorrectly show a version number alongside the update in the logs
  • Certain unusual situations when syncing templates and items of that template will no longer throw a null reference due to stale caches

Upgrading

Upgrading from 3.1.4 is just a NuGet upgrade away. There are some config changes, so overwrite files and merge anything custom back in. Or even better use config patches and leave the default config alone :)

Note that Unicorn 3.1.5 changes the storage format for checkbox fields so that unchecked checkboxes are stored as ‘0’ instead of Sitecore’s standard blank value. This makes sync more reliable, however it does means that the first sync after serializing a new item with an unchecked checkbox will result in a ‘change’ as the zero value is pushed back into the Sitecore database. Checkboxes that are already serialized with blank values will continue to work just fine as they are.

Credits

Unicorn is a team effort of the Sitecore community. I’d like to thank the following folks who contributed to this release:

Dana Hanna
Kasper Gadensgaard
Mark Cassidy
Mike Carlisle
Søren Kruse
WizX20

And extra special thanks to Robin Hermanussen, Richard Seal, Alex Washtell, Mark Cassidy, and everyone else I missed for tirelessly answering questions in #unicorn on Sitecore Slack when I’m busy or not around!

The Solr Cannon: Rapid Solr Setup for Sitecore

Have you ever wished that standing up a Solr server for Sitecore was easy? I have. So I made a script to make it so.

The Solr Cannon automates the setup and configuration of Solr on Windows (Note: Linux may be a better production choice if you have the option), including installation as a background service, Sitecore schema installation, core creation, and generating a Sitecore config patch file that will point Sitecore at your shiny new Solr server and its cores.

What you’ll need

  • A copy of the Bitnami Solr Stack for Windows (I used 5.5.0-1)
  • A copy of the script and the solr schema file (the schema was generated with Sitecore 8.1 Update 1, you might need to generate your own if it doesn’t work)
  • PowerShell 3.0 or later

Let’s do this

  • Copy the Solr stack and scripts to a folder on the server that will run Solr
  • Review the Install Solr.ps1 script to make sure the variables are to your liking (project name, ports, solr stack installer path, etc)
  • Open an administrative PowerShell prompt
  • Execute Install Solr.ps1
  • Kick back for a couple minutes
  • Bask in the glow of your new Solr server and all of its Sitecore cores

But what about the rest?

Ok ok, that’s not all you need to do in order to configure Solr and Sitecore.

  • Check out Patrick Perrone’s post to help you swap in Solr as the search provider and configure the Solr IoC container. Note: I’d recommend Windsor for the IoC. Autofac and Ninject were less than awesome install experiences…
  • Once you have the Sitecore configuration flipped to Solr, you can grab the Solr.config file that the Solr Cannon produced when it installed Solr (written to the script directory). This config patch will:
    • Point Sitecore at your Solr server
    • Make Sitecore look at the correct core names for each index
    • Enable SwitchOnRebuildSolrIndex which allows you to rebuild indexes without any index downtime (great for production…or dev)
  • Load up Sitecore, open the indexing manager, and rebuild all your indexes. With a bit of luck, you’ll be good to go!

Multi-tenant Solr

Suppose you’re working on your own machine and you’ve got more than one dev site that’s using Solr. The Solr Cannon scripts can act to create a new config set and cores for several Sitecore installations on the same Solr server. Edit the script to have a different $ProjectName set, run it again, and say no when asked if you need to install a Solr server. In this mode, the script will build out a config set and cores for a new Sitecore site, as well as the patch file.

Have fun :)

Branch Datasource Presets

Ever stored rendering component data source items under a page? For example:

/sitecore/content/Page
/sitecore/content/Page/Datasources/ComponentDataSourceItem

It’s a good practice and it seems to work quite well for page specific components. I use it all the time.

But have you ever wished branch templates understood that pattern in a logical fashion? What if this:

/sitecore/templates/branches/Foo/$name
/sitecore/templates/branches/Foo/$name/Datasources/ComponentDataSourceItem

…expanded out to have the branch create the hierarchy and re-link the layout details on the instiantiated branch item to point to the right relative child data source item, instead of the child item under the branch template?

Doing this lets you use branch templates to create preset rendering hierarchies, including page specific data source items.

Sound good? Well you’ve found the right place to get the code to do just that. This requires Sitecore 8 or above with the pipeline-based item provider to operate.

Unicorn 3.1.4 Released

Unicorn 3.1.4 and Rainbow 1.2 are released to NuGet. This is a maintenance release which contains a few UX improvements and minor bug fixes and improvements.

What’s new?

Logging Verbosity

The most obvious change is that you can now select a logging verbosity in the control panel. If you’ve ever run a sync where many, many changes were synced you may have run into the issue where browsers start to crawl when fed that many log messages. Now you can choose to log only warnings or errors for that kind of situation. The Sitecore log files always get full verbosity, so you can see what exactly happened even if it was not written to the browser. Your verbosity setting is stored in a persistent cookie, so your selection is remembered.

Verbosity does not affect automated tools that invoke Unicorn. They always receive full verbosity. (#112)

XML Field Formatting Improved

XML fields are now stored by default with attributes on new lines. This further improves mergeability of Rainbow serialized items by allowing attribute merges. If you hate this idea, you can turn it off in Rainbow.config. (#12)

Rainbow 1.1:

<r xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <d id="{FE5D7FDF-89C0-4D99-9AA3-B5FBD009C9F3}" l="{0DAC6578-BC11-4B41-960A-E95F21A78D1F}" />
</r>

Rainbow 1.2 (Unicorn 3.1.3+):

<r xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <d
    id="{FE5D7FDF-89C0-4D99-9AA3-B5FBD009C9F3}"
    l="{0DAC6578-BC11-4B41-960A-E95F21A78D1F}" />
</r>

This formatting change is completely backwards compatible and no reserialization is required. As items are saved, they will be rewritten to use attributes on newlines.

Fixes & Minor Improvements

  • The UX of the control panel has been streamlined to result in less vertical space for each configuration, a boon for setups like Habitat with lots of configurations.
  • Unicorn auto publishing now functions correctly on items with versions in several languages (#114)
  • Using the “Copy to” function will no longer result in an incorrect serialized parent ID on the copied item. Other forms of duplication were not affected. (#119)
  • Fixed a case where NullReferenceException was incorrectly thrown when using transparent sync (#117)
  • Fixed an error when duplicating items that did not exist in the database when using transparent sync (#116)
  • Rainbow YAML formatting is more friendly to third party YAML parsers with the way it escapes values. This does mean that some values that did not previously get wrapped in double quotes (like GUIDs), will now have it going forward. Format is backwards compatible, no reserialize required. Items will upgrade to the new version on save. (#11)

Upgrading

To upgrade simply update your NuGet packages. There are no other steps required for this release unless coming from Unicorn 3.0.x.

If you wish to completely normalize your items to the latest Rainbow 1.2 format, you may reserialize them. But it is not a requirement, as Rainbow 1.2 can parse 1.1 serialized items.

Credits

As usual, Unicorn updates are a community effort. Big thanks to the folks below for reporting bugs, sending pull requests, and general encouragement.

Unicorn 3.1 Released

Unicorn 3.1 and Rainbow 1.1 are released to NuGet. These updated versions bring several new features and a bunch of fixes. Upgrading is recommended for all users of Unicorn 3.0 due to the fixing of some data issues.

What’s new?

Configuration Dependencies

Unicorn configurations may now declare dependency relationships between each other. This is useful for example when one configuration may contain templates and another configuration might contain items based on those templates. With a dependency relationship the dependent configurations are guaranteed to always go first when configurations are synced. Transitive relationships (e.g. A -> B -> C) are supported.

Dependencies are defined using a dependencies attribute on the configuration.

Dependencies do not force dependent configurations to sync. If config A depends on config B and you choose to sync only config A, B will not be synced. However if you sync both A and B, then B will always sync first due to the dependency.

Include/Exclude Predicate Enhancements

The predicate system has been significantly overhauled in Unicorn 3.1 to allow more advanced include and exclude scenarios.

  • Exclude entries may now use paths relative to the parent <include> by omitting a leading slash. This would exclude /foo/bar:

    <include path="/foo">
        <exclude path="bar" />
    </include>
    
  • A new explicit grammar for excluding all children of an include has been introduced. This replaces the previous implicit grammar of adding a trailing slash (which still works but is deprecated).

    <include path="/foo">
        <exclude children="true" />
    </include>
    
  • The explicit children grammar supports <except> syntax to exclude all children except for specific ones.

    <include path="/somechildren">
        <exclude children="true">
            <except name="tests" />
            <except name="fests" />
        </exclude>
    </include>
    
  • A new explicit grammar for excluding only children of a specific sub-path has been added. This grammar also supports <except> syntax and relative paths.

    <include path="/foo">
        <exclude childrenOfPath="/foo" />
        <exclude childrenOfPath="bar"> <!-- /foo/bar/* -->
            <except name="quux" /> <!-- include /foo/bar/quux -->
        </exclude>
    </include>
    

Take a look at the test predicate configuration for examples of all available grammars.

Control Panel Tool Authentication

The method used to authenticate automated tools to Unicorn has been updated to be significantly more secure using a CHAP implementation. This also comes with a handy PowerShell module that you can use to simply invoke the handshake from a build script.

Should you require the legacy shared secret method, that is still available but it is not the default. Security is also now pluggable, if you wanted to use your own implementation.

The remote API ships disabled by default and you must activate it by generating a random shared secret and adding it to the config. This same secret must be known to invoke the API, and unlike the legacy API the secret is never transmitted directly.

Breaking Changes

Unicorn 3.1 has some breaking changes compared to Unicorn 3.0, so upgrading requires reserializing your items to get them in a consistent format with the latest version. Unicorn 3.1 does understand Unicorn 3.0-formatted items, but Unicorn 3.0 does not understand Unicorn 3.1-formatted items. There will also be some repeated changes in the logs if you sync 3.0 items with 3.1, so usage with 3.0 items is not recommended.

Introducing breaking changes was a hard decision to make, but I think the saner defaults and new capabilities will justify the slightly more difficult upgrade process. So what’s breaking?

  • The item’s database name is now stored in the serialized data (#7). This enables parsing serialized items in contexts where the database cannot easily be inferred, such as Courier.
  • Unversioned fields have official support and are nested under the Language instead of under a Version within the language which was incorrect. Prior to reserializing, syncing a 3.0-formatted item with 3.1 may result in messages that the field has been reset to standard values and then set again. IItemData now has an UnversionedFields property to store unversioned fields by language.
  • The name on serialized fields has been converted from a comment in the YAML into an actual parsed value. Note that this value is just a hint and if a template field is renamed it is NOT updated automatically.
  • The default setting of Rainbow.SFS.ExtraInvalidFilenameCharacters has been changed to "$" to make Rainbow compatible by default with TFS, and the TFS config example that previously set this has been removed.
  • The default setting for Rainbow.SFS.SerializationFolderPathMaxLength has been increased from 90 to 110 based on real world experience that 90 is too short.
  • The default setting for Rainbow.SFS.MaxItemNameLengthBeforeTruncation has been reduced from 100 to 30 based on real world experience that 100 was long enough to break the path length limitation for items with near 100 length names.
  • Automated deployment tools calling Unicorn have a new security API. See below for details on this, but it’s now more secure.

Other Improvements:

  • Syncing dictionary items will now cause the dictionary cache to be cleared at the end of the sync, forcing the synced entries to be loaded.
  • The Unicorn Control Panel is now pipeline-based and extensible so that modules may define their own verbs or alter the UI if needed.
  • Error messages when serializing and deserializing XML-based fields (layout, rules) have been improved.
  • Unicorn and Rainbow both now utilize the C# 6.0 syntax and require the C# 6.0 compiler to build. They still target .NET 4.5.
  • Config file comments and documentation have been updated to cover the new features of Unicorn 3.1.

Bug fixes:

  • Renaming or moving items which are serialized on loopback paths will no longer cause the serialized children to be deleted.
  • Missing field errors on the root item of a tree now result in a retry and non fatal error as expected.
  • A non fatal warning when syncing multiple configurations will no longer result in the sync stopping at the end of the configuration with the non fatal warning.
  • When using the New Items Only evaluator, you will no longer receive sync conflict warnings when saving items created by the evaluator.
  • Copying items when Transparent Sync is enabled and the source item is not in the Sitecore database now works correctly.
  • Empty field values will no longer cause problems when Transparent Sync is enabled.
  • XML-based fields (layout, rules) with empty values will no longer cause exceptions. #6, #90
  • Items installed by Sitecore packages or update packages into Unicorn configuration areas will no longer lose some field values in the serialized items. #92
  • Predicate include entries and SFS trees are no longer confused by paths which are different but start with similar bases, such as /sitecore/content and /sitecore/content monkey
  • Plain text Unicorn logs returned to automated tool calls will no longer include HTML-encoded values such as &gt;
  • Naming items a Windows reserved file name.aspx) such as CON or COM1 will no longer result in an error. #8
  • Fields with a blank value in Sitecore that were not included in the serialized item were not being correctly reset to standard values. This has been fixed.

Upgrading

Ok so there are breaking changes, how do I upgrade? Thankfully it’s not too hard.

  1. Before you upgrade, run a full sync of all configurations. If some have Transparent Sync enabled, turn that off during the upgrade. (you can reenable after step 4)
  2. Upgrade your NuGet packages to Unicorn 3.1. (which will also install Rainbow 1.1)
  3. Run a build of your solution, and if you develop out of webroot publish your changes.
  4. Using the Unicorn control panel, Reserialize all configurations to flush them to disk as 3.1-format items.
  5. If you use the automated tool API to do automated execution of Unicorn, migrate to the PowerShell module to invoke the new API.
  6. Have fun!

Contributors

Unicorn 3.1 has seen contributions from many community members. Thank you all for your support, testing, bug reports, and contributions!

The Benchmark Show: Sitecore Precompilation

I’m mildly obsessed with Sitecore performance, so when I found a new trick I had to see what it could do: aspnet_compiler.exe. This is a really old tool that was designed to compile your .aspx pages into an assembly - and then later your cshtml as well. This improves startup time because no dynamic compilation needs to take place at runtime.

Getting aspnet_compiler to run against Sitecore, by which I mean executing it against an entire deployment ready web root including both your code and the sitecore folder, is a bit of a challenge. Because you cannot tell aspnet_compiler to ignore anything, and because one of the cshtml files in Sitecore 8.1 Update-1 apparently has a compiler error (for the curious, add @using Sitecore.ListManagement.Client.Web.UI.Controls.ImportMapTo to /sitecore/shell/client/Applications/ListManager/Controls/ContactFields/ContactFields.cshtml). You also have to comment out the @Html.Sitecore()s in /sitecore/shell/Templates/layout.cshtml (don’t worry it’s just a template used when you add a new MVC layout).

Once you get the fixes made, running aspnet_compiler is as simple as:

"c:\Windows\Microsoft.NET\Framework\v4.0.30319\aspnet_compiler.exe" -v / -p "c:\path\to\deploy\ready\files" "c:\output\path"

It will take a while - probably at least a minute - while it compiles every aspx and cshtml in the whole place into assemblies.

Cool trick. Does it work?

Let’s look at some hard data. I took my base Sitecore site (which runs Performance.config) and executed my deployment scripts over it, then precompiled that to a new folder with aspnet_compiler. Then I devised several tests to tease out both the effects of precompilation and whether Sitecore’s built in SPEAK precompilation has improved since its 8.0 days - where it could make startup take 140 seconds. The tests were in most cases run both with a cold ASP.NET Temporary Files folder (where .NET puts compiled Razor caches) and with a warm one for comparison. Cold would be the first startup on a new machine, whereas warm would be after an app pool recycle thereafter.

Test 1: Home Page

This is a simple test of loading the home page, which is a fairly simple Sitecore MVC layout with about 4 view and controller renderings and several partials. All tests begin with the app pool stopped and no w3wp executing.

Test 2: SPEAK Media Library Dialog

The SPEAK media browser (what you get when browsing from an image field, for example) has been historically a pain point performance wise. For this test I loaded the dialog iframe URL into a Chrome tab and measured the time to finish page load with the Chrome developer tools.

Results

Home Page (cold)

Baseline (Performance.config): 22.25s
SC Precompiler: 24.8s
Precompiled: 21.5s

As expected the precompiled is marginally fastest. Sitecore 8.1’s precompilation function, as we’ll see throughout the tests, seems to add 2-4 seconds to startup time. Without preexisting compiled temp files, startup is extremely slow in all cases - unexpectedly so for the supposedly ‘precompiled’ site. My guess is that it copies the precompiled files to the temp folder, which expends I/O, but that’s a guess.

Home Page (warm)

Baseline: 9.71s
SC Precompiler: 13.97s
Precompiled: 9.51s

With the temporary files warm, startup gets much faster. Once again the Sitecore precompiler is in last place.

SPEAK Media (cold)

Baseline: 29.6s
SC Precompiler: 33.8s
Precompiled: 22.93s

The effects of precompilation are strong when loading the very complex, partial-heavy SPEAK dialog - coming in 47% faster than the Sitecore default.

SPEAK Media (warm)

Baseline: 12.2s
SC Precompiler: 16s
Precompiled: 12.82s
SC Precompiler (warmed): 13.23s

Performance is nearly identical once the temporary compiled files are cached the first time for SPEAK. In this case there is also a test for having warmed the Sitecore precompiler and allowed it to finish precompiling in the background, then restarted the cold app pool.

Analysis (TL;DR)

Overall the effects of using aspnet_compiler prior to deployment are relatively minor in most real world cases. Initial startup is ideally a rare case, and once their caches are heated all options perform quite similarly (as an example, the second hit to the media dialog is ~250ms to finish). For more elastic deployments, such as cloud setups, or for larger clusters it is probably worth the effort since you burn the CPU cycles once and it does significantly improve first hit to SPEAK dialogs as well as marginal overall performance improvements. Rendering heavy Sitecore MVC implementations would also likely see similar gains to SPEAK. In addition to performance, using the aspnet_compiler validates the C# in your Razor views - which is a nice plus in the event of a Razor syntax error which would normally be discovered at runtime.

The big downside of precompiling is that it’s fairly slow (several minutes on fast hardware, probably much slower otherwise or without SSDs), and that it has to re-copy the whole web root to another output location (lots of I/O). So overall build time on CI may increase significantly.

Another option: Use the Sitecore precompiler

If you look at the results, startup time with the Sitecore precompiler is always a bit longer. But once the Sitecore precompiler completes its asynchronous run - usually within a couple minutes after startup - the results are very close to a natively precompiled site (see the warm media dialog results). This is overall a much simpler option. It does mean every instance has to compile for itself, but it does not require adaptation of the build process nor long times spent copying files.

And here’s the best part: you can use it for your own views, too. It’s just a Razor precompiler pipeline processor, and you can add one - or more - for your sites’ MVC views. For example:

<processor type="Sitecore.Pipelines.Initialize.PrecompileSpeakViews, Sitecore.Speak.Client">
    <Paths>/Areas/MySite/Views</Paths> <!-- virtual path to your views -->
</processor>

This will cause your views to all get precompiled asynchronously on startup - if they are not already - which writes their assemblies into the Temporary ASP.NET Files folder. So you trade a couple seconds of startup time, and an initial spike of CPU cycles, for consistent performance thereafter. However, you lose the protection against syntax errors provided by build-time precompilation.

Next time on the benchmark show: Common Sitecore development tasks, benchmarked on a variety of hardware.

Unicorn 3.0.2 Released

Unicorn 3.0.2 (and Rainbow 1.0.1) have been released to NuGet.

This release includes some critical fixes to the serialization system, and upgrading is highly recommended for all users of Unicorn 3.

Improvements

  • Unicorn auto-publish now reports its activities to the sync console, and operates synchronously to guarantee published state for automated builds.
  • Guidance on some Transparent Sync pitfalls has been added to the wiki.
  • The new Rainbow.SFS.ExtraInvalidFilenameCharacters setting enables TFS users to successfully serialize branch templates which begin in $, a reserved character by TFS. The Rainbow.TFS.config.example file is now shipped with Rainbow to illustrate its usage for TFS.
  • An example configuration file to demonstrate how to move your serialized items elsewhere has been added.

Fixes

  • CRITICAL Saving item versions that were not in the default Sitecore language was not updating the serialized item. This has been fixed. #76
  • CRITICAL Serialization of significant empty field values, such as unchecked checkboxes when the standard value is checked, has been fixed. #74
  • Items that are restored from the Sitecore recycle bin or archive to Unicorn-controlled locations now result in the item being serialized on restore. #72
  • Resetting a field to standard values now correctly results in the field value being removed from the serialized item. #74
  • Query string order is always respected when syncing multiple configurations at once using the control panel.
  • The serialization conflict dialog (if disk doesn’t match base item data on save) now formats correctly using HTML for Sitecore 8+
  • Using the Dump Item and Dump Tree commands in Sitecore to reserialize partial item trees now correctly serializes an item to all configurations that it may belong to, instead of just the first one.

Upgrading

Upgrading from Unicorn 3.x is as simple as updating your NuGet packages. If it prompts to overwrite any changed config files, I’d suggest doing so and using a diff with your source control to resolve differences.

If you suspect that you have serialized items that may be missing significant empty values, you should reserialize those after upgrading to get their values normalized.

The Credits

Unicorn isn’t just my project. Thanks a lot to all the community members who reported bugs, sent pull requests, and diagnosed issues to help make Unicorn better.

Unicorn 3.0.2 is brought to you by…

Robin Hermanussen
Thomas Eldblom
Nathanael Mann
@GeorgeKarbyn
Kevin Williams
@amitthakur2014
@WizX20
@kamsar
…and the rest of #unicorn on Sitecore Slack

As usual, if you run into trouble you can submit an issue on GitHub, hit me up on Sitecore Slack, or tweet at me.

Unicorn 3.0.1 Released

The first maintenance release for Unicorn 3.x has been released to NuGet. Unicorn 3.0.1 brings with it a small set of bug fixes and improvements:

  • Data provider config patch is compatible with Sitecore Commerce and other modules that add data providers (thanks @jonnekats!)
  • Fixed a bug in Transparent Sync where syncing an item based on a template that only existed in transparent sync (and not the database) would act like the template did not exist. Ironically this was fixed in 3.0.0 but some n00b forgot to merge the config to activate it! (thanks Kevin Williams!)
  • Fixed a race condition that could cause SFS trees to incorrectly reinitialize during a reserialize operation and cause an error (thanks @akshaysura13!)
  • Content editor warnings have been improved:
    • The name of the Unicorn configuration that includes the item is now shown so it’s easy to trace back why something is included.
    • For transparent sync items, there is an indication of whether the item is on disk only or both on disk and in the database as well. Note that items in both places may well have different field values, there is no equality check.

Due to the fix to Transparent Sync, this update is recommended for anyone using that feature.

Have fun!