I’ve pushed Blade 2.1 out to NuGet. This release updates the MVC references to MVC5, and the Razor references to Razor 3.
This release also uses the latest bits for Sitecore NuGet, which resolve some issues with local IIS site connections.
There are many times where you need to extend the Sitecore data architecture in some way. One of the most obvious, and most used, is attaching an event handler to one of the item events (e.g. item:saved, item:renamed) but this is not by any means the only way to manipulate items before they get to the database.
This blog post came out of a discussion I had with Alex Shyba about the DataEngine, a few hours on a plane, and a lot of decompilation. I’m going to attempt to show what happens when an item gets saved and where you can plug into that.
The most high level API is of course the Item class. Saving an item using this API is very simple:
using(new EditContext(item))
{
item["MyField"] = "newValue";
}
When the EditContext goes out of scope, this causes item.Editing.AcceptChanges() to be invoked, and our journey down the rabbit-hole begins there.
Our next step is the highest level item data API in Sitecore: the ItemManager. At first blush ItemManager appears to be a rather ugly giant static class, with tons of manipulaton methods: SaveItem, AddVersion, etc. However, it’s not as bad as it looks. ItemManager is basically a static facade around the current ItemProvider.
Here’s where things get interesting. You can register multiple item providers in the config under itemManager/providers. While this appears to provide a lot of power as an extension point - being able to plugin your own high level provider - it unfortunately seems to be rather ungainly. If multiple providers are registered you can access them directly using ItemManager.Providers, but the other ItemManager methods only execute against the default provider. You could replace the default provider with your own, but this is less than ideal as it can lead to contention for whose item provider gets used if more than one module wants to extend using it.
Extension points aside, the ItemProvider is the most broad data API Sitecore provides. It deals in high level objects and abstracts lower level concepts like databases and item definitions away. Most all methods in the ItemProvider result in invocations of the next lower level construct, the DataEngine.
UPDATE: Nick Wesselman pointed out that some queries will bypass the Item Provider and go directly to the Data Engine. This gist has some examples of the types of things that bypass the Item Provider. In light of this I’d say the item provider is generally not a good candidate for extending things.
DataEngine (generally accessed by databaseObject.DataManager.DataEngine) is a very interesting component for extension. Unlike the item provider, the DataEngine is database-specific. This is also the layer at which item event handlers (e.g. item:saved) are processed, but more interestingly is also a place where you can hook events in an uninterruptible fashion. Regular event handlers, such as item:saved, are vulnerable to being disabled when someone scopes an EventDisabler. Generally this is good, as events are disabled for performance reasons during things like serialization bulk load operations. But occasionally, such as in Unicorn, you need an event handler that cannot be disabled. Having a Unicorn event not fire would likely mean data being lost.
The internal architecture of DataEngine works something like this:
DataEngineCommand<> object, using a stored prototype of the commandDataEngineCommand has its Execute method invoked.The EngineCommand is a generic type, and each action that the engine can take has its own implementation - for example, there is a Sitecore.Data.Engines.DataCommands.SaveItemCommand class. Each of these command types is bootstrapped by a prototype system on the DataEngine itself. The engine, by way of its Commands property, stores copies of each command type. When a new command is needed, the existing prototype command is cloned (Clone()), initialized (Initialize()), and then used to execute the action. These prototypes are settable, including via configuration, so this allows you to inject your own derived implementation of each command and thus inject ‘event handler like’ functionality into the DataEngine.
Item Buckets utilizes this type of functionality injection, replacing the AddFromTemplatePrototype with its own extended implementation. This is the relevant section of Sitecore.Buckets.config:
<database id="master" singleInstance="true" type="Sitecore.Data.Database, Sitecore.Kernel">
<Engines.DataEngine.Commands.AddFromTemplatePrototype>
<obj type="Sitecore.Buckets.Commands.AddFromTemplateCommand, Sitecore.Buckets" />
</Engines.DataEngine.Commands.AddFromTemplatePrototype>
</database>
This same functionality could be used to inject non-disableable event handler type functionality into other events, for example by replacing the SaveItemPrototype. Unfortunately this method also suffers from possible contention issues if multiple modules wished to patch these commands, so be careful when using them.
DataEngineCommandOnce the DataEngine creates and executes a command, the command generally does two things:
DataApi to be handledMost of the methods on DataEngineCommand are virtual so you could extend them. The most obvious candidate, of course, is the DoExecute method that performs the basic action of the command.
Further down the rabbit-hole we arrive to the Nexus APIs.
DataApiThe Nexus assembly, due to it containing licensing code I believe, is the only obfuscated assembly in Sitecore. This makes following what the data API is doing rather difficult, but I’m pretty sure I traced it back out as a call to the databaseItem.DataManager.DataSource APIs, which thankfully are not obfuscated. It’s worth noting that the Nexus APIs appear to also be using a separate command-class based architecture internally. Given the sensitive nature of the Nexus assembly however, I would look elsewhere for extension points.
DataSourceThe Sitecore.Data.DataSource appears to largely be a translation layer between slightly higher level APIs, where things like Item are used, and the low level API of the data providers (which use constructs like IDs and ItemDefinition instead). The data source is very generic and does not appear to be designed for extension, which is fine because we can extend both above and below it.
Methods within DataSource eventually make calls down to static methods on the DataProvider class.
There are two faces to the DataProvider class. The internal static methods that the DataSource is invoking are helper methods that invoke non-static methods on all of the data providers attached to the database. (Yes, you can have multiple data providers within the same database, which may not even look at the same backend database at all…hehe) The actual data provider instances are the lowest level data APIs in Sitecore. They deal with primitive objects and have a lot of unspoken rules about them that make them tougher to implement than most other extension points. However they are also the most powerful extension point there is. Using a data provider you can manipulate the content tree in nearly any fashion for example Rhino, a data provider that makes serialized item files on disk appear to be real content items in Sitecore.
Hopefully this is a useful post for some crazy nuts like myself who like to bend the guts of Sitecore. It’s definitely a long and byzantine road from saving an item to it getting to the database. Rather amazing that Sitecore is as fast as it is with all of these layers. Part of me suspects that some are baggage from Sitecore 4 for backwards compatibility ;)
Put me on a plane to the Sitecore MVP Summit and I’m going to write software! All of these updates are available on GitHub for source and NuGet for packages.
This is an extension to Synthesis.Blade that enables automagical constructor dependency injection to any presenter using the Ninject IoC container. I previously blogged about how to do this (it’s not that complex) but now doing DI with Blade is as simple as a NuGet package. Simply wire up Ninject with your dependencies, add some dependencies to a presenter constructor, install this package, and you’re good to go. Here’s a simple example of a presenter using DI to get a repository dependency:
public class TestPresenter : SynthesisPresenter<TestModel, ITestTemplateItem>
{
private readonly ITestRepository _repository;
// this constructor parameter will be automatically set by Blade via Ninject
public TestPresenter(ITestRepository repository)
{
_repository = repository;
}
public override TestModel GetModel(IView view, ITestTemplateItem item)
{
var model = new TestModel();
model.Frob = _repository.GetFrob(item);
return model;
}
}
Note: It’s trivial to implement the same constructor injection with a different IoC container, or without using Synthesis integration. Take a look at the source for Synthesis.Blade.Ninject and it should be pretty obvious what to do :)
Note2: Make sure to use the above Synthesis.Blade 7.0.1 configuration file, or the installation may not work correctly and patch the wrong thing.
This is a minor release with enhancements to testability and bug fixes.
IView now has a ViewContext property that exposes the HttpContextBase for the current request, enabling testable access to request and response data within presentersIView became covariant in Blade 2, which meant that IView<BaseClass> became similar to IView<DerivedClass>, and the base class presenter might be selected first in some cases) - fix courtesy of @gobinerThis is a minor release that corrects an oversight in 7.0.0 where some of the new field type interfaces did not have setters for their values. This prevented editing items when using field type interfaces. Bug report courtesy of @gobiner.
This is a minor release that resolves an issue with the default configuration file if multiple patches were present to the Blade PresenterFactory. To upgrade an existing installation, simply edit Synthesis.Blade.config and change the presenter factory patch line to read like so (changing the ‘instead’ attribute):
<presenterFactory patch:instead="*" type="Synthesis.Blade.Configuration.SynthesisPresenterFactory, Synthesis.Blade" />
One of my pet peeves about the Sitecore Web Forms for Marketers module is that it manages the forms database connection string in its own config file, instead of the regular ConnectionStrings.config file.
Fortunately this is easy to fix if you want to manage all your connection strings in one place. First, you need to override the default WFMDataProvider like so:
Then, you modify the data provider declaration in App_Config\Include\forms.config (replace the namespace and assembly with wherever you place the class above in your project):
Finally, simply add a connection string named forms (or whatever name you configured in forms.config above) to your ConnectionStrings.config and you’re good to go!
It seems like most folks these days are using jQuery on their sites. Sitecore, in fact, uses jQuery for the page editor UI. I discovered an interesting case where it can conflict with your own jQuery version that I thought I’d share.
The scenario where it happens is this:
jQuery has a feature called noConflict that is designed to allow you to isolate it from other libraries, such as Prototype, that also claim the “$” global variable. The feature can also be used to run two versions of jQuery on the same page, and Sitecore in fact does exactly this. However, the case that noConflict does not prevent is one where a second copy of jQuery is loaded on top of an existing one.
What happens here is that your jQuery loads in the heading, then Sitecore loads its jQuery and calls noConflict on it. Unfortunately, the loading of Sitecore’s JS overwrites the global jQuery variable, resulting in this mess:
jQuery = Sitecore jQuery (1.5.5 for SC7 Update-1, pretty old)$ = your jQueryNow load a jQuery plugin that uses an IIFE to load itself like:
(function($) {
// plugin load code here
})(jQuery);
Yeah that’s right, the plugin invokes its function passing in jQuery aliased as $. Which means that the plugin just loaded to Sitecore’s jQuery, not yours! Now if you invoke the plugin you’ll see that it’s undefined in your jQuery!
Fortunately it’s relatively easy to manually fix this issue. You simply need to:
<script>var my_jquery = jQuery;</script>webedit placeholder explicitly on your layout, so you can control where Sitecore will load its version of jQuery:<sc:Placeholder runat="server" Key="webedit"/>webedit placeholder definition:<script>jQuery = my_jquery;</script>!Sitecore.Context.PageMode.IsNormal as well if you want clean output.If you can put your copy of jQuery in the footer, you don’t need to resort to any of this hackery. Hope this helps someone :)
Synthesis 7 is now available on NuGet. This version brings a number of new features and bug fixes largely aimed at supporting unit testing of sites that are built with Synthesis. Version 7 is compatible with Sitecore 7 Update-1 and Update-2.
Previous versions of Synthesis would generate classes that had concrete field types, such as RichTextField, that were strongly tied to the Sitecore item APIs or search APIs. This prevented easily creating mock versions of the class for testing purposes. Synthesis 7 changes that by allowing you to define both a private concrete field type mapping and and public interface mapping, such as this:
[IndexField("title")]
public ITextField Title {
get { return new TextField(/* ... */);
}
Now each field can be easily mocked with a mocking framework of your choice, enabling you to create totally Sitecore API free instances of Synthesis items. For example, using Moq:
var item = new Mock<IFooItem>();
item.SetupGet(x => x.Title).Returns(new TestTextField("isn't this nice?"));
string result = item.Object.Title.RawValue; // "isn't this nice?"
But wait! What about those pesky metadata properties like statistics, database, and editing? Those are annoying, huh. Well Synthesis 7 uses adapter classes to make those mockable too.
public IDatabaseAdapter Database { get; }
It’s very easy to add a public interface to a field type mapping; you simply add an interface attribute to the mapping:
<map field="Single-Line Text" type="Synthesis.FieldTypes.TextField, Synthesis" interface="Synthesis.FieldTypes.Interfaces.ITextField, Synthesis" />
There is now a Synthesis.Testing package on NuGet that contains some dummy field type implementations to help you write cleaner mocks of Synthesis item types. The TestTestField class referred to in the Moq example above is part of the testing package.
Previously you were able to map a Synthesis field class to a Sitecore field type. Synthesis 7 enables you to get more specific and map a field class to a specific field on a template. This enables you to provide custom behavior for specific fields.
<templateFieldMappings hint="raw:AddTemplateMapping">
<map template="Content Section" field="Content" type="Synthesis.FieldTypes.TextField, Synthesis" interface="Synthesis.FieldTypes.Interfaces.ITextField, Synthesis" />
</templateFieldMappings>
The template specification can be the template name, full path, or ID interchangeably. The field can be either the field name or field ID.
If you’ve used Synthesis long, you’ve probably seen this one. You add a template field, generate your model, and forget to publish the field. Suddenly you get a really helpful “Parameter cannot be null” error that doesn’t tell you anything about the field name that actually caused it. Synthesis 7 fixes that problem, and reports the field name that was missing instead of a generic error message.
LinkProvider has AlwaysIncludeServerUrl enabled. Previously, links like /http://foo/~/media/bar would be generated. Fix contributed by Dave PetersonGetSynthesisQueryable() to work, because it was fixed in Sitecore 7 Update-1.Item instances. This enables decoupling the IDatabaseAdapter from the Sitecore API.FileListField (File Drop Area) field type has been removedWordDocumentField field type has been removedIFieldMappingProvider interface signature has changed to enable injecting custom field types on a field level. Custom providers may need minor refactoring.SynthesisPresenter.GetItem() method has changed from protected to public (a breaking change for all presenters based on this). Unfortunately, this was by far the simplest and cleanest avenue to allow for injecting arbitrary Synthesis item data sources for testing purposes.Upgrading to Synthesis 7 is a process that takes a bit of work.
First, upgrade the NuGet package to Synthesis 7
After upgrading the NuGet package, you should remove the field mappings in Synthesis.config to the removed Word Document field and File Drop Area field types. The Word field mapping may not exist depending on what version you’re coming from.
<map field="File Drop Area" type="Synthesis.FieldTypes.FileListField, Synthesis" /> <!-- remove this from Synthesis.config -->
Next, you will need to regenerate the Synthesis model. Do not build the project at this point, because the model format has changed in Synthesis 7 and the existing model will be full of build errors. Hit /synthesis.axd and force a regeneration, which will remove any build errors not caused by a breaking change as outlined above.
httpRequestBegin pipeline handler) is causing an error before you even get to synthesis.axd temporarily disable it for the purposes of regenerating the model.By default, a Synthesis 7 upgrade will not enable using interfaces for field types (unless you overwrite Synthesis.config during the install, which is not recommended). Should you wish to enable interfaces, simply look at the default config file and merge the interface attributes in under <fieldMappings> to your config, then regenerate.
If using Synthesis.Blade, you will need to change the visibility on GetItem() for any presenters that use SynthesisPresenter as their base class to public, instead of protected.
Verify that the case of your content search config patch at the bottom of Synthesis.config matches the case in your Sitecore.ContentSearch.Lucene.DefaultIndexConfiguration.config. The case of the default config changed in Update-2, but depending on how you upgraded yours might not have. See this blog post for details as to why this is necesary. If the case is incorrect, Synthesis LINQ queries will not work correctly.
Fix any remaining build issues caused by breaking changes (if any), and test the site.
I’d love to hear from you if you have trouble or ideas. Send me an issue on GitHub!
As always, the Synthesis source is available under the MIT license, so feel free to hack away - and send me a pull request ;)
I’m happy to announce that Blade v2 is now available from NuGet. There are some very nice new features in this release.
Blade 2.0 replaces the RazorEngine library with directly calling MVC4 APIs. Doing this is much cleaner (and faster!) than the previous Razor implementation. This does introduce some minor breaking changes so read up before you upgrade.
@model to declare the model type in razor views instead of the much more verbose @inherits RazorRendering<TModel> previously required. The old way still works fine too.BladeHtmlHelper is gone, replaced by the normal MVC HtmlHelper - note that pieces of the helpers that do not make sense in a pseudo-MVC context, such as ActionLink, may behave unpredictably.~/Foo/Bar/Baz.cshtml) unless you are storing your Razor files in /Views like MVC doesRazorTemplate web control, you must add .cshtml on the end of view paths, whereas previously that was optionalPresenters can now handle postback in a clean fashion by using ASP.NET MVC model binding to bind posted values to your ViewModel automatically. There is also explicit support for handling XHR (AJAX) postbacks separately, enabling easy creation of simple callbacks to the same page that return JSON data.
[Required]) and validation controls are fully supported for Razor views, as are MVC HTML helpers (@Html.TextBoxFor(x=>x.MyStringProperty), for example). User Controls and Web Controls support validation, but they do not have ModelState to pass it around to so you’d need to manually parse the validation result in the presenter for those.SitecorePresenter or SynthesisPresenter, simply override the HandlePostBackWithModelBinding method. The method is invoked if HTTP POST is requested, and model binding is automatically executed. Similar methods exist for XHR and non-automatic-modelbinding for both.GetModel() will have already been called on the presenter.ModelBinders in Global.asax, just like in MVC, should work perfectly.In Blade itself, the SitecorePresenter has been modified to allow injection of a custom Item as the data source of the presenter (via the SetDataSource() method), enabling you to decouple it from data source resolution logic. However, you probably should not be testing with Item instances, as those require a lot of Sitecore API presence. To get around this issue, Synthesis.Blade’s SynthesisPresenter enables injection of a custom Synthesis item interface directly into its GetItem method for completely Sitecore-independent tests! Check out this trivial example of testing a SynthesisPresenter, using Moq:
var item = new Mock<ITestTemplateItem>();
var view = new Mock<IView>();
// todo: setup mocks appropriately
var presenter = new TestPresenter();
var model = presenter.GetModel(view.Object, item.Object);
// todo: assert something about the resultant model
Views are not particularly testable due to them using the MVC Razor engine, which requires a web context to execute. Hypothetically you could write tests against them using a web-based test runner, but it seems that most people do not recommend testing the view due to the relative fragility of the tests involved.
To upgrade from Blade 1.5 to Blade 2.0, simply upgrade the NuGet package. You should definitely test all of your razor views after the upgrade to make sure they are running correctly against the new MVC-based razor implementation. If you are not using razor, the upgrade should be pretty much seamless.
Because Blade uses XDT-transformation-based installation routines, you will need NuGet 2.6 or later to install it.
If you are also using Synthesis with Blade, you will need to upgrade to Synthesis/Synthesis.Blade 7.0 to be compatible with Blade 2.0.
I’d love to hear from you if you have trouble or ideas. Send me an issue on GitHub!
As always, the Blade source is available under the MIT license, so feel free to hack away - and send me a pull request ;)
Sitecore 7 Update-2 was released today, and it brought with it a surprise: Synthesis’ LINQ query support broke. Well, that’s unusual for a Sitecore update.
Fortunately it is easy to fix. In the Sitecore.ContentSearch.Lucene.DefaultIndexConfiguration.config config patch file, Update-2 changes all of the element casing from PascalCase to camelCase. So the old XPath /configuration/sitecore/contentSearch/configuration/DefaultIndexConfiguration turned into /configuration/sitecore/contentSearch/configuration/defaultIndexConfiguration. DOH! Unfortunately for us, XML - and Sitecore config patching - is case senstive. This means the config tweaks made to search in Synthesis.config no longer applied!
The release notes for Update-2 do not mention this casing change, so if you were manually patching you might not catch this. But the stock DefaultIndexConfiguraion.config file available for download on that page, as well as in the zip distro of Update-2, both change a lot of case in the config.
Fortunately it’s pretty easy to fix Synthesis running against a re-cased Update-2 config file. At the bottom of your Synthesis.config find the
<contentSearch>
<configuration>
<defaultIndexConfiguration>
<indexDocumentPropertyMapper type="Synthesis.ContentSearch.SynthesisDocumentTypeMapper, Synthesis" />
<fields hint="raw:AddComputedIndexField">
<field fieldName="_templatesimplemented" storageType="yes" indexType="untokenized">Synthesis.ContentSearch.ComputedFields.InheritedTemplates, Synthesis</field>
</fields>
</defaultIndexConfiguration>
</configuration>
</contentSearch>
Unfortunately this leaves Synthesis in a less than optimal situation where the default distribution is more or less Sitecore patch-specific. Synthesis 5, released with Sitecore 7 RTM, only works with RTM. Synthesis 6, released with Update-1, only works with Update-1 due to the moving of a Sitecore class in the update. It requires you to find this blog post to get it to work correctly with Update-2 by way of the patch above. The upcoming Synthesis 7 will have configuration by default for Update-2 - which means it will break if installed under Update-1.
Don’t get me wrong, I love Sitecore 7, but the search APIs are more than a little unstable at the moment for library authors!
A few months ago I read a very interesting article on Ars Technica titled How I became a password cracker. Seriously, go read it. These days it’s astonishing how simple it is to brute force even long, random-looking passwords. That 1337-5p33k p@55w0rd? You might as well use dictionary words because there are rules that check for that. Using a passphrase of english words (like correct horse battery staple)? Well it’s easy to test for combinations of dictionary words too, so those are a lot less secure than they look.
The core problem is that most passwords are stored using hash algorithms such as SHA and MD5 that were originally designed to verify file integrity - so speed was a primary concern. If a hash algorithm is fast, so is brute-forcing passwords that use it. Modern GPUs can compute hashes obscenely fast (spend a bit and you can hit 1 billion hashes per second), so a good way to thwart attackers is simply to use a hash algorithm that is slow. Several algorithms exist that are designed to be utilized specifically for passwords (such as PBKDF2 and BCrypt), and they are all much slower to compute for this very reason.
These algorithms also usually have a “work factor” that allows you to adjust the time it takes to compute the hash, which provides even more future-proofing against even faster password cracking hardware that is yet to be developed.
To be clear this is not a remotely-exploitable vulnerability. It’s also not a Sitecore-specific vulnerability - it affects most all sites running ASP.NET’s Membership provider which until very recent providers (Universal Providers) defaults to the SHA1 hash algorithm to store passwords. In order to exploit what I’m describing here, an attacker would have to gain access to your databases, which they would then use an offline password cracker like oclHashcat-plus to break the passwords to login to Sitecore.
There’s a class in the .NET framework that implements the PBKDF2 algorithm already, but it does so in a way that makes it difficult to add to a membership provider. There’s a very nice NuGet package called Zetetic.Security that wraps this functionality into a KeyedHashAlgorithm that can be used easily with membership. It’s very easy to install PBKDF2 into a membership provider using it - you have to install the NuGet package and change two configurations.
The only problem is that existing users will be unable to login because the existing hashes are all in SHA1 (Sitecore, and membership’s, default algorithm) and the membership provider now expects PBKDF2 hashes. If there are only a few users you can just reset passwords and be done with it. But if you have an existing userbase, you probably wish you could allow existing users to keep signing in, and new users (or password changes) to convert to PBKDF2. I’ve implemented a prototype extension of the SqlMembershipProvider that does exactly this by allowing the ValidateUser method to attempt SHA1 if PBKDF2 validation fails.
The solution discussed in this post was tested against Sitecore, but is a generic method you could use with any .NET app that uses SQL Membership to store user info. I hope this random crypto rant has been useful to you :)
