Project work in progress – Part 3: Getting technical

In our ‘Project work in progress — Part 1‘ and ‘Part 2‘ blogs we looked at our process and the team; now it’s time to get technical and talk about the solution, environments and releases.

Early in the project, our client selected Umbraco as their chosen content management tool after comparing the associated features and costs with those of Sitecore. Although we have a lot of experience developing with Umbraco and many positive things to say about it, there had been some issues with releases since version 6 so we had some concerns over performance under load and whether the platform would scale up to an enterprise solution. This meant the technical architecture needed careful consideration.

Identifying issues

The first step was to test the Umbraco release under load and identify areas of weakness. To do this, we loaded a blank version of Umbraco v6 on to our development server and load tested it through Loadster, ramping up to 400 concurrent users. The results showed a really high initial spike in RAM which then settled down over time. As we increased the amount of content, the response times grew significantly.

Based on these results, we came to these conclusions:

1. The application needed to be in an environment where it could ‘warm up’ before applying load.
2. We should defer load away from Umbraco wherever possible.
3. We needed a LOT of caching!

Separating the editor environment

We made a decision to separate the Umbraco editor environment from the web-facing sites for resilience purposes; removing the editor from live sites leads to less regular publishing, which means a less volatile cache. Separate environments also allow us to fine tune each for performance, and we can scale the web environment up as needed.

This means that we now needed to move changes from the editor environment to the web. Rather than trying to use Courier in a solution of this scale, we wrote our own sync tool that compares databases using Redgate SQL compare. We have added custom dashboard components to Umbraco to allow users to configure and monitor the sync tool (you can view the status and detailed logs from each synchronisation).

The tool works in two modes — sync mode (moving published content from editor to web) and release mode (moving Umbraco changes from development to UAT to live). For this project we have decided not to promote content between environments but as we have seeded identifiers differently we could support this in future by making minor changes.

By using the sync tool with Octopus, we are able to perform releases in around one minute with no manual steps required. (If you would like to know more about the sync tool, get in touch via our Twitter or Facebook page).

Reducing load on Umbraco

To reduce the load server-side, we wanted to read content from a data source external to Umbraco. We implemented Elastic Search, pushing content into the index as it is published through a custom indexer. We use a search service to return data wherever possible, rather than relying on the Umbraco API. The performance of pages built through the search is impressive and it drastically reduces the load on Umbraco. This model also resolves content dependency issues in a distributed environment.

Caching, caching and more caching

Here are a few caching layers that we have added…

At an application level we cache by component using MVC child actions with output caching.
We store Umbraco media items in a database and have built a custom provider for image resizer to serve and cache media.
Using Cassette with a CDN domain and an additional caching layer reduces the number of front end requests. (More details on our front end solution coming in a future post).
On top of this, we have added an additional proxy layer using Varnish servers to provide page level caching for key pages where there is no customisation. (Again, more info on infrastructure coming soon for those that are interested.)

The result?

We are now over halfway through the site build and the client has started to add content to the production environment. We have been continually performing load testing and monitoring performance throughout the build phase but, so far, results look promising.

[0]:/blog/2013/04/project-work-in-progress-–-part-1-from-pitch-to-sprint-0/ “Project work in progress – Part 1: From pitch to sprint 0”
[1]:/blog/2013/06/project-work-in-progress-part-2-solution-design-review/ “Project work in progress – Part 2: Solution design & review”