TicketDesk 3 Dev Diary – Update: One AspNet and Aspnet.Identity

Since most of what I’m working on now is being done in my private repository, I wanted to give everyone a quick update on TicketDesk 3′ s progress.

I’m still working on TD3, but late in the fall it became apparent that I needed to wait on the RTM version of Visual Studio 2013 and the new Asp.net Web API 2 and Identity bits. Now that this stuff has all been released, and most of the dependencies have caught up, I have resumed work on TD3.

The main challenge is incorporating the new Aspnet.Identity framework. I am thrilled that Microsoft finally replaced the old security providers, but the transition to Aspnet.Identity is not all roses. The new framework is not as well documented as I’d like, and guidance on advanced uses is still thin. It is also fairly complex framework that requires a good bit of manual coding, especially when used in conjunction with a SPA front-end. Fortunately for me, Yago Pérez Vázquez has created a template project called DurandalAuth that does exactly what I’ve been trying to do with TicketDesk…. combine the bleeding edge versions of Aspnet.Identity and Web API 2 with the bleeding edge versions of Durandal, Breeze, and Bootstrap.

In fact, his template is so good, that I’m pretty much building TicketDesk 3 on top of his template instead of just porting the authentication stuff into my existing TD3 project… and this is why the project hasn’t been pushed into the public repository just yet… it’s a new project that doesn’t yet have all the features from the old one.

There are a few things about the DurandalAuth template that I’m not so sure about; the use of StructureMap instead of Ninject for IoC, and fact that he’s layered the back end with a custom repository and unit of work pattern… but overall, the design is generally the same as what I had been designing for TD3; except that he’d also implemented the new asp.net identity bits. The template also includes some SEO stuff that isn’t relevant to TicketDesk 3, though I may leave it there in case people want to make use of it.

At present, I’m in the process of combining the new project with the code I’ve already written for TD3, and adapting the design to TD3’s particular needs (internationalization for example). This will take a few more weeks, but once I’m done I will be able to push the new project to the public GitHub repository for everyone else to look at.

The technology stack for TD3 is now complete; and includes the following:

  • Asp.net Web API 2
  • Durandal 2 & Breeze
  • i18Next
  • Aspnet.Identity
  • Bootstrap 3
  • SignalR 2

The only major design element that I’ve yet to work out completely is using ACS and/or ADFS security. The identity implementation for Web API 2 uses a different mechanism (organization accounts) for integrating with ACS and ADFS; so I’ll have to find a way to smash the two options together; and provide enough internal abstractions to where either configuration is possible with minimal additional code.

TicketDesk 3 Dev Diary – MEF, IoC, and Architectural Design

TicketDesk 2 and TicketDesk 3 have some key architectural differences. Both enforce a strict separation of concern between businesses and presentation layers, but there are major architectural differences within each layer. In this installment, I’d like to talk about how the back-end architecture will evolve and change.

TicketDesk 2 – Decoupled design:

The most significant technology that shaped TicketDesk 2’s class library design was the use of the Managed Extensibility Framework (MEF). The use of MEF in TicketDesk 2 was not about modularity, at least not in a way that is relevant to business requirements. TicketDesk 2 was never intended to support plug-ins or dynamic external module loading. I used MEF for two reasons; I was giving test driven development (TDD) another shot, and I had planned to write a Silverlight client for TicketDesk 2.

MEF was originally built by the Silverlight team. It had a lot of potential for other environments, but didn’t play well with MVC back then. It took some dark magic and hacking to just make it work there. MEF is an extensibility framework first, but an IoC container only by accident. While MEF can do the job of an IoC container, it wasn’t particularly good in that role.

As an extensibility framework, MEF actually has more in common with require.js than traditional server-side IoC frameworks. As a Silverlight technology, the primary purpose was to enable clients to download executable modules from the server on demand when needed. This is exactly what require.js does for JavaScript in HTML applications. The truly interesting thing is that TicketDesk 2 did not use MEF in this way at all. Asp.Net MVC is a server-side environment following a request-response-done type execution flow. Deferred module loading isn’t relevant in that kind of environment. TicketDesk used MEF only for its secondary IoC features — runtime composition and dependency injection.

Considering the difficulty in getting MEF working, and the fact that there are better IoC frameworks for MVC, I should have scrapped MEF in favor of Ninject –which has made me very happy in dozens of other projects. I stuck with MEF partly because it would pay off when I got to the Silverlight client, and partly because I liked the challenge that MEF presented.

Sadly, I was only three weeks into development on TicketDesk Silver, the Silverlight client, when Microsoft released Silverlight’s obituary. I had two other projects under development with Silverlight at the time, so that was a very bad summer for me.

The modular design of TicketDesk’s business layer is mostly about testability. EF 4 was quite hostile to unit testing, so I did what everyone else was doing… I wrapped the business logic in unit-of-work and repository patterns, and made sure the dependencies targeted abstract classes and interfaces. If you want to get all gang-of-four about it, the service classes in TD2 are more transaction script than unit-of-work, but it gets the same job done either way. This gave me the level of testability I needed to follow a (mostly) TDD workflow.

One thing I have never liked about heavy unit testing, and TDD in particular, is having to implement complex architectures purely for the sake of making the code testable. I’ll make some design concessions for testability, but I have a very low tolerance for design acrobatics that have nothing to do with an application’s real business requirements.

TicketDesk 2 walks all over that line. I dislike that there are a dozen or more interfaces that would only ever have one (real) concrete implementation. Why have an interface that only gets inherited by one thing? I also dislike having attributes scattered all over the code just to describe things to an IoC container. Neither of those things make TicketDesk work better. It just makes it more complex, harder to understand, and harder to maintain.

On the flip-side, I was able to achieve decent testability without going too far towards an extreme architecture. The unit tests did add value, especially early in the development process –They caught a few bugs, helped validate the design, and gave me some extra confidence.

If you noticed that the current source lacks unit tests, bonus points to you! My TDD experiment was never added to the public repository. I was pretty new to TDD, and my tests were amateurish (to be polite). They worked pretty well, and let me experience real TDD, but I didn’t feel that the tests themselves made a good public example of TDD in action.

TicketDesk 3 – Modularity where it matters:

A lot has changed for the better since I worked on TicketDesk 2.

Some developers still write their biz code in a custom unit-of-work and repository layer that abstracts away all the entity framework stuff; which is fine. But when EF code-first introduced the DbContext, it became much friendlier towards unit testing. The DbContext itself follows a unit-of work pattern, while its DbSets are a generic repository pattern. You don’t necessarily need to wrap an additional layer of custom repository and unit-of-work on top of EF just to do unit testing anymore.

I plan to move most of the business logic directly into the code-first (POCO) model classes. Extension methods allow me to add functionality to any DbSet<T> without having to write a custom implementation of the IDbSet interface for each one. And the unit-of-work nature of the DbContext allows me to put cross cutting business logic in the context itself. Basically, TD 3 will use something close to a true domain model pattern.

As for dependency injection, the need to target only interfaces and abstract types has been reduced. An instance of a real DbContext type can be stubbed, shimmed, or otherwise mocked most of the time. In theory, I should be able to target stubbed/shimmed instances of my concrete types. If I find the need to target abstracts, I can still refactor the DbSets and/or DbContext to inherit custom interfaces. There still isn’t a compelling need to wrap the business logic in higher layers of abstraction.

In TicketDesk 3, I will not be using a TDD workflow. I love unit testing, but am traditionally very selective about what code I choose to test. I write tests for code that will significantly benefit from them –complex and tricky code. I don’t try to test everything. Using TDD as a design tool is a neat thought process, but I find that design-by-test runs counter to my personal style of design. I can easily see how TDD helps people improve their designs, but I personally tend to achieve better designs when I’m coding first and testing later.

When I do get to the need for dependency injection, I plan to run an experimental branch in TicketDesk 3 to explore MEF 2 a bit further. I think they have fixed the major issues that made MEF 1 hard to use in web environments, but it is almost impossible to find good information online about MEF 2. The documentation, when you can find it, is outdated, contradictory, and just plan confusing. What I have found suggests that MEF 2 does work with MVC 4, but still requires some custom infrastructure. What I don’t know is how well it works.

With the need for dependency injection reduced, few compelling extensibility requirements on the back-end, and no plans to do heavy unit testing, I am more inclined to go with Ninject. They care enough to write top-notch documentation, and it was designed explicitly for the purpose of acting as an IoC container… which is the feature set TicketDesk actually needs.

TicketDesk 3 Dev Diary – Localization and Internationalization

worldOne of the most frequently requested features for TicketDesk 2 was support for localization. TicketDesk is a stringy application; lots of system generated text that will end up in the user’s face at some point. Localizing TD2 required combing through the source code, line-by-line, translating magic strings by hand.

Clearly, this is not an acceptable approach with TicketDesk 3.

Since localization is thorny, and a weak spot in my own skill-set, I consider it essential to designed for localization as early in the process as possible… and now that the code has gotten to the point where it draws a useful UI, it is time to get started.

In the typical Asp.Net application, localization is mostly just a matter of creating resource files that contain the text translations, then making sure the code only gets text from those resources. There is a lot of support in .Net around localization, cultures, and resource files, so this is pretty easy to do. The only difficult part, for the mono-lingual developer, is getting the text translated into those other languages in the first place.

TicketDesk 3 is a SPA application, which presents a different problem. The UI is mostly HTML and JavaScript, so all that nice .Net localization stuff is unavailable when generating the screens that users will actually see. So the first step was to find a JavaScript library for localization; something that does the same job as .Net resource files. The second challenge was connecting that JavaScript library to the server-side .Net resource files.

Fortunately, there is a fantastic JavaScript library called i18next that fits the bill.

Translations in TicketDesk 3:

i18next follows a pattern similar to server-side .Net resource files. You supply it with json files that contain translated text. Once i18next has the text, it takes care of binding it to the UI via an HTML data-* attributes, or through javascript functions directly. As a bonus, i18next is easy to use in conjunction with knockout’s own UI binding.

TicketDesk performs text functions on the server too, so it still needs resource files, so I wanted to be able to pipe the contents of the resource files to i18next directly, rather than maintaining separate translation files for the server and client. For this, I leveraged Asp.Net Web Api. Configuring i18next to get its json translations from Web Api is simple –just map the URLs it uses to Web Api endpoints.

The Web Api controller itself was a bit more complex. It has to detect the specific language that i18next is requesting, then build an appropriate response in a format i18next can consume. The controller loads a ResourceSet for the requested culture, then loops through the properties to build a dynamic key/value object with all the translated text. Once that’s done, it outputs the dynamic collection as a json response.

i18next has a richer set of capabilities than straight .Net resource files. Resource files are just simple name/value lookups. With i18next, the translation files can have nested properties, replacement variables, and there are features for interpolation (plural forms, gender forms, etc.). These features are available in .Net with some advanced language frameworks, but the basic resource files don’t go that far. Fortunately, TicketDesk only needs the basic set of features, so a flat name/value lookup should be sufficient to get the job done; though it doesn’t leverage some of i18next’s more advanced features.

Localization is more than text translations. TicketDesk also deals with numbers occasionally, and there are some dates too. Fortunately, it isn’t a number heavy application, nor are are there user editable dates or numbers.  The moment.js library easily handles local date display formatting, and numeral.js can handle the couple of numbers.

The main weak point in TicketDesk 3’s localization will be an absence of structural language transformations. Once you get into right-to-left languages and other exotic forms, the physical layout of the entire UI has to change. Sadly, I do not have the expertise to correctly design for such languages.  HTML 5 and CSS 3 do have decent support for this kind of cultural formatting though, so my hope is that anyone needing to localize for these languages can do so themselves without difficulty.

Internationalization:

My intention for TicketDesk 3 was simple localization; the admin would tell the server what language to use, and the system would just generate the UI in that language for all users. I did not initially expect to support dynamic internationalization — the ability to support multiple languages based on individual user preference.

When I got into the details of the i18next implementation though, it quickly became apparent that TicketDesk 3 could easily achieve real internationalization… in fact, internationalization would be about as easy as static localization.

The result is that TicketDesk 3 will be internationalized, not just localized. It will detect the user’s language and dynamically serve up a translated UI for them –as long as resource files exist for their language. If translations for their specific language and culture aren’t available, it will fall back to the best-matching language, or to the the default if no better match exists.

State of the Code:

I have the plumbing for internationalization in place in the alpha already. It auto-detect’s the browser’s language, or you can override it via the query string (e.g. ?setLng=es-MX). Since I don’t speak any other languages, I can’t provide any real translations myself. For the short term, I have created a generic Spanish resource file, into which I copied the English text surrounded by question marks. This isn’t real localization, but it serves to validate that the localization support works correctly.

For dates, I’m using moment.js, so it should adapt to the browser’s language settings automatically, but I haven’t setup moment to use the querystring override yet… I’ll get to that soon though. I’m not doing any number formatting yet, but when I do I’ll implement numeral.js or a similar library.

When TicketDesk 3 gets into beta, and I have the full list of English text strings in resource files, then I will get a native Spanish speaker to help generate a real set of translations. Hopefully, the community will pitch-in to provide other languages too.

If you want to take a look at the early alpha version, I have published a TicketDesk 3 demo on Azure. I can’t promise that it will be stable, and it certainly isn’t a complete end-to-end implementation. But feel free to play around with it. To play with localization, either change your browser’s language to something spanish (es, or es-MX, or similar), or use the querystring override: ?setLng=es

TicketDesk 3 Dev Diary – SignalR

One of the overall design goals of TicketDesk since version 1 has been to facilitate near-frictionless, bi-directional communication between help desk staff and end-users. Tickets should evolve as a natural conversation, and the entire history of those conversations should be right there in the ticket’s activity log. TicketDesk has done a reasonably good job in this area, but SignalR presents an opportunity to take this idea to a whole different level.

The goal behind the SignalR library is to give the server a live, real-time channel to code running on the client. The server can push notifications whenever things change, and that information is available to the user immediately. The techniques that SignalR use to achieve this are not entirely new, but have historically been difficult to implement.

TicketDesk 3 uses breeze on the client, and breeze tracks all the entities it has retrieved from the server. Knockout is used to bind those entities to the UI for display. The beauty of this combination is that Knockout automatically updates the UI anytime the entities in breeze change.

With SignalR, the browser can listen in the background for updates from the TicketDesk server. When the server notifies the client that a ticket has changed, the client can then choose to fetch the new data in the background, and update the local copy being tracked by Breeze… and Knockout will automatically refresh the display to show that updated data to the user.

The best thing about SignalR is that it is trivially easy to setup, and with the combination of Breeze and Knockout it is super simple for the UI to respond intelligently.

As a proof of concept, I have coded up a simple SignalR hub that will tell all connected clients when a ticket changes (and what the ID of the changed ticket is). The client will check to see if it is tracking a ticket with that ID, and if so it will automatically fetch a new copy of the ticket from the server. Anything on the screen that is bound to that ticket will automatically update to show the changes. This was not only very easy to implement, but it seems to work very well.

I then took it a step further, and coded up several special routines for the ticket list view. Not only does it update the tickets displayed on screen, but it also responds intelligently to changes in the number of open tickets, or changes of the order of the tickets within the list.

This list view, as currently implemented in the alpha, is a paged list showing 5 items on screen at a time. Because the list is sorted by the last updated date, anytime a ticket is updated the order of items in the list changes too. If a ticket is closed or re-opened, the number of items will grow or shrink change. Pager buttons may need to be disabled or enabled, items on the current page may change as tickets are shuffled around, and the current page might not even be valid anymore if the number of open tickets shrinks enough.

With very little effort, I was able to code up the list view that dynamically responds to real-time changes on the server, and keeps itself current without the user ever needing to explicitly refresh the screen.

I plan to use the set of capabilities around SignalR to make the entire application behave in near real-time. The Ticket activity log will behave like a real-time chat conversations, lists will automatically adjust as things change, and notifications will appear to keep the user informed.

If you want to take a look at the early alpha version, I have published a TicketDesk 3 demo on Azure. I can’t promise that it will be stable, and it certainly isn’t a complete end-to-end implementation. But feel free to play around with it.

To see the SignalR behavior in action, just open the site in two browsers at the same time. Make changes to a ticket in one, and watch the other browser update itself.

TicketDesk 3 Dev Diary – Hot Towel

toweliconFor TicketDesk 3, what I most hope to achieve is an improvement in the overall user experience. Since I wrote TicketDesk 2, much has happened in the JavaScript world. New JavaScript frameworks have matured, and enable deeper user experiences with much less development effort than ever before. TicketDesk is a perfect candidate for Single Page Application (SPA) frameworks, so all I had to do was pick a technology stack and learn to use it.

I have decided to start from the wonderful Hot Towel SPA by John Papa. Hot Towel is a visual studio project template that combines packages from several different client and server frameworks. It includes Knockout.js for UI data-binding, Durandal for routing and view management, and Breeze for talking to the ASP.NET Web Api backend.

My main reasons for choosing Hot Towel are:

  • It is a complete end-to-end SPA Template.
  • It is well documented.
  • The components it relies on are reasonably mature, and well maintained.
  • There are good sample applications built on Hot Towel.
  • John Papa published an excellent, and highly detailed, video course for Hot Towel at Pluralsight.
  • It is very easy to learn and use.

One of the disappointments when switching from server-side Asp.Net to a SPA framework is that the UI is pure JavaScript, HTML, and CSS. It makes almost no use of MVC controllers or views, which always makes me feel like I’m wasting valuable server capabilities. A SPA does make heavy use of Asp.Net Web Api for transferring data, but the UI leaves all that wonderful Asp.Net and Razor view engine stuff behind.

Once I learned by way around Hot Towel, I was surprised to find that working with Knockout, Durandal, and Breeze on the client is much easier than working with Asp.Net on the server. I’m no fan of JavaScript as a language, but the current crop of JavaScript frameworks are truly amazing.

Now that I’ve learned my way around Hot Towel’s various components, I’ve been able to develop a fairly advanced set of UI features very quickly. The current UI is very raw and only provides a primitive set of features, but it has already exceeded my initial expectations by several orders of magnitude.

If you want to take a look at the early alpha version, I have published a TicketDesk 3 demo on Azure . I can’t promise that it will be stable, and it certainly isn’t a complete end-to-end implementation. but feel free to play around with it.

Asp.net 4.5 mvc or webforms – model binding dropdownlist to an enum with description attributes

One of the more common problems people encounter when working with any .Net user application is the need to put a UI on top of some enumeration. Normally you need to present a friendly list of all the possible items in the enumerator list, then allow the user to pick one of them. This UI typically takes the form of a drop down list, combo box, list box, or similar.

Enums are wonderful in C#, but unlike some other languages, they are also a very thin type. Enums define a collection of named constants. By default, each enumerator in the list equates to an underlying integer value.

Here is an example Enum, and for clarity I’ve specified the value for each enumerator:

public enum CarMake
{
    Ford,     //0
    Chevy,    //1
    Toaster   //2
}

Enums are lightweight, highly efficient, and often very convenient –until you start trying to map them to a UI anyway.

Each of the items, enumerators, within the enum have a name, but the name cannot contain white space, special characters, or punctuation. For this reason, they are rarely user-friendly when converted to a string and slapped into your dropdown lists.

Enter the DescriptionAttribute (from the System.ComponentModel namespace). This attribute allows you to tag your enumerators with a nice descriptive text label, which will fit the UI pretty well if only you can dig the value up. Unfortunately, reading attributes is a cumbersome job involving reflection.

Here is the same enum decorated with descriptions:

public enum CarMake
{
	[Description("Ford Motor Company")]
	Ford,     //0

	[Description("Chevrolet")]
	Chevy,    //1

	[Description("Kia")]
	Toaster   //2
}

To bind up an enum to a drop down list, many developers tend to just manually hard-code the list with user-friendly text values and corresponding integer values, then map the selected integer to right enumerator on the back-end. This works fine until someone comes along later and changes the enum, after which your UI is horribly busted.

To get around this mess, I’ve put together a set of extensions that solves this problem for the common enum to drop down list cases.

Note: I’m using the SelectList class, which comes from Asp.net MVC, as an intermediary container. I then bind the SelectList to the appropriate UI control. You can use SelectList in Asp.net webforms, and most other UI frameworks as well, but you’ll need to implement the code for SelectList. The easiest way to do this is to include the source files for SelectList into your own projects.

The code for SelectList can be found on the AspNetWebStack project page over at CodePlex. Here are the three files needed for SelectList :

The first step in solving the problem is to have an extension method that takes care of reading the description from the enumerators in your enum.

public static string GetDescription(this Enum enumeration)
{
	Type type = enumeration.GetType();
	MemberInfo[] memberInfo = type.GetMember(enumeration.ToString());

	if (memberInfo != null && memberInfo.Length > 0)
	{
		var attributes = memberInfo[0].GetCustomAttributes(typeof(DescriptionAttribute), false);

		if (attributes.Length > 0)
		{
			return ((DescriptionAttribute)attributes.First()).Description;
		}
	}
	return enumeration.ToString(); ;
}

To get an enumerator’s description using this extension method:

string text = CarMake.Ford.GetDescription();

The next challenge is to build a select list for the enum.

public static SelectList ToSelectList(this Enum enumeration, object selectedValue = null, bool includeDefaultItem = true)
{
	var list = (from Enum e in Enum.GetValues(enumeration.GetType())
				select new SelectListQueryItem<object> { 
				   ID = Enum.Parse(enumeration.GetType(), Enum.GetName(enumeration.GetType(), e)), 
				   Name = e.GetDescription() }).ToList();

	if (includeDefaultItem)
	{
		list.Insert(0, new SelectListQueryItem<object> { ID = null, Name = "-- select --"});
	}
	return new SelectList(list, "ID", "Name", selectedValue);
}
internal class SelectListQueryItem<T>
{
    public string Name { get; set; }
    public T ID { get; set; }
}

To get the select list using this extension, you just new-up the enum and call the method:

var carSelectList = new CarMake().ToSelectList();

This extension has an optional parameter to set which item in the list should be selected, and another optional parameter that will include a default item in the SelectList (you may want to adjust this extension to match your own convention for default items).

Here is an example that sets a selected value and includes a default item:

var carSelectList = new CarMake().ToSelectList(CarMake.Ford, true)

I’ve been working in C# for over ten years, and until just this week I had no idea that you could instantiate an enum with the new keyword. Of course I cannot think of reason why you’d normally want to new-up an enum either, but the capability is handy for this extension.

You can also use this extension by calling it on one of the specific enumerator items too. This example does exactly the same as the previous example:

var carSelectList = CarMake.Ford.ToSelectList(CarMake.Ford, true);

I personally prefer the new-up pattern in this case, since it isn’t intuitive that calling ToSelectList on a specifc item would return the list of ALL items from the containing enum.

Now that we have the SelectList, all we have to do is bind it up to a DropDownList.

In Asp.net WebForms 4.5, using model binding, this looks like this:

<asp:DropDownList ID="CarMakeDropDown" runat="server" 
	SelectMethod="GetCarMakeDropDownItems"
	ItemType="MyApp.SomeNameSpace.SelectListItem" 
	DataTextField="Text" 
	DataValueField="Value"
	SelectedValue="<%# BindItem.CarMakeValue %>" 
/>
protected SelectList GetCarMakeDropDownItems()
{   
	return new CarMake().ToSelectList();
}

For more in-depth examples of various techniques for binding SelectList to DropDownList in webforms, see my previous article titled “Asp.net 4.5 webforms – model binding selected value for dropdownlist and listbox“.

In Asp.net MVC model binding to a SelectList is a very common and routine pattern, so I’m not going to provide a detailed example here. Generally though, you make sure your model includes a property or method for getting the enum’s SelectList, then you use the Html.DropDownList or Html.DropDownListFor helper methods. This would look something like this:

@Html.DropDownListFor(model => model.CarMakeValue, Model.CarMakesSelectList)

 

 

Asp.net 4.5 webforms – model binding selected value for dropdownlist and listbox

With the Asp.net 4.5 release, webforms inherited several improvements from its Asp.net MVC cousin. I’ve been enjoying the new model binding support lately. Since none of the old datasource controls fully support Entity Framework’s DbContext anyway, model binding is quite handy when using code-first EF models.

For the most part, model binding is straight forward. The basic usage is covered in the tutorials on the asp.net web site, but other resources are rare and hard to find. In the case of DropDownList and similar controls, I found that model binding in webforms was not as straight-forward as I would have thought — especially when trying to set the selected value.

Before I begin, let me explain about the SelectList class.

These examples are valid no matter what data you bind to, but the ItemType that I’m showing in these examples use an implementation of the SelectList class, which I’ve borrowed from asp.net MVC. I don’t like to reference MVC assemblies in my webforms applications, so I just copy in source files to my webforms application. Using SelectList gives you a consistent and strongly typed collection, which tracks each item’s display text, value, and its selected state. SelectList acts as a sort of miniature view-model.

The code for SelectList can be found on the AspNetWebStack project page over at CodePlex. Here are the three files needed for SelectList :

How you bind your dropdownlist depends a lot on if it appears inside some other model bound control or not. To bind a dropdownlist inside of a model bound container (repeater, listview, formview, etc) looks something like this:

*.aspx.cs

<asp:DropDownList ID="MyDropDown" runat="server" 
	SelectMethod="GetMyDropDownItems"
	ItemType="MyApp.SomeNameSpace.SelectListItem" 
	DataTextField="Text" 
	DataValueField="Value"
	SelectedValue="<%# BindItem.EdCodeType %>" 
/>

*.aspx.cs

protected SelectList GetMyDropDownItems()
{
    var items = from t in someDbContext.AllThings
    select new { ID = t.ID, Name = t.Name };
    return new SelectList(items, "ID", "Name");
}

Note: SelectedValue property does NOT show up in intellisense. This appears to be a bug caused by the fact that this property was marked with the BrowsableAttribute set to false (for mysterious reasons). 

When working with a dropdownlist that is not conveniently nested within a model bound container control, binding the dropdown is still fairly simple. You have three options. You can explicitly declare a selected value, if you know what it is at design-time and it never changes. If that isn’t the case, then you can set the SelectedValue property to the results of some method call, or wire up an ondatabound event handler to set the selected item. Here are the examples:

Declarative example: Set SelectedValue to a known value (rarely helpful):

*.aspx.cs

<asp:DropDownList ID="MyDropDown" runat="server" 
	SelectMethod="GetMyDropDownItems"
	ItemType="MyApp.SomeNameSpace.SelectListItem" 
	DataTextField="Text" 
	DataValueField="Value"
	SelectedValue="1234" 
/>

Declarative example: Set SelectedValue to the result of a method call:

*.aspx

<asp:DropDownList ID="MyDropDown" runat="server" 
	SelectMethod="GetMyDropDownItems"
	ItemType="MyApp.SomeNameSpace.SelectListItem" 
	DataTextField="Text" 
	DataValueField="Value"
	SelectedValue="<%# GetSelectedItemForMyDropDown()%>"
/>

*.aspx.cs

private SelectList myDropDownItems;

protected SelectList GetMyDropDownItems()
{
	//store the selectlist in a private field for use by other events/methods later
	if(myDropDownItems == null)
	{
		var items = from t in someDbContext.AllThings
					select new { ID = t.ID, Name = t.Name };

		var selectedItems = from t in someDbContext.SelectedThings
					select new { ID = t.ID};

		myDropDownItems = new SelectList(items, "ID", "Name", selectedItems);
	}

	return myDropDownItems;
}

protected string GetSelectedItemForMyDropDown()
{
	var selected = GetMyDropDownItems().FirstOrDefault(i => i.Selected);
	return (selected != null) ? selected.Value : string.Empty;
}

Event example: Set Selected item from an event handler

*.aspx

<asp:DropDownList ID="MyDropDown" runat="server" 
	SelectMethod="GetMyDropDownItems"
	ItemType="MyApp.SomeNameSpace.SelectListItem" 
	DataTextField="Text" 
	DataValueField="Value"
	OnDataBound="MyDropDown_DataBound" 
/>

*.aspx.cs

private SelectList myDropDownItems;

protected SelectList GetMyDropDownItems()
{
	//store the selectlist in a private field for use by other events/methods later
	if(myDropDownItems == null)
	{
		var items = from t in someDbContext.AllThings
					select new { ID = t.ID, Name = t.Name };

		var selectedItems = from t in someDbContext.SelectedThings
					select new { ID = t.ID};

		myDropDownItems = new SelectList(items, "ID", "Name", selectedItems);
	}

	return myDropDownItems;
}

protected void MyDropDown_DataBound(object sender, EventArgs e)
{
	var ddl = (DropDownList)sender;
	var selectedValue = GetMyDropDownItems().FirstOrDefault(i => i.Selected);
	if(selectedValue != null)
	{
		ddl.Items.FindByValue(selectedValue.Value).Selected = true;
	}
}

With the ListBox control, and controls similar, you can employ the same techniques as long as you only allow single item selection. If you need to support multiple selection though, you can’t just set SelectedValue. Instead, you would use the DataBound event to loop each item to selecting the appropriate ones.

*.aspx

<asp:ListBox ID="MyListBox" runat="server"
	SelectionMode="Multiple"
	SelectMethod="GetMyListBoxItems"
	ItemType="Weber.Vfao.Inside.Web.SelectListItem"
	DataTextField="Text" 
	DataValueField="Value" 
	OnDataBound="MyListBox_DataBound"
/>

*.aspx.cs

private SelectList myListBoxItems;

protected SelectList GetMyListBoxItems()
{
	//store the selectlist in a private field for use by other events/methods later
	if(myListBoxItems == null)
	{
		var items = from t in someDbContext.AllThings
					select new { ID = t.ID, Name = t.Name };

		var selectedItems = from t in someDbContext.SelectedThings
					select new { ID = t.ID};

		myListBoxItems = new SelectList(items, "ID", "Name", selectedItems);
	}

	return myListBoxItems;
}

protected void MyListBox_DataBound(object sender, EventArgs e)
{
	var lb = (ListBox)sender;
	foreach (var item in GetMyListBoxItems())
	{
		if (item.Selected)
		{
			b.Items.FindByValue(item.Value).Selected = true;
		}
	}
}

TicketDesk 3 Dev Diary – Thoughts on Security

hunts point, the bronx, new yo... by andre dos santosIn this post, I’m brainstorming new approaches to handling security in TicketDesk 3, and reviewing the limitations of TicketDesk 2’s old approach.

Security in  TicketDesk 2:

TicketDesk 2 shipped with support for either active directory, or local security. Local security via the built-in membership providers worked reasonably well for most deployments, but the AD option has always been problematic.

Windows authentication, in conjunction with the WindowsTokenRoleProvider, works well enough for the basic question, “who are you?”, but windows authentication severely limits in the amount of information you can obtain about the user. You get the windows account name and a list of the user’s AD groups –and that’s pretty much all you get.

Some features in TD required more than that. It needed display names and email addresses, and it needed to list all users in some AD groups; information the built-in providers are incapable of obtaining.

To support these requirements, TicketDesk 2 used a complicated background processes for AD concerns. It maintains a local cache of AD data to reduce the number of queries, and moved AD query processing off of the user servicing request threads.

Even with the abstractions in System.DirectoryServices, querying AD is painfully obtuse. In addition, response times for AD queries are often measured in the tens of seconds –even in single domain environments. You can’t reasonably query AD in real-time without bringing the entire application to a standstill. The custom module alleviates these issues, but was a  complete nightmare to code and maintain.

The other significant problem has been the difficulty of extending the old system to support more features, such as custom roles and permissions. This is why TicketDesk still uses hard-coded roles with pre-defined permissions –I simply never had the time or energy to re-factor the AD side of the system.

Security technologies of the present:

I’ve played a bit with the SimpleMembership and Universal Providers, which have replaced the original SqlMembership providers. ASP.NET’s security system is still based on the same provider framework that shipped with .Net 2.0 back in 2005. Sadly, the framework is a fundamentally flawed system, and in need of a complete redesign.

In recent years, there have been two attempts to modernize the ASP.NET providers, but neither really addresses the underlying problems of the core framework itself. Both are just extensions layered on top of the original infrastructure.

Universal Providers are an evolutionary upgrade of the original SQL providers from ASP.NET 2.0. They mainly just move data access into Entity Framework, which allows them to work with any database back-end supported by EF. Otherwise, there isn’t a much of a difference between them.

SimpleMembership was created for ASP.NET WebPages, but has become the officially recommended replacement in MVC projects as well. This is the default security provider in the newer MVC project templates that come with Visual Studio 2012. As I wrote in another post though, this system has some deep design flaws of its own. You almost have to learn how to hack up a customized version of SimpleMembership just to make it useful in any real-world scenario.

The good news is that both SimpleMembership and Universal Providers include oAuth support backed by the popular DotNetOpenAuth libraries. So, while flawed and build on a shaky foundation, at least they have been modernized.

Traditional windows authentication itself has not changed in any significant way, and there have been no official replacements for the windows security providers that I know of. ASP.NET still uses the same windows provider options that shipped in .Net 2.0, all of which are spectacularly terrible.

The major action for Active Directory authentication has been with Active Directory Federation Services. This is an STS token server that exposes AD to clients that support WS-federation. The Windows Identity Foundation (WIF), originally introduced in WCF, was pulled directly into the core of the .Net framework with the 4.5 release. This provides the infrastructure ASP.NET needs to leverage federated authentication and authorization. So, if you setup federation in ASP.NET you can easily connect it to ADFS to authenticate your AD users. Best of all, ADFS can share any information stored in AD.

In Windows Azure land, you have the Access Control Service (ACS), which is also a WS-Federation token service. From ASP.NET, connecting to ACS is similar to connecting to ADFS. The real advantage of ACS is that it can act as an aggregator and proxy for any number of other external authentication sources. For example, you can connect your ACS to your domain’s ADFS service and to oAuth providers.

The new identity features in .NET 4.5 also incorporate claims based identity as a pervasive and normative feature used throughout the entire framework. You don’t have to use WS-federation in order to use claims based identities, since all identities are now claims based.

Ideas for TicketDesk 3:

For TicketDesk 3, I hope to support the following authentication options:

  • local user accounts (forms with user registration)
  • oAuth (sign-in with google, facebook, twitter, etc.)
  • Windows (AD or local system accounts)
  • WS-Federation (ADFS, Azure ACS, etc.)

My challenge is to find a way to bring all these pieces together in TicketDesk 3 in a sane way.

I want TicketDesk’s business logic to make use of claims based identity for its authorization decisions. Since claims are baked in, this seems to be the most sensible way of doing things. The key will be to map all users, regardless of how they are authenticated, to a common set of claims that TicketDesk understands. The business logic can then rely on having those claims attached to every user’s identity.

For authentication, I have little choice but to rely on existing mechanisms. I am not a crypto expert, and have no intention of rolling my own password and user verification code. But, to support multiple authentication sources, I will likely need to roll my own module to intercept user requests, and dynamically route them to the appropriate pre-built authentication mechanism as needed at runtime.

Each of the possible authentication systems will produce a ClaimsIndentity, but the structure and content of those claims varies. Some authentication generates very few usable claims, while others may contain all the claims TicketDesk could ever need. To bridge the gap, TicketDesk will need an identity transformer. This component will map incoming identities, obtained during authentication, to a TicketDesk identity with all the claims TicketDesk needs. Since some claims cannot be obtained during authentication, TicketDesk will need to obtain any missing data from the user, or an administrator, then store and manage that data locally.

As you can tell, this is all in the early conceptual stages. I’m doing a lot of experiments and research, but I’m at the point where I have a good grasp of most of the pieces that I’ll need to smash together.

I have several other projects with similar security requirements, and I’ve grown tired of approaching each new project with the question, “how much hacking will ASP.NET membership need this time?” So, no matter what I come up with, I hope to roll it into a re-usage package.

ASP.NET MVC SimpleMembership – Initilization and New Project Template Issues

The ASP.NET MVC new project templates have been upgraded in Visual Studio 2012. Among the many improvements, was the change to using SimpleMembershipProvider. Simple they may be named, but I’ve found using it to be anything but simple. There are some deep design flaws in the provider, as well as how it is used in the stock MVC project templates.

The issues start with web.config. If you have worked with asp.net in the past, the most obvious difference in new web.config files will be the lack of configuration elements related to security providers.

Old templates had something like this:

<system.web>
       ... stuff ...
    <membership>
        <providers>
            <add name=".... />
        </providers>
    </membership>
    <profile>
        <providers>
            <add name=".... />
        </providers>
    </profile>
    <roleManager>
        <providers>
            <add name=".... />
        </providers>
    </roleManager>
    ... stuff ...
</system.web>

The new templates have none of these elements. There is a default connection in web.config, called DefaultConnection, and the templates use this connection for membership as well application data by default.

So far, so simple.

If you need to change the default behavior of SimpleMembership though, where do you make those changes? For example, I don’t like the default password length requirements, so I’ve always changed that in config. You can explicitly add the SimpleMembership provider to web.config, but doing so is useless. SimpleMembership just ignores settings in web.config. If you want to change the behavior, you have to track down the entity classes SimpleMembership is using (models/AccountModels.cs in the stock templates), and manually change the attributes in code.

While modifying settings in code isn’t hard, it is a truly horrible design. Your security configuration is scattered all over your model classes, and the runtime security policy is buried deep within compiled code, instead of a centralized configuration file where your server admin can get at it. This design also eliminates the option of using config transformations during build, publish, or deployment to customize the security policy for different target environments.

If you want to change the connection used by SimpleMembership, you no longer do this through web.config either. Instead, you have to track down the code that initializes the security system, which is inconveniently located in the /Filters/InitializeSimpleMembershipAttribute.cs file. Here, you need to update the call to WebSecurity.InitializeDatabaseConnection to pass in the correct name of your connection string. Not very intuitive, but that’s not the truly weird part either…

The weird part is that initialization for SimpleMembership, in the stock project templates, is done from an ActionFilterAttribute. This attribute decorates the AccountController class, and only that class. The result is that none of the SimpleMembership system will be initialized until a real user physically visits one of your account pages; login, register, or whatever.

If you write code in a page that allows anonymous access, or in a background or start-up routine, and it tries to do something related to membership, roles, or profiles you will get bizzare behavior. You would think that the code would just fail –the providers haven’t been initialized right?

The truth is much, much worse.

If code tries to do security stuff before SimpleMembership is explicitly initialized, asp.net will initialize the default security providers automatically. Instead of the SimpleMembership provider that you expected, it will initialize a SqlMembershipProvider using a connection named “LocalSqlServer”. This stuff is configured in the machine.config file, which includes the default configuration settings for SqlMembership, and points it at the local SQLExpress instance.

So, lets say you open up your new MVC project, which you created from the VS 2012 default template. You go to the HomeController, and edit the Index action to call Membership.GetAllUsers(). That’s all. You don’t need to do anything with the users, just ask for a list of them.

Then you run your application.

What do you think happens?

If your system is setup like most, you probably have SQL express installed. So, suddenly a mysterious aspnet.mdf file shows up in your app_data folder, and gets attached to your SQLExpress instance… and this all probably happens without you noticing it. If you don’t have a SQLExpress instance, your application will hang on startup for a long while, then timeout with “The system cannot find the file specified”.

I don’t know about anyone else, but I REALLY hate when an application starts making new databases on SQL servers that it shouldn’t even know exist.

Overall, I am not impressed with SimpleMembership, nor how the stock templates implement it. The providers are prone to unexpected and unintended behavior unless you have deep knowledge about their internal workings. Customizing SimpleMembership is the opposite of simple. And for configuration, it is confusing that it plugs into a standard asp.net feature, then ignores the traditional means of configuring that feature. And to top it off, requiring basic security configuration to be handled in code is a severely deranged design choice.

TicketDesk 3 Dev Diary – EF Migrations & TD2 Upgrades – Part 3

dead-birdsIn this last installment, I’ll describe TicketDesk’s custom database initializers, which allow TicketDesk 3 to automatically choose which migration should run on startup. I’ll also confess to a major development blunder!

In my last post, I talked about creating the initial migration for new TicketDesk databases. I used the MigrateDatabaseToLatestVersion<T> initializer in global.asax.cs. This initializer works well for normal EF migration usage patterns, but TicketDesk needs initializers with a bit more spunk.

EF also ships with a feature called “automatic migrations”. I will NOT be talking about this feature here. Automatic migrations are, in my opinion, an abortion-in-progress for anything but the most trivial of applications. I advise you to read up on this feature, and then do your best to avoid it. What I will be talking about here is automatically executing a “code-based migration”.

Great primer on Automatic migrations here, and for Code-base migrations here.

Initializers:

A common technique for customizing database initialization is to write a custom initializer that inherits one of the standard initializers. Sadly, the MigrateDatabaseToLatestVersion initializer is ill-suited for inheritance. There are no overridable methods or extension points to plug into. Instead, I’ll roll my own initializers, implementing IDatabaseInitializer<T>, and take direct control over the migrations process. Fortunately, the EF APIs makes this pretty simple.

Initially, I built one big do-it-all initializer, but later switched to separate initializers that operate independently — in keeping with the SRP.

On Application_Start, TicketDesk will run both initializers. Here is how the process should go:

1) Legacy Initializer Runs
    1a) Check if there is a database
    1b) If a database exists, check the version number (stored in the dbo.settings table)
    1c) If version 2.0.2, run the legacy initial migration (else, do nothing)
2) Regular Initializer Runs
    2a) Run all standard migrations

While the initializers don’t need a mediator, or other coordinating object, the order in which they are invoked is relevant to achieve the desired behavior.

The trick to step 2 is that, if the DB was upgraded in step 1c, then the regular migration will think its own initial migration has already run (remember, both use the same ID and hash value). The regular initializer will still run all additional migrations added later, even if the legacy initializer did the initial migration.

The initializers themselves are rather simple. They instantiate EF’s DbMigrator, which is the code equivalent to using the Package Manager Console’s Update-Database command.

The regular initializer:
public class TicketDeskDatabaseInitializer : IDatabaseInitializer<TicketDeskContext>
{
    private string ConnectionName { get; set; }
    public TicketDeskDatabaseInitializer(string connectionName)
    {
        ConnectionName = connectionName;
    }

    public void InitializeDatabase(TicketDeskContext context)
    {
        var config = new Configuration();
        config.TargetDatabase = new DbConnectionInfo(ConnectionName);
        var migrator = new DbMigrator(config);
        migrator.Update(); //run all migrations 
    }
}
The legacy initializer:
public class LegacyDatabaseInitializer : IDatabaseInitializer<TicketDeskLegacyContext>
{

    private string ConnectionName { get; set; }

    public LegacyDatabaseInitializer(string connectionName)
    {
        ConnectionName = connectionName;
    }

    public void InitializeDatabase(TicketDeskLegacyContext context)
    {
        //if existsing TD 2.x database, run upgrade; creates migration history table 
        if (context.Database.Exists() && IsLegacyDatabase(context))
        {
            var upgradeConfig = new Configuration();
            upgradeConfig.TargetDatabase = new DbConnectionInfo(ConnectionName);

            //this will do nothing but add the migration history table 
            //  with the same migration ID as the standard migrator.
            var migrator = new DbMigrator(upgradeConfig);
            migrator.Update("Initial"); //run just the initial migration
        }
    }

    public static bool IsLegacyDatabase(TicketDeskLegacyContext context)
    {
        // TicketDeskLegacyContext has no DbSets, directly execute select query
        var oldVersion = context.Database.SqlQuery<string>(
                  "select SettingValue from Settings where SettingName = 'Version'");
        return 
        (
            oldVersion != null && 
            oldVersion.Count() > 0 && 
            oldVersion.First().Equals("2.0.2")
        );
    }
}

The Accidental Integration Test:

You will note that the constructor for both initializers requires an explicit connection name, which is used to set the target database for the migrator. This is the result of another oversight in how EF migrations were implemented internally –I consider it an oversight anyway.

The initialize method takes an instantiated DbContext as a parameter, but there is no way to pass that context to a DbMigrator. Instead, DbMigrator always creates a new instance of the DbContext, and it always uses the DbContext’s default constructor. So, in cases where you want to use non-default connections, you must explicitly pass that information into the DbMigrator, otherwise it will use whatever connection your DbContext’s default constructor uses.

I discovered this issue when trying unit test the legacy initializer. My unit test used a custom connection string when it instantiated the DbContext (pointing to a localdb file database in the unit test project). I would run the test, but the DB in the test project refused to change.

Eventually, I discovered that the migrator was making a new DbContext from the default constructor, which in this case was hard-coded to a default connection named “TicketDesk”. This connection string was present in my app.config file, but I had, unwisely, left it pointed at my production database server…  the real TicketDesk 2 database that my company’s IT staff uses!

Yikes!

The test was, indeed, migrating the crap out of my production database!

As a testament to the migrator’s effectiveness, and the similarity between the TD3 and TD2 schemas, TicketDesk 2 didn’t skip a beat. The users never knew their DB had been upgraded, and everything kept working. Later, I manually un-migrated the production database using the PM console commands, which also worked without a hitch.

Lesson learned! Again…

Planting Seeds

Another interesting facet to the design of EF Migrations is how it handles seed data. For migrations, seeding is done through an override method in the migration configuration class. The odd thing about this seed method is that it runs every time migrations are run, even if there are no pending migrations to apply to the database. In TicketDesk’s case, both seed methods will run (from legacy and regular configuration classes).

In my opinion, there are two kinds of “seed” data –there is “default data” which is data that should be pre-set when the DB is created, then there is “test data”, which is used for testing or just pre-loading some example data. EF uses “seed data” in the sense of “test data”.

As a practical example of the difference, TicketDesk has a bunch of default settings that are stored in its settings table. These values are required, or the application fails, but admins can change the default settings later if they choose. This is not, in the sense EF uses the term, “seed data”. It is “default data”, for which EF has no explicit support.

For default data, the best solution seems to be inserting the data during the migration itself –by adding insert operations to the migration class’s up and down methods. Unfortunately, there aren’t any convenient helper methods for doing inserts, so you have to issue raw SQL insert commands as strings.

For “test data”, you can rely on the seed method in configuration, as long as you are comfortable with your data being reset every time the application starts. At least you can use Linq helper methods and your strong-typed entities there.

At this point, I have working database initialization and full migrations support in TicketDesk 3, at least for the core model entities. Security, and security database support, may prove more challenging.