it makes me want to gouge my eyes out with a cheese grater
magnify
formats

TicketDesk 3 Dev Diary – Thoughts on Security

hunts point, the bronx, new yo... by andre dos santosIn this post, I’m brainstorming new approaches to handling security in TicketDesk 3, and reviewing the limitations of TicketDesk 2′s old approach.

Security in  TicketDesk 2:

TicketDesk 2 shipped with support for either active directory, or local security. Local security via the built-in membership providers worked reasonably well for most deployments, but the AD option has always been problematic.

Windows authentication, in conjunction with the WindowsTokenRoleProvider, works well enough for the basic question, “who are you?”, but windows authentication severely limits in the amount of information you can obtain about the user. You get the windows account name and a list of the user’s AD groups –and that’s pretty much all you get.

Some features in TD required more than that. It needed display names and email addresses, and it needed to list all users in some AD groups; information the built-in providers are incapable of obtaining.

To support these requirements, TicketDesk 2 used a complicated background processes for AD concerns. It maintains a local cache of AD data to reduce the number of queries, and moved AD query processing off of the user servicing request threads.

Even with the abstractions in System.DirectoryServices, querying AD is painfully obtuse. In addition, response times for AD queries are often measured in the tens of seconds –even in single domain environments. You can’t reasonably query AD in real-time without bringing the entire application to a standstill. The custom module alleviates these issues, but was a  complete nightmare to code and maintain.

The other significant problem has been the difficulty of extending the old system to support more features, such as custom roles and permissions. This is why TicketDesk still uses hard-coded roles with pre-defined permissions –I simply never had the time or energy to re-factor the AD side of the system.

Security technologies of the present:

I’ve played a bit with the SimpleMembership and Universal Providers, which have replaced the original SqlMembership providers. ASP.NET’s security system is still based on the same provider framework that shipped with .Net 2.0 back in 2005. Sadly, the framework is a fundamentally flawed system, and in need of a complete redesign.

In recent years, there have been two attempts to modernize the ASP.NET providers, but neither really addresses the underlying problems of the core framework itself. Both are just extensions layered on top of the original infrastructure.

Universal Providers are an evolutionary upgrade of the original SQL providers from ASP.NET 2.0. They mainly just move data access into Entity Framework, which allows them to work with any database back-end supported by EF. Otherwise, there isn’t a much of a difference between them.

SimpleMembership was created for ASP.NET WebPages, but has become the officially recommended replacement in MVC projects as well. This is the default security provider in the newer MVC project templates that come with Visual Studio 2012. As I wrote in another post though, this system has some deep design flaws of its own. You almost have to learn how to hack up a customized version of SimpleMembership just to make it useful in any real-world scenario.

The good news is that both SimpleMembership and Universal Providers include oAuth support backed by the popular DotNetOpenAuth libraries. So, while flawed and build on a shaky foundation, at least they have been modernized.

Traditional windows authentication itself has not changed in any significant way, and there have been no official replacements for the windows security providers that I know of. ASP.NET still uses the same windows provider options that shipped in .Net 2.0, all of which are spectacularly terrible.

The major action for Active Directory authentication has been with Active Directory Federation Services. This is an STS token server that exposes AD to clients that support WS-federation. The Windows Identity Foundation (WIF), originally introduced in WCF, was pulled directly into the core of the .Net framework with the 4.5 release. This provides the infrastructure ASP.NET needs to leverage federated authentication and authorization. So, if you setup federation in ASP.NET you can easily connect it to ADFS to authenticate your AD users. Best of all, ADFS can share any information stored in AD.

In Windows Azure land, you have the Access Control Service (ACS), which is also a WS-Federation token service. From ASP.NET, connecting to ACS is similar to connecting to ADFS. The real advantage of ACS is that it can act as an aggregator and proxy for any number of other external authentication sources. For example, you can connect your ACS to your domain’s ADFS service and to oAuth providers.

The new identity features in .NET 4.5 also incorporate claims based identity as a pervasive and normative feature used throughout the entire framework. You don’t have to use WS-federation in order to use claims based identities, since all identities are now claims based.

Ideas for TicketDesk 3:

For TicketDesk 3, I hope to support the following authentication options:

  • local user accounts (forms with user registration)
  • oAuth (sign-in with google, facebook, twitter, etc.)
  • Windows (AD or local system accounts)
  • WS-Federation (ADFS, Azure ACS, etc.)

My challenge is to find a way to bring all these pieces together in TicketDesk 3 in a sane way.

I want TicketDesk’s business logic to make use of claims based identity for its authorization decisions. Since claims are baked in, this seems to be the most sensible way of doing things. The key will be to map all users, regardless of how they are authenticated, to a common set of claims that TicketDesk understands. The business logic can then rely on having those claims attached to every user’s identity.

For authentication, I have little choice but to rely on existing mechanisms. I am not a crypto expert, and have no intention of rolling my own password and user verification code. But, to support multiple authentication sources, I will likely need to roll my own module to intercept user requests, and dynamically route them to the appropriate pre-built authentication mechanism as needed at runtime.

Each of the possible authentication systems will produce a ClaimsIndentity, but the structure and content of those claims varies. Some authentication generates very few usable claims, while others may contain all the claims TicketDesk could ever need. To bridge the gap, TicketDesk will need an identity transformer. This component will map incoming identities, obtained during authentication, to a TicketDesk identity with all the claims TicketDesk needs. Since some claims cannot be obtained during authentication, TicketDesk will need to obtain any missing data from the user, or an administrator, then store and manage that data locally.

As you can tell, this is all in the early conceptual stages. I’m doing a lot of experiments and research, but I’m at the point where I have a good grasp of most of the pieces that I’ll need to smash together.

I have several other projects with similar security requirements, and I’ve grown tired of approaching each new project with the question, “how much hacking will ASP.NET membership need this time?” So, no matter what I come up with, I hope to roll it into a re-usage package.

 
 Share on Facebook Share on Twitter Share on Reddit Share on LinkedIn
1 Comment  comments 
formats

ASP.NET MVC SimpleMembership – Initilization and New Project Template Issues

The ASP.NET MVC new project templates have been upgraded in Visual Studio 2012. Among the many improvements, was the change to using SimpleMembershipProvider. Simple they may be named, but I’ve found using it to be anything but simple. There are some deep design flaws in the provider, as well as how it is used in the stock MVC project templates.

The issues start with web.config. If you have worked with asp.net in the past, the most obvious difference in new web.config files will be the lack of configuration elements related to security providers.

Old templates had something like this:

<system.web>
       ... stuff ...
    <membership>
        <providers>
            <add name=".... />
        </providers>
    </membership>
    <profile>
        <providers>
            <add name=".... />
        </providers>
    </profile>
    <roleManager>
        <providers>
            <add name=".... />
        </providers>
    </roleManager>
    ... stuff ...
</system.web>

The new templates have none of these elements. There is a default connection in web.config, called DefaultConnection, and the templates use this connection for membership as well application data by default.

So far, so simple.

If you need to change the default behavior of SimpleMembership though, where do you make those changes? For example, I don’t like the default password length requirements, so I’ve always changed that in config. You can explicitly add the SimpleMembership provider to web.config, but doing so is useless. SimpleMembership just ignores settings in web.config. If you want to change the behavior, you have to track down the entity classes SimpleMembership is using (models/AccountModels.cs in the stock templates), and manually change the attributes in code.

While modifying settings in code isn’t hard, it is a truly horrible design. Your security configuration is scattered all over your model classes, and the runtime security policy is buried deep within compiled code, instead of a centralized configuration file where your server admin can get at it. This design also eliminates the option of using config transformations during build, publish, or deployment to customize the security policy for different target environments.

If you want to change the connection used by SimpleMembership, you no longer do this through web.config either. Instead, you have to track down the code that initializes the security system, which is inconveniently located in the /Filters/InitializeSimpleMembershipAttribute.cs file. Here, you need to update the call to WebSecurity.InitializeDatabaseConnection to pass in the correct name of your connection string. Not very intuitive, but that’s not the truly weird part either…

The weird part is that initialization for SimpleMembership, in the stock project templates, is done from an ActionFilterAttribute. This attribute decorates the AccountController class, and only that class. The result is that none of the SimpleMembership system will be initialized until a real user physically visits one of your account pages; login, register, or whatever.

If you write code in a page that allows anonymous access, or in a background or start-up routine, and it tries to do something related to membership, roles, or profiles you will get bizzare behavior. You would think that the code would just fail –the providers haven’t been initialized right?

The truth is much, much worse.

If code tries to do security stuff before SimpleMembership is explicitly initialized, asp.net will initialize the default security providers automatically. Instead of the SimpleMembership provider that you expected, it will initialize a SqlMembershipProvider using a connection named “LocalSqlServer”. This stuff is configured in the machine.config file, which includes the default configuration settings for SqlMembership, and points it at the local SQLExpress instance.

So, lets say you open up your new MVC project, which you created from the VS 2012 default template. You go to the HomeController, and edit the Index action to call Membership.GetAllUsers(). That’s all. You don’t need to do anything with the users, just ask for a list of them.

Then you run your application.

What do you think happens?

If your system is setup like most, you probably have SQL express installed. So, suddenly a mysterious aspnet.mdf file shows up in your app_data folder, and gets attached to your SQLExpress instance… and this all probably happens without you noticing it. If you don’t have a SQLExpress instance, your application will hang on startup for a long while, then timeout with “The system cannot find the file specified”.

I don’t know about anyone else, but I REALLY hate when an application starts making new databases on SQL servers that it shouldn’t even know exist.

Overall, I am not impressed with SimpleMembership, nor how the stock templates implement it. The providers are prone to unexpected and unintended behavior unless you have deep knowledge about their internal workings. Customizing SimpleMembership is the opposite of simple. And for configuration, it is confusing that it plugs into a standard asp.net feature, then ignores the traditional means of configuring that feature. And to top it off, requiring basic security configuration to be handled in code is a severely deranged design choice.

 
Tags: ,
 Share on Facebook Share on Twitter Share on Reddit Share on LinkedIn
1 Comment  comments 
formats

TicketDesk 3 Dev Diary – EF Migrations & TD2 Upgrades – Part 3

dead-birdsIn this last installment, I’ll describe TicketDesk’s custom database initializers, which allow TicketDesk 3 to automatically choose which migration should run on startup. I’ll also confess to a major development blunder!

In my last post, I talked about creating the initial migration for new TicketDesk databases. I used the MigrateDatabaseToLatestVersion<T> initializer in global.asax.cs. This initializer works well for normal EF migration usage patterns, but TicketDesk needs initializers with a bit more spunk.

EF also ships with a feature called “automatic migrations”. I will NOT be talking about this feature here. Automatic migrations are, in my opinion, an abortion-in-progress for anything but the most trivial of applications. I advise you to read up on this feature, and then do your best to avoid it. What I will be talking about here is automatically executing a “code-based migration”.

Great primer on Automatic migrations here, and for Code-base migrations here.

Initializers:

A common technique for customizing database initialization is to write a custom initializer that inherits one of the standard initializers. Sadly, the MigrateDatabaseToLatestVersion initializer is ill-suited for inheritance. There are no overridable methods or extension points to plug into. Instead, I’ll roll my own initializers, implementing IDatabaseInitializer<T>, and take direct control over the migrations process. Fortunately, the EF APIs makes this pretty simple.

Initially, I built one big do-it-all initializer, but later switched to separate initializers that operate independently — in keeping with the SRP.

On Application_Start, TicketDesk will run both initializers. Here is how the process should go:

1) Legacy Initializer Runs
    1a) Check if there is a database
    1b) If a database exists, check the version number (stored in the dbo.settings table)
    1c) If version 2.0.2, run the legacy initial migration (else, do nothing)
2) Regular Initializer Runs
    2a) Run all standard migrations

While the initializers don’t need a mediator, or other coordinating object, the order in which they are invoked is relevant to achieve the desired behavior.

The trick to step 2 is that, if the DB was upgraded in step 1c, then the regular migration will think its own initial migration has already run (remember, both use the same ID and hash value). The regular initializer will still run all additional migrations added later, even if the legacy initializer did the initial migration.

The initializers themselves are rather simple. They instantiate EF’s DbMigrator, which is the code equivalent to using the Package Manager Console’s Update-Database command.

The regular initializer:
public class TicketDeskDatabaseInitializer : IDatabaseInitializer<TicketDeskContext>
{
    private string ConnectionName { get; set; }
    public TicketDeskDatabaseInitializer(string connectionName)
    {
        ConnectionName = connectionName;
    }

    public void InitializeDatabase(TicketDeskContext context)
    {
        var config = new Configuration();
        config.TargetDatabase = new DbConnectionInfo(ConnectionName);
        var migrator = new DbMigrator(config);
        migrator.Update(); //run all migrations 
    }
}
The legacy initializer:
public class LegacyDatabaseInitializer : IDatabaseInitializer<TicketDeskLegacyContext>
{

    private string ConnectionName { get; set; }

    public LegacyDatabaseInitializer(string connectionName)
    {
        ConnectionName = connectionName;
    }

    public void InitializeDatabase(TicketDeskLegacyContext context)
    {
        //if existsing TD 2.x database, run upgrade; creates migration history table 
        if (context.Database.Exists() && IsLegacyDatabase(context))
        {
            var upgradeConfig = new Configuration();
            upgradeConfig.TargetDatabase = new DbConnectionInfo(ConnectionName);

            //this will do nothing but add the migration history table 
            //  with the same migration ID as the standard migrator.
            var migrator = new DbMigrator(upgradeConfig);
            migrator.Update("Initial"); //run just the initial migration
        }
    }

    public static bool IsLegacyDatabase(TicketDeskLegacyContext context)
    {
        // TicketDeskLegacyContext has no DbSets, directly execute select query
        var oldVersion = context.Database.SqlQuery<string>(
                  "select SettingValue from Settings where SettingName = 'Version'");
        return 
        (
            oldVersion != null && 
            oldVersion.Count() > 0 && 
            oldVersion.First().Equals("2.0.2")
        );
    }
}

The Accidental Integration Test:

You will note that the constructor for both initializers requires an explicit connection name, which is used to set the target database for the migrator. This is the result of another oversight in how EF migrations were implemented internally –I consider it an oversight anyway.

The initialize method takes an instantiated DbContext as a parameter, but there is no way to pass that context to a DbMigrator. Instead, DbMigrator always creates a new instance of the DbContext, and it always uses the DbContext’s default constructor. So, in cases where you want to use non-default connections, you must explicitly pass that information into the DbMigrator, otherwise it will use whatever connection your DbContext’s default constructor uses.

I discovered this issue when trying unit test the legacy initializer. My unit test used a custom connection string when it instantiated the DbContext (pointing to a localdb file database in the unit test project). I would run the test, but the DB in the test project refused to change.

Eventually, I discovered that the migrator was making a new DbContext from the default constructor, which in this case was hard-coded to a default connection named “TicketDesk”. This connection string was present in my app.config file, but I had, unwisely, left it pointed at my production database server…  the real TicketDesk 2 database that my company’s IT staff uses!

Yikes!

The test was, indeed, migrating the crap out of my production database!

As a testament to the migrator’s effectiveness, and the similarity between the TD3 and TD2 schemas, TicketDesk 2 didn’t skip a beat. The users never knew their DB had been upgraded, and everything kept working. Later, I manually un-migrated the production database using the PM console commands, which also worked without a hitch.

Lesson learned! Again…

Planting Seeds

Another interesting facet to the design of EF Migrations is how it handles seed data. For migrations, seeding is done through an override method in the migration configuration class. The odd thing about this seed method is that it runs every time migrations are run, even if there are no pending migrations to apply to the database. In TicketDesk’s case, both seed methods will run (from legacy and regular configuration classes).

In my opinion, there are two kinds of “seed” data –there is “default data” which is data that should be pre-set when the DB is created, then there is “test data”, which is used for testing or just pre-loading some example data. EF uses “seed data” in the sense of “test data”.

As a practical example of the difference, TicketDesk has a bunch of default settings that are stored in its settings table. These values are required, or the application fails, but admins can change the default settings later if they choose. This is not, in the sense EF uses the term, “seed data”. It is “default data”, for which EF has no explicit support.

For default data, the best solution seems to be inserting the data during the migration itself –by adding insert operations to the migration class’s up and down methods. Unfortunately, there aren’t any convenient helper methods for doing inserts, so you have to issue raw SQL insert commands as strings.

For “test data”, you can rely on the seed method in configuration, as long as you are comfortable with your data being reset every time the application starts. At least you can use Linq helper methods and your strong-typed entities there.

At this point, I have working database initialization and full migrations support in TicketDesk 3, at least for the core model entities. Security, and security database support, may prove more challenging.

 
 Share on Facebook Share on Twitter Share on Reddit Share on LinkedIn
No Comments  comments 
formats

TicketDesk 2.1.1 – Official Release

TicketDesk 2.1 LogoTicketDesk 2.1.1 has been officially released at codeplex, and the source code has been merged and pushed to the public mercurial repository.

This is a platform refresh of the TicketDesk 2 project. The source code now supports development with Visual Studio 2012, and the application has been updated to target the .Net Framwork 4.5 and Asp.net MVC 4.

The databases have not been modified, but have been verified for compatibility with SQL 2012, including localdb.

There are no new user-facing features in this release.

 
Tags:
 Share on Facebook Share on Twitter Share on Reddit Share on LinkedIn
2 Comments  comments 
formats

TicketDesk 3 Dev Diary – EF Migrations & TD2 Upgrades – Part 2

Fixing MigrationsIn part 1, I outlined TicketDesk’s database management requirements, and how I hoped to use EF Migrations to meet them.

In this part, I will describe how I setup EF Migrations for new databases or TicketDesk 2 upgrades.

Migrations for a new TD3 database:

Ignoring TD2 upgrades, my first mission was to enable migrations for fresh TicketDesk 3 installations.

Bizarrely, Visual Studio 2012 did not include any tooling to support EF 5 migrations –odd since it has tooling for other EF 5 features. Developers interact with EF Migrations through the Package Manager Console; poor man’s tooling, but at least the commands are straight forward and well documented.

The Enable-Migrations command successfully created a migrations folder, generated the initial migration files, and added a migrations configuration class. As I mentioned before, there are database objects you can’t define with data annotations or the fluent API, so the next step was to edit the initial migration to include those pesky default constraints.

Once I had the initial migration finished, I was able to execute it from the PM console to produce the new database. The command line tools are fine for developers, but I also want the web application to initialize the database and run the migrations for me.

EF includes a migrations aware database initializer (MigrateDatabaseToLatestVersion), so I switched my app_start to use it. This also worked fine and the web application will make a fresh database on startup when it needs to.

Great! I’d be done if it weren’t for the need to support TicketDesk 2 databases upgrades.

Migration internals and limitations:

Before I get into how I Migrations work for TicketDesk 2 upgrades, I need to expalin a little about how migrations work internally, and point out some of its limitations.

From the developer’s viewpoint, EF Migrations are pretty simple. The initial migration is just a regular class that inherits from DbMigration. It has two method overrides, one for migrating up (containing create commands), and one for migrating down (containing drop commands). There are two other files associated with the initial migration. These track the parent/child relationships between migrations, but their main purpose is to manage the all-important model hash.

Initial-Migration

The hash is a compressed version of the entire code model at the time the migration was created. When the migration is run, the hash lands in the _MigrationHistory table in the database. This is how EF knows which migration was last used on the DB, and what the model looked like at that time.

The important thing to understand is that EF Migrations never look at your physical database schema. When EF generates a new migration, it does so by combobulating a model from the old hash value. Then it combobulates another model based on the hash of the current code. With before and after models in memory, it uses a complicated diff routine to figure out what needs to be changed to make the models match.

Since the real DB schema isn’t involved in migrations, you can modify the physical database manually, at your own peril, without migrations needing to know about it. In addition, you can modify the statements in the migration class in any way you want; EF will assume that whatever you changed will still result in a database that supports the storage needs of your EF model. Your physical database can also contain other tables and objects that are not related to the model being managed by EF.

The most glaring oversight in the design of EF 5 Migrations is that the _MigrationHistory table can only handle migrations for one model. If you have two different models and DbContexts, you cannot enable migrations on both unless they target different physical databases. If you try to share the same database, the migrations will overwrite each other’s records in the _MigrationsHistory table.

EF 6, currently an Alpha release, will fix this issue with a feature called “multi-tenant migrations”, but I’m not keen on using alpha bits just now.

TicketDesk 2 – Legacy Migration

While I’ve been careful to make sure the initial TD3 database is the same as a TD2 database, there are still minor differences. EF should run a different initial migration against an existing TD2 database. I will call this the “legacy migration” from here out. The legacy migration will just make a few alterations to eliminate the remaining schema differences.

I’ve already talked about how model hashes work, so here is why it matters… Each migration keeps track of its parent migration, which is how EF knows the correct order in which to run them. It is vital that both initial migrations use identical hashes, and have the same migration ID; otherwise, EF will get confused when I add new migrations later, and the linear migration path will be broken.

It took some experimentation to get two initial migrations to co-exist peacefully. Since both must share the same ID, class name, and hash value, it was simplest to put them in different assemblies –TicketDesk.Legacy for the legacy migrations, with regular migrations remaining in the TicketDesk.Domain project.

The legacy migration is a complete forgery, so I didn’t generate it with the PM console commands. Instead, I copied the initial migration files from TicketDesk.Domain into the legacy project. I then changed their namespaces, and gutted the up/down methods. This results in a legacy migration that does nothing, but when executed will add a _MigrationHistory table, and set the migration ID and hash values.

In order to run the legacy migration, EF will need a DbContext and migration configuration class. So, I made a LegacyDbContext. There is no actual model for this project, and we will never generate additional migrations for it, so the LegacyDbContext is empty, except for the constructors. It exists only so EF can open a connection to the database and run this one migration. I made a copy of the standard configuration file too, modifying its namespace and pointing it at the LegacyDbContext.

Legacy-Migration

The last step was writing custom commands into the legacy migration class’s “up” method to take care of those last few schema differences. For completeness, I also added appropriate commands to the “down” method that restores the database to the stock TD2 schema.

At this point, I was able to use the PM console commands to run either the legacy migration, or the regular initial migration, and both work as expected. Looking at the databases with the schema compare tool, there were only cosmetic differences between a fresh DB and an upgraded one. Perfect!

In the next part of this series, I will setup an initializer to selectively run the appropriate initial migration at runtime, and explore a few more gotchas with the EF 5 migrations framework.

 
 Share on Facebook Share on Twitter Share on Reddit Share on LinkedIn
1 Comment  comments 
formats

TicketDesk 3 Dev Diary – EF Migrations & TD2 Upgrades – Part 1

Sandhill Crane Migration by Serge MelkiTicketDesk 3 may be starting from a fresh code-base, but it is not a new application. There are organizations using TicketDesk in the real world. New releases of TicketDesk have always supported in-place database upgrades, and later versions automated them. I plan to continue this tradition in TicketDesk 3, but would rather not need to roll my own upgrade code, or rely on hacked up T-SQL scripts.

How Migrations Help:

EF code-first is highly capable of generating a well structured SQL database from a pure-code driven model –most of the time. But, EF’s standard database initializers only handle two operations; drop a database, and generate a new database. When you make a model change, EF can automatically drop the DB, and then create a new one that reflects the current model.

This is all fine when developing locally, and is acceptable when deploying a new installation to production. But once you have a production database that real people are using, it is a bad idea to drop and re-create it to deploy a new version of the code.

Customers often dislike having their production databases dropped.

This is where EF Migrations come in. The migrations framework allows EF to modify an existing database in-place, keeping existing data intact. Customers often prefer this approach.

EF make some assumptions about how you will use migrations. It expects a linear, ordered evolution of your database, starting from some “initial state” and ending with the “current state” of your model. The first migration creates a base-line database, then other migrations modify that base-line to reflect any model changes that come along later.

What TicketDesk needs from EF Migrations:

New TicketDesk 3 installations work fine with the normal linear pattern expected by EF migrations. The initial migration will create the database, and fill it with tables. Since I was careful to start with a model that closely resembles TicketDesk 2, the initial migration will create a complete, and reasonably mature, database schema. The TD3 model shouldn’t experience the kind of rapid churn you’d see when building a brand-new product. Still, some model changes will be necessary as work on TD3 progresses, and additional migrations will be needed. The standard design of EF Migrations fully supports this pattern of model evolution.

In my earlier post, I mentioned that there are some database objects that EF can’t automatically generate from data annotations, or the fluent API –default constraints being the main culprit. TicketDesk 3 could get by without default constraints, and the POCO classes could just handle setting defaults in code. But I want my DB to be as similar to the stock TD2 DB as possible. Fortunately EF Migrations does allow you to add real default constraints as part of the migration routine, though you have to manually edit the code to add them.

When upgrading from a TD2 database things gets more complicated. The linear pattern EF migrations expect is not sufficient for upgrades. EF should not attempt to run the regular kind of initial migration against an existing TD2 database. Such a migration would try to create DB objects that already exist, and would fail.

When upgrading, EF will need to run a different initial migration routine –one designed only to modify the DB where necessary to match the normal TD3 schema. Since I was careful to ensure that the two schemas are nearly identical, all the special migration has to do is clean up a few minor variations (e.g. mark an FK as “cascade on delete”, create a non-clustered index, etc).

In theory, once the special migration runs, the old database should be functionally identical to a regular TD3 database. No matter which initial migration routine runs, the same set of subsequent migrations will be able to upgrade the initial DB to reflect model changes that occur later in development.

So that’s what I want from EF Migrations. In the next post, I’ll try to make it happen.

 
 Share on Facebook Share on Twitter Share on Reddit Share on LinkedIn
1 Comment  comments 
formats

TicketDesk 3 Dev Diary – EF Code-First Model

Nowhere to stand, nowhere to hide by Pete Wright Photography, on Flickr

It is often difficult to decide where to start when building a new application, but for TicketDesk 3 the obvious first step is to create the database and entity model.

Until now, the TicketDesk database has followed an unbroken evolution from the original TD1 database to TicketDesk 2.0.3. I built the original database by hand, then used it to generate a Linq to SQL model diagram. When I put TicketDesk 2 together, I just switched to EF 4 entity model diagrams; again, building the model from the existing database.

For TicketDesk 3, I’ll once again switch data access technologies, this time using EF 5′s DbContext with a code-first model (POCO). I could continue with *.edmx models, and generate the DbContext and POCO classes from the diagram, but Microsoft seems to be moving away from *.edmx, and I strongly prefer the code-first approach anyway.

Initially, I want to start TicketDesk 3 with a model as similar to the TicketDesk 2 database schema as possible. Due to the way EF generates T-SQL, there will be cosmetic differences, but I don’t want any significance structural variations –not yet.

To create the initial POCO classes, I was able to leverage the old entity model diagrams. The tooling in VS 2012 allowed me to open the old *.edmx files and add the “EF 5.x DbContext Generator”. This creates T4 templates, which in turn automatically produce generated DbContext and POCO classes based on the diagram. All I had to do was pluck out the generated class files and drop them into the TicketDesk 3 project.

Interestingly, the POCOs created by the generator include the model properties, but not the annotation attributes that identify key fields, required fields, and so on. Normally I’d have to annotate them all by hand, but I found a shortcut for this too. TicketDesk 2 originally used another T4 template, called AutoMetaData, which produces meta classes for all its entities; and those meta classes contain the data annotation attributes. The old EntityContext stuff doesn’t need those annotations, but generating them was useful to enable UI validations in the MVC front-end. So, all I had to do was copy the attributes from the TD2 meta classes, and drop them on the corresponding properties in the new TD3 model classes. Tedious, but easy.

TD 2 also included custom extensions (via partial classes) for a few entities. Mostly these just had a few non-persistent helper properties, and a custom method to two. I simply copied these into new POCOs too, and then made sure they were annotated correctly (adding the NotMapped attribute to most).

By leveraging the work I’d already done in TD 2, I was able to create a nearly complete EF code-first model for TD 3 without writing any code by hand. Sweet!

The last step was to test DB generation from the new model and DbContext.

My web project was created from the stock MVC SPA template that shipped with the “ASP.NET and Web Tools 2012.2 update“. I setup a new localdb connection string in web.config, configured EF to use the DropCreateDatabaseAlways initializer, and then called the Database.Initialize() method on my new DbContext during the application_startup event. Running the web application triggered the database creation without any problems.

I then used the schema compare tool in VS 2012 to compare the new DB against an unmodified copy of the TD2 database. There were a few differences, which I had expected, but structurally it was the same DB that TicketDesk 2 used. The only significant variation was the lack of default constraints in the new database.  EF 5 doesn’t support default constraint definitions via attributes, nor the fluent model builder APIs. Fortunately, EF migrations can handle defaults, so I will solve that problem later on.

The stock TD2 database also included SQL membership, role, and profile tables, which are not a part of  the new entity model. I’ll be addressing the security system pretty soon,  and then I can decide how best to handle the security DB objects.

And that is pretty much it for the initial TD 3 code-first entity model. As I develop TD 3, I will likely make changes, but I can rely on EF to manage the database schema going forward. I’ll need to setup EF migrations, deal with seed data, and add upgrade support for legacy TD2 databases… but those are topics for future posts.

 
 Share on Facebook Share on Twitter Share on Reddit Share on LinkedIn
1 Comment  comments 
formats

TicketDesk 3 Dev Diary – Source Control and Source Hosting

source code hostingBefore any good project gets further than a mission statement, it needs source control. Typically, I don’t even create a project file without first creating a source repository in which to put it. But I’ve not been that happy with CodePlex, so I’ve decided to move the project to GitHub and switch to the Git source control system.

Source Control Systems

TicketDesk 1 and 2 were hosted at CodePlex using Microsoft’s Team Foundation source control system. After 5 years of trying, I eventually developed a deep and lasting hatred for all things TFS. When CodePlex added support for Mercurial, I had TicketDesk migrated as soon as I could.

I use both Mercurial and Git, but I tend to prefer Mercurial. Both are highly capable systems, but for windows development, the experience around Mercurial is much better. Mercurial installs easily, runs smoothly, is easy to learn, easy to use, and comes with a great GUI out of the box.

In contrast, running Git on windows requires installing a whole mess of junk to bridge the gap between Linux and Windows… and that just gets you to the command-line tools. There are several third party Git GUIs, but most of them are terrible. GitExtensions is the best, though still ugly and counter-intuitive compared to TortoiseHG.

Fortunately, there are great extensions available for Visual Studio no matter which system you go with.

About command lines… I love having the option of a command line, but I hate using a command line as my primary user interface. Command lines are fine for advanced scenarios, automation, or propping up a nice GUI… but for something I interact with as often as my source control system, I want a good GUI!

Repository Hosting

I like CodePlex, but it lags behind BitBucket and GitHub in both features and popularity. CodePlex’s release management features are way ahead of the competition, and the site makes a better end-to-end product hosting service than most of its competitors.

CodePlex elegantly handles product information, downloadable releases, source code, support forums, bug tracking, and documentation. I am completely comfortable sending an end-user to TicketDesk’s CodePlex site to submit bugs or ask questions. In contrast, I would never send an end-user to GitHub or BitBucket to get support for one of my products.

As much as I like CodePlex though, I’ve also had many bad experiences with them, particularly around support and reliability. As an example, my routine request to have TicketDesk converted to mercurial went unanswered for more than 3 months, and it wasn’t handled very well when they did get to it. CodePlex often doesn’t perform very well, though this seems to have improved some recently. CodePlex also isn’t very popular, which makes projects there less discoverable, and people less likely to fork or contribute.

I use Bitbucket extensively for my professional code. Bitbucket lagged behind GitHub for a long time, but in the last year it has improved rapidly. Now, it rivals or exceeds GitHub in most of the areas I care about most. They’ve added support for Git repositories too, though I’ve only used it with Mercurial. I like BitBucket’s pricing model, and it supports organizational team accounts, as well as individual users  –perfect for corporate projects where an individual doesn’t directly own the repositories.

GitHub is crazy popular, and for “social coding” popularity is everything. If you are using Git, and your project is public, then GitHub is the obvious choice. It has amazing features, and sets the bar by which everyone else is measured. The only major complaint I have is the lack of a release management system. GitHub doesn’t make the best full-service project hosting site, but it does a bang-up job with the core mission of source code collaboration. GitHub’s pricing is perfect for open source projects and private repositories, but awkward for some corporate scenarios.

Why Git and GitHub for TicketDesk 3?

    • Git
      • I use Git a lot when working with other peoples’ projects, but I have not used it extensively for my own. I could use more end-to-end experience with it.
      • The typical workflow with Git is perfect for public projects like TicketDesk. Private branches, history culling, and similar features all help keep the public repository clean, and focused only on commits that moved the project forward.
      • Even among the .Net developer community, Git is the more popular option. Using Git will hopefully encourage more people to contribute.
      • Git has better integration with Windows Azure, and I do hope to make TicketDesk 3 an Azure deployable application. This is a minor plus, but worth a mention.
    • GitHub
      • Since I’ve chosen Git, GitHub is the inevitable choice for hosting.
      • Hosting at GitHub should result in higher visibility for the project, which I hope translates into more people discovering TicketDesk.
      • GitHub’s best features are those designed to enable independent programmers to contribute and collaborate on public projects; something I’d like to see happen more frequently with TicketDesk.

What about the CodePlex project?

I do plan to keep the CodePlex project around. I will likely continue to push packaged releases there, especially since GitHub doesn’t do much for release management. I may push occasional source updates to CodePlex when the code reaches major milestones.

The TicketDesk 3 repository is already on GitHub, but is private for now. Once I have a good start, I’ll open it up to the public.

 
 Share on Facebook Share on Twitter Share on Reddit Share on LinkedIn
No Comments  comments 
formats

TicketDesk 3 Dev Diary – The development plan

TicketDesk3LogoTicketDesk 3 is mostly about re-architecting the core system for current asp.net webstack technologies. It will not be a feature heavy upgrade, though it will come with a completely new user interface on top of the existing feature set.

The following is an incomplete high-level feature list. Several aspects will change once I’ve done more research in certain areas.

New Features:

  • Ticket Subscribers: anyone can subscribe to a ticket’s notifications
  • Editable lists in TicketCenter: a TD1 feature that never got back into TD2
  • New Security System
    • Local users (forms)
    • oAuth support
    • Active Directory via ADFS
    • Legacy Account Migrations (TD2 upgrades)
  • New User Interface
  • Localization

Platform:

  • Entity Framework 5
    • Code-First Model
    • EF Migrations
      • Legacy Migrations (TD2 upgrades)
  • SQL Server 2012
    • localdb, Express, and full editions
    • SQL Azure
    • SQL CE 4 (tentative)
  • MVC 4
  • SPA Based UI
    • HTML 5 & JavaScript
    • Responsive Design
    • Knockout.js & related
  • WebAPI
    • oData (tentative)
  • Windows Azure deployability
  • IoC
    • MEF 2 or Ninject (tentative)
  • Security
    • SimpleMemebership or Universal Providers (tentative)
    • oAuth
    • AD integration via ADFS
      • WIF (tentative)

Other Considerations:

Mobile is of course the hot-topic these days, and to a lesser extent so are native Windows 8 applications. I intend to design TicketDesk 3 around WebApi, which should go a long way towards enabling support for many different front-end clients. I’m looking at responsive design to enable the initial web application to scale around different screen sizes, but I’m not sure responsive design alone can bridge the gap all the way down to the small-screen devices (phones and mini-tablets).

After the initial TD3 release, I hope to build mobile specific HTML 5 views. These should enable a good experience on any modern smartphone, but I don’t want to get into building a native app for each phone platform.

I do hope to build a native Windows 8 metro (or whatever they call it this week) client app.

I plan to support upgrades for existing TicketDesk 2 installations, and the databases will continue to be self-initializing and self-upgrading.

Localization has also been a highly requested feature, much to my surprise. I’m monolingual, and have very little experience with localization, so don’t expect too much. I do plan to use localizable resource files for all the application’s various text, and will support localized date, time, and number formats.

 
 Share on Facebook Share on Twitter Share on Reddit Share on LinkedIn
1 Comment  comments 
formats

TicketDesk 3 Dev Diary – Getting TicketDesk 2 back in shape

TicketDesk 2 LogoWhen developing a new version of an existing product, it really helps if you can run the old version and use it as a reference.

TicketDesk 2 works well, but it was designed for MVC 2 running on .Net 4, which would be fine if I still maintained a local development environment compatible with the old code. But I’ve ditched Visual Studio 2010 and 2008 entirely.

So, the first order of business is migrating TicketDesk 2 to the newer asp.net webstack.  I’d also like to update the source on codeplex so others can use it with current dev tools too.

I will not be upgrading the application to use MVC 4 or .Net 4.5 specific technologies or techniques; I’m not switching it to the razor view engine or upgrading to EF 5. The goal is to just get it running on current platforms with as few changes as possible.

The first step was finding the asp.net MVC 2 to MVC 3 project converter. Once I tracked that down, and did a bit of tweaking to get it to work, the conversion ran without a problem. This allowed the project to open  in VS 2012 at least, though it wouldn’t build or run.

Next I moved the web project up to MVC 4, and re-targeted both projects to compile against .Net 4.5. Re-targeting was just a matter of switching the setting in the project properties dialog. Then I just pulled down the asp.net MVC 4 NuGet packages, along with the three dependent packages. TicketDesk uses the old ASPX view engine, and doesn’t use WebApi or anything fancy, so I didn’t need most of the other NuGet packages that you’d see in most MVC 4 applications.

TicketDesk 2 used MEF 1. This was probably the riskiest decision I made when building it originally. MEF 1 was intended for Silverlight development, and I had a LOT of trouble getting it to work correctly for asp.Net MVC (I wrote about those issues at length here and here). Had my only need been dependency injection, I would have switched to Ninject. In the end though, I did get MEF to work for TicketDesk 2. But to be honest, the experience was not very inspiring. Since then, I’ve stuck with Ninject, and been the happier for it.

Now we have MEF 2, but it is nearly impossible to find coherent guidance on how to use it with web apps. There are tons of articles, blog posts, and discussions all over the net, but they all use different techniques, none of the examples actually work beyond demo land (most don’t work there either), and most of the info is based on obsolete pre-release versions. The MSDN docs are of particularly low quality, and frustratingly out of date too.

I’m sure MEF 2 is amazing, but if MS doesn’t do something serious about cleaning up the documentation and providing some coherent guidance appropriate for real world applications, then I doubt it’ll get much use among the asp.net developer community.

I spent several hours working with the best example of MEF 2 with MVC that I could find. In the end though, I still couldn’t get it to play nice with TicketDesk 2. But I got lucky and was able to get what I had built originally to work with very few changes. I’m sure the old pattern misses out of some nice MEF 2 specific improvements, but the goal here is to get TicketDesk 2 working again not optimize it for the new platforms.

I’ll probably take a deeper look at MEF 2 for TicketDesk 3 before I decide if I’ll switch to Ninject, or give the new MEF another shot.

I no longer have SQL Express installed locally. I am using SQL 2012 localdb, and have a full instance of SQL 2012 developer edition. So, I switched the TD 2 connection strings to localdb, and have had no problems with it. I’ll put them back to SQL express before I merge the code back into the default branch, and there are no databases changes for the new build.

This got me to up a working version of TicketDesk 2 that can run on MVC 4 using the .Net 4.5 framework. Mission accomplished!

Once I had it working though, I went ahead and upgraded the jQuery stuff. I just couldn’t resist. This required a tiny tweak to the corners plug-in to fix a compatibility problem with the newer jQuery releases; but overall it seems to work great.

At this point, I have what appears to be a fully functional update of TicketDesk 2. All that remains it to put it through some live testing, and update the documentation. Then I will merge the new code in source control, and push it up to codeplex along with a new downloadable release.

Best of all, I now have a working local copy I can use as a reference while I build TicketDesk 3.

 
 Share on Facebook Share on Twitter Share on Reddit Share on LinkedIn
No Comments  comments 
formats

TicketDesk 3 Dev Diary – Getting started

TicketDesk 3 LogoI’ve been putting off working on TicketDesk for far too long.

There were many reasons for the delay. TicketDesk 2 was originally written for MVC2, but changes in the newer versions of asp.net, MVC, EF, and MEF have made the task of upgrading TicketDesk to the modern asp.net stack a challenge. Additionally, there are new technologies that I’d rather be using that require re-architecting most of the existing code-base.

Until recently though, the asp.net stack was moving too fast, and wasn’t stable enough, for everything I wanted to do with TicketDesk. Every time I considered diving back into development, there was a new-hotness in the works worth waiting on. First it was EF code-first, then EF migrations, then WebApi, then oAuth, then the SPA framwork, and so on. Looking at the current asp.net landscape, it seems that most of the technologies that I want to use are now done and stable –or stable enough at least.

Rather than develop against the TicketDesk 2 code-base though, I’ve decided it will be easier to just start over with fresh projects. A lot of the old code can be re-used of course, but there are sweeping architectural changes that are simpler to implement from scratch.

I do plan to hack-together a working VS 2012 and .Net 4.5 compatible build of TicketDesk 2. This isn’t as easy as it sounds. MEF in particular has undergone numerous changes that are incompatible with the version TicketDesk 2 originally targeted. Additionally, the MVC 2 project itself can’t even be opened in VS 2012. I will need to upgrade it to at least MVC 3, then figure out how to get it to play with the current version of MEF. I don’t plan to add any new functionality to TicketDesk 2, but I do hope to have a working code-base that I can maintain with VS 2012.

 

 
 Share on Facebook Share on Twitter Share on Reddit Share on LinkedIn
No Comments  comments 
formats

Java, ten years later – a C# developer’s perspective.

javaTen years ago, I was about a year into my first full-time programming gig. Prior to that, I’d just been a hobbyist developer. I had a reasonable understanding of C#, VB 6, VBScript, JScript.NET, ASP 3.0, and even dabbled a little in PHP. I was also pretty wiz-bang with Perl 5 regular expressions, though not so much Perl itself. I spent most of my spare time on the new .Net framework, but my skills were strongest with JavaScript.

My JavaScript background landed me my first full-time programming job, where I wrote complex drag-n-drop form designers, fancy-pants grid controls, and lots of out-of-band XML stuff (later known as AJAX). Basically, I wrote the kinds of stuff you’d just buy from a 3rd party component vendor now days. The application itself was a colossal enterprise EAM product, and the server side was all Java… lots and lots of all Java.

Given that my paycheck came from a Java shop, learning Java would have probably been the smart move. But the more I looked at it, the less enthusiasm I could muster. I had no problem with the language itself. Java and C# were comparable at the time, and I was comfortable with static typing, case sensitivity, and the C-style syntax in general. But, a variety of issues kept putting me off Java.

  • My interest was web development, but JSP was slow, clunky, and annoying.
  • The run-anywhere promise had a certain appeal, but the price of portability was poor performance and lowest common denominator functionality  With web apps, you wound up coupled a particular application server anyway, so portability was a low value proposition.
  • I disliked how SUN conducted business. They’d refused to turn the language over to an independent standards body, and were quick to file lawsuits. Their licensing and prices had grown increasingly draconian as well.
  • I wasn’t impressed with the direction many Java libraries were taking, and enterprise JavaBeans was a total mess.
  • The tooling was terrible too. I prefer IDEs to command line compilers and plain text editors. Eclipse was the best Java IDE around, but it was not fun. On my high-end workstation, it often took five minutes to launch; and once open it was still slow, buggy, and feature poor.

In the end, I decided to skip making a deeper dive into Java. I rode out my JavaScript job for another year, and then jumped into a full-time ASP.NET position as soon I could. I have never regretted that decision.

Ten years later, I’m going back to college to grab that long overdue CS degree. Naturally, my curriculum centers around Java. I still have no love for the language, but I am looking forward to learning my way around it more… it never hurts to branch out.

The last ten years have radically changed C#. A C# developer from 2003 would probably not recognize most code written today. So, when I took another look at Java, I’d expected to see a similar progression. I was wrong.

Using Java today feels eerily like being back in 2003  – cranking out blocky-ass code, casting everything six ways from Sunday, and with precious little of the expressiveness and elegant fluidity I’ve grown accustomed to. The only surprising thing about Java is how stubbornly it has resisted change.

Java’s  most significant advance was the addition of generics. After digging deeper though, I find Java’s generics a bit half-baked. They don’t support primitive types, which is super annoying. You can use the reference wrappers, and rely on auto-boxing/unboxing, but it feels clumsy. The syntax for generics is also a bit odd in places. The wildcards are especially interesting, but I’m reserving judgment for now. While confusing, wildcards seem like a good way to handle covariance and contravariance. C# also struggled with those, and its solution wasn’t very intuitive either.

When generics were first introduced in .Net, it gained a significant performance boost across the entire framework. However, Java implemented generics purely as a compiler trick using type erasure, which is an… er… interesting choice. This means Java can’t really optimize around generics like .Net can.

Additionally, Java’s implementation prevents runtime reflectivity over generic types. In C#, I have used generic reflection heavily in a few cases, and was able to produce elegant solutions to some really thorny problems. I’m not even sure how to approach those kinds of problems in Java without generic reflectivity.

Java did pick up some smaller improvements over the years, but they don’t seem to add up to very much:

  • Foreach loop. A simple feature, and a solid win.
  • Annotations. Weak-sauce compared to C# attributes, but just as essential for tooling, code generation, and reflection.
  • Automatic boxing/unboxing. Especially useful given that generics don’t support primitives, and Java has no type lifting (nullable value types to us C# guys). At least auto-boxing/unboxing cuts down the casting cruft. The implementation on the unboxing side gives me the heebeegeebees, but I’ve had no trouble from it in actual practice.
  • Enumerations. One of the few things Java does, arguably, better than C#. Java enums are full reference types, and can implement methods and such. Nice!
  • Varargs (params to us C# folks). A minor improvement, but a handy simplification to a common code pattern.
  • Static imports. Yeah, sure… ok. I don’t often use aliases in C# either.
  • Strings in switch statement. How was that not there all along? More to the point, why does Java’s switch statement still allow fall-through? Why doesn’t it work with all types? Overall, the switch statement in Java still seems horribly broken, but at least it works with strings now… so that’s progress I guess.
  • Automatic resource management via the Try statement. The same as a “using” statement in C#. A useful little feature to have around.
  • Binary integer literals. Anyone who uses binary constants probably loves this, so I imagine there are at least two happy developers out there somewhere.
  • Underscores in numeric literals. Sure, why not!
  • Type inference. Limited to generic types. A more comprehensive type inference mechanism would have been nice, but some is better than none. Complex usages of generics can produce some gnarly expressions, so if you were going to half-ass tackle type inference, generics would be the place to start.

Java is seriously conservative in adopting changes. There are good things to say about stability and predictability, but there are a few other things from C# that I REALLY miss when working in Java.

First-class Properties:

This drives me insane, and there is no excuse for Java not having added native property syntax by now. Properties are easier on the developer, and they make the intent behind your code much more apparent. Java’s convention driven naming pattern of manually written getter/setter methods is a piss-poor substitute.

Extension Methods:

I didn’t really appreciate these when they first appeared in C#, but after getting used to them I find them indispensable for framework development. Extension methods let me quickly add functionality to existing types, even 3rd party components, without any need to modify the original source, write wrapper classes, or create some intermediate abstraction layer.

Lambdas/Closures:

I completely get why these aren’t in Java yet, but at the same time their usefulness is also unavoidably obvious. The sheer number of amazing capabilities C# picked up after adding lambdas is staggering. This feature is slated for the next release of Java, and I sincerely hope they don’t kick the can down the road any further.

Most of the arguments within the java community over this feature are plain silly, and those arguing against lambdas are just plain idiots. I respect Java’s inner and anonymous class implementations. They are slick, and come close to providing full featured closures already. Enhance what’s already there, and full closure support might be a done deal… but if that’s all that’s done, it would fall far short of the full potential. Lambdas are not just about the functionality; the real benefit is providing an elegant and expressive syntax for using the functionality.

I dearly miss lambdas when working in Java, and I’m 100% sure that Java developers will love this feature as much as C# developers do. I just hope the final implementation isn’t as half-baked as generics were.

Ten years ago C# was just a shiny new clone of Java. Since then though, C# has grown and changed rapidly. It has picked up many of the most useful features of functional and dynamic programming, and balanced them successfully against its own static-typed roots.

Java’s lack of progress should  make learning my way around it a lot easier. I’ll have to acclimate to the “Java way” of course. Tackling the design patterns and core libraries of any language takes time and patience, but at least the language itself is simple, predictable, and familiar. I just wish java felt more… alive.

After having used C#, Ruby, and modern JavaScript for so many years, Java just seems old and stagnant. Hopefully, a bit more time working with it will improve my opinion.

 

 
 Share on Facebook Share on Twitter Share on Reddit Share on LinkedIn
No Comments  comments 
formats

College at 40

I’ve just wrapped up my first semester at college, so I thought I’d write a bit about the experience so far.

I decided to go for that Computer Science degree I should have gotten in my 20s. I’m 39 now, and have spent 18 years in the IT field — 11 as a professional software developer. Strictly speaking, I don’t need a degree. I’m comfortably employed, well paid, and still enjoy slinging code for a living. But, I’m getting old. Should I need to seek new employment, I’d be going up against people half my age with twice my credentials. Experience matters, but programming is still seen as a young person’s field. Most companies look for people with a “BS in Computer Science, or equivalent degree”, but HR usually doesn’t know what “equivalent” looks like. They have dozens of other applicants who do have the expected degree, so my resume goes to the bottom of the pile. Also, I will not be able to sling code at this pace forever. I’ve often considered teaching as a way to wrap up my career, but for that I’ll need degrees.

So, now I’m a freshman at University of South Carolina’s Upstate campus. I could have gone with an online school, but after years of solo career building and self-study, I really wanted to experience real teachers and living, breathing classmates for a change. USC Upstate is a nice hybrid between a tech/community college and a traditional university. It is a small school, but focuses on real 4 year degrees. There is a sizable population of students living on-campus, but also plenty of commuter and non-traditional students too — though not as many my own age as I’d expected.

The CS program at USC Upstate is very small, so most of the core classes are only offered during the day. I’m very lucky that my job affords a flexible schedule, and that my employer encourages continuing education. I couldn’t attend daytime classes otherwise.

As a late applicant, I ended up with a horrible schedule — five classes, five days a week. The first challenge was finding time to fit in 40+ hours at work on top of a full-time course load. I’m used to 75+ hour work weeks, but still, the combined workload turned out to be too much. I ended up switching my hardest class to an audit, but even at 12 credit hours, I was worn to the bone by the end of the semester. Going forward I’ve had to reduce my course load to part-time (3 classes), and restricted attendance to just three days a week. At his slower pace, I’ll need more than five years to complete the degree, but I’m not in a rush.

Many people going back to school at my age complain that it’s hard for them to keep up with the pace of learning. Learning does become a bit harder as people age, but I have not had as much trouble with this as I thought I would. The nature of my job, and years of self-education, have kept my mind pretty sharp.

The main area where I feel my age is with mathematics. As a programmer, I work “with” math a lot, but it has been a very long time since I’ve worked problems by hand. I usually get the right answers, but I’m really slow at it. I would not have passed my trig class, so I switched it to an audit when it became necessary to free up more time for work. Computer Science is a math heavy curriculum though, so I’ll need a refresher in basic algebra before I can tackle the advanced math classes for real.

In other classes, my age helps some. In particular, my English skills have improved over the years. I’ve written quite a lot professionally, and I type like a fiend. In contrast, the majority of my peers are fresh out of high school, which has apparently fallen way behind the writing standards of my day. My younger peers can do amazing things with math, and their reading comprehension is good, but they turn out papers that would have caused my 9th grade English teacher to commit suicide.

The bar for English 101 was so low that the class was outright painful. As it turns out, English 101 and 102 are just glorified writing workshops. The professor spends a lot of one-on-one time with each student, tutoring and advising them as needed. I respect what the school is doing, but it’s sad that high schools are producing such poorly trained writers that two entire courses have to be wasted on writing workshops. This was hugely disappointing to me. I’d once considered an English major. Even though I chose a different path, I was still looking forward to a refresher on contemporary grammar, and maybe picking up some new writing styles and techniques. I’m not the best writer in the world, but my current skills are so far ahead of the expectations that the classes offer nothing of value to me. I’ve attempting to test out of English 102 just to avoid another class of empty paper writing.

As for my core comp-sci classes, I don’t expect to get that much out of the undergraduate curriculum. I’ll learn a bit of Java, which I managed to avoid professionally, but Java isn’t fundamentally different from C# where I already have deep expertise. I plan to take the C++ course, just to have an excuse to actually do something with the language. Overall, I’ll have to wait until grad school before I really get deep into areas of comp-sci where my self-education was lacking.

This semester, I took Introduction to Computer Science. I don’t perform well when under-challenged, but I managed to stay engaged and interested for most of the course. The bulk of the class was spent programming with Alice, a GUI based development environment designed to teach the fundamental concepts of programming, without tripping students up with text editors and syntax issues. Basically, Alice is the new Logo. I enjoyed messing around with it, but was disappointed to find that Alice isn’t open source. I was unable to contribute fixes for several bugs I’d found, nor could I examine the source code to see how functions and features were really implemented. For me, the hardest part of the whole course was answering test questions based only on material presented in the textbooks, which often do not conform to reality, nor my own considerable personal experience.

On the social front, things have been better than I’d expected. I’m twice the age of most of my peers, and there have probably never been two generations of Americans with such radical differences in their upbringing. I am from the last generation born before the public internet and cell phones. Most of my peers were born after the net went public, and had cellphones in grade-school. I’m also from the under-parented, Beavis and Butthead generation, while my peers were helicopter parented and some still can’t  take a shit without parental guidance. I plan to write a separate post on this topic later, but it has been interesting to interact closely with this particular age group. Overall though, my younger peers are not so different from my own age group, despite the significant social changes of the last 20 years.

While I am old, I don’t quite look my age — though I am starting to show signs of rot at the edges. People tend to see what they expect to see though, so most students assume I’m younger than I really am. I’m still much older in their eyes, but not quite uncle-creepy old.

I’ve never been good at fitting in, but I’ve been pleasantly surprised that the age gap hasn’t been much of an issue. On days when I’m dressed in business-casual for work, I am sometimes mistaken for a professor, occasionally even by other professors. The most significant difference is that instructors tend to treat me with a little more respect, and take my opinions more seriously. Of course, they also expect a little more from me sometimes, but it’s a fair trade-off.

Juggling work and school can be exhausting at times, but so far I’m enjoying being a student, learning new things, and meeting new people. I’ve managed a 4.0 GPA this semester, which is a nice way to start things off.

 
 Share on Facebook Share on Twitter Share on Reddit Share on LinkedIn
1 Comment  comments 
formats

Thoughts on the Windows 8 Release Preview

Published on June 7, 2012 by in Code

Windows 8 Release Preview dropped to the public last week. This release gives us a concrete look at the final form of the new OS. Between now and final release, there will only be a few bug fixes and tweaks.

Before I begin, I want to qualify that I’ve been using Windows 8 as my primary OS since February. I am not running this in a VM, and I spend 60+ hours a week using this system. I run everything on this machine; development tools, database servers, web servers, admin tools, utilities,  games,  videos, music, social networking, etc. My opinions are my own, but they are based on months of extensive use.

What I learned from the release preview is that Microsoft’s Windows 8 team is convinced that all the negative reaction to Windows 8 is simply change resistance. They appear convinced that once people get used to the new user interface, everyone will love it. For traditional laptop and destop users, I completely disagree, but the ship has sailed. What we see now is what we’re going to get.

I’ve already written at length about Windows 8 on a tablet, and the release preview doesn’t change any of my expectations there. So, here, I’ll concentrate on Windows 8 for x86 based desktops, laptops, and hybrid devices.

The metro UI is still horrible with a mouse and keyboard, and metro apps even more so. There have been minor improvements since the beta versions, but the fundamental nature of metro is still a highly touch-first design, and that isn’t going to change. The horizontal scrolling in metro is obnoxious, and even after months of use I still hate it. If you are using large screen monitors, or a multi-monitor systems, the full-screen windowless nature of metro apps is outright silly and wasteful. The inability to run multiple metro apps on multiple monitors is insulting  for a multi-tasking OS.

The stock Metro apps have been improved to show off what metro apps can do. Of course, that also shows off how limited these apps are compared to their traditional desktop cousins. Metro apps waste space like crazy, and common commands are still hidden away most of the time. Getting to the swipe out command bars always seems to take twice as many clicks as it should. I’m always wishing for a way to tell the apps to just leave the command bar open so I don’t have work so hard to get at them. I also hate the absence of a system-wide task or  status bar in metro. I’d like to see what’s going on with my other apps, see a clock, and most importantly status indicators that show my remaining battery time and network connection statuses.

And of course, metro apps are still sandboxed to hell and back, and are still unable to cooperate with desktop apps. Your metro apps don’t show on the desktop taskbar, which is REALLY annoying. The hot-corner switch list will show all the running metro apps, but not your individual desktop apps; it only gives you one item for the entire desktop and all apps running there. Switching between metro app and desktop app is still disorienting and jarring, though the transition is better animated in the release preview.

The only thing positive I can say about metro apps is that they can be quite pretty. Not pretty in any useful way, but visually appealing in a shallow and vain way.

The good news is that, other than the start screen, you can mostly just ignore metro apps and the entire metro UI. I have not seen anything that a metro app does that traditional desktop apps can’t do just as well or better, so ignoring metro is quite easy. The desktop, and the desktop applications, work just fine and work way they always have in the past.

In my opinion, Microsoft  had finally gotten the start menu right in Windows 7, but the new metro start screen is a sad regression that you are stuck with unless you obtain a hack that puts the start menu back (I have not done this, forcing myself to get used to the start screen instead). You can get used to the start screen and be productive with it. It has all the features necessary to do the job. I suspect that some users might come to like it more than the start menu, since it is actually bit easier for non-professional users to manage and organize. Of course, most desktop applications written before windows 8 dump a truck-load of useless and ugly icons all over the start screen, so you have an incentive to learn how to organize and customize it.

As for running traditional desktop applications, Windows 8 is quite nice. It is a good bit faster than windows 7, and there are a few minor touches that qualify as improvements here and there. The traditional file explorer has a ribbon interface that makes setting viewing options a bit easier. The control panel is still a total fucking usability disaster, but some of the utilities in the control panel have been slightly improved.

One thing striking about the the release preview is in how well it renders on screen. I’m using the same video card and driver that I was using in the last beta, but something about the release preview made even traditional desktop apps much prettier than they’ve ever appeared. It’s a subtle, but very distinct improvement.

I am still annoyed by the new, and over-hyped, File History feature. File history is obviously inspired by Apple’s time machine for real-time continuous backup and file versioning. File history also works well with the new system reset feature that can put your OS back to stock without requiring a format and re-install. Obviously, Microsoft is pushing this feature as a replacement for traditional disk and image backups, and it is a fine feature I suppose. But they’ve done their best to hide the full windows image backup tools. If you can find it in the control panel, it’s called Windows 7 File Recovery. But if want a full system backup that lets you restore your whole system after a complete disk crash, then this is still the better tool by far.

Hyper-V is another nice feature, and it finally lets you run 64bit virtual OSes. I’m glad to see Hyper-V on Windows 8, but this is pretty much the entire Hyper-V service you see on Windows Server. That means it has all the power you could want, but it also means that it is nearly impossible for anyone that isn’t an IT professional to use. I was hoping to see a more friendly front-end admin tool for it, something similar to Virtual PC/XP Mode on Windows 7 but with the full power of Hyper-V behind it. Setting up virtual networks and drives via the full server admin tools is not trivial, especially on a laptop that sometimes uses WiFi, sometimes is docked to a hard-line, and other times uses a built-in mobile broadband card. Fortunately, VMs are a feature that few non-professionals really need, so I don’t think this will hurt sales or customer approval in any way.

Overall, I don’t predict success for Windows 8 on traditional laptops and desktops, especially not in the enterprise. Metro running side-by-side with the desktop will be confusing for the average home consumer. For businesses, it will a training and support nightmare that most shops will simply avoid.

A lot of people are talking up touch-screen monitors for desktop and laptops, but I am not sure that’ll fly either. Holding your hands out like a zombie to use a touch screen is physically exhausting, ergonomically unhealthy, and very inconvenient. I personally have no interesting in smearing up my screen with my grubby paws, and cleaning a laptop or desktop screen is much more inconvenient than rubbing a tablet across your shirt every now and again. And all that’s ignoring the fact that touch screens much more expensive, and wont be widely available on most end-user PCs for while yet.

The only possible bright spot I really see for Windows 8 is in hybrid, or so-called convergence, devices. There are already a lot of very interesting devices with new and experimental designs being shown off. It will be neat to see how the hybrid device market shakes out over the next year or two. The real issue will be finding an actual customer demand for hybrid devices. I suspect that there may be some opportunities for a successful hybrid devices, but only time will tell. The sad part is, if this market doesn’t materialize, then I don’t see Windows 8 amounting to anything less than a complete disaster.

I predict that Windows 8′ will be a huge sales boost for Apple’s MacBook line, but no help to Microsoft’s own market share. I further predict that Steve Ballmer will be forced out as CEO next year. I will personally continue using Windows 8, while ignoring the annoying metro end of things. For experimentation, I will likely buy a hybrid windows 8 device of some sort in the near future.

At this time I could not recommend Windows 8 to any of my non-professional friends and associates.

 
 Share on Facebook Share on Twitter Share on Reddit Share on LinkedIn
No Comments  comments 
formats

Windows 8, a failure on tablets too.

Published on May 23, 2012 by in Code

I’m not a big fan of Windows 8, but I’m not going to waste time on what I do and don’t like about it as a desktop OS; that ground has been covered to death elsewhere on the net.

What I do want to talk about are my many doubts about its future as a tablet operating system… I don’t see this schizophrenic bastardization doing well, even on pure tablet devices.

What Window 8 is up against:

Microsoft got caught off-guard by the explosion in mobile and tablet devices, which is a fine testament to the poor state of their management. Tablets are a critical market that will decimate the established consumer PC space during the next several years. Microsoft knows this, even if they appear clueless as to how to proceed with that knowledge.

Most people don’t need a full-featured, general-purpose computer at home. They just need something that can show movies, browse the web, play music, check messages, display books, and play games. They’ll gladly give up the complexity of today’s general purpose PCs and laptops in favor of highly portable, user friendly, and easy to manage tablets. The consumer power-user of the future is just someone with a dock and keyboard for their tablet.

Microsoft is also, probably, smart enough to recognize that a large chunk of the business market will turn to tablets. Most corporate users are information consumers, not content and data producers. Tablets are a better fit for those users for most of the same reasons –they are cheaper, more portable, and orders of magnitude easier to support and maintain.

So Windows 8 is aimed at giving Microsoft a competitive tablet before the PC market collapses on itself.

The tablet market today:

Apple and Google have had years to establish and define the tablet. Both companies have decided on, fundamentally, the same approach. They up-ported phone OSes that run sand-boxed apps purchased from centralized app stores. Those apps are full-screen affairs with a heavily touch-centric UI. Alternate inputs (stylus, keyboard, mouse, etc.) are marked off as unimportant and poorly supported legacy options.

More importantly, Apple and Google have grown large and rapidly maturing software ecosystems for their tablets. They have also established a positive mind-share among the general public. Apple in particular has a substantially mature product with a vast array of capable apps. Android is struggling a bit still, but is coming along at a breath-taking pace, with major new releases appearing every few months.

Windows 8 on ARM:

On ARM devices, Windows 8 will be running hardware that is fairly comparable to that used by Apple and Google tablets. While less powerful than traditional PCs, ARM hardware promises to run cooler, have longer battery life, and be powerful enough to smoothly run a single full-screen foreground app with a few background tasks.

The key thing about ARM hardware is that these are the devices cell phone carriers will be selling. Make no mistake; those carriers are the single most important factor for competing in the tablet market.

Carriers provide ubiquitous connectivity. WiFi may be good enough for the take-it-out, set-it-up, and use-it-hard laptop world, but it falls short with use-it-anywhere tablets. Having constant connectivity is a necessity for the rapid user adoption of tablets.

Carriers also heavily discount the up-front costs of the hardware in exchange for lucrative multi-year data contracts. This makes purchasing a tablet much more appealing for most users, and it will be the deciding factor that drives many people’s decision to give tablets a try.

Microsoft doesn’t have a particularly good relationship with cell carriers these days. Based on what we’ve seen from Windows Phone 7, Verizon doesn’t even return Microsoft’s calls. AT&T will sell just about anything, but they haven’t really been promoting Microsoft’s products. And, sadly, Sprint and T-Mobile just aren’t big enough to put a dent in the overall tablet market.

As for Windows 8 itself, on ARM it is limited to running only metro apps. The metro UI is actually quite good, especially for a version 1 OS. The WinRT platform that powers metro apps is fantastic, and the developer tools are light-years ahead of the competition. And, naturally, the core Windows kernel is still an impressive feat of engineering, especially after the internal refactoring that’s been done over the last 10 years or so (yes, I hear the Linux guys grinding their axes).

Metro is, in my opinion, where the heart of Windows 8′s pending tablet failure resides.

Metro is, fundamentally, the same approach to tablets that everyone else has taken. Sandboxed full-screen apps, obtained from a centralized app store, with a heavily touch biased UI. ARM devices can’t run traditional PC apps and has no traditional desktop. Basically, apps on Windows 8 ARM devices behave just like apps on every other tablet.

Since you can’t run any of your old Windows PC software on ARM hardware, you’ll have to buy all new apps and learn how to use them. You can’t even re-use your existing Windows skills, since Metro is completely new and quite different from the traditional Windows experience of the past. A Windows 8 tablet will be just as unfamiliar to Windows users as an iPad or Android tablet.

Add to that the fact that Microsoft has a horrible public image. Lots of people love Google and Apple, but public approval of Microsoft rivals only that of the U.S. congress; and even there it’s a neck and neck race to the bottom.

To sum it up… Windows 8 on ARM will be a new and untested OS that is completely alien in how it works. It is made by a company you haven’t enjoyed doing business with in the past. It will be running on a new and untried hardware platform. It will have a small number of quality apps, and you’ll have to buy your software all over again. You’ll have to learn how to use the new apps from scratch. You’ll have to buy them from a tightly controlled app market that you’ve never done business with before. And your cell provider is probably going to do their damnedest to talk you into getting an iPad instead.

Does that sound like a recipe for success to you?

Windows 8 on x86:

Windows on x86 tablet hardware is quite different from the ARM devices. It is on x86 where the schizophrenic nature of the operating system is most apparent.

Hardware for x86 tablets is, generally, the same stuff used in laptops today. The x86 hardware will tend to run hotter, be a bit bulkier, and have much less battery life. On the plus side, x86 hardware is much more powerful, especially for running multiple desktop applications, or for heavily multi-threaded apps.

To achieve even modest battery life in the small form-factor of a tablet, most x86 tablets will have to stick with low-end components compared to traditional laptops. Metro apps will do fine, even on this lower-end hardware, but traditional desktop applications that were smooth and fast on your old laptop, will be sluggish on a tablet.

Other than expensive developer machines, I don’t expect to see a lot of high-end x86 tablets, the hardware costs too much, takes up too much space, run too hot, and requires more battery than a reasonably sized tablet has room for.

A major problem with x86 tablets is that cell carriers are not interested in selling them. They have to support the devices they choose to sell, and since these are still full-featured, general-purpose OSes, they still have all the same problems that have plagued PCs in the past; viruses, conflicting and buggy software, bum drivers, security vulnerabilities, etc. Cell carriers learned their lesson during the brief days of netbook computers, and I don’t see them lining up for another customer support ass-kicking.

The primary sales channels for x86 tablets will be the same as for traditional laptop PCs. Without cell carriers to discount the hardware costs, these will have higher up-front costs.

Getting mobile broadband working on these tablets will be just as complicated, expensive, and confusing as it is for laptop users today. You will either have to select a tablet that has the right built-in broadband card for your preferred carrier, or you’ll have to go down to the phone shop and buy a mobile hotspot. Either option adds significant up-front costs, requires specialized knowledge to setup, and still locks you into an expensive 2 year data contract (with no hardware discount to show for it).

As for Windows 8 itself, on x86 it can run both traditional desktop software, as well as the new touch friendly metro apps. While this may seem like a selling point, it presents a whole mess of new problems. Windows 8 doesn’t do much to make traditional applications touch-friendly. The on-screen keyboard also doesn’t go very far. Hopefully most of these tablets will support a stylus, which will make a decent mouse substitute for desktop applications, but the OS doesn’t  do near enough to make the stylus a compelling alternative. You can always use an external mouse and keyboard, but at the cost of the portability that makes tablets appealing in the first place.

Metro apps work fine on x86 tablets, so most of what was said about them in the Windows 8 on ARM section applies here. But metro apps have a totally different look and feel. Switching from desktop applications to metro apps is a jarring and uncomfortable experience. Furthermore, metro apps can’t interact with to their non-metro counterparts in any meaningful way, which leads to near endless user frustration. The whole experience around mixed metro and traditional applications on the same device is disorienting and counter-intuitive, and that’s the polite description.

To sum it up, Windows 8 on x86 tablets will give you an over-powered tablet for metro apps, but under-powered for desktop apps. They’ll be more expensive, bulkier, and have crap for battery life. It will be made by a company you haven’t enjoyed doing business with in the past. It will run desktop software and metro style apps at the same time, but the dual nature of the interface will probably drive you insane. Desktop applications will be difficult to use on a touch-screen, and metro apps are plain obnoxious when using a mouse and keyboard. You’ll have to buy metro apps from a tightly controlled app market that you’ve never done business with before. You will have to buy the tablets from traditional PC distributors at full price, and getting them online with mobile broadband will be difficult and significantly more expensive.

Where’s my cheese grater?

What Windows 8 could have been:

The depressing thing about Windows 8 is that it could have been great, had Microsoft just taken a different approach.

What I, and many others, really wanted from Windows 8 was an OS that brought real general purpose computing to tablets. We wanted a tablet that let us get real work done with real “applications”, not another device that just runs little finger-paint apps…

What we wanted was a tablet built for adults.

Even more depressing is the fact that Microsoft was well on their way to producing that kind of tablet on two separate occasions during the last 10 years.

First was the old Tablet PC OS, a variant of Windows XP. It was a good start that failed mostly due to internal apathy and politics within Microsoft itself; though some blame falls to the hardware, which wasn’t quite up to the job of making cost-effective tablets 10 years ago.

Later there was the courier project, a dual screen book-like tablet with a completely fresh and unique approach to the UI. It would have combined touch, stylus, and on-screen keyboard with a task-driven, workflow oriented application design. Courier had massive support and interest from the public before Microsoft killed it in early 2010.

The key thing about Microsoft’s past attempts was that they did NOT concentrate exclusively on finger-based input. Instead, a stylus was used for detailed work, while fingers were used when accuracy wasn’t an issue. Handwriting recognition via the stylus competed with an on-screen keyboard for data entry. And of course, external keyboard and mouse was a fully supported option.

Had windows 8 continued in this tradition, instead of just copying what Apple and Google were doing, it could have brought touch, stylus, keyboard, and mouse together with full-featured desktop applications. And it could have done so by extending a familiar Windows user interface, rather than chucking out 25+ years of graphical user interface evolution.

Microsoft could still have introduced WinRT, and the other technologies that are aimed at creating touch-first apps, yet let those apps run side-by-side, comfortably, and cooperatively on the same desktop with traditional applications. They could have still had an app market, without needing to force it on everyone. And they could have still forked a version of the OS to port over to ARM hardware without needing to support the older desktop software.

I mourn the OS that could have been, and I suspect that after Windows 8 falls on its face, Microsoft may never again be in a position to produce a compelling tablet OS.

 
 Share on Facebook Share on Twitter Share on Reddit Share on LinkedIn
1 Comment  comments