I had a veryinterestingissuesubmitted over the weekend; one of several which boiled down to the same issue. The application against which it was submitted has a number of categories. Some of the categories are empty, which is just fine in the application. We made the mistake of underestimating our users and we started to hide empty categories. We assumed that when filtering users wouldn’t want to see categories which would give them empty results. Why would you want to see empty categories? Well as it turns out there are a ton of uses cases:

• When adding a new category it will be empty by default and users would like to see that their addition of a category worked
• When generating reports users want to have proof that the category’s contents are missing
• When listing categories users have an expectation that none will be missed

We hid the categories to reduce options in a drop down and to generally clean up the UI. Now I’ve had a number of issues on this topic submitted by users we’re going to remove the empty category filters. We’re actually hurting our users by trying to guess at what they want instead of asking them and letting them try things out. Lessons learned in software development, I suppose.

In yesterday’s post I talked about how to change out the JSON serializer in ASP.net MVC. That was the first step to serializing an NHibernate model out to JSON. The next issue I camacrosswas that the serialization was really slow. What’s the deal with that? I looked at the JSON which was produced by the serialization and found that a single record was 13KiB! Even a modest result from this action was over 250KiB. This is a pretty significant amount of data to serialize. There was really no need to serialize that much data. In the end the data was being used to populate a couple of columns in a table.

The reason the data was so huge is that the NHibernate object which was being serialized was fully populated with data fromseveraltables. There was even a collection or two of records in there.

Projections to the rescue!

The beauty of returning JSON is that it isn’t strongly typed. This means that all you need to do to trim down the traffic is to use object builder notation to project from the collection into a new object. To do this you need only do something like

results.Select(x=>new{ID=x.ID,Client=x.Client.Name,UserName =x.User.Name,Name=x.Name,Date=x.Date})

In my case this reduced the JSON payload from over 250KiB to 2.5KiB. This reduced not only the serialization time but also the time taken to send the traffic over the network.

This is a really easy optimization and it is something that tends to slip the mind. This is why it’s important to test with large dataset.

Today I stumbled onto some code which was serializing an NHibernate model to JSON and returning it to the client. The problem with directly serializing the object from NHibernate is that it may very well contain loops. This is fine in a C# object graph because only references are stored. However JSON doesn’t have the ability to store references. So a serializer attempting to serialze a complex C# object must explore the whole object graph. The loop would make the object infinatly large.

The code in question returned a ContentResult instead of a JSON result. I didn’t like that, why not just override the Json method? So I went to override the call and found, to my chagrin, that the base class, Controller, did not declare all the signatures as virtual. Bummer. I hate this sort of thing. Now serialization becomes something of which developers need to be aware. There are a couple of methods you can override but that’s not a good solution as unless developer know about the custom serializer they won’t know which methods they can an cannot call. I’m not sure what the motivation is around not providing a place to plug in a custom serializer but it does seem to me to be a pretty common extension point. The ASP.net MVC team are pretty good a what they do so I imagine there is a reason.

The best solution I could come up with in short order was to override the two signatures I could. To do that I wrote

https://gist.github.com/anonymous/5268866

I also created a NewtonSoftJsonResult which extended the normal JsonResult. This class provides the serialization implementation. In our case it looks like

https://gist.github.com/anonymous/5268878

I still don’t like this solution but it does work. I’m going to think on this one.

I’m way behind the times on mobile development. It is one of those things I’ve been meaning to get into but just haven’t had the cycles. There is a mobile app on my horizon so I’ve been looking a bit into how I would go about it. From what I can see there are currently two good options for building mobile applications at the moment. Ishouldstop and explain mycriteriabefore somebody shives on me for jumping to conclusions.

First is that it needs to run on as many devices as possible. Realistically an Android tablet and iPad would be sufficient(sorry, Surface). This is obviously a pretty common use case these days. The tablet market has been pretty much owned by iPad but recently Android has been making inroads. I have a Nexus 7 which is really a fantastic device. Of course I’m comparing it with a first generation iPad so things on the Apple side might have come along some way.

Second is that it needs to be easy to develop. I’ve never been very impressed with Objective C and developing for the Dalvik VM using Java is grim. I’m really sad about where Java’s going as of late. There are Java security flaws all over the web as of late and the autoupdate mechanism install highly questionable software along with the updates. I still think that the JVM is a good platform, I just think that Java has lost its way.

With these criteria in mind I think the two great options for developing for mobile are Mono and Phone Gap. Mono started off as a port of the .net framework to Linux but has now really got legs. You can now write the majority of your application in C# and have it run on Android or iOS. Phone Gap allows you to write your application in HTML5 but still have access to most of the underlying device’s hardware. For the things that don’t have an API yet you can write bindings, but they have to be donenativelyand, obviously, they need to be written for every platform.

In my mind both Phone Gap and Mono provide powerful programming tools and models for phone development. There arecertainlysome things for which you might want to write a native app but I bet they make up less than 5% of the apps on phones. If I had to develop an app today without being given any opportunity for further research I think I would probably jump into Phone Gap. I’m really excited about all the good work going on in the HTML/JavaScript space at the moment. I think the ecosystem for HTML based phone applications will greatlybenefitfrom the larger world of HTML and JavaScript development.

The mobile space is getting really interesting, far more interesting than it was even two years ago. I bet it will be even more exciting in two years.

I blog about quite a few subjects. They are mostly software andtechnologyrelated but from time to time I talk about business or why youabsolutelyhave to put orange zest in cranberry orange muffins. Within the technology camp I talk about all sorts of things because I consider myself to be a generalist. I don’t want to be stuck using any one technology because I worry that something will happen to that technology and I’ll be unable to find a new job. Then I’ll be unable to work and things which just spiral down until I’m the star of some show on TLC: “TechWennie to Crack Deal” or “Early Adopter to Laggard:Victimsof Rogers Diffusion Model”

Sure there are some “safe” technologies like Oracle or SAP which areunlikelytodisappearbut you never know”¦ I think about Siverlight which was the future for many years until HTML5 pretty much killed it.

I think that being a generalist is a great move. I work with all sorts of technologies and it affords me the ability to apply ideas form one technology to another. I think I come up with someinterestingsolutions in a problem space because I’ve seen how it is done by people in an unrelated space. I am constantly exposed to things outside of my comfort zone, I’m always learning. However, it has its costs: I probably don’t earn anywhere near what I could as a deep specialist in a field. Because I have a lack of really deep knowledge I don’t have theopportunitiesto travel to conferences or give training as an expert. Basically I’m trading being a super-massive star with a short lifespan for being an average main sequence star with a long lifespan. I won’t shine as bright but I’m alsoprobablynot going to be on TLC.

There has always been a room in my house which doesn’t get good WI-FI coverage. It’s odd because my house isn’t that big and a single WI-FI point should cover it easily. I’velargelyjust been living with it but the other week I decided enough was enough. I found a great little app for my Nexus 7 which did WI-FI analytics.

WI-FI

It is a great program and I used it to confirm that, yes, my one room is almost devoid of signal from the two WI-FI points I have. I could also use this app to find WI-FI channels which have more or less traffic on them(channel 11 is pretty crummy near me). As it was I just used the signal meter which shows the strength of a single WI-FI point and walked around the house. There is literally only onedead spotin the entire house ““ that one bedroom.

I think it might be because there is a giant blob of metal in the room in the form of a bed. I tried moving router around the house, raising them up, lowering them down, changing channels and always the signal in this bedroom was grim. So I bought the biggest, most powerful WI-FI point I could find and stuck it in that room.

Problem solved.

WI-FI sucks.

If your application needs to write out Excel files then your best option is to make use of the near magical EPPlus. I’ve written about EPPlus beforeand today’s post actually stems from that one. See the default databars which EPPlus puts in are a little limited. The specific issue the customer had was that when a value of 0 was entered the databar still showed a little bit. If you manually add databars through Excel the problem doesn’t exist. I was curious about what options were missing in the excel file created by EPPlus which existed in the Excel created file. Could I duplicate them in EPPlus?

Excel is a pretty old application dating back to when I was just a wee lad. As you might expect over time the file format became more and more complicated as new features were added and hacks put in (ask me about how dates work in excel). In Office 2007 Microsoft threw out their old file format and moved to a new, XML based, file format. They did it for all the major office applications: Word, Excel and Powerpoint. Access is still a disaster, but you should be using LightSwitch, right? It is a far better file format than the old one even though it is XML.

Excel file are actually a collection of XML files all zipped up with a standard zip algorithm. That means that you can just rename your .xlsx file to .zip and you can open it up with your fully registered version of WinZip. Inside you’ll find a bunch of files but the one we’re intersted in is sheet1.xml.

Excel Contents

I put in some simple databars and opened up this file’s XML. I won’t include the full contents here as it is pretty long but you can look at it in a gist. Comparing this with what EPPlus generated the difference became apparent: the whole extLst section was missing. My understanding is that these extList and ext sections are used to provide extensions to the file format. So while the conditional formatting section existed in EPPlus’ version it didn’t have the extensions and it was the extensions which provided the styling I wanted.

Fortunately there is a solution inside EPPlus. When defining the worksheet there is a property called WorksheetXML which contains the full XML document which will be exported when the excel package is saved. I tied into this XML using the Linq2XML API and injected the appropriate extLst information and the line in the conditional formatting to link to it. It took me a little while to get everything set up the way it needed to be but that was mainly because of my mental block around XML namespaces.

It certainly isn’t a pretty way to manipulate Excel documents but if you need some functionality which hasn’t been bound by EPPlus yet then it’s a pretty good solution. Being able to open an existing Excel file and make small changes then compare it with the previous version gives an easy path to reverse engineer the file format.

I had to write a small bit of throw away code today to validate some database entries. I’ve been playing a bit with F# at night and this would have been an ideal application of what I’ve been learning.Unfortunatelywork has a strict C# only policy. I’m sure this isn’t uncommon. Policies like this are in place so that other developers can jump into an application quickly and help out or pick up where others left off. On the surface it seems to be a great idea and I’ve probably suggested putting policies like this in place in a previous life. Today I got to thinking about the policy and decided that it is actually terribly flawed.

The premise is that the hardest thing about picking up a new project is learning the language in which it is written. Theinterestingthing about that is that if you ask any developer how long it takes to pick up a new language they will say “an afternoon” or “a couple of days”.Admittedlysome of that may be developer hubris but even if the estimate is out by a factor of two it still isn’t very long. The vast majority of the learning curve for joining a new project is in understanding the architecture and the design as well as figuring out where to make changes. Knowing the language in which the application is written is going to be of minor help in that case. Good architecture can and does exist in any language.

On the other side of the equation preventing developers from trying out new languages narrows their field of expertise. Supposing that the language is only used by the developer briefly for one project they may still learn new practices or patterns which are unique to the language.

The principle oflinguistic relativityholds that the structure of alanguageaffects the ways in which its speakers conceptualize their world, i.e. theirworld view, or otherwise influences theircognitive processes.

-Wikipedia

I know that learning typescript and F# has opened my eyes to different ways of doing things in my regular C# code. There are also some tasks which are easier in one language or another. I find it very difficult to justify spending more money on the development of an application in C# if the same business needs can besatisfiedfor half the price in another language, say Python or Scala. Just as only planting a single crop is dangerous so is supporting only a single language.

As an added bonus experimenting with other languages is a great way to keep developers. We devs love learning new things and if your company can’t keep developers through high pay then cool technology a good alternative.

Developing in other languages isn’t something to be feared, it is something which should be cautiously explored.

I could not believe it when I checked and found that I don’t have a post on how to write a bug report. Over the years I’ve seen a lot of bug reports. Of course most of them aren’t real bugs as that would imply that the code I write isn’t perfect. Aridiculousproposition. The bug reports are usually substandard in some way.

The purpose of a bug report is to explain to the developers or support group exactly what went wrong. To do this I like to see bugs which tell a story:

I was attempting to load the add user form from the main menu but when I clicked on “Add User” I was taken to the user list screen. I expected to be taken to a screen where I could add the user.

This story contains the key elements which a developer would need to reproduce the bug. Obviously this is a very simple scenario and one in which it is apparent what the issue is. As the scenarios get more complicated the information needed from the user increases.Fortunatelymuch of this information can be gained from examining logs instead of pestering the user with questions like “What server were you on?” and “At what time did the failure occur?”.

Make no mistake writing bug reports is a pain and for every issue you get a report of there are likely to be a dozen users who just give up. Frustratedusers aren’t what you want. Making reporting as simple as possible for users should be a key part of your user interaction strategy. To streamline this process I usually suggest using a tool like jing. This allows users to quickly record their screen so they don’t even need to type in a bug report.

This still requires that users let you know when they have problems. I just stumbled across MouseFlowwhich is the actualization of an idea that I had some time back for tracking user interactions in the browser. Traditional web analytics let you know what users are doing on your site, where they are clicking and what they’re looking at. MouseFlow allows you to look at how users are acting on the page. As the web evolves more of the user interactions are moving within a page using Ajax or even single page applications. These are more difficult to profile than traditional websites. Some interactions don’t need to make server trips so the user’s activities are lost. MouseFlow captures these interactions and gives you some great tools for analyzing them.

I haven’t used MouseFlow but it looks amazing. When I do get around to trying it out then I’ll post back with my experiences.

Well I had quite the adventure setting up builds for AngelaSmith. CodeBetter are kind enough to host a TeamCity instance on which you can build open source projects. I got an account and started to set up the build. The first part went okay, I got builds running and unit tests going as well as code coverage. I mostly put in code coverage as a metric to annoy Donald Belchamwho doesn’t believe in metrics.

Then I turned my attention to building nuget packages. I decided that I was going to do things right and build not just release packages but also symbol packages on the off chance that somebody would like to debug into our very simple library. I was also going to have the build upload packages directly to nuget.org.

The first thing I did was create a nuspec file.

I didn’t specify any explicit contents for it. Instead I use a convention based directory structure which allows me to add files to the package easily. In the build Icopy the output of the build into a lib directory. I copy both the .dll and the .pdb file which is needed for constructing a source package. I also create a src directory adjacent to my lib directory. Into that directory I copy the source code too as this needs to be in the symbol package. To do this I just use a bat script. Sometimes the simple solution is still the best.

TeamCity has a great set of nuget package tasks. I started with the one for assembling the package.

TeamCity Nuget Package Settings

This is very simple, you just need to set the path to the nuspec file. I added a version number based on the build number and a “-beta”. When I took this screenshot I was pretty sure that this followed semantic version numbering. As it turns out it doesn’t. As these are builds not for release I should be numbering them as +build${buildNumber}.Unfortunatelynuget does not allow for proper semantic versioning so we have to live with a ““ instead of a +.

The next step was to publish the packages to nuget.org. To do this I set up a new account on nuget.org just for publishing AngelaSmith packages. The secret key is entered into TeamCity which takes care of uploading both the standard package as well as the symbol packages. I don’t know how secure the key is which is one of the reasons I used a standalone account. If it is compromised none of the other nuget packages I maintain will be vulnerable.

With this all set up the AngelaSmith builds now push directly to nuget.org.