The great cloud accounting software search of 2014 (part 4)

This is a continuation of my search from the previous posts (part 1, part 2, part 3, part 3b).

At this point I decided to look in more detail at the user interface and our typical usage pattern around time capturing. A large part of our business is consulting services where we log and bill time to clients. This is therefore something that every employee will use on a daily basis and from past experience we know that this can cause major frustration and disruption to the business if it doesn't work optimally. And of course logging time doesn't exactly add value to our customers so the less time our people spend on this activity the better.

To test this I devised a simple test in three parts as follows:

1. I log in, create two projects with three tasks each, log out.
2. I log in, capture four time entries with different project/task combinations, log out.
3. A colleague logs in on a typical staff account and logs eight time entries on various project/task combinations, then logs out.

For each part of the test, the tester (either me or the colleague) rates the user friendliness subjectively on a scale of 1 to 10 and for each part of the test we also time how long it takes.

We then come up with a combined score for each app that combines our subjective user friendliness ratings with an overall time spent rating. The time spent rating combines the time entries with weighting factors to estimate the total person hours that our organisation might spend on time capturing in a typical month. The user friendliness scores are added together (and in the case of time capturing averaged) and multiplied by a weighting factor representing the number of users who would be exposed to this functionality. We then take the combined user friendliness rating and divide it by the estimated man hours per month spent on capturing to get to an overall score and pick the top 3 from this.

This test revealed some interesting points. For instance, our user-friendliness ratings for Zoho wasn't particularly high, but it did very well on timing, suggesting that our subjective rating is definitely subjective.

I also noticed very quickly that the app that is tested first is at a disadvantage as I got to know the data I was capturing better with each repetition. To counter this effect, we did everything twice and did the second run in reverse order (i.e. if Freshbooks was first in the first run, it would be last in the second run) and then used average time readings.

The end result of this process gave us the following scores:

• FreeAgent: 8.69
• Freshbooks (+ Kashflow): 10.16
• Triggerapp (+ Xero): 2.36
• Zoho: 9.70
• Paymo (+ Kashflow): 8.45
• Harvest (+ Xero): 10.46

You will notice that Harvest scored strongest. Their branding is also orange, so my initial reaction was that I must have been biased. However, my user-friendliness rating for them on the first part was joint-lowest and for the second part was joint in a 4-way tie for second (Freshbooks being the only one that scored higher and Triggerapp being the only one that scored lower), so I don't really think that I favoured them unfairly in any of my subjective ratings.

I should mention that Triggerapp scored particularly low because it seems to be designed on the premise that you will always (or at least the majority of the time) use the timer to start/end time recordings. In our case I don't foresee our users actually doing this. This had a significant impact on our ratings and times as it meant we had to start timer, stop timer, edit time to capture what would otherwise simply have been creating an entry. I'm not sure if we just missed something obvious, but neither of us could find another way of doing this. The app itself actually looked really nice and had some funky ideas, so if the timer start/stop mechanism is your cup of tea it's well worth a look.

To conclude, the top 3 that move on to the final round of my evaluation process are therefore Xero + Harvest, Kashflow + Freshbooks, and Zoho. 

In the next round of evaluation, I'm going to generate invoices from time captured (this will also test the Freshbooks-Kashflow and Harvest-Xero integration) and do some accounting entries that I know we require and that would stretch something that isn't proper accounting software.