Statisfaction

A few months ago I kicked up a fuss about the usability and interaction design of the first New Zealand Online Census.

Just having completed my own design projects for the SJS registration forms and E-Government compliance for Grants Online, I was surprised at how the Online Census differed so radically from what I considered sensible design for online forms and E-Government Web Guidelines compliance.

To fuel the fire several other usability experts agreed with my observations, and after hearing stories of others struggling with the online census I publicly criticised Statistics New Zealand on this blog.

At the time I told Statistics New Zealand about the issues and below is their response.

Note: Statistics New Zealand have agreed to let me publish this letter. I appreciate the Census team sharing their reasoning behind their design decisions and I hope you’ll also find these insights of interest.

May 31, 2006

Hi Zef,Here are the responses to the comments you made about the Online Census.

You may be interested to know that we have been named as a finalist in the Computerworld Excellence Award 2006 under the category of “Excellence in the Use of IT in Government”.

[Zef] Let’s start with the strange and long website address you need to type into your browser to access the census website – http://www.stats.census2006.govt.nz/ – why so long and complicated? Why not just make the address simply http://www.census.govt.nz?

[Stats] We agree that a website address should be as concise as possible. However a number of factors determined the structure of the URL for the Online Census Option. It was a branding requirement that both the Government agency that was running the event (Statistics New Zealand), and the name of the survey (2006 Census of Population and Dwellings) were represented within the URL for the website. Note that www.stats.census.govt.nz was the information website used to inform the public of New Zealand about the census.In all of the feedback that we collected (from helpdesk enquiries, survey’s and media extracts during the course of the 2006 Census), we do not see that the length of the website address was an issue.

[Zef] Tabbing – the main form of navigation between form fields on the web – was inconsistent and in some cases, simply broken. …And a big blooper – on several steps the tab key would leap-frog over whole sections I’m sure I was meant to fill out (in once case it jumped over 20 questions!).

[Stats] While we are aware that there may have been some keyboard navigation issues with certain browsers, we have no evidence that this created a major problem for respondents. We note your comment that using the Safari browser, a key tab action would jump over numerous questions on the online form. We advise that we were unable to recreate this issue in our testing. We are sure you are aware that the online form had JavaScript design features to purposely skip irrelevant questions, based on previous responses.

[Zef] When entering my internet ID I deliberately made an error to see how the system would react. Well it did react and wasn’t very friendly – first of all I got a warning message in a dialogue box (pop-up) – then on returning me to the page it had erased the valid numbers I had entered. I had to start again…

[Stats] The system would intentionally erase the incorrectly input Identification Number and PIN, to maximise security of the login step against automated processes. We agree that this process would force the user to re-input the complete pair of login credentials again.

[Zef] When I tried the form with JavaScript turned off (which is the situation for up to 5% of people out there), the system presented me with this message – after 15 minutes it was still going – so I gave up…

[Stats] Our records show that 99% of our responses came from the JavaScript version of the Online Census Option. The feedback that we have received does not show any evidence that delays in loading pages caused an issue for our respondents, either on Broadband or Dial Up connections. We apologise for the delays you encountered and trust they were an isolated occurrence.

[Zef] The census was mad on pop-ups – pop-ups for error messages, pop-ups for the privacy policy, pop-ups for help, pop-ups for questions – it was pop-ups pop-ups pop-ups all over the place…

[Stats] We note your concerns regarding the use of “pop-up” windows within websites and acknowledge that respondents did report some issues, where they were unaware census form pages were being blocked by their 3rd party “Pop-up Blocker” applications. We will review the use of pop-up windows in any future designs of online forms.

[Zef] Extra questions you won’t find on the paper form have made their way to the online form – such as ‘Where are you filling out this form?’, followed by a question asking your address…

[Stats] The extra questions were included for respondents who may have been completing forms away from their home address (eg. internet cafes), who needed instructions which would help to clarify later questions (ie. questions which refer to “this dwelling”).

[Zef] A ridiculously long page for the blue (Individual) form – whatever happened to the widely used step-by-step wizard?

[Stats] The long page (scrolling format) was deliberately chosen to provide a similar context to paper forms in order to maximise data comparability. Our objective was not necessarily to give the user a “slick, web-saavy experience”, but simply to provide the public with an alternative user-friendly means to complete their statutory obligation. In addition, we needed to ensure that we could still collect reliable / useable information (ie. data that would not differ in any significant way from paper data). From the respondents perspective, it was designed to provide the user witha feeling of control when completing the questionnaire (for instance – they could easily scroll back to view responses to associated questions). The use of the single scrolling page has been welcomed by Don Dillman – a recognised expert in the field of questionnaire design. A single scrolling page for the online form also allowed us to reduce the number of calls required to the host webservers, so reducing the potential load on census night during peak activity. Once the form was loaded the respondent could complete the entire form without interruption or waiting for an additional page to load. In addition, if the respondents internet connection was interrupted prior to submission, then the completed form data would be retained without potential loss of input whilst the con
nection was re-established.

[Zef] And did it validate for web code compliance and accessibility? Absolutely not…

[Stats] Our application vendor’s brief was to optimise the usability of the Online Census website for use with a wide range of browsers and operating systems. Statistics New Zealand is more than happy with the range of the public’s computers that were able to access the Online Census website. They concede that this optimisation may have caused conflicts with Code compliance in certain areas.

[Zef] ‘A cultivated census’…

[Stats] We appreciate the efforts you have made to construct your own version of the Online Census application. There were a number of business requirements given to our application vendor, that we understand you would have been unaware of.

[Zef] The online census was a flop…

[Stats] Statistics New Zealand considers the Online Census Option to be a successful project. The objectives set prior to implementation were all about offering an efficient and secure online option for completion of the 2006 Census of Population and Dwellings. In our view, these success criteria were fully met. We agree that the uptake was not as high as our initial expectations, but we feel that this was due more to respondent behaviour, than any failing within the application or its associated processes. Up until the day after census, the uptake was projecting towards a 15 to 20% figure. It was the period after census day, where we saw a fall off from our projected volumes. This was a significant change in respondent behaviour, than that seen in our previous public testing. We can now use this valuable information in our future projections and have passed on to other worldwide organisations currently implementing online solutions for census collection.

[Zef] The online census did not follow web design best-practise for online forms…

[Stats] We believe that we delivered the best design solution, considering the various business requirements that needed to be met. The online census creators did not follow a user-centred design process. The design of the online census questionnaire followed an iterative process of usability testing with members of the public (on numerous prototype systems) to define an easy to use, intuitive and robust online census option. The large bulk of feedback we have received would suggest that the New Zealand public who took the online option, thought it was a relatively user friendly and efficient way to complete their census forms. Interim results from the post census survey indicate a very high satisfaction level. A large majority of those who completed their census forms online, or would have if they had been offered the option, indicated that they would use the online option in the 2011 Census.

[Zef] People like the aesthetics of filling out a form on paper…

[Stats] For the 2006 Census, we focussed on offering an additional option for completing the census form. We acknowledge that a large percentage of the population will prefer to complete their census form on paper.

Best regards

Ian Smith
Project Manager Online Census
Statistics New Zealand