Researching Usability

Roundup 19th Feb: Not what you say you’ll do but what you actually do

Posted by: Lorraine on: February 16, 2010

• In: Evaluation | Round-up
• 1 Comment

Heuristic report

This week the heuristic inspection report has been published and is available to read. If you would like to read it feedback is very welcome. The document is available in Word or as a PDF from the NeSC digital library: http://bit.ly/ux2inspectionreport. It is a sizeable document so thanks in advance for taking the time to read it! 🙂

Not what you know, nor who you know, but who you know already

This is a research paper which was a collaboration between myself, Hazel Hall and Gunilla Widén-Wulff. The research was undertaken when I first graduated from my Masters in 2007 and this week I received the good news that it will be published  in Libri: International Journal of Libraries and Information Services at some point this year. The paper examines online information sharing behaviours through the lens of social exchange theory. My contribution was the investigation into the commenting behaviour of undergraduate students at Edinburgh’s Napier University as part of their coursework. I’m very excited by this news as it is only my second publication. I look forward to seeing it in print and will provide details here if it becomes available online.

TAM part 2: revised acceptance model by Bernadette Szajna

Another paper which I read this week was ‘Epirical Evaluation of the Revised Technology Acceptance Model’ by Bernadette Szajna (1996). In this paper Szajna uses the revised Technology Acceptance Model (TAM) from Davis et al. (1989) to measure user acceptance of an electronic mail system over a 15 week period in a longitudinal study. By collecting data from participants at different points in the study she was able to reveal that self-reported usage differed from actual usage and that as a consequence it may not be appropriate as a surrogate measure. This supports what those who’ve been running usability tests have been saying for a while: what users say and what they do are seldom the same. In user research terms this means that observing what users do during their interaction with a system is as important as what they say about their experience.

In addition the paper revealed that “unless users perceive an IS as being useful at first, its ease of use has no effect on the formation of intention”. This struck a chord with me because as a usability professional I often assume that ease of use is a barrier to the usefulness of a system; if a user does not know how to manipulate the interface they are unable to discover the (possibly useful) information below the surface. Then when I was considering the usefulness of Twitter groups I realised that it began to follow the same pattern.  Twitter groups is a recent addition to Twitter and available to users. It allows those you are following to be categorised into self-named groups. For example, it’s best application is a means for users to differentiate their professional connections from personal ones. In theory it is a good idea and one which I thought I might use as a way of separating out different networks would certainly make them easier to monitor. I can’t imagine it being too difficult to set up a group if I so wished but the problem is that I never considered it useful for me to do so and consequently I  never did (note: I created a private group today to test my theory). The reason in this case is that I rarely use Twitter’s website to monitor or communicate with those I’m following. There are many different client managers such as TweetDeck who can do this for me. I’m sure there are a few people out there who have created groups and view them regularly but could these people be in the minority? I’d be interested to test my theory so any comments on your own Twitter group behaviour is welcome.

My conclusion is that (for me) the usefulness of the groups tool was a greater barrier to use than the ease of creating a group, verifying Szajna’s findings. This illustrates how important usefulness is to the user acceptance of technology and is therefore something that should be evaluated in every system to ensure success.

Mendeley Webinars

Lastly Mendeley directors are hosting webinars which will provide an introduction to its features including inserting citations and using the collaborative tools. The webinars will be held on Tuesday, February 23, 2010 5:00 PM – 6:00 PM GMT and Wednesday, February 24, 2010 9:00 AM – 10:00 AM GMT respectively. I have signed up for the webinar on Wednesday and look forward to learning more. So far I’ve managed to add items to my library and connect with others online but don’t feel I have exploited its features fully and am having difficulties amending my bibliography in Word. Hopefully this webinar will provide help and advice.

Weekly Round-Up: November 27th

Posted by: Lorraine on: November 27, 2009

• In: Round-up
• 1 Comment

Mendeley

You might notice that I’ve put a link to my Mendeley profile in the right-hand column of the blog recently. I finally got round to taking a closer look at it after hearing good things about it back in September. I’ve created a public folder for the ux2 project which I want to use to keep track of papers relevant to the project. For those who do not know what I’m talking about Mendeley is a network which allows researchers to create a bibliographic database (there is also a video presentation on YouTube). So far it seems to be a good way of organising papers because they are tagged and categorised for you. Recommendations aren’t available yet but I can see it being very useful and a good way to find new material.

I did run into a couple usability issues when trying to find other members and build my network. Firstly, if you want to check your email for contacts it is not possible to do so with your University email address (which is likely to be a common problem for academics). Currently you can only search Gmail, Yahoo, Hotmail, AOL and GMX. Secondly, the search system within ‘Find People’ was not as intuitive to use as I was hoping. Turns out the search form  searches your contacts by default. I did not realise this at first for a couple reasons:

1. The search is very responsive and immediately starts searching your contacts as you type letters by highlighting where the letters appear in all of your contacts. Responsiveness is not necessarily a bad thing but I wrongly assumed that this meant it was only searching my contacts and started to think that I had to go elsewhere to search the whole system.
2. Secondly, I did not read the message which appeared once I submitted my search and as a result, drew the wrong conclusions again (see image). I did what users often do and that is to scan the words quickly. The first part I read was ‘None of your contacts match this search term’ which only helped to reinforce my conclusion that this search form only searched my existing contacts. I therefore did not bother to continue reading the second line which said ‘Click Search to include all Mendeley users in the results’. The word ‘Search’ was linked making it a different colour and as a result it stood out from the rest of the text. Again I only read ‘Click Search’ and quickly dismissed it because I felt that I had already completed this action when I selected Search this first time. I spent quite a while looking around the rest of the site to find out how to search all members and eventually came to the conclusion that it wasn’t possible yet. To the credit of Mendeley, I tweeted my difficulties on Twitter and got a quick response. This showed how proactive they are at fixing problems. It wasn’t until I asked a colleague for help that I finally realised that the message asks users to click Search a second time to search all members. I think that it would be more intuitive for users if it did this by default. In addition it should give users the ability to choose whether they want to search all users or just their contacts, otherwise it is likely to be confusing. If something requires instructions to be used correctly it often means that it is not intuitive!

👁 mendeley

Anyway, I still believe that Mendeley has the potential to be a useful tool for researchers and I will continue to use it. Please feel free to connect with me if you also have an account.

COI Usability Toolkit

This week COI announced that they had developed a usability toolkit aimed at public sector websites (although anyone can access it). The toolkit provides good-practice guides on a variety of topics including search form design and search results design. After briefly looking at some of the guides they are effective in communicating the main points in a clear and concise manner with annotated illustrations to help time-poor users get a good understanding very quickly. There is also a section where you can test your knowledge and this helps to reinforce user’s learning. It is predominantly aimed at those with limited or no  previous knowledge of usability and avoids using any techy words or code.

To use the toolkit visit http://usability.coi.gov.uk/.

Infomaki

Infomaki is an open-source, lightweight usability testing tool developed by The New York Public Library’s Digital Experience Group which I came across this week. It is based on the ideas of fivesecondtest, a tool which I have come across in the past. Based on the same premise, it asks users to answer one question. The questions can either be multiple choice or a design question asking users to state where they would click on a page to complete a specified task. You can also make comparisons between two designs and test the user’s recall of features. The beauty of the concept is that it does not require a lot of the users time. Answering one question can take only a few seconds and this has been shown to be attractive to users as the response to the survey was very high. So much so that the developers found that 90% of users wanted to answer more than one question – behaviour which is difficult to elicit through traditional market research methods.

Evidence suggests that tools like this are extremely successful in gathering a large volume of quantitative data which can help to back up one-to-one usability test data. The developers also plan to incorporate features that collect demographic data. This will add even more value to the tool as it will help in the construction of user persona’s.

Tools such as Infomaki are particularly useful to those working in digital libraries without a dedicated user research team. Open source means it is free and it can be set up by anyone interested in gathering data about their digital library.

There is more information on the software available in addition to the article on Code4Lib by Michael Lascarides which can be found here: http://journal.code4lib.org/articles/2099.

Usability Week, Berlin

My UX2 colleague, Boon returned for Nielsen Norman Group Usability Week in Berlin with lots of knowledge to share. One of the many things that came out of his time there was the idea of user persona creation for digital libraries. After a couple leads we were pointed to the work by Max Planck Digital Library and the persona’s they created. If anyone else knows of other work that has been conducted in persona creation for digital libraries please let us know, thanks.