Case 3: evaluating e-Learning Lab's website

In the evaluation of eLL’s website usability testing will be the approach when finding issues to approve on the website. Usability testing was the most used method for testing in the 1980’s and is still a very popular method. Today methods as heuristic evaluation and field studies are also dominant when designing and evaluating software ( Rogers et. al. 2002, p. 341). In usability testing typical users are tested under controlled conditions where they are given a set of predefined tasks that they have to complete. Users performance are for example measured in time spend on a task, number of errors or clicks. Most often users are given a questionnaire or are interviewed after the test (ibid.).

One of the characteristics of usability testing is that it is controlled by the evaluator and takes place under laboratory conditions where there are no disturbing elements. The user is recorded when doing the task and the video is used for later analysis.  User performance is then quantified and put up in tables or usability specifications that are used for evaluating the software. Even though data is quantified the number of users are too small to do a statistical analysis (ibid. p. 342).

Considerations about our approach

A script that shortly introduces the test should be made and presented for each user so that all the users are carried through the same procedure and differences are eliminated (ibid. 434).

Ideas for introduction script:

·      Tak fordi du vil deltage I usability test af eLL’s website

·      Data fra testen vil blive brugt til at evaluere og forbedre websiden

·      Der vil blive udarbejdet en rapport, som kun bliver delt inden for AAU’s “fire vægge”

·      Du får en række opgaver, som vi vil bede dig løse på hjemmesiden

·      Vi skal bede dig tænke højt mens du udfører opgaverne og testlederen vil stille spørgsmål undervejs

·      Du skal ikke skynde dig igennem opgaverne, bare tag dig den tid du har brug for og sig til, hvis du ikke kan løse en opgave, så går vi videre til den næste

·      Testen vil blive videofilmet og skærmen du udfører opgaverne på bliver ligeledes filmet, videoerne bliver ikke offentliggjort og du må være anonym I rapporten såfremt du ønsker det

·      Vi skal påpege at vi ikke ønsker at teste dine færdigheder inden for IT, men udelukkende hvor brugervenlig web siden er

·      Deltageren får evt. en samtykkeerklæring som han/hun skal underskrive

Rogers et al. also recommend letting the participants start with an explorations task to get familiar with the web site (ibid. p. 435). We have decided that the test should last maximum 30 min. so that we would not take too much of the participants’ time and thereby make it easier to find participants. With the limited time we will not spend too much time on an exploration task. Besides that the users chosen for the usability test are all familiar with eLL’s website or at least eLL as organisation. Instead of at exploration task we will start with an easy task that is loosely defined and very much up to the user how to complete.

According to Rogers et. al. a script for the evaluator to use during the test should also be carried out (ibid.). By following the script an even procedure for all the participants are ensured. Besides that the evaluator are reminded to stay within the time schedule so that the participant does not spend too much time on one task. An estimated amount of time for each task has to be decided by the evaluators before the test.

Ideas for evaluator script:

·      Hvad tænker du om X, når du klikker på X, når du ser X?

·      Kan du fortælle mig hvad du tænker nu?

·      Er du gået I stå?

·      (Hvis opgaven overskrider den estimerede tid:) Jeg bliver desværre nødt til at bede dig om at gå videre til næste opgave.

·      Evaluatoren skal være opmærksom på følgende:

o   Yd først assistance til en opgave når deltageren har gjort adskillige forsøg selv og er gået helt I stå

o   Giv ikke hjælpende hints undervejs

o   Mere?

When making the specific tasks for the test Morville & Rosenfeld have some recommendations for the evaluator (2007, p. 260). Inspired by these recommendations we will, as mentioned, let the participants start out with an open “task”. In relation to that Morville & Rosenfeld recommends to make some topical questions and not only specific tasks (ibid.). Even though the usability test is an artificial setting the evaluator should build in some realistic tasks (ibid.). In the evaluation of eLL’s web site it could be necessary to make specific task for the specific user group taking this recommendation into consideration.   

Events

We are working with four pages in the app for startvaekst.dk. The pages are: a wiki, events, contact and news. I have been working with the page, events, and made a suggestion for a blueprint, wireframe and content inventory list.

The blueprint:

The functions that should be available at this page are:

• Arrangementer i listevisning med mulighed for:
• Føj til favoritter
• Pop-op med arr. beskrivelse
• Herover en header bestående af:
• Hjem (man kommer tilbage til forsiden)
• Vælg uge nr.
• Favoritter
• Find et arr. tæt på mig (GPS)
•  Under favoritter er der ligeledes en listevisning med mulighed for:
• Pop-op med arr. beskrivelse
• Headeren i Favoritter består af
• Hjem (man kommer tilbage til forsiden)
  
• Vælg uge nr.
• Tilbage (til arrangementer, listevisning)
• Find et arr. tæt på mig (GPS)
• Under Find et arr. tæt på mig er der ligeledes en listevisning med mulighed for:
• Føj til favoritter
• Pop-op med arr. beskrivelse 
• Headeren i Find et arr. tæt på mig består af:
• Hjem (man kommer tilbage til forsiden)
  
• Vælg uge nr.
• Favoritter
• Tilbage (til arrangementer, listevisning)
  

Playing in the sandbox

Today we kick-started the design phase with a little play in the sandbox – a tool we use in the design lab to generate ideas.  

Here is a little summary of what we talked about.

There are different where the app for Startvækst can be useful. For example when you are on the go or just have to check something quickly and don’t want to bother turn on your computer.

When a thought strokes your mind you can quickly find relevant information with the app and maybe get additional information sent to your email.

When you get a sudden idea - for example on a café talking to your friends - you can get information from the app.

There a two situations where the app can be useful: 1) when you get a sudden idea and have to get information about something or 2) when you have some time waiting on the train, the bus or for the wife to be ready to go, you can read the latest news or check when there is an event close to you. 

Literature from webcredible.com

Useful literature when creating an app or mobile site

7 usability guidelines for websites on mobile device

1.     Reduce the amount of content

2.     Single column layout – use the whole width of the screen and scroll downwards not across

3.     Different navigation – place navigation at the top or bottom, dropdowns, back-bottons

4.     Minimize text entry – minimize typing, use stored details, pin instead of password,

5.     Consider more mobile sites – for mobiles with different screen and processing power

6.     Design for both touch screen and non-touchscreen users – separate object to be clicked on

7.     Take advantage for inbuilt functionalities – calls, map, GPS, QR-codes

How to study mobile behaviour – an ethnographic perspective

Qualitative study of peoples’ mobile behaviour on the go

Insights about mobility (quoted)

·      A powerful motivation for using smartphones is the need for civil inattention, i.e. to appear engaged with something while travelling, waiting, or in situations when we are forced to be together with others while on our own.

·      Activities undertaken on the go are often interrupted and performed in short spurs, sometimes while also doing something else.

·      The time between stops is, in a subjective way, both the commuter's unit of time on the tube and also the length of their attention span when consuming content or performing a task on their phones.

Findings about mobile shopping behaviour (quoted)

·      Participants wanted to see some products in a big screen or, even better, in store before buying them, but some participants used their phone to buy something online soon after they saw it in a store!

·      Smartphones were used for "having a little browse" anytime participants felt too lazy to go downstairs and switch on the computer, but they still used their computer to carry out "important" tasks such as purchasing something very expensive.

·      As with most things designed, participants took "ownership" of their apps and used them in a way that makes more sense to them: we talked to people that use Foursquare just to read the reviews instead of checking in, or Twitter to find cheap flights.

·      Many participants were keen to use apps that replace human interaction, such as paying a their bill via the Pizza Express app instead of waiting for the waiter to come back with a card machine

·      People struggled a bit to find the right apps among all those installed on their phones, or at least to find them fast enough to use them on the go. Some of them devised interesting strategies to classify and remember them.

Six mistakes to avoid in mobile usability

Most known mistakes that people make when they are creating a mobile presence for their business

1.     Not adapting to the mobile platform – making mobile web-sites as if designing for at normal web site

2.     Missing the point – conduct a survey of what your customers want

3.     Failing to ask the target audience – test on target users!

4.     Over complicating things – keep it simple, used the most up-to-date software, limit hyperlink on each page to max 10

5.     Bloated sites – make it quickly accessible, not too much content which is slowly downloaded

6.     Jumping in without research – see what your competitors are doing (benchmarking)

Research: Content

We are approaching the research phase from three angles: context, user and content following the frame work recommended by Morville & Rosenfeldt

When researching content it is recommendable to look at either the homepage that has to be improved or related homepages. Even though the navigation and content of a homepage seems inexpedient there can be valuable objects to work with.

Designing an app is very different from designing a homepage, but still I think Morville and Rosenfeldts “content analysis” can inspire us to make our own content analysis of starvaeks.dk.

Our approach

• Find objects on startvaekst.dk, which could be useful content of an app – mention all you could think of!

• Have the following guidelines in mind: 1) Reduce the content, 2) Minimize the need of typing, 3) Use the functionalities of the mobile device (call, address to map, GPS, quick response - QR)

• When objects have been found, discuss:

• What is this object?
• How can I describe this object? 
• What distinguishes this object from others? 
• How can I make this object findable?

• Look for patterns and relationships
• Are certain groupings of content becoming apparent? 
• Are you seeing clear hierarchical relationships? 
• Are you recognizing the potential for associative relationships, perhaps finding disparate items that are linked by a common business process?

• Benchmarking
• In which ways can we use the structure of the homepage? 
• Are there already designed apps that we can use as bench markers? 

Findings from data conducted by Starvækst to improve their homepage:
Increased focus on the local networks
Overview of events – maybe in a calendar
Podcasts etc.
Reference work – maybe under themes
”Situation entrances”
Articles often misunderstood (news)
Registration
Contact
FAQ


Discuss: what are the goals of this app that we are designing?

Case 2: designing an app for Startvækst

Yesterday we were introduced to the second case that we are going to work with this semester on information architecture. As the headline says, we are going to design an app for Startvækst – a site for Danish entrepreneurs. The casework runs from now on until the 26th this month where we are going to present a suggestion for a design and a report with an explanation and argumentation for the design.

Our approach to the process will be following Morville & Rosenfeldts recommendations about the process of designing an information architecture of a web site.

Reflections about the DAVID project

Last week we finished the DAVID project by writing a paper about the challenges that are by overcoming risks in mega project. The paper focussed on the role of the information architect in mega projects. More concrete how the information architect can help identify potential risks already in the decisions making phase.

I was thrown into the DAVID project a few weeks after Helene and Karina started working with it and it has been a confusing process not really knowing exactly what we were doing. But I also think it has been a process very close to one I could meet in my future work life.

Ellen told us to reflect upon the knowledge and the competences that we have gained and try to separate these two concepts. I think this is a difficult thing to do because knowing what to do theoretically does not mean that you are capable of doing that in a real situation. In this project we have had the chance to actually act in a “real” context and that has been really educational. In the following I will shortly try to list the knowledge and competences that I think I have gained during this process:

Knowledge

• Insight in the potential risks in mega projects and a theoretical knowledge from the book: Flyvbjerg et al: Risk management in megaprojects.
• Knowledge about the information architects role in mega projects and how the information architect can help optimizing the process by identifying risks. Especially by making sure that a performance specification is carried out.
• A theoretical framework for how to make sure that at project group share a common language and works towards common goals

Competences

• Identifying the information architects role in a mega project and keep the focus on the specific task instead of loosing the overview because of information overload
• Be critical and always keep an eye on worst case scenarios
• Be specific about goals by illustrating where the project is heading
• See the project in a context and furthermore a long-term perspective