Designing for participatory projects: emergent best practice, getting discussion started

I was invited over to New Zealand (from Australia) recently to talk at Te Papa in Wellington and the Auckland Museum.  After the talks I was asked if I could share some of my notes on design for participatory projects and for planning for the impact of participatory projects on museums.  Each museum has a copy of my slides, but I thought I'd share the final points here rather than by email, and take the opportunity to share some possible workshop activities to help museums plan audience participation around its core goals.

Both talks started by problematising the definition of a 'museum website' - it doesn't work to think of your 'museum website' as purely stuff that lives under your domain name when it's now it's also the social media accounts under your brand, your games and mobile apps, and maybe also your objects and content on Google Art Project or even your content in a student’s Tumblr.  The talks were written to respond to the particular context of each museum so they varied from there, but each ended up with these points.  The sharp-eyed among you might notice that they're a continuation of ideas I first shared in my Europeana Tech keynote: Open for engagement: GLAM audiences and digital participation.  The second set are particularly aimed at helping museums think about how to market participatory projects and sustain them over the longer term by making them more visible in the museum as a whole.

Best practice in participatory project design

• Have an answer to 'Why would someone spend precious time on your project?'
• Be inspired by things people love
• Design for the audience you want
• Make it a joy to participate
• Don't add unnecessary friction, barriers (e.g. don't add sign-up forms if you don’t really need them, or try using lazy registration if you really must make users create accounts)
• Show how much you value contributions (don't just tell people you value their work)
• Validate procrastination - offer the opportunity to make a difference by providing meaningful work
• Provide an easy start and scaffolded tasks (see e.g. Nina Simon's Self-Expression is Overrated: Better Constraints Make Better Participatory Experiences)
• Let audiences help manage problems - let them know which behaviours are acceptable and empower them to keep the place tidy
• Test with users; iterate; polish

Best practice within your museum

• Fish where the fish are - find the spaces where people are already engaging with similar content and see how you can slot in, don't expect people to find their way to you unless you have something they can’t find anywhere else
• Allow for community management resources - you’ll need some outreach to existing online and offline communities to encourage participation, some moderation and just a general sense that the site hasn’t been abandoned. If you can’t provide this for the life of the project, you might need to question why you’re doing it.
• Decide where it's ok to lose control. Try letting go... you may find audiences you didn't expect, or people may make use of your content in ways you never imagined. Watch and learn and tweak in response – this is a good reason to design in iterations, and to go into public or invited-beta earlier rather than later. 
• Realistically assess fears, decide acceptable levels of risk. Usually fears can be turned into design requirements, they’re rarely show-stoppers.
• Have a clear objective, ideally tied to your museum’s mission. Make sure the point of the project is also clear to your audience.
• Put the audience needs first. You’re asking people to give up their time and life experience, so make sure the experience respects this. Think carefully before sacrificing engagement to gain efficiency.
• Know how to measure success
• Plan to make the online activity visible in the organisation and in the museum. Displaying online content in the museum is a great way to show how much you value it, as well as marketing the project to potential contributors.  Working out how you can share the results with the rest of the organization helps everyone understand how much potential there is, and helps make online visitors ‘real’.
• Have an exit strategy - staff leave, services fold or change their T&Cs

I'd love to know what you think - what have I missed?  [Update: for some useful background on the organisational challenges many museums face when engaging with technology, check out Collections Access and the use of Digital Technology (pdf).]

More on designing museum projects for audience participation

I prepared this activity for one of the museums, but on the day the discussion after my talk went on so long that we didn't need to use a formal structure to get people talking. In the spirit of openness, I thought I'd share it. If you try it in your organisation, let me know how it goes!

The structure - exploratory idea generation followed by convergence and verification - was loosely based on the 'creativity workshops' developed by City University's Centre for Creativity (e.g. the RESCUE creativity workshops discussed in Use and Influence of Creative Ideas and Requirements for a Work-Integrated Learning System).  It's designed to be a hackday-like creative activity for non-programmers.

In small groups...

• Pick two strategic priorities or organisational goals...
• In 5 minutes: generate as many ideas as possible
• In 2 minutes: pick one idea to develop further

Ideas can include in-gallery and in-person activity; they must include at least two departments and some digital component.

Developing your idea...
Ideas can include in-gallery and in-person activity; they must include at least two departments

• You have x minutes to develop your idea
• You have 2 minutes each to report back. Include: which previous museum projects provide relevant lessons? How will you market it? How will it change the lives of its target audience? How will it change the museum?
• How will you alleviate potential risks?  How will you maximise potential benefits?
• You have x minutes for general discussion. How can you build on the ideas you've heard?

For bonus points...

These discussion points were written for another museum, but they might be useful for other organisations thinking about audience participation and online collections:

What are the museum’s goals in engaging audiences with collections online?

• What does success look like?
• How will it change the museum?
• Which past projects provide useful lessons?

How can the whole organisation be involved in supporting online conversations?

• What are the barriers?
• What small, sustainable steps can be taken?
• Where are online contributions visible in the museum?

Geek for a week: residency at the Powerhouse Museum

I've spent the last week as 'geek-in-residence' with the Digital, Social and Emerging Technologies team at the Powerhouse Museum. I wasn't sure what 'geek-in-residence' would mean in reality, but in this case it turned out to be a week of creativity, interesting constraints and rapid, iterative design.

When I arrived on Monday morning, I had no idea what I'd be working on, let alone how it would all work. By the end of the first day I knew how I'd be working, but not exactly what I'd focus on. I came in with fresh questions on Tuesday, and was sketching ideas by lunchtime. The next few days were spent getting stuck into wireframes to focus in on specific issues within that problem space; I turned initial ideas into wireframes and basic copy; and put that through two rounds of quick-and-dirty testing with members of the public and Powerhouse volunteers. By the time I left on Friday I was able to handover wireframes for a site called 'conversations about collections' which aims to record people's memories of items from the collection. (I ran out of time to document the technical aspects of how the site could be built in WordPress, but given the skills of the team I think they'll cope.)

The first day and a half were about finding the right-sized problem. In conversations with Paula (Manager of the Visual & Digitisation services team) and Luke (Web Manager), we discussed what each of us were interested in exploring, looking for the intersection between what was possible in the time and with the material to hand.

After those first conversations, I went back to Powerhouse's strategy document for inspiration. If in doubt, go back to the mission! I was looking for a tie-in with their goals - luckily their plan made it easy to see where things might fit. Their strategy talked about ideas and technology that have changed our world and stories of people who create and inspire them, about being open to 'rich engagement, to new conversations about the collections'.

I also considered what could be supported by the existing API, what kinds of activities worked well with their collections and what could be usefully built and tested as paper or on-screen prototypes.  Like many large collections, most of the objects lack the types of data that supports deeper engagement for non-experts (though the significance statements that exist are lovely).

Two threads emerged from the conversations: bringing social media conversations and activity back into the online collections interfaces to help provide an information scent for users of the site; and crowdsourcing games based around enhancing the collections data.
The first was an approach to the difficulties in surfacing the interesting objects in very large collections. Could you create a 'heat map' based on online activity about objects to help searchers and browsers spot objects that might be more interesting?

At one point Nico (Senior Producer) and I had a look at Google Analytics to see what social media sites were sending traffic to the collections and suss out how much data could be gleaned. Collection objects are already showing up on Pinterest, and I had wild thoughts about screen-scraping Pinterest (they have no API) to display related boards on the OPAC search results or object pages...

I also thought about building a crowdsourcing game that would use expert knowledge to data to make better games possible for the general public – this would be an interesting challenge, as open-ended activities are harder to score automatically so you need to design meaningful rewards and ensure an audience to help provide them. However, it was probably a bigger task than I had time for, especially with most of the team already busy on other tasks, though I've been interested in that kind of dual-phased project since my MSc project on crowdsourcing games for museums.

But in the end, I went back to two questions: what information is needed about the collections, what's the best way to get it?  We decided to focus on conversations, stories and clues about objects in the collections with a site aimed at collecting 'living memories' about objects by asking people what they remember about an object and how they'd explain it to a kid.  The name, 'Conversations about collections' came directly from the strategy doc and was just too neat a description to pass up, though 'memory bank' was another contender.
I ended up with five wireframes (clickable PDF at that link) to cover the main tasks of the site: to persuade people (particularly older people) that their memories are worth sharing, and to get the right object in front of the right person.  Explaining more about the designs would be a whole other blog post, but in the interests of getting this post out I'll save that for another day... I'm dashing out this post before I head out, but I'll update in response to questions (and generally things out when I have more time).

My week at the Powerhouse was a brilliant chance to think through the differences between history of science/social history objects and art objects, and between history and art museums, but that's for another post (perhaps when if I ever get around to posting my notes from the MCN session on a similar topic).
It also helped me reflect on my interests, which I would summarise as 'meaningful audience participation' - activities that are engaging and meaningful for the audience and also add value for the museum, activities that actually change the museum in some way (hopefully for the better!), whether that's through crowdsourcing, co-curation or other types of engagement.

Finally, I owe particular thanks to Paula Bray and Luke Dearnley for running with Seb Chan's original suggestion and for their time and contributions to shaping the project; to Nicolaas Earnshaw for wireframe work and Suse Cairns for going out testing on the gallery floor with me; and to Dan Collins, Estee Wah, Geoff Barker and everyone else in the office and on various tours for welcoming me into their space and their conversations.

Usability: the key that unlocks geeky goodness

This is a quick pointer to three posts about some usability work I did for the JISC-funded Pelagios project, and a reflection on the process. Pelagios aims to 'help introduce Linked Open Data goodness into online resources that refer to places in the Ancient World'. The project has already done lots of great work with the various partners to bring lots of different data sources together, but they wanted to find out whether the various visualisations (particularly the graph explorer) let users discover the full potential of the linked data sets.

I posted on the project blog about how I worked out a testing plan to encourage user-centred design and set up the usability sessions in Evaluating Pelagios' usability, set out how a test session runs (with sample scripts and tasks) in Evaluating usability: what happens in a user testing session? and finally I posted some early Pelagios usability testing results. The results are from a very small sample of potential users but they were consistent in the issues and positive results uncovered.

The wider lesson for LOD-LAM (linked open data in library, archives, museums) projects is that user testing (and/or a strong user-centred design process) helps general audiences (including subject specialists) appreciate the full potential of a technically-led project - without thoughtful design, the results of all those hours of code may go unloved by the people they were written for. In other words, user experience design is the key that unlocks the geeky goodness that drives these projects. It's old news, but the joy of user testing is that it reminds you of what's really important...

Design constraints and research questions: museum metadata games

Back in June I posted parts of my dissertation project outline in 'Game mechanics for social good: a case study on interaction models for crowdsourcing museum collections enhancement'. Since then, I've been getting on with researching, designing, building and evaluating museum metadata games (in my copious spare time after work, in a year when we launched three major galleries).

I'm planning to blog bits of my dissertation as I write it up so there'll be more posts over the next month, but for now I wanted to contextualise the two games I'm evaluating at the moment.  In the next post I'll talk about the changes I made after the first solid round of evaluation.

Casual games
The two games, nicknamed 'Dora' and 'Donald' are designed as casual games - something you can pick up and play for five minutes at a time.  Design goals included: an instantly playable game that provides stress relief, supports a competitive spirit (but not necessarily against other people), inherently rewarding experience, simple game play and puts 'fun before do-gooding'.  The games were designed around a specific research-based persona ('Meet Janet', pdf link) - hopefully it's exactly right for some people who are close to the persona in various ways, and quite fun for a wider group.  It won't suit everyone, not least because definitions of 'fun' and expectations around 'games' can be deeply individual.

Design constraints
The games are also designed to test ideas about the types of objects and records that can be used successfully, and the types of content people would be able to contribute about the less charismatic and emotionally accessible reaches of science, technology and social history collections - this means that some of the objects I've used are quite technical, not all the images are great and small variations on object records are repeated (risking 'not another bloody telescope').  While this might match the reality of museum catalogues, would it still allow for a fun game?

The realities of a project I was building in my free time and my lack of graphic design and illustration skills also provided constraints - it had to be browser-based, it couldn't rely on a critical mass of concurrent players to validate actions or content, it had to help the player dive straight into playing and overcome any fears about creating content about museum objects, and it had to use objects ingested through available museum APIs (I selected broad subjects for testing but didn't individually select any objects).

I then added a few extra constraints by deciding to build it as a WordPress plugin - I wanted to take advantage of the CMS-like framework for user logins, navigation and page layout, and I wanted the code I wrote to be usable by others without too much programming overhead.  I'll need to tidy up the code at the end, but once that's done you should be able to install it on any hosted WordPress installation.  I'm making a related plugin to help you populate the database with objects (also part of an experiment in the effectiveness of letting people choose their own subject areas or terms to select playable objects).  I'll talk more about how I worked with those constraints and how they informed the changes I made after evaluation in a later post.

Different games for different purposes
I've been thinking about a museum metadata game typology, which not only considers different types of fun, but also design constraints like:

• the type and state of the collection (e.g. art works, technical/specialist and social history objects; photographs and other media vs objects; reference collections vs selected highlights; 'tombstone' vs general vs interpretative records)
• the type of data sought including information curators could add if they had infinite time (detail on the significance of the object, links to other subjects, people, events, objects, collections, etc); information that can be extrapolated from the existing catalogue record; things curators couldn't know (personal history, experiential accounts about the design, manufacture, use, disposal etc of objects); emotional responses; external specialist knowledge; amateur/hobbyist specialist knowledge; synonyms in every day language; terms in other spoken languages

I've also been playing with the idea of linking different game types to different 'life stages' of museum collection metadata.  For example, some games could help a museum work out which of its catalogued items seem more interesting to the public, others help gather tags, create links between items or encourage players to research objects and record new information or links about them, and others still could work well for validating data created in earlier games.  The data I gather through evaluating the games I've designed will help test this model.

So, all that said, if you'd like to play (and help with my evaluation), the two games are:

Donald's detective puzzle - find a fact about an object
Dora's lost data - a simpler tagging game

What do you mean by 'wireframe'?

This post on 'The future of wireframes?' chimed a few bells, not only because I'm revising for a Requirements Engineering exam but because I've been in the start-up phase for projects of all sizes lately and have been thinking hard about the best way to understand and communicate requirements. In doing so, I've realised that 'wireframes' has become one of those terms that mean different things to different people - and that of course, it's an entirely new term to people who haven't worked on a design phase of a digital project before. This summed up past and current definitions neatly:

For many years the primary role of wireframes was to specify software. We now use wireframes to investigate and explore how people will interact with a site. Using a ‘just enough’ approach, we often create a series of simple interactive prototypes to try out a variety of approaches to solving a problem. These prototypes can be made in HTML or they can be as simple as a series of Keynote slide for someone to click through.

This is a very different approach to wireframing. Rather than simply documenting where a link goes, the goal is to model and start experiencing what moving around a site feels like as quickly as possible. The prototype can then be tested and the results used to iteratively improve the end solution.

Of course, sites still need to be specified, but wireframes aren’t always the right tool for doing this.

Here's a list of wireframe and prototype tools - do you have any favourites?

A rare post from me - I've been completely caught up in work and my MSc for the past few months. Normal service will be resumed soon - I've still got to report on UKMW09 and a trip to Oslo to give a lecture on social media and museums, libraries and archives.

Get thee to a wiki - the great API challenge in action

Help us work on an informal, lightweight way of devising shared data, API standards for museum and cultural heritage organisations - museum-api.pbwiki.com is open for business.

You could provide examples of APIs you've used or produced, share your experience as a consumer of web services, tell us about your collections.

Commenting on other people's queries and content is an easy way to get started.  I'd particularly love to hear from curators and collections managers - we should be working together to enable greater access to collections.  If you check it out and none of it makes any sense - be brave and say so!  We should be able to explain what we're doing clearly, or we're not doing it right.

Some background: as announced on the nascent museumdev blog, the Science Museum is looking at releasing an API soon - it'll be project-specific to start with, but we're creating it with the intention of using that as an iterative testing and learning process to design an API for wider use. We could re-invent the wheel, but we'd rather make it easy for people to use what they've learnt using other APIs and other museum collections - the easiest way to do that is to work with other museums and developers. The Science Museum's initial public-facing collections API will be used for a 'mashup competition' based on object metadata from our 'cosmos and culture' gallery.

Speaking of museumdev, I started it as somewhere where I could ask questions, point people to discussions, a home for collections of links and stuff in development.  It's also got random technical bits like 'Tip of the Day: saving web.config as Unicode' because I figure I might as well share my mistakes^H^H^H^H^H^H^H^H learning experiences in the hope that someone, somewhere, benefits.

What would you create with public (UK) information?

Show Us a Better Way want to know, and if your idea is good they might give you £20,000 to develop it to the next level.

Do you think that better use of public information could improve health, education, justice or society at large?

The UK Government wants to hear your ideas for new products that could improve the way public information is communicated.

Importantly, you don't need to be a geek:

You don't have to have any technical knowledge, nor any money, just a good idea, and 5 minutes spare to enter the competition.

And they've made "gigabytes of new or previously invisible public information" available for the project, including health, crime and education data (but no personal information).

Nice information design/visualisation pattern browser

infodesignpatterns.com is a Flash-based site that presents over 50 design patterns 'that describe the functional aspects of graphic components for the display, behaviour and user interaction of complex infographics'.

The development of a design pattern taxonomy for data visualisation and information design is a work in progress, but the site already has a useful pattern search, based on order principle, user goal, graphic class and number of dimensions.

Play with your customer profiles

It's a bit early for a random Friday fun link, but this Forrester 'Build Your Customers' Social Technographics Profile' interactive counts as work too.

Companies often approach Social Computing as a list of technologies to be deployed as needed — a blog here, a podcast there — to achieve a marketing goal. But a more coherent approach is to start with your target audience and determine what kind of relationship you want to build with them, based on what they are ready for. You can use the tool on this page to get started.

You can pull down menus to change the age group, country and gender of your target audience, and the graph below updates to show you how many are in each 'Social Technographics' group.

The definitions of the 'Social Technographics' groups are given in a slideshow.

Hat tip to Nina Simon. [Update to get Nina's name right, I'm very sorry!]

Let's help our visitors get lost

In 'Community: From Little Things, Big Things Grow' on ALA, George Oates from Flickr says:

It's easy to get lost on Flickr. You click from here to there, this to that, then suddenly you look up and notice you've lost hours. Allow visitors to cut their own path through the place and they'll curate their own experiences. The idea that every Flickr visitor has an entirely different view of its content is both unsettling, because you can't control it, and liberating, because you’ve given control away. Embrace the idea that the site map might look more like a spider web than a hierarchy. There are natural links in content created by many, many different people. Everyone who uses a site like Flickr has an entirely different picture of it, so the question becomes, what can you do to suggest the next step in the display you design?

I've been thinking about something like this for a while, though the example I've used is Wikipedia. I have friends who've had to ban themselves from Wikipedia because they literally lose hours there after starting with one innocent question, then clicking onto an interesting link, then onto another...

That ability to lose yourself as you click from one interesting thing to another is exactly what I want for our museum sites: our visitor experience should be as seductive and serendipitous as browsing Wikipedia or Flickr.

And hey, if we look at the links visitors are making between our content, we might even learn something new about our content ourselves.

Notes from 'Unheard Stories – Improving access for Deaf visitors' at MCG's Spring Conference

These are my notes from the presentation 'Unheard Stories – Improving access for Deaf visitors' by Linda Ellis at the MCG Spring Conference. There's some background to my notes about the conference in a previous post.

Linda's slides for Unheard Stories – Improving access for Deaf visitors are online.

This was a two year project, fit around their other jobs [and more impressive for that]. The project created British Sign Language video guides for Bantock House. The guides are available on mp3 players and were filmed on location.

Some background:
Not all 'deaf' people are the same - there's a distinction between 'deaf' and 'Deaf'. The notation 'd/Deaf' is often used. Deaf people use sign language as their first language and might not know English; deaf people probably become deaf later in life, and English is their first language. The syntax of British Sign Language (BSL) is different to English syntax. Deaf people will generally use BSL syntax, but deaf people might use signs with English grammar. Not all d/Deaf people can lip-read.

Deaf people are one of the most excluded groups in our society. d/Deaf people can be invisible in society as it's not obvious if someone is d/Deaf. British sign language was only recognised as an official language in March 2003.

Their Deaf visitors said they wanted:
Concise written information; information in BSL; to explore exhibits independently; stories about local people and museum objects; events just for Deaf people (and dressing up, apparently).

Suggestions:
Put videos on website to tell people what to expect when they visit. But think about what you put on website - they're Deaf, not stupid, and can read addresses and opening hours, etc. Put a mobile number on publicity so that Deaf people can text about events - it's cheap and easy to do but can make a huge difference. If you're doing audience outreach with social software, don't just blog - think about putting signed videos on YouTube. Use local Deaf people, not interpreters. Provide d/Deaf awareness training for all staff and volunteers. Provide written alternatives to audio guides; add subtitles and an English voice over signed video if you can afford it.

Notes from 'Rhagor - the collections based website from Amgueddfa Cymru' at MCG's Spring Conference

There are my notes from the presentation 'Rhagor - the collections based website from Amgueddfa Cymru' by Graham Davies at the MCG Spring meeting.

This paper talked about the CMS discussed in Building a Bilingual CMS.

'Rhagor' is Welsh for more - the project is about showing more of the collections online. It's not a 'virtual museum'.

With this project, they wanted to increase access to collections and knowledge associated with those collections; to explain more about collections than can be told in galleries with space limitations; and to put very fragile objects online.

[He gave a fascinating example of a 17th century miniature portrait with extremely fragile mica overlays - the layers have been digitised, and visitors to the website can play dress-up with the portrait layers in a way that would never be physically possible.]

The site isn't just object records, it also has articles about them. There's a basic article structure (with a nice popout action for images) that deals with the kinds of content that might be required. While developing this they realised they should test the usability of interface elements with general users, because the actions aren't always obvious to non-programmers.

They didn't want to dumb down their content so they've explain with a glossary where necessary. Articles can have links to related articles; other parts of the website and related publications, databases etc. Visitors can rate articles - a nice quick simple bit of user interactivity. Visitors can share articles on social networking sites, and the interface allows them to show or hide comments on site. Where articles are geographically based, they can be plotted onto a map. Finally, it's all fully bilingual. [But I wondered if they translate comments and the replies to them?]

In their next phase they want to add research activities and collections databases. They're also reaching out to new audiences through applications like Flickr and Google Earth, to go to where audiences are. If the content is available, audiences will start to make links to your content based on their interests.

The technology itself should be invisible, user has enriched experience through the content.

Questions:
Alex: to what extent is this linked with collection management system (CollMS)? Graham: it's linked to their CMS (discussed in earlier papers), not their CollMS. They don't draw directly from CollMS into CMS. Their CollMS is working tool for curators, needs lots of data cleaning, and doesn't necessarily have the right content for web audiences; it's also not bilingual.

More on cultural heritage and resistance to the participatory web

I've realised that in my post on 'Resistance to the participatory web from within the cultural heritage sector?', I should have made it clear that I wasn't thinking specifically of people within my current organisation. I've been lucky enough to meet a range of people from different institutions at various events or conferences, and when I get a chance I keep up with various cultural heritage email discussion lists and blogs. One way or another I've been quietly observing discussions about the participatory web from a wide range of perspectives within the cultural heritage and IT sectors for some time.

Ok, that said, the responses have been interesting.

Thomas at Medical Museion said:

This are interesting observations, and I wonder: Can this resistance perhaps be understood in terms of an opposition among curators against a perceived profanation of the sacred character of the museum? In the same way as Wikipedia and other user-generated content websites have been viewed with skepticism from the side of many academics — not just because they may contain errors (which encyclopedia doesn’t?), but also because it is a preceived profanation of Academia. (For earlier posts about profanation of the museum as a sacred institution, see here and here.). Any ideas?

I'm still thinking about this. I guess I don't regard museums as sacred institutions, but then as I don't produce interpretative or collection-based content that could be challenged from outside the institution, I haven't had a vested interest in retaining or reinforcing authority.

Tom Goskar at Past Thinking provided an interesting example of the visibility and usefulness of user-generated content compared to official content and concluded:

People like to talk about ancient sites, they like to share their photos and experiences. These websites are all great examples of the vibrancy of feeling about our ancient past.

For me that's one of the great joys of working in the cultural heritage sector - nearly everyone I meet (which may be a biased sample) has some sense of connection to museums and the history they represent.

The growth of internet forums on every topic conceivable shows that people enjoy and/or find value in sharing their observations, opinions or information on a range of subjects, including cultural heritage objects or sites. Does cultural heritage elicit a particular response that is motivated by a sense of ownership, not necessarily of the objects themselves, but rather of the experience of, or access to, the objects?

It seems clear that we should try and hook into established spaces and existing conversations about our objects or collections, and perhaps create appropriate spaces to host those conversations if they aren't already happening. We could also consider participating in those conversations, whether as interested individuals or as representatives of our institutions.

However institutional involvement with and exposure to user-generated content could have quite different implications. It not only changes the context in which the content is assessed but it also lends a greater air of authority to the dialogues. This seems to be where some of the anxiety or resistance to the participatory web resides. Institutions or disciplines that have adapted to the idea of using new technologies like blogs or podcasts to disseminate information may baulk at the idea that they should actually read, let alone engage with any user-generated content created in response to their content or collections.

Alun wrote at Vidi:

Interesting thoughts on how Web 2.0 is or isn’t used. I think one issue is a question of marking authorship, which is why Flickr may be more acceptable than a Wiki.

I think that's a good observation. Sites like Amazon also effectively differentiate between official content from publishers/authors and user reviews (in addition to 'recommendation'-type content based on the viewing habits of other users).

Another difference between Flickr and a wiki is that the external user cannot edit the original content of the institutional author. User-generated content sites like the National Archives wiki can capture the valuable knowledge generated when external people access collections and archives, but when this user-generated content is intermingled with, and might edit or correct, 'official' content it may prove a difficult challenge for institutions.

The issue of whether (and how) museums respond to user-generated content, and how user-generated content could be evaluated and integrated with museum-generated content is still unresolved across the cultural heritage sector and may ultimately vary by institution or discipline.

Integrating Accessibility Throughout Design

Integrating Accessibility Throughout Design is a great resource for thinking about how to incorporate accessibility testing in user-centered design processes. It's available as a website and a book, that covers:

• The basics of including accessibility in design projects
  • Shortcuts for involving people with disabilities in your project
  • Tips for comfortable interaction with people with disabilities
• Details on accessibility in each phase of the user-centered design process (UCD)
  • Examples of including accessibility in user group profiles, personas, and scenarios
  • Guidance on evaluating for accessibility through heuristic evaluation, design walkthroughs, and screening techniques
  • Thorough coverage of planning, preparing for, conducting, analyzing, and reporting effective usability tests with participants with disabilities
  • Questions to include in your recruiting screener
  • Checklist for usability testing with participants with disabilities

Two unrelated posts I've liked recently, on Navigators, Explorers, and Engaged Participants as user models; and going back to basics for digital museum content:

We don't need new technologies to attract teen audiences, we need, if anything, to revisit how we (and others) interpret our collections and ideas and then decide what new technologies can best convey the information

Usability articles at Webcredible

The article on 10 ways to orientate users on your site is useful because more and more users arrive at our sites via search engines or deep links. Keeping these tips in mind when designing sites helps us give users a sense of the scope, structure and purpose of a website, no matter whether they start from the front page or three levels down.

How to embed usability & UCD internally "offers practical advice of what a user champion can do to introduce and embed usability and user-centered design within a company" and includes 'guerrilla tactics' or small steps towards getting usability implemented. But probably the most important point is this:

The most effective method of getting user centered design in the process is through usability testing. Invite key stakeholders to watch the usability testing sessions. Usability testing is a real eye-opener and once observed most stakeholders find it difficult to ignore the user as part of the production process. (The most appropriate stakeholders are likely to be project managers, user interface designs, creative personnel, developers and business managers.)

I would have emphasised the point above even if they hadn't. The difference that usability testing makes to the attitudes of internal stakeholders is amazing and can really focus the whole project team on usability and user-centred design.

Collected links and random thoughts on user testing

First, some links on considerations for survey design and quick accessibility testing.

Given the constraints of typical museum project budgets, it's helpful to know you can get useful results with as few as five testers. Here's everybody's favourite, Jakob Nielsen, on why you can do usability testing with only five users, card sorting exercises for information architecture with 15 users and quantitative studies with 20 users. Of course, you have to allow for testing for each of your main audiences and ideally for iterative testing too, but let's face it - almost any testing is better than none. After all, you can't do user-centred design if you don't know what your users want.

There were a few good articles about evaluation and user-centred design in Digital Technology in Japanese Museums, a special edition of the Journal of Museum Education. I particularly liked the approach in "What Impressions Do People Have Regarding Mobile Guidance Services in Museums? Designing a Questionnaire that Uses Opinions from the General Public" by Hiromi Sekiguchi and Hirokazu Yoshimura.

To quote from their abstract: "There are usually serious gaps between what developers want to know and what users really think about the system. The present research aims to develop a questionnaire that takes into consideration the users point of view, including opinions of people who do not want to use the system". [my emphasis]

They asked people to write down "as many ideas as they could - doubts, worries, feelings, and expectations" about the devices they were testing. They then grouped the responses and used them as the basis for later surveys. Hopefully this process removes developer- and content producer-centric biases from the questions asked in user testing.

One surprising side-effect of good user testing is that it helps get everyone involved in a project to 'buy into' accessibility and usability. We can all be blinded by our love of technology, our love of the bottom line, our closeness to the material to be published, etc, and forget that we are ultimately only doing these projects to give people access to our collections and information. User testing gives representative users a voice and helps everyone re-focus on the people who'll be using the content will actually want to do with it.

I know I'm probably preaching to the converted here, but during Brian Kelly's talk on Accessibility and Innovation at UKMW07 I realised that for years I've had an unconscious test for how well I'll work with someone based on whether they view accessibility as a hindrance or as a chance to respond creatively to a limitation. As you might have guessed, I think the 'constraints' of accessibility help create innovations. As 37rules say, "let limitations guide you to creative solutions".

One of the points raised in the discussion that followed Brian's talk was about how to ensure compliance from contractors if quantitative compliance tests and standards are deprecated for qualitative measures. Thinking back over previous experiences, it became clear to me that anyone responding to a project tender should be able to demonstrate their intrinsic motivation to create accessible sites, not just an ability to deal with the big stick of compliance, because a contractors commitment to accessibility makes such a difference to the development process and outcomes. I don't think user testing will convince a harried project manager to push a designer for a more accessible template but I do think we have a better chance of implementing accessible and usable sites if user requirements considered at the core of the project from the outset.

Why Your Web App Sucks lists basic but important reasons why your web application might suck.

Is Web 2.0 user-centred design in action?

An interesting perspective from Mike Ellis and Brian Kelly at MW2007:

Trawling the Web finds the following phrases recurring around Web 2.0: “mashup”, “de-centralisation”, “non-Web like”, “user generated content”, “permission based activity”, “collaboration”, “Creative Commons” … What sits at the heart of all of these, and one of the reasons Web 2.0 has been difficult for bigger, established organisations (including museums) to embrace, is that almost all the things talked about put users and not the organisation at the centre of the equation. Organisational structures, departmental ways of naming things, the perceived ‘value’ of our assets, in fact, what the organisation has to say about itself - all are being challenged.

Web 2.0: How to Stop Thinking and Start Doing: Addressing Organisational Barriers

Notes on usability testing

Further to my post about the downloadable usability.gov guidelines, I've picked out the bits from the chapter on 'Usability Testing' that are relevant to my work but it's worth reading the whole of the chapter if you're interested. My comments or headings are in square brackets below.

"Generally, the best method is to conduct a test where representative participants interact with representative scenarios.
...
The second major consideration is to ensure that an iterative approach is used.

Use an iterative design approach

The iterative design process helps to substantially improve the usability of Web sites. One recent study found that the improvements made between the original Web site and the redesigned Web site resulted in thirty percent more task completions, twenty-five percent less time to complete the tasks, and sixty-seven percent greater user satisfaction. A second study reported that eight of ten tasks were performed faster on the Web site that had been iteratively designed. Finally, a third study found that forty-six percent of the original set of issues were resolved by making design changes to the interface.

[Soliciting comments]

Participants tend not to voice negative reports. In one study, when using the ’think aloud’ [as opposed to retrospective] approach, users tended to read text on the screen and verbalize more of what they were doing rather than what they were thinking.

[How many user testers?]

Performance usability testing with users:
– Early in the design process, usability testing with a small number of users (approximately six) is sufficient to identify problems with the information architecture (navigation) and overall design issues. If the Web site has very different types of users (e.g., novices and experts), it is important to test with six or more of each type of user. Another critical factor in this preliminary testing is having trained usability specialists as the usability test facilitator and primary observers.
– Once the navigation, basic content, and display features are in place,
quantitative performance testing ... can be conducted

[What kinds of prototypes?]

Designers can use either paper-based or computer-based prototypes. Paper-based prototyping appears to be as effective as computer-based prototyping when trying to identify most usability issues.

Use inspection evaluation [and cognitive walkthroughs] results with caution.
Inspection evaluations include heuristic evaluations, expert reviews, and cognitive walkthroughs. It is a common practice to conduct an inspection evaluation to try to detect and resolve obvious problems before conducting usability tests. Inspection evaluations should be used cautiously because several studies have shown that they appear to detect far more potential problems than actually exist, and they also tend to miss some real problems.
...
Heuristic evaluations and expert reviews may best be used to identify potential usability issues to evaluate during usability testing. To improve somewhat on the performance of heuristic evaluations, evaluators can use the ’usability problem inspector’ (UPI) method or the ’Discovery and Analysis Resource’ (DARe) method.
...
Cognitive walkthroughs may best be used to identify potential usability issues to evaluate during usability testing.
...
Testers can use either laboratory or remote usability testing because they both elicit similar results.

[And finally]

Use severity ratings with caution."