Research Ethics: If you're a researcher, be that a PhD Student, Research Staff (Assistant / Associate / Fellow), or you're supervising a PhD student, or are a Principle Investigator, an Independent Scholar, or any other type of Researcher conducting work with human participants, then this is the section for you!
We would expect most researchers to be using methodologies that we do not have initial approval for because the situation, context, or method are non-standard or non-trivial. In this case we would expect that the applicant will know the methodology to be used along with the analysis required. While, we have included some starting points below, the domain is so broad that a complete list of methodologies including their research nuances that, we expect the applicant to have researched these by themselves.
If you only require some very specific evaluations then you can follow a very constrained set of procedures that we have pre-approval (i.e., approval can be given automatically via our online system, without referral to a central committee) for these from the University's central Ethics Committee. You should however, read and understand all text presented in the online form fields, and you can get immediate help by selecting the blue 'I' information button next to each question.
You may change any part of the ethics application, but this will mean that you must attend one of the weekly ethics meetings to justify your changes. We would only suggest that you do this if your work requires it and if you are fully conversant with the changes you will propose.
Once you have ethical approval, you MUST follow the procedures set out in the ethics form when actually doing your work with participants; if you do not follow these procedures then you are not legally covered or insured and could potentially be liable in the event of anything going wrong.
You need to make an application via the online system and you will be given an approval insurance number once complete - this should take you no more than 20 minutes to do. The online system should still be used - but it is very likely that immediate approval will not be granted, and that you will need to receive approval for the application to go to committee (from the CS Ethics Liaison) and then attend one of the weekly central ethics meetings to justify your application.
If anything goes wrong contact the CS Ethics Liaison immediately.
It is sometimes difficult to understand exactly what the user is thinking or, in some cases, doing when they are navigating a complex interface. This is especially the case when the user is familiar with the interface and interaction, and may even be undertaking different, but related, tasks at the same time as the primary task. In this case, to understand explicitly the activities and thoughts of the user, as they are performing the interaction, the think aloud methodology can be used.
The think aloud methodology is a classic of the HCI evaluation process evolving mainly from design based approaches. It produces qualitative data and often occurs as part of an observational process, as opposed to a direct measurement of participant performance, as would be normal in laboratory settings. While it is true that think aloud requires tasks to be completed, the object is not the direct measurement of those tasks. Instead, it is the associated verbalisations of the participants as they progress through the task describing how they are feeling and what they think they need to do.
Think aloud is intended to produce data which is deeper than standard performance measures in that some understanding of the thoughts, feelings, and ideas that are running through the mind of the participant can be captured. The main problem with think aloud is also its strength in that, it is very easy to set up and run and therefore the design aspect of the tasks can be ill conceived. In this way it is often easy to implicitly influence the participant into providing outcomes that are positive regardless of the true nature of the interface or interaction. Indeed, the very act of verbalising their thoughts and feelings means that participants often change the way they interact with the system. It is for this reason that think aloud should not be used as a methodology on its own but should provide the qualitative aspects lacking in other quantitative or performance-based measures.
So if you want to test out your software after you have developed it, then you should use software usability evaluation techniques and in this case we think the 'Think Aloud Method' provides the most straightforward way of doing this.
Imagine working for a company which is trying to specify the functionality for a new computer system. This computer system will move the individual paper-based systems of the sales department and those of the dispatch department into one streamlined computerised sales and dispatch system. In this case you know some of the pieces, you know some of the systems which already exist, you know your own company's business processes and have an understanding of the organisational knowledge. However, you do not know the detailed processes of the sales and dispatch departments separately; and there is no documentation to describe them. In addition, you do not know the interface which currently exists between these two departments and so you cannot understand the flow of information and process that will be required in a unified system. In this case, it is useful to employ one-to-one interview techniques with people while will use the final system.
Interviewing is useful method in qualitative research, however, while sociologists practice this in a structured interview style the anthropologist uses unstructured interviews, often described as `conversations with a purpose'. These often take place as friendly exchanges between participant and the observer in order to find out more about the organisation or the participants job. Interviewing techniques are best applied when: the issues under investigation are resistant to observation; when there is a need for the reconstruction of events; when ethical considerations bar participant observation; when the presence of a participant observer would result in reactive effects; because it is less intrusive in people's lives, because there is also a greater breadth of coverage; and finally, because there can be a specific focus applied. In this way, both structured and unstructured interviews are useful to the computer scientist because they enable a direct confirmation of partial hypotheses that are being formed as the study continues.
So if you want to find something out or allow users to influence your software design before you start coding, then you should use the One-to-One Interview.
How do you find out if the system or interface that you have designed and deployed is useful and has useful features? What kinds of improvements could be made and in what order should these improvements be prioritised? To answer these kinds of questions it is useful to talk to a large number of people, far more than you could expect to recruit for a laboratory experiment. In this case, you may decide to use a questionnaire based survey, recruiting as many users as you possibly can.
Question based surveys are usually designed to provide statistical descriptions of people and their activities by asking questions of a specific sample and then generalising the results of that survey to a larger population. This means that the purpose of the survey is to produce statistics and that the main way of collecting information is by asking people questions.
For your postgraduate taught project then we would expect that you can design an appropriate questionnaire using a Likert scale as a way for participants to rate their answers to your questions. And that you can identify and use the appropriate statistical analysis; at least for descriptive statistics.