cybertree

I went through a lot of stuff lying on my desktop and I found this list of questions. It was put together by Andrew Broad. He has more advice on his website to prepare you for your viva, but this list is what people have asked me for again and again. So I decided to put it up here:

1.    What is the area in which you wish to be examined? Particularly difficult and important if your thesis fits into several areas, or has several aspects, or seems to fit into an area of its own.

2.    In one sentence, what is your thesis? Resist the temptation to run from the room!

3.    What have you done that merits a PhD?

4.    Summarise your key findings.

5.    What are you most proud of, and why? This may be asked (again) towards the end of the viva.

6.    What’s original about your work? Where is the novelty? Don’t leave it to the examiners to make up their own minds - they may get it wrong!

7.    What are the contributions (to knowledge) of your thesis?

8.    Which topics overlap with your area?

9.    For topic X:

9.1. How does your work relate to X?

9.2. What do you know about the history of X?

9.3. What is the current state of the art in X? (Capabilities and limitations of existing systems)

9.4. What techniques are commonly used?

9.5. Where do current technologies fail such that you (could) make a contribution?

9.6. How does/could your work enhance the state of the art in X?

9.7. Who are the main `players’ in X? (Hint: you should cluster together papers written by the same people)

10. Who are your closest competitors?

11. What do you do better than them? What do you do worse?

12. Which are the three most important papers in X?

13. What are the recent major developments in X?

14. How do you expect X to progress over the next five years? How long-term is your contribution, given the anticipated future developments in X?

15. What did you do for your MPhil, and how does your PhD extend it? Did you make any changes to the system you implemented for your MPhil?

16. What are the strongest/weakest parts of your work?

17. Where did you go wrong?

18. Why have you done it this way? You need to justify your approach - don’t assume the examiners share your views.

19. What are the alternatives to your approach?

20. What do you gain by your approach?

21. What would you gain by approach X?

22. Why didn’t you do it this way (the way everyone else does it)? This requires having done extensive reading. Be honest if you never thought of the alternative they’re suggesting, or if you just didn’t get around to it. If you try to bluff your way out, they’ll trap you in your own words.

23. Looking back, what might you have done differently? This requires a thoughtful answer, whilst defending what you did at the time.

24. How do scientists/philosophers carry out experiments?

25. How have you evaluated your work?

26. Intrinsic evaluation: how have you demonstrated that it works, and how well it performs?

27. Extrinsic evaluation: how have you demonstrated its usefulness for a specific application context?

28. What do your results mean?

29. How would your system cope with bigger examples? Does it scale up? This is especially important if you have only run your system on `toy’ examples, and they think it has `learned its test-data’.

30. How do you know that your algorithm/rules are correct?

31. How could you improve your work?

32. What are the motivations for your research? Why is the problem you have tackled worth tackling?

33. What is the relevance of your contributions? To other researchers? To industry?

34. What is the implication of your work in your area? What does it change?

35. How do/would you cope with known problems in your field? (e.g. combinatorial explosion)

36. Have you solved the field’s problem that you claim to have solved? For example, if something is too slow, and you can make it go faster - how much increase in speed is needed for the applications you claim to support?

37. Is your field going in the right direction? For example, if everyone’s been concentrating on speed, but the real issue is space (if the issue is time, you can just wait it out (unless it’s combinatorially explosive), but if the issue is space, the system could fall over). This is kind of justifying why you have gone into the field you’re working in.

38. Who are your envisioned users? What use would your work be in situation X?

39. How do your contributions generalise?

40. To what extent would they generalise to systems other than the one you’ve worked on?

41. Under what circumstances would your approach be useable? (Again, does it scale up?)

42. Where will you publish your work? Think about which journals and conferences your research would best suit. Just as popular musicians promote their latest albums by releasing singles and going on tour, you should promote your thesis by publishing papers in journals and presenting them at conferences. This takes your work to a much wider audience; this is how academics establish themselves.

43. Which aspects of your thesis could be published?

44. What have you learned from the process of doing your PhD? Remember that the aim of the PhD process is to train you to be a fully professional researcher - passing your PhD means that you know the state of the art in your area and the directions in which it could be extended, and that you have proved you are capable of making such extensions.

45. Where did your research-project come from? How did your research-questions emerge? You can’t just say “my supervisor told me to do it” - if this is the case, you need to talk it over with your supervisor before the viva. Think out a succinct answer (2 to 5 minutes).

46. Has your view of your research topic changed during the course of the research?

47. You discuss future work in your conclusion chapter. How long would it take to implement X, and what are the likely problems you envisage? Do not underestimate the time and the difficulties - you might be talking about your own resubmission-order!

Ethnographic research in information systems research is not new, but what is it?

It is a very in-depth research method and a way of describing the real world. The researcher usually spends several months “in the field”, in this research area the field usually is a company. Typically the researchers immerse themselves in the life of the people they study. It has been used to investigate social and organizational contexts of information systems.

In ethnography data is collected through interviews, documentary evidence, participant observation and informal social contact.

The Benefits of ethnographic research are the in-depth understanding of people, organization and context of work. The researcher sees what people are doing and hears what they are saying. This enables the researcher to question what is taken for granted. For example people might have the perception that they follow a certain process, while observation might unveil they are following a completely different process or none at all. Your observation might cause you to question what standard and best practice is and it unravels the actual practice.

The Disadvantages of ethnographic research are that the data collection and the analysis process is long. The research has a narrow focus, as it concentrates on one company.

For the researcher this means it is important to think about what are the parameters which make the company comparable to others and are these parameters relevant for the findings. However, if it is valid to generalize from case study research, it is similarly valid to apply this principle here.

I am preparing ethnographic research to explore requirements for a software engineering framework. So I read a bit about it and talked to some ethnographic experts I know.

After talking to people who have done ethnographic studies in information systems research, I came up with the following list of issues to consider:

1.       Get involved, get a job to do in the company that is of use to the people you are observing

2.       Follow one project from the beginning as far as your timeline allows

3.       Introduce an observation instrument from the beginning:

·         Keep a diary; write ½ hour at the end of each day

·         Analyze and then follow the structure after the 1^st week
Write about people, profiles, tasks, projects

·         Collect data so it suits your research; have a plan!

·         Interviews should be written up at the same day with at least a brief summary - if there is a recording.

·         Regularly review your ideas and develop ideas; write analytic memos

4.       Organize a workshop and report what you found or saw

·         A happened like that

·         B happened like that

·         Then ask:

a.       What is best for you? (ask people in the company)

·         Ask yourself:

a.       What are the parameters to consider?

·         Come up with a prototype process together with the people involved

·         Be prepared that people might not like what you report back

Some things to consider: things written in your diary might be without much reflection, just to document them and you will know that, but someone else reading it might be upset.

Reporting your findings

To report the findings, methods of participatory design have been used. In particular Eva Brandt has done a lot of research on that.

Ulrike Schultze wrote a useful tutorial on methodologies for ethnographic research. That’s where the next entry will continue J

References

Myers, M.D. (1999). Investigating Information Systems with Ethnographic Research. In: Communications of the Association for Information Systems, Volume 2.

Ely, M., Anzul, M., Friedman, T., Garner, D. & McCormack Steinmetz, A. (2003) Doing Qualitative Research: Circles within Circles. Taylor & Francis eLibrary. Available from http://www.scribd.com/doc/49851619/27/Analytic-Memos [Accessed 20 October 2011]

Schultze, U. (2000). A Confessional Account of an Ethnography about  Knowledgework. In: MIS Quarterly 24 (1) p. 3-41

Brandt, E. (2006).Designing exploratory design games: a framework for participation in Participatory Design? In: Proceedings of the ninth Participatory Design Conference

ACCESSIBLE is an EU project that ran out in September 2011. The main goal was the development of a methodology to allow designers to make initial accessibility assessments before involving users in evaluations. The effectiveness of guidelines like WCAG 2.0 is questioned. Instead a scalable simulation system that consists of methodology and tools has been proposed. The methodology aims at automated exploitation of existing guidelines. The simulation system will integrate combinations of disabilities. The website supports a sourceforge page where a number of its tool sare available for download. In particular it provides a web accessibility tool (opens in new window) and a mobile web accessibility tool (opens in new window). The project also presents a number of links to project-related ontologies (opens in new window); however, most of those do not work..