Methods are truly important when you do research, and depending on your research tradition there are indeed many strong opinions about what is a good and bad method. Generalizability, validity and reliability are for some the only quality criteria possible. And if you do an interview study you need an enormous number of interviews according to some people.
Some would argue that human-computer interaction is indeed a melting pot of different opinions and perspectives on methods. Indeed, I have seen some very harsh comments on some of the methods that I have used from people with a completely other world-view. The problem when I get these comments is that I don’t really know what to write as an answer. Please take a course in qualitative research and relevant quality criteria for that kind of research is perhaps not the most politically correct rebuttal text to write.
One of my favorite stories related to methods is from my supervisor, Jan Gulliksen who was a part of a long review process where one other researcher claimed that the method was un-important:
I don’t care that the method is flawed. I like the results!
Before the Christmas holiday Christiane Grünloh presented a workshop paper that we wrote together that presents a mixed methods approach to analyzing interviews.
“Human-Computer Interaction (HCI) is a melting pot of different theories and methods. The combination of qualitative and quantitative methods in studies is still quite rare, but has become more and more common. In this paper, we present our experiences from doing a combination of quantitative and qualitative analysis to shed some light on patient accessible electronic health records. We conducted a national survey to patients to learn about their experiences of accessing their electronic health records. The questionnaire was informed by previous interviews with physicians related to effects on their work environment, and we made use of identified themes from that study as a lens to analyse survey data.”
You find the paper free for download here.