PhD seminar:

Designing Rigorous and Impactful Empirical Research

Background

The main objective of this PhD course is to provide doctoral students with basic and advanced knowledge about research designs and methodologies for rigorous and impactful research. The PhD course focuses on the fundamental ideas of various empirical research designs that can be used to investigate research questions with a specific focus on topics at the intersection between business and technology. Building on the foundations of scientific knowledge development and engaged scholarship, this course covers a range of qualitative and quantitative research designs. This course aims to introduce PhD students to the key considerations in designing rigorous and impactful research and thereby provides a solid basis for more advanced courses on specific research designs and methodologies.

The course covers the following main topic areas:

1. Foundations of scientific knowledge development at the intersection between business and technology, and implications for designing rigorous and impactful empirical research
2. Design science and action research (designs)
3. Qualitative research designs and methodologies
4. Quantitative and experimental research designs and methodologies

Learning objectives

After completion of this PhD course, participants should be able to:

• Understand the basic considerations of designing rigorous and impactful empirical research on problems at the intersection of business and technology.
• Understand fundamental ideas behind various empirical research designs and methodologies for disciplinary and interdisciplinary research.
• Learn about research designs and methodologies that combine academic knowledge creation with practical, real-world intervention.
• Develop a research design for PhD research projects that aim to advance knowledge in the intersection between business and technology and potentially combine practical, real-world intervention with academic knowledge creation.
• Choose and implement the appropriate research designs and methodologies with specific application to issues in interdisciplinary research at the intersection of business and technology according to the formulated research question.
• Learn how to critically assess different research designs and methodologies in scientific research at the intersection of business and technology.

Format and schedule

This course is comprised of seven days of teaching in addition to preparing readings and assignments per session. Participants are expected to have carefully read the recommended articles prior to class. The mandatory articles are indicated (*) in the references list of the course. It includes a combination of seminal and instrumental articles on the qualitative and quantitative methodology discussed, as well as the examples of articles implementing those methods.

The teaching format includes readings for each session, lectures, hands-on practical exercises in the classroom, and mandatory final assignment with an optional presentation.

Each day is comprised of three to four sessions (details to be defined by the teacher):

• Session 1: 09:00 am – 10:30 am
• Session 2: 11:00 am – 12:30 pm
• Lunch
• Session 3: 01:30 pm – 03:00 pm
• Session 4: 03:30 pm – 05:00 pm

Final assignment: Research design proposal

This assignment aims to develop student skills in an elaborate, rigorous, and robust research design of a study. Each student must submit a detailed proposal for a research design utilizing the elements and the methodologies covered in the course. Appropriate methodologies (i.e., qualitative, quantitative, or the combination of both) should be described according to the research problem and examine research questions. This research design may be used for the entire PhD thesis or one of the research papers in the PhD thesis.

Style guide for the final assignment:

The document can include five to seven pages of text plus one page of references; single-spaced; 11 Times New Roman points.

Supporting literature:

• Aguinis, H. (2024). Research Methodology: Best Practices for Rigorous, Credible, and Impactful Research. SAGE Publications, Inc. Thousand Oaks, CA: USA.
• Menken, S., & Keestra, M. (2016). An Introduction to Interdisciplinary Research: Theory and Practice. Amsterdam University Press. Amsterdam: The Netherlands.
• Repko, A. F., & Szostak, R. (2025). Interdisciplinary Research: Process and Theory (5 ed.). SAGE Publications. Thousand Oaks, CA: USA. 
• Van de Ven, A. H. (2007). Engaged Scholarship: A Guide for Organizational and Social Research. Oxford University Press. Oxford: United Kingdom.

Course workload

The following pages provide detailed information for each day.

Day 1: Foundations

Topics and objectives for this session:

The session will provide students with an understanding of different conceptions of knowledge in the approaches integrating the research areas of Social Science and Technology. The session will discuss the implications of different approaches to the framing of research problems and the repercussions for the PhD project.

Topics:

• Philosophical perspectives in social and technical research
• Ontology, epistemology, and methodology
• Framing research problems and critical questions

Class preparation:

• Read the recommended materials to gain a basic understanding of the role of philosophical perspectives in social and technical research.
• Write down the research question (and sub-questions, if existing) guiding your research.
• Before the class, students will receive detailed guidelines for the in-class activity consisting of a short presentation and peer group discussion on the philosophical perspectives covered.

In class activities:

• Short presentation on philosophical perspectives in social and technical research, research questions, and the role of ontology, epistemology, and methodology.
• Student presentations of research topics and questions, and peer group discussions of the philosophical assumptions and limitations.

Fundamental readings (*):

• Bechara, J. P., & Van de Ven, A. H. (2007). Philosophy of Science Underlying Engaged Scholarship. In A. H. Van de Ven (Ed.), Engaged Scholarship: A Guide for Organizational and Social Research (pp. 36–70). Oxford University Press.
• Leonardi, P. M. (2012). Materiality, Sociomateriality, and Socio-Technical Systems: What Do These Terms Mean? How Are They Related? Do We Need Them? In P. M. Leonardi, B. A. Nardi, & J. Kallinikos (Eds.), Materiality and Organizing: Social Interaction in a Technological World (pp. 25-48). Oxford University Press. Available at SSRN: https://ssrn.com/abstract=2129878 or http://dx.doi.org/10.2139/ssrn.2129878
• Orlikowski, W. J. (2010). The Sociomateriality of Organisational Life: Considering Technology in Management Research. Cambridge Journal of Economics, 34(1), 125-141. https://doi.org/10.1093/cje/bep058
• Sandberg, J., & Alvesson, M. (2011). Ways of Constructing Research Questions: Gap-Spotting or Problematization? Organization, 18(1), 23-44. https://doi.org/10.1177/1350508410372151

Complementary readings:

• Textbooks on philosophy of science to be indicated.

Day 2: Intervention-based research designs

Topics and objectives for this session:

The session introduces students to research approaches where the researcher seeks to influence, control, or design aspects of a case organization. Focus is placed on Design Science Research (DSR) and Action Research (AR), as both approaches combine practical, real-world intervention with the creation of academic knowledge.

The session will explore how these approaches shape the framing of research problems and discuss their implications for the design and execution of PhD projects. The day will be highly interactive, so preparation in advance is essential. Please pay special attention to the texts for which you have been assigned the roles of presenter or opponent

Topics:

• Pragmatism as an intellectual foundation in DSR and AR
• Generating knowledge through changing practice in AR
• Designing and evaluating artifacts that support practice in DSR
• Similarities and differences between DSR and AR

Class preparation:

• Read the recommended materials to gain a basic understanding of Action Research (AR) and Design Science Research (DSR).
• Prepare to act as a presenter of one paper and as an opponent on a second paper. Information about which papers you are assigned to present and oppose will be announced before the course.
• Formulate your PhD project’s research question (and sub-questions, if applicable) and reflect on how these might be framed when applying DSR or AR.
• Before the session, you will receive detailed guidelines for an in-class activity, including the format for a short presentation of an AR or DSR article and your role as an opponent.

In class activities:

• Short presentation on the intellectual legacy of DSR and AR.
• PhD student presentations of AR and DSR articles, and how their research should be framed if applying DSR and AR. Feedback will be provided by PhD students and the teacher.
• Wrap up in plenum and reflect on key learnings and how your project could be framed as an AR or DSR project. 

Fundamental readings (*):

• Avison, D., Kock, N., & Malaurent, J. (2017). Special Issue: Action Research in Information Systems. Journal of Management Information Systems, 34(3), 630-632. https://doi.org/10.1080/07421222.2017.1372995

• Kock, N., Avison, D., & Malaurent, J. (2017). Positivist Information Systems Action Research: Methodological Issues. Journal of Management Information Systems, 34(3), 754-767. https://doi.org/10.1080/07421222.2017.1373007
• Gregor, S., & Hevner, A. R. (2013). Positioning and presenting design science research for maximum impact. MIS Quarterly, 37(2), 337-355.
• Hevner, A., R , March, S. T., Park, J., & Ram, S. (2004). Design science in information systems research. MIS Quarterly, 28(1), 75-105.

Complementary readings:

• Action Research: Exploring Perspectives on a Philosophy of Practical Knowing. Academy of Management Annals, 5(1), 53-87. https://doi.org/10.5465/19416520.2011.571520
• Van Aken, J. E. (2004). Management research based on the paradigm of the design sciences: the quest for field‐tested and grounded technological rules. Journal of Management Studies, 41(2), 219-246.

Day 3: Qualitative research designs (Madalina Pop)

Topics and objectives for this session:

This session aims to equip students with a comprehensive understanding of qualitative research designs and the foundational principles that underpin qualitative inquiry. It will delve into the theoretical underpinnings of qualitative research, various research designs, and the crucial concept of ensuring trustworthiness in qualitative research. By the end of the session, students should be able to critically assess different qualitative research approaches and develop a solid foundation for designing their own qualitative studies.

Topics:

• Theory and foundations of qualitative research
• Designing qualitative research: grounded theory, ethnography, and case studies
• Choosing between qualitative research methods: interviews, observations, and other options
• Finding fit between research philosophy, theory, research design, method, and research question
• Assessing trustworthiness in qualitative research: credibility, transferability, dependability, confirmability

Class preparation:

• Read mandatory and recommended materials.
• Prepare questions or thoughts about the role of theory in guiding qualitative research.
• Form groups and be ready to discuss the main theories that underpin qualitative research.
• Choose one of the following example articles, and read for in-class assignment:
  • Jia, N., Luo, X., Fang, Z., & Liao, C. (2024). When and How Artificial Intelligence Augments employee Creativity. Academy of Management Journal, 67(1), 5-32. https://doi.org/10.5465/amj.2022.0426
  • Arvidsson, V., Holmström, J., & Lyytinen, K. (2014). Information Systems Use as Strategy Practice: A Multi-Dimensional View of Strategic Information System Implementation and Use. The Journal of Strategic Information Systems, 23(1), 45-61. https://doi.org/10.1016/j.jsis.2014.01.004
  • van den Broek, E., Sergeeva, A., & Huysman, M. (2021). When the Machine Meets the Expert: An Ethnography of Developing AI for Hiring. MIS Quarterly, 45(3), 1557-1580. https://doi.org/10.25300/MISQ/2021/16559
  • Dale, M., & Scheepers, H. (2020). Enterprise Architecture Implementation as Interpersonal Connection: Building Support and Commitment. Information Systems Journal, 30(1), 150-184. https://doi.org/10.1111/isj.12255
  • Kranz, J. J., Hanelt, A., & Kolbe, L. M. (2016). Understanding the Influence of Absorptive Capacity and Ambidexterity on the Process of Business Model Change – the Case of on-Premise and Cloud-Computing Software. Information Systems Journal, 26(5), 477-517. https://doi.org/10.1111/isj.12102

In-class activities:

• During the session, each group will present a brief overview of one paper with qualitative research design and analyze it against the principles discussed in the classroom.
• Engage in discussions about research design fit

After-class assignment:

• Explore the strategies to ensure trustworthiness in your proposed qualitative research design, providing justifications for each strategy.

Fundamental readings (*):

• Creswell, J. W., & Poth, C. N. (2023). Qualitative Inquiry and Research Design: Choosing among Five Approaches (5th ed.). SAGE Publications. Thousand Oaks, CA: Sage
• Harley, B., & Cornelissen, J. (2022). Rigor with or without Templates? The Pursuit of Methodological Rigor in Qualitative Research. Organizational Research Methods, 25(2), 239-261. https://doi.org/10.1177/1094428120937786
• Orlikowski, W. J., & Baroudi, J. J. (1991). Studying Information Technology in Organizations: Research Approaches and Assumptions. Information Systems Research, 2(1), 1-28. https://doi.org/10.1287/isre.2.1.1
• Orlikowski, W. J., & Scott, S. V. (2008). Sociomateriality: Challenging the Separation of Technology, Work and Organization. Academy of Management Annals, 2(1), 433-474. https://doi.org/10.5465/19416520802211644

Complementary readings:

• Alammar, F. M., Intezari, A., Cardow, A., & Pauleen, D. J. (2019). Grounded Theory in Practice: Novice Researchers’ Choice between Straussian and Glaserian. Journal of Management Inquiry, 28(2), 228-245. https://doi.org/10.1177/1056492618770743
• Bradshaw, M.; Stratford. E. (2010). Qualitative research design and research. I. Hay (Ed.), Qualitative research methods in human geography, Third Edition, Oxford University Press Canada, pp. 69-80.
• Ormston, R., Spencer, L., Barnard, M., & Snape, D. (2014). The foundations of qualitative research. Qualitative research practice: A guide for social science students and researchers, 2(7), 52-55.
• Maxwell, J. A. (2004). Qualitative research design: An interactive approach. Thousand Oaks, CA: Sage Publications, Incorporated.
• Sinkovics, R. R., Penz, E., & Ghauri, P. N. (2008). Enhancing the Trustworthiness of Qualitative Research in International Business. Management International Review, 48(6), 689-714. https://doi.org/10.1007/s11575-008-0103-z
• Small, M. L. (2009). How many cases do I need? On science and the logic of case selection in field-based research. Ethnography, 10(1), 5-38
• Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Thousand Oaks, CA: Sage Publications.

Day 4: Quantitative research designs

Topics and objectives for this session:

The session will provide students with an introduction to econometric and causal inference methods for designing and applying rigorous quantitative empirical research. Students will learn to match research questions to appropriate datasets and appropriate quantitative methods. The day progresses from understanding available data sources and causal inference fundamentals through an introduction of empirical design issues, including quasi-experimental methods that exploit natural variation (IV, RD, DiD), to observational methods and other research design concerns.

Topics:

• Fundamentals of quantitative data, data structures, and sources
• Replication of social science research
• Hypothesis/theory testing and quantitative data
• Methods: Instrumental variables (IV), Regression discontinuity (RD), Difference-in-differences (DiD), Correlational evidence (OLS)
• Role of sensitivity analysis, heterogeneous treatment effects, triangulation of methods, pre-analysis plans, and reporting standards

Class preparation:

It is essential that the students familiarize themselves with the fundamental readings. The students should bring one quantitative research paper in their field that they would like to replicate, or that they particularly like for discussion purposes.

In-class activities:

Lectures and discussions.

Fundamental readings (*):

• Balafoutas, L., Celse, J., Karakostas, A., & Umashev, N. (2025). Incentives and the Replication Crisis in Social Sciences: A Critical Review of Open Science Practices. Journal of Behavioral and Experimental Economics, 114, 102327. https://doi.org/10.1016/j.socec.2024.102327
• Bettis, R., Gambardella, A., Helfat, C., & Mitchell, W. (2014). Quantitative Empirical Analysis in Strategic Management. Strategic Management Journal, 35(7), 949-953. https://doi.org/10.1002/smj.2278
• Angrist, J. D., & Pischke, J. (2015). Mastering Metrics: The Path from Cause to Effect. Chapter 5. Princeton University Press.
• Wooldridge Ch 3 (p 69-94) + 14: Introductory Econometrics: A modern Approach. Cengage Learning (W3 & W14).

Day 5: Experimental research designs and modelling

Topics and objectives for this session:

The session will provide PhD students with an introduction to different types of experiments (laboratory experiments, field experiments, and natural or quasi-experiments) as well as models involving systems and/or subjects studies. 

Topics:

• Models and modelling: The cyclical nature of mathematical modeling, the relevance of sensitivity, of the modeling, and of the solution and its integration to empirical research design
• Experiments: understand the approach to experimental research and the role of the solution in cyclical mathematical modelling and applications
• Relevance of data acquisition in experimental research, fundamental theories in system measures (e.g., uncertainty of the data, randomness, probability)
• Subject-based experimentation framework, data gathering to test solutions or data to identify hypothesis

• Valid approaches and procedures in literature to perform measurement and empirical research approach.

Class preparation:

The class preparation comprises of the readings for this session:

• Guala, F. (2002). Models, Simulations, and Experiments. In L. Magnani & N. J. Nersessian (Eds.), Model-Based Reasoning: Science, Technology, Values (pp. 59-74). Springer US. https://doi.org/10.1007/978-1-4615-0605-8_4

In class activities:

• Design in research-related studies, modelling, measures;
• In-class discussion on methods and approaches for empirical research design.

Fundamental readings (*):

• Grootswagers, T. (2020). A Primer on Running Human Behavioural Experiments Online. Behavior Research Methods, 52(6), 2283-2286. https://doi.org/10.3758/s13428-020-01395-3
• Knudsen, T., A. Levinthal, D., & Puranam, P. (2019). Editorial: A Model Is a Model. Strategy Science, 4(1), 1-3. https://doi.org/10.1287/stsc.2019.0077
• Radder, H. (2009). The Philosophy of Scientific Experimentation: A Review. Automated Experimentation, 1(1), 2. https://doi.org/10.1186/1759-4499-1-2

Complementary readings:

• DeVellis, R. (2006). Classical Test Theory. Medical Care, 44(11), S50-S59. https://doi.org/10.1097/01.mlr.0000245426.10853.30
• McMillan, D., & Lee, R. (2010). A Systematic Review of Behavioral Experiments Vs. Exposure Alone in the Treatment of Anxiety Disorders: A Case of Exposure While Wearing the Emperor's New Clothes? Clinical Psychology Review, 30(5), 467-478. https://doi.org/10.1016/j.cpr.2010.01.003