EVALUATION OF RESEARCH DESIGN AND METHODOLOGY Foreword The evaluation of research design and the methods for a research paper is used throughout the process of data collection to data analysis

EVALUATION OF RESEARCH DESIGN AND METHODOLOGY
Foreword The evaluation of research design and the methods for a research paper is used throughout the process of data collection to data analysis, in which the major concerns of scientific research. Researchers hypothesize much importance to a solid design and careful choice of methods those have straight effects on the quality of a research initiative. As King et al. (1994) argues that “valid inferences about social and political life depend on appropriate design of research. A researcher must have good command in the required elements of research design and the methodological tools and techniques in order to conduct study in a systematic way following scientific rules”. The main purpose of this essay is to observe research design and methodology. I have chosen a doctoral dissertation to review and criticize with an aim to learn from this research design and methodology that would be vastly helpful for my future research struggle. The dissertation topic named as “Partisan Competition and Democratic Transition and Consolidation in South Asia: A Comparative Study of Democracy in India, Pakistan and Nepal” by Pramod Kumar Kantha (2000), University of Missouri-Colombia, USA, is a qualitative research. The key focus of this research paper was to make inferences on how multiparty political competition developed and how it affected intra-party competition, governmental actors, stakeholders, institutions and ultimately the democratization process. I will examine the research design and methodological features of this dissertation along with the quality valuation in terms of validity and reliability concerns those depend on the research design followed by the researchers.
Overview Research Design and Methodology Researcher has to make a number of critical choices regarding the research design and the methods used to convey the study successfully and scientifically. Research design and methodology is the long plan of a study that includes a list of actions to execute that plan. Scholars classify those elements in different ways, as Creswell (2014) defines “it as the intersection or combination of three elements- philosophical ideas, strategies of inquiry, and specific methods to be used for data collection, analysis and interpretation.” And Robert K. Yin (2009) addresses as “it as a logic of sequence or a logical plan that make a bridge of the initial research questions to the best possible answers of those questions (p. 26). King et al (2013) focus on four components of research design: formulation of research question, use of theory, data collection and analysis.” Again similarly, Yin (2009) mentions about “five components of research design. The first three components; study’s questions, propositions and units of analysis, indicate what data to be collected whereas the rest two components; data link with the propositions and the data analysis, tell about what to be done after the data collection.” “Research methodology includes various actions or ‘research strategies’ (Creswell, 2014) to be followed in a study and the justification of those actions.” Similarly Schensul (2012) categorizes those actions as methodological decisions that include from selecting world view to data analysis process.”
So research design and methodology is a comprehensive process where researchers selects the steps from very beginning to the end of research study. As scholars acknowledged research design and methodology in different components with some overlapping decisions and actions plan, it is really quite struggling to cover all those elements in analyzing the selected dissertation.
Research Design of the Dissertation The author of this doctoral dissertation recognizes his research design as the combination of an “Historical-Analytical Approach” based on multiple case study research (Kantha, 2000, p. 4). “Philosophical worldview, although mostly remains hidden in research (Slife & William, 1995 cited in Creswell, 2009), it can be assumed the researcher of this dissertation hold social constructivist worldview where qualitative inquirers try to understand the context in which they live and work through developing subjective meanings of their experiences formed in interaction with society, history and culture (Creswell, 2009).” The writer applies qualitative research method and follows largely inductive way of research. The most data resources are published materials on political history and the interpretation is based on historical deliberations and explanations to make valid inferences. This essay will critically covers these choices and decision made by researcher in his dissertation.
Research Questions Creswell (2014) says “statement of problems in a research acts as ‘narrative hook’ that allure readers for details of the study (Creswell, 2014).” Having discussed the major focuses of dissertation, the researcher presents three central questions:
How multi-party political competition emerged in all three countries?
How the multi-party political dynamic has impacted on intra-party competition?
How such competition effects governmental actors, institutions and ultimately the democratization process itself?
“The role of specific research questions is to guide the researcher what data to be collected about any case. It would be quite difficult or in most cases impossible to cover “everything” of any unit of research. Therefore, narrowing down the scope with specific research questions will make a research process doable (Yin, 2009, p. 29).” Here, the author demonstrates his expertise in identifying and specifying research questions to a narrow focus that is confined with multiparty political competition. He adds a new dimension (multiparty dynamic) to look into the political history in the process of democratization. So it can be justified to claim that the research questions have the importance in the real world and at the same time it would make specific contribution in political science literature. Thus the research questions meet the two criteria suggested by King et al. (1994).
Case Selection Ragin (1992) states that “every social phenomenon analyzed with specific time and place can be considered as case. In a study, the case can be an individual or group, an event or entity, any phenomenon, a geographical area or country.” Again, Yin (2009) argued that “in a case study research, the important conditions for a case are: it should be specific, real-life phenomenon, contemporary events, and no distinguishable boundaries between phenomenon and context.”
Kantha selects three countries, India, Pakistan and Nepal, as three cases for in-depth comparative investigation on historical development of partisan competition in the process of democratization. However, he does not mention any specific reasons or arguments in favor of the choices of these three cases. “He grows up in India, completes higher education in Nepal and works in Pakistan (Kantha, 2000, p. 5).” This directly indicates and influences the deliberate selection of cases by the investigator resulting selection bias that is quite common in small-N research (Collier, 1995; King et al. 1994). He chooses multiple countries as cases that are beneficial for comparative assessment as “Yin (2009) argues that multiple case design offers analytic benefits and it is superior to single case design.” Again “Ragin (1992) states that selection of countries as cases is a traditional way of research in comparative and historical social science. He also argues that instead of countries, sequences of events or social processes or historical outcomes or narratives should be the cases. Kantha could define his case as democratization process or similar political phenomenon that is the outcome of his historical deliberations in the dissertation.
Another challenge of Kantha’s cases is that the selected three countries “are quite diverse in terms of different aspects including geography, population, religion, culture, political and historical context (Jamil et al., 2013).” Such differences may raise question, as Kantha also acknowledges (p. 6), about the tenability for comparative assessment. Although there are few similarities as well in some respects, like all are developing countries, belong to South Asian region, share common colonial history (except Nepal), and start democratizing process almost in same period at the end of 1940s. However, such cases can also be treated advantageous for comparative assessment with the strategy of Most Different Systems Design (MDSD). “Przeworski and Teune came up with this idea of MDSD in 1970 that argues to select the most different cases in terms of extraneous variables (Anckar, 2008).”
Unit of Analysis “Unit is one of the many elements to be observed in a study” (King et al., 1994, p. 76).” Kantha selects a multiple comparative case design where he does not explicitly mention about the unit to be observed. But party system of each case (country) is the main focus of the study. “Clear and specific research questions would be essential precondition to identify the appropriate unit of analysis in a study (Yin, 2009).” Author exactly sets his research questions and specifically focuses on partisan competition in different party systems- dominant, two party, multiparty and bipolar in nature. This specifies that he develops debates considering ‘party system’ as the central unit of his dissertation.
Theoretical Analysis
“Layder (2011) argues that research can never be theory-neutral and theorizing is a continuous process that may be done at any phase of the research process- before and after the data collection, but better to associate with data analysis. Quantitative study has extensive focus on theory at the very beginning to set a theoretical framework, while qualitative approach put less efforts on it. As the qualitative study is exploratory in nature, the researcher depends mostly on participants and finding from the field (Creswell, 2009, p. 26) rather than variables and hypotheses derived from particular theory.” In this qualitative dissertation, Kantha emphasizes on historical events and findings instead of being bogged down to theoretical insights. In this dissertation, “Kantha does not use any particular theory to build his theoretical framework, instead he discussed some broader theoretical literatures as ‘theoretical cross-system analysis’ (Kantha, 2000, p. 20).” “Creswell (2009) discusses two main uses of theories in qualitative research- either placing at the end as grounded theory or putting at the beginning to be used as lens that shape the major focus of the study and research questions.” In his qualitative research, Kantha shows some universal theories at the beginning where he moderately discusses several aspects of theories relevant to the thematic areas of his dissertation. He mainly highlights factors that influence democratization process such as ‘Litmus Test of Democracy’ or alternations of power; development and democracy; role of elite interactions; democratic transition and consolidation; democratic values and political parties; and challenges for the ‘third wave1’ democracy.
Collection of Data “Historical researchers usually depend on the available data in the form of documents, literatures, reports, newspapers, websites, paintings, artworks, structures, and the like. Such data can be derived both primary sources- produced by the first persons or eyewitnesses, and secondary sources that deliberated by any non-eyewitness persons (Lundy, 2012)” “who may belong to second or third generation (Rees & Howells, 1999).” “Data and information from secondary sources are comparatively less accurate, less reliable than primary sources (Burns & Grove, 1995, cited in Rees & Howells, 1999).” The author mostly collected data through published documents such as election reports, previous studies of party politics in the three countries, journal articles, newspapers, magazines, periodicals, and internet sources. He also explores election studies edited by Gould and Ganguly (1998) and Roy and Wallace (1999) that are purely based on Indian elections. “Actually in historical research published materials are the main data for analysis and to address the research questions (Rees & Howells, 1999).” But the data from observation, interview and other direct methods, that Kantha does not apply, can provide better room for explanations. In identifying the data sources, Kantha does not distinguish among primary and secondary sources of data. Another challenge “in this method is analyzing not only the content of documents, but proper attention should also be paid on how documents are assembled, how they are used as resources, how they function in social interaction (Atkinson & Coffey, 2011; Prior, 2011).” The most important rule for data collection, as King et al. (1994) mentions, is “to report how the data were created and how we came to possess them” (p. 51). Kantha does not explains the background of his data sources. “Again social science data in general are susceptible to produce bias information (King et al. 1994).” Therefore, the dependence on purely published materials as sources of data can raise questions against the dissertation of Kantha. Moreover, internet is also one of the major sources of data in this dissertation. However, “a researcher may also face serious challenges to be selective and find quite difficult to analyze the vast data particularly in qualitative approach that requires inductive, extensive analysis and depth understanding (Markham, 2011).”
Analysis of Data “Historical research method is not just a searching for facts about the historical events but it focuses on the understanding and interpreting the inner meaning of those events (Lundy, 2012).” “The analysis of historical evidences contributes to generate contextual ideas where actual practices and behavior are embedded (Lee, 2013 & Roberts, 1996).” Kantha mainly accepts historical deliberations with explanatory arguments as the technique for data analysis with an effort to present the understanding of democratization process in the context of three Asian countries. He takes theoretical contrast of data analysis based on his discussions on broader theory based literatures. “It is usual in qualitative studies that do not involve any explicit theoretical framework and developing essence from findings draws utmost importance for researchers (Creswell, 2014).” “Qualitative analysis focus on exploring and explaining the data essence, patterns, process and structures (Rapley, 2011).” “Historical evidences explain longitudinal practices and
provides a details of background events to bring about something (Lee, 2013).” In the dissertation, Kantha also puts his effort to explain the essence of historical events that took place over long period of time. However, “analyzing data with historical-analytical method may have embedded biasness problem where he deliberates historical events solely based on documents that may produce prejudice explanations and lack of detailed contextual understandings (Seale, 2011).”
Research Results Kantha clarifies political history in different aspects of democratization process in three studied countries. He emphases on procedural democracy (elections, voting rights, democratic freedom, etc.), and alternation of power (also called ‘Litmus Test of Democracy’) that is measured as necessary condition for democracy. Political elite’s role is another area where Kantha enlightens that political actors (e.g. incumbent and opposition elites), their strategies, interests, and perception about gaining power in democratic setup played a crucial role for the survival of democracy. Thus the theoretical insights are the basis for his explanations building to make valid inferences on the effects of multiparty competition for democratization process as “valid inference requires the perfect combination of theory and observation that demonstrate a causal effect (Laitin, 1995).” Finally, the researcher points that although democracies is far from consolidation in these three counties, but the emergence of new parties and new stage of multiparty comp
Evaluating Research Quality The research quality always depends on assortment of research design and methodology, use of scientific tools, instruments and techniques for data collection and data analysis. I am reviewing here the broader choices made by the author in terms of research design of this dissertation.
Research Way: Inductive The author of this dissertation follows inductive way of research in this dissertation where he generates meanings and patterns of data based on historical discussion of political development. “A social science research can never be theory-neutral (Layder, 2011) and researcher cannot get into the field with completely blank mind.” Kantha discourses the general theoretical literatures not to be guided by those in framing research questions and hypotheses, but to place those in direct contact with evidence. Author arises the research questions in an inductive way, and based on his findings and logic of arguments, his emphases on specific patterns of evidence in the studied three countries. He starts with more specific research questions that he elaborates analyzing historical evidence contrasting with theoretical essence to generalize in a comparative manner. Here, the deductive approach is merged with some elements of inductive during construction of evidences. “It would not be possible for a study to be “100% inductive” because researcher has to have some preconception to make specific focus on some particular aspects of phenomenon and avoid many other possible explanations (Anckar, 2008, p. 393).” I possibly discovery this dissertation a good study of inductive research based on historical-analytic approach which is not quite common in social science research areas. The research questions and hypotheses are subject to be refined at the end according to the findings and data analysis in an inductive way of research.