Abstract: The indefinite nature of the test fairness (McNamara et al., 2019) and different interpretations and definitions of the concept has stirred much controversy over the years, necessitating the reconsideration of the concept. Kunnan’s (2004, 2008, 2010) test fairness model as one of the most well-known theorizations stirred considerable interest in constituting factors of test fairness during the past decades. On this basis, the present study was designed to examine the empirical quantitative evidence as to the empirical validity of Kunnan’s (2004, 2008, 2010) test fairness model by developing a test fairness questionnaire and assessing five underlying factors, including validity, absence of bias, access, administration, social consequences, and 16 subcomponents of the framework. Based on obtained values and feedbacks of the content evaluation panel and the pilot testing data, some items were removed or revised. In the main phase of the study, 206 participants were requested to fill out the questionnaire. The Cronbach alpha index of internal consistency or mean inter-item correlation of all subscales except one were satisfactory, with the total scale Cronbach’s Alpha of .885. Construct validity of the questionnaire, and Kunnan’s (2008) test fairness model was examined using a series of exploratory and confirmatory factor analyses (CFA). The results indicated that Kunnan’s (2004, 2008, 2010) postulated five-factor model framework was not empirically valid in the study context, emphasizing the necessity for some reconsideration of Kunnan’s (2008) conception of test fairness in Iranian educational context. Next, the research aimed to revisit the established test fairness conceptualization following the principles of grounded theory, and accordingly, a series of interviews were designed to explore the stakeholders’ viewpoints on the qualities required for a fair test. To this end, 10 university lecturers of TEFL, 20 high school English language teachers, 15 Ph.D.
