Development of a program to control response biases and assessment of its usefulness in typical performance measures
- Autores: David Navarro González
- Directores de la Tesis: Urbano Lorenzo Seva (dir. tes.), Andreu Vigil i Colet (codir. tes.)
- Lectura: En la Universitat Rovira i Virgili ( España ) en 2018
- Idioma: español
- Tribunal Calificador de la Tesis: Vicente Ponsoda Gil (presid.), Fàbia Morales Vives (secret.), Hendrik Albert Lambertus Kiers (voc.)
- Programa de doctorado: Programa de Doctorado en Salud, Psicología y Psiquiatría por la Universidad de Almería y la Universidad Rovira i Virgili
- Materias:
- Enlaces
- Tesis en acceso abierto en: hdl.handle.net TDX
- Resumen
The present work focus in one relevant problem that psychologist have to deal when trying to assess personality: to what extent the measures applied reflects accurately the latent trait of interest. Personality measures are usually obtained using self-reports, where the participant’s response may be reflecting not only the content assessed but also other systematic effects.
These effects are known as response bias, which Paulhus (1991, 2017) defined as a systematic tendency to answer the items on some other basis than the specific item content. The two best known response bias in typical response measures are Acquiescence (AC) and Social Desirability (SD).
Acquiescence can be defined as the tendency of responders to agree with a statement regardless of its content (Paulhus and Vazire, 2005), while Social Desirability can be defined as people’s tendency to present themselves in a generally favorable fashion (Holden, 2010).
The impact of SD and AC on different aspects of the measures have been studied for a while. They affect the validity of self-reports (Ones, Dilchert, Viswesvaran, & Judge, 2007; Ones, Viwesvaran, & Reiss, 1996; Salgado, 2005) and the factor structure (Bentler, Jackson, & Messick, 1971; Rammstedt, Goldberg, & Borg, 2010; Soto, John, Gosling, & Potter, 2008), generating distortions that should be avoided.
There are some methods available for controlling the impact of response biases, but none of them was able to control SD and AC at the same time using a Factor Analysis approach since Ferrando, Lorenzo-Seva, & Chico (2009) develop a method for assessing SD and AC effects simultaneously. The method, however, was not implemented in any statistical software, hindering his potential benefits. That was one of the main objectives of the present work: create a software to allow researchers to use the method when creating a questionnaire free of response biases.
Additionally, our second objective was to illustrate the utility of the procedure implemented in the Psychological Test Toolbox in two main issues related with the response biases. First, our intention was to investigate the impact of each bias separately and simultaneously for illustrating the differential impact on the factor structure of each one in self-assessed reports. We hypothesize that the factor structure of the self-assessed questionnaires would be heavily distorted by the impact of response biases, and especially by AC, since it generates a bigger distortion on the inter-correlation matrix than the originated by SD. In addition, controlling the impact of both response biases will improve the factor structure simplicity and coherence.
The second issue concerning response biases distortion is the relationships between response biases and the effects associated with the personality differentiation hypothesis across ability levels (PDH). The PDH assumes that people with higher level of ability have a more differentiated personality structure (Brand, Egan, & Deary, 1994). Different explanations for the PDH were formulated, one of them being the differential reliability associated with ability levels (DRAAL), which implies that people with high level of ability present higher levels of reliability, and this could partially explain the differences in personality differentiation postulated by the PDH. Some authors suggested that response biases are responsible for the DRAAL, so our objective was to investigate the relationship between response biases and intelligence measures. Our hypothesis is that response biases will not be a clear responsible of DRAAL, since the evidence that SD and AC are related to intelligence are weak (De Fruyt, Aluja, García, Rolland, & Jung, 2006, Meisenberg & Williams, 2008).
Our first objective was accomplished: we developed The Psychological Test Toolbox, a program that allow researchers to apply the method mentioned above It was developed using MATLAB, and it is distributed by two ways: (a) as stand-alone application, which requires the installation of MATLAB runtime library, available from the Mathworks website; (b) as a MATLAB toolbox, which can be executed by MATLAB users from the code files.
The program is free to use, and it is distributed under the General Public License version 3. The first version was released in 2016, and it has been regularly updated to include new features and correct bugs. It was specially designed for being easy to use, so we developed a Graphical User Interface (GUI) that guides the user through all the required steps to perform the analysis.
The manuscript with all the information of the program is currently accepted (Navarro-González, Vigil-Colet, Ferrando, & Lorenzo-Seva, in press), and we hypothesize that once is finally published, the overall interest for the program will grow, allowing more researchers to benefits from the potential advantages of the proposed method.
Concerning the second objective, first we analyzed the impact of SD and AC on the factor structure of two self-reports: a personality inventory (OPERAS, Vigil-Colet, Morales-Vives, Camps, Tous, & Lorenzo-Seva, 2013) and an aggression inventory (IDAQ, Ruiz-Pamies, Lorenzo-Seva, Morales-Vives, Cosi, & Vigil-Colet, 2014). The results (Navarro-González, Lorenzo-Seva, & Vigil-Colet, 2016) demonstrated that both biases impact differently the factor structure of the questionnaires. AC generate a bigger distortion than SD in both questionnaires, and the difference between them is more appreciable in the OPERAS, which supports our hypothesis, and once controlling the structure of the questionnaires, their interpretability and simplicity are improved.
Finally, concerning our last investigation (Navarro-González, Ferrando, & Vigil-Colet, 2018), our results showed that none of the SD measures showed relationship with intelligence, and a small relationship was found between AC and intelligence. Our hypothesis was on point, since we cannot conclude that response biases explain the DRAAL, but maybe AC could be a candidate for partially explain the phenomena.
In summary, the scientific community now have a tool to apply the response bias control method proposed by Ferrando, Lorenzo-Seva, & Chico (2009), and we also provide more evidence about the impact of SD and AC in two specific situations.
