Psychology: Research and Review
From: Scale development: ten main limitations and recommendations to improve future research practices
Study | Scale | Step 1 | Step 2 | Step 3 | N step 3 | Initial item pool | Final item pool | Main limitations reported |
---|---|---|---|---|---|---|---|---|
Aagja and Garg (2010) | PubHosQual Scale | LR/ES/I | EJ | EFA/CFA/NV/CV/DV/ICR | 401 | 59 | 24 | LG |
Ahmad et al. (2009) | Service Quality Scale | LR/FC | EJ | CFA/CV/DV/S-RR/ICR | 413 | 31 | 10 | LG |
Akter et al. (2013) | MHealth Service Quality Scale | LR/ES/FC/I | EJ | EFA/CFA/NV/PV/CV/DV/I-JR/I-T-CR/ICR | 305 | 29 | 22 | LG/CSM |
Alvarado-Herrera et al. (2015) | CSRConsPerScale | LR/ES | EJ | CFA/CV/DV/NV/ICR | 1087 | 73 | 18 | LG/Lack of the PV |
Armfield (2010) | IDAF-4C+ | LR/ES | EJ | EFA/CtV/PV/T-RR/ICR | 1083 | 29 | 8 | LG/Lack of the CV/SRM |
Atkins and Kim (2012) | Smart Shopping Scale | LR/FC/I | EJ/TPJ | EFA/CFA/NV/CV/DV/ICR | 1.474 | 62 | 15 | LG |
Bagdare and Jain (2013) | Retail Customer Experience Scale | LR/EP | EJ/TPJ | EFA/CFA/CV/ICR | 676 | 45 | 12 | LG/This study has not established DV and NV |
Bakar and Mustaffa (2013) | Organizational Communication Scale | LR/FC | EJ | EFA/CFA/CtV/CV/DV/T-RR/ICR | 596 | 386 | 38 | LG/Inadequate choose variables to be correlated |
Beaudreuil et al. (2011) | URAM | EP/I | EJ/TPJ | EFA/CV/DiV/T-RR/ICR | 138 | 52 | 9 | LG/SSS |
Bhattacherjee (2002) | Individual trust in online firms scale | LR/ES | TPJ | CFA/CV/DV/NV/ICR | 269 | 18 | 7 | WBS |
Blankson et al. (2007) | International consumers’selection of banks scale | LR/FC | EJ/TPJ | EFA/CFA/CV/PV/NV/ICR/I-T-CR | 773 | 60 | 18 | LG |
Blankson et al. (2012) | Scale measuring college students’ choice criteria of credit cards | FC/ES | EJ | EFA/CFA/CV/DV/S-RR/ICR | 405 | 59 | 19 | LG/CSM |
Bolton and Lane (2012) | IEO | LR/ES | TPJ | EFA/IV/EV/CV/DV/I-T-CR/ICR | 1162 | NCR | 10 | LG/Lack of the CFA |
Bova et al. (2006) | HCR | I/FC. | EJ | EFA/T-RR/ICR | 99 | 58 | 15 | LG/Scale was administered in a face-to-face interview/SSS. |
Boyar et al. (2014) | CESS | LR | EJ | CFA/DV/ICR | 446 | 140 | 52 | CSM |
Brock and Zhou (2005) | OIU | LR/I | EJ | DV/PV/NV/ICR | 112 | NCR | 7 | LG |
Brun et al. (2014) | Online relationship quality scale | LR, and ES | EJ/TPJ | EFA/CFA/CV/DV/PV/ICR | 476 | 33 | 21 | LG |
Butt and Run (2010) | SERVQUAL model scale | LR/EP | EJ | EFA/CFA/CV/DV/ICR | 340 | 17 | 17 | LG |
Caro and García (2007) | Perceived service quality in urgent transport service scale | LR/I/ES | EJ/TPJ | EFA/CFA/DV/CV/NV/I-T-CR/ICR | 375 | 68 | 38 | LG/Lack of the CV or DV |
Chahal and Kumari (2012) | CPV Scale | LR/ES | EJ/TPJ | EFA/CFA/CV/I-T-CR/ICR | 515 | 32 | 27 | LG |
Chen et al. (2009) | Process Orientation Scale | LR/I | EJ | EFA/CFA/CV/DV/I-I-CR/ICR | 356 | NCR | 6 | LG/SSS/Lack of the NV |
Choi et al. (2011) | Measure of dyspnea severity and related functional limitations | LR/I/EP | EJ/TPJ | EFA/CFA/CV/DiV/T-RR/ICR | 608 | 364 | 33 | CSM |
Christophersen and Konradt (2012) | Reflective and formative usability scales | LR/EP/ES | EJ | CFA/CtV/PV/EV/ICR | 378 | 80 | 54 | CSM/SRM |
Cicero et al. (2010) | ASI | LR/EP | EJ/TPJ | EFA/CFA/CV/DV/ICR | 1281 | NCR | 29 | Items are not reverse-scored |
Coker et al. (2011) | IPPR | LR/I | EJ/TPJ | EFA/CFA/CtV/NV/DV/ICR | 1200 | 65 | 11 | LG |
Coleman et al. (2011) | B2B service brand identity scale | LR | EJ/TPJ | EFA/CFA/DV/I-T-CR/ICR | 210 | 119 | 15 | LG/Deductive approach to scale development |
Colwell et al. (2008) | Measure of service convenience | LR/I | EJ/TPJ | EFA/CFA/CV/DV/NV/ICR | 201 | 30 | 17 | LG/CSM |
Cossette et al. (2005) | Caring Nurse–Patient Interactions Scale | LR/ES | EJ | EFA/CV/CtV/ICR | 332 | 121 | 70 | CSM |
Dennis and Bocarnea (2005) | Servant leadership assessment instrument | LR/EP | EJ | EFA/CtV/ICR | 293 | 71 | 42 | MD |
Devlin et al. (2014) | Fairness measurement scale | LR, and ES | EJ/TPJ | EFA/CFA/CV/DV/NV/ICR | 3100 | 9 | 8 | LG |
Dunham and Burt (2014) | Organizational Memory Scale | LR/ES | NCR | EFA/CFA/CV/T-RR/ICR | 431 | 72 | 21 | SRM |
Edwards et al. (2010) | STL | FC | TPJ | EFA/CV/CtV/ICR | 270 | NCR | 84 | LG |
Feuerstein et al. (2005) | Response to work in those with upper extremity pain scale | LR/FC/ES | TPJ | EFA/T-RR/ICR | 282 | 136 | 136 | LG/SSS |
Fisher et al. (2014) | Entrepreneurial Success Scale | LR/I | EJ | EFA/CFA/ICR | 213 | 9 | 4 | SSS/Subjective Analysis/SRM |
Flight et al. (2011) | Characteristics-based innovation adoption scale | LR | EJ/TPJ | EFA/CFA/ICR/EV/CV/DV/NV/ICR | 430 | 122 | 43 | LG |
Forbush et al. (2013) | EPSI | LR | NCR | EFA/CFA/CV/CtV/DV/T-RR/ICR | 1528 | 160 | 45 | LG |
Foster et al. (2015) | GNS | LR/ES | NCR | EFA/CFA/ICR | 2259 | 35 | 33 | Lack of the validity |
Franche et al. (2007) | RRTW | LR/EP | EJ | EFA/CFA/CrV/PV/IV/EV/ICR | 632 | NCR | 22 | SSS/CSM |
Gesten (1976) | HRI | LR/EP/ES | EJ | EFA/T-RR/ICR | 592 | 79 | 54 | LG |
Gibbons et al. (2013) | MULTIPleS | LR/ES/QER | TPJ | EFA/T-RR/ICR | 490 | 53 | 22 | LG |
Gligor and Holcomb (2014) | SCA | LR/ES/I | EJ/TPJ | EFA/CFA/CV/DV/EV/ICR | 151 | NCR | 21 | CSM |
Glynn et al. (2015) | PFS | ES/QER | NCR | EFA/CV/T-RR/ICR | 1496 | 26 | 10 | LG/MD |
Gottlieb et al. (2014) | Consumer perceptions of trade show effectiveness scale | LR/I | NCR | EFA/CFA/CV/DV/NV/I-T-CR/ICR | 739 | 13 | 11 | LG/Items ambiguous/Difficult to control variables |
Hall et al. (2002) | General Trust in physicians scale | LR/FC/EP | EJ/TPJ | EFA/CV/CtV/ICR | 502 | 25 | 11 | LG/CSM |
Han et al. (2011) | Scale of switching barriers in full-service restaurants | LR/FC | EJ/TPJ | EFA/CFA/CV/NV/I-JR/ICR | 401 | NCR | 17 | LG |
Henderson-King and Henderson-King (2005) | ACSS | LR | TPJ | EFA/DV/CV/T-RR/ICR | 1288 | 26 | 15 | LG |
Hernandez and Santos (2010) | Development-based trust | LR/I | TPJ | EFA/CFA/CV/DV/NV/ICR | 238 | 30 | 27 | CSM |
Hildebrandt et al. (2004) | MDDI | LR/ES | NCR | EFA/CV/DiV/T-RR/ICR | 245 | 21 | 20 | LG/Lack of the DV |
Ho and Lin (2010) | Scale for measuring internet banking service quality | LR/I/ES | TPJ | EFA/DV/CV/ICR | 130 | 30 | 17 | SSS |
Jong et al. (2014) | CRIQ | LR/ES | EJ | EFA/CFA/T-RR/ICR | 310 | 120 | 120 | Lack of the CFA - the CFI fit is below the 0.90 |
Kim et al. (2011) | CEI | LR | TPJ | EFA/CFA/CV/DV/ICR | 397 | 134 | 26 | LG/Lack of the validity/WBS |
Kim et al. (2014) | SAPS | LR | EJ | CFA/CtV/CV/ICR | 795 | 29 | 15 | Lack of the DV |
Kwon and Lennon (2011) | Brand Association Scale | LR | EJ | EFA/CFA/CV/DV/I-JR/ICR | 671 | 28 | 14 | LG |
Lin and Hsieh (2011) | SSTQUAL Scale | LR/FC/I | EJ | EFA/CFA/CV/DV/NV/I-T-CR/ICR | 1238 | 75 | 20 | LG/subjectivity in the EFA and CFA |
Lombaerts et al. (2009) | SRLTB | LR | EJ | EFA/CFA/ICR | 952 | 39 | 10 | Initial unsatisfactory factor analysis output |
Lucas-Carrasco et al. (2011) | QOCS | LR/FC | TPJ | EFA/CFA/CV/DV/ICR | 3772 | 44 | 17 | Recruitment of a larger number of interviewers |
Mahudin et al. (2012) | Measuring rail passenger crowding | LR/ES | EJ/TPJ | EFA/CFA/CV/DV/ICR | 525 | 9 | 20 | Lack of the CtV/SRM |
Medina-Pradas et al. (2011) | BDSEE | ES/EP | EJ | EFA/CV/CtV/ICR | 77 | 14 | 14 | SSS/CSM |
Morean et al. (2012) | AEAS | LR/ES | EJ/TPJ | EFA/CFA/CV/CtV/DV/T-RR/ICR | 546 | 40 | 22 | LG/SRM/CSM |
Morgado et al. (2014) | SAS-EB | LR/FC | EJ/TPJ | CFA/CV/DV/ICR | 318 | 33 | 18 | Lack of the use of a validated scale in the CV |
Nagy et al. (2014) | Scale to measure liabilities and assets of newness after start-up | LR/I | EJ | EFA/CFA/DV/ICR | 260 | 235 | 19 | LG/SSS |
Napoli et al. (2014) | Consumer-based brand authenticity scale | LR | EJ/TPJ | EFA/CFA/CV/DV/PV/ICR | 762 | 157 | 14 | Lack of a more robust LR |
Negra and Mzoughi (2012) | OCPS | LR/I | EJ | EFA/CFA/CV/DV/NV/I-T-CR/ICR | 512 | 77 | 5 | Widely heterogeneous sample/Brevity of the scale. |
Ngorsuraches et al. (2007) | TRUST-Ph | LR/FC/EP/ES | EJ | EFA/CV/CtV/ICR | 400 | 40 | 30 | LG/SSS/MD/social desirability bias |
Oh (2005) | Affective reactions to print apparel advertisements scale | LR/FC/ES | TPJ | EFA/CFA/CV/DV/CtV/ICR | 128 | 66 | 54 | LG |
Olaya et al. (2012) | ISAD | EP/ES | EJ | CV/DiV/T-RR/ICR | 76 | 20 | 17 | LG |
Omar and Musa (2011) | LPSQual | LR/FC | EJ | EFA/CFA/CV/DV/NV/ICR | 655 | 57 | 26 | LG/Lack of the NV/CSM |
Pan et al. (2013) | PMGS | I | EJ/TPJ | EFA/CFA/CV/S-RR/I-I-CR/I-T-CR/ICR | 554 | 71 | 14 | LG/SRM/Lack of the T-RR |
Patwardhan and Balasubramanian (2011) | Measurement scale for brand romance | LR/ES/QER | TPJ | EFA/CFA/CV/DV/CtV/NV/ICR | 711 | 70 | 12 | LG |
Pimentel et al. (2007) | EPM | NCR | EJ/TPJ | EFA/CFA/ICR | 480 | 13 | 13 | LG/Lack of the CV and DV |
Pommer et al. (2013) | PCQ | EP/FC | TPJ | EFA/CFA/CV/ICR | 953 | 391 | 18 | CSM |
Reed et al. (2011) | ESLS | ES | EJ | EFA/CFA/CV/DV/ICR | 218 | 55 | 25 | LG/SRM/WBS |
Rice et al. (2013) | MDRS-22 | LR | EJ | EFA/CFA/ICR | 1176 | 82 | 22 | LG/Lack of the T-RR/Lack of the CV |
Riedel et al. (2011) | RSM-scale | LR/ES | EJ/TPJ | DV/T-RR/ICR | 136 | 43 | 36 | LG |
Rodrigues and Bastos (2012) | Organizational Entrenchment Scale | EP/ES | EJ | EFA/CFA/I-T-CR/I-I-CR/ICR | 721 | 31 | 22 | LG |
Rodríguez et al. (2013) | VEDAS | ES | NCR | EFA/CFA/CV/CtV/T-RR/ICR | 1034 | 40 | 20 | Long time between the test and retest/Lower Cronbach’s alpha |
Rosenthal (2011) | IEK | EP | EJ | EFA/CV/CrV/I-T-CR/I-I-/T-RR/ICR | 292 | 54 | 21 | LG/SSS |
Saxena et al. (2015) | UCLA Hoarding Severity Scale | LR/EP | EJ | EFA/CV/DV/I-I-CR/ICR | 127 | NCR | 10 | Lack of the T-RR/Lack of the instructions for raters in the initial version of the scale |
Schlosser and McNaughton (2009) - | I-MARKOR scale | LR/FC/I | EJ/TPJ | EFA/CFA/CV/DV/NV/ICR | 138 | 71 | 20 | SSS/CSM. |
Sewitch et al. (2003) | PPDS | LR | EJ | EFA/CrV/CV/ICR | 200 | 10 | 10 | LG/CrV was limited/content validity was not formally assessed |
Sharma (2010) | Personal Cultural Orientations Scale | LR/I | EJ | EFA/CFA/NV/CV/PV/DV/ICR | 2332 | 96 | 40 | LG/Lack of the PV |
Sharma and Gassenheimer (2009) | SPC Scale | LR/EP/I | EJ | EFA/CFA/CV/DV/ICR | 511 | 8 | 17 | Lack of the EV |
Shawyer et al. (2007) | VAAS | LR | EJ | CV/T-RR/ICR | 41 | 61 | 31 | Lack of a more robust demonstration of the validity/SSS |
Sin et al. (2005) | CRM | LR | EJ | EFA/CFA/CV/DV/NV/ICR | 641 | 78 | 18 | LG/CSM |
Sohn and Choi (2014) | Expected Interactivity Scale | LR/EP/I | EJ | EFA/CFA/CV/DV/CtV/T-RR/ICR | 378 | 50 | 12 | Lack of the empirical test |
Song et al. (2011) | SECI | LR | EJ | EFA/CFA/CV/I-I-CR/ICR | 469 | 26 | 17 | LG/deductive approach |
Staines (2013) | Investigative Thinking Styles Scale | LR | TPJ | EFA/CV/CtV/ICR | 545 | 68 | 16 | LG |
Sultan and Wong (2010) | Performance-based servisse quality model scale | LR/FC/ES | EJ | EFA/CFA/ICR | 362 | 67 | 67 | The study uses three sources to collect data |
Swaid and Wigand (2009) | E-Service Quality Scale | LR | TPJ | EFA/CFA/CV/DV/ICR. | 557 | NCR | 28 | Online survey |
Tanimura et al. (2011) | DDLKOS | LR/I | EJ | EFA/CFA/CtV/ICR | 362 | 48 | 14 | Inadequate choose variables to be correlated with that of the study |
Taute and Sierra (2014) | Brand Tribalism Scale | LR/ES | NCR | EFA/CFA/CV/DV/ICR | 616 | 35 | 16 | LG |
Tombaugh et al. (2011) | SEW | LR/EP | NCR | EFA/CFA/CV/DV/PV/ICR | 348 | 5 | 5 | CSM/Brevity of the scale |
Turker (2009) | CSR | LR/FC/ES | TPJ | EFA/I-I-CR/I-T-CR/ICR | 269 | 55 | 18 | LG |
Uzunboylu and Ozdamli (2011) | MLPS | LR/I/EP | EJ | EFA/S-RR/ICR | 467 | 31 | 26 | LG |
Van der Gaag et al. (2013) | DACOBS | EP | EJ | EFA/CV/ICR/S-RR/T-RR | 257 | 70 | 42 | SSS/Validation performed with patients/Inappropriate choice of the instruments for validation |
Von Steinbüchel et al. (2010) | QOLIBRI | LR/ES | EJ | EFA/CFA/T-RR/ICR | 2449 | 148 | 37 | SSS |
Voon et al. (2014) | HospiSE | LR/FC | EJ/TPJ | EFA/CFA/CV/DV/CtV/ICR | 1558 | NCR | 21 | LG/CSM |
Walshe et al. (2009) | DIP | LR/I/ES | TPJ | Ecological validity/ICR | 31 | 48 | 48 | SSS/Lack of the DV, CV and T-RR |
Wang and Mowen (1997) | SC | LR | EJ | EFA/CFA/CV/DV/PV/I-T-CR/ICR | 140 | 60 | 9 | SSS |
Wepener and Boshoff (2015) | The customer-based corporate reputation of large service organizations scale | LR/ES/FC | EJ | EFA/CFA/NV/CV/DV/ICR | 2551 | 78 | 19 | LG |
Williams et al. (2009) | SCSC | LR/I | EJ/TPJ | EFA/CFA/CV/DV/PV/I-T-CR/ICR | 162 | 5 | 5 | LG; b) WBS. |
Wilson and Holmvall (2013) | Incivility from customers scale | LR/FC | EJ | EFA/CFA/CV/DV/CtV/ICR | 439 | 27 | 10 | LG/CSM/SRM |
Yang et al. (2014) | BLOG-S-INNO Scale | EP | TPJ | EFA/CFA/CV/DV/ICR | 498 | 517 | 18 | LG |
Zhang and Hu (2011) | Farmer-buyer relationships in China Scale | LR/ES | EJ/TPJ | EFA/CFA/CV/I-I-CR/ICR | 210 | 39 | 22 | LG |
Zheng et al. (2010) | DPEBBS | LR/FC | EJ | EFA/CFA/CtV/T-RR/I-T-CR/ICR | 269 | 51 | 24 | LG/SSS/EFA and CFA - same sample/Reliability coefficients - unsatisfactory. |