Measuring and evaluating nurses’ performance are vital to identify areas for improvement in maintaining quality of service delivery and ensuring sustainability of current practices. This study attempts to examine the content validity of the nurses’ performance scale. It is also aimed to achieve acceptable criteria for content validity of this instrument. Construct and content domain of nurses’ performance were identified followed by items generation and instrument formation. Subsequently, assessments of content validity were performed based on content validity Index (CVI), Inter-rater agreement percentage (IRA%) and modified Kappa statistic. Two level of judgement were performed using the lay expert panel and research expert panel. Criteria were established based on these indices as basis for item reduction process. Pilot study was conducted on 50 respondents to assess the internal consistency using Cronbach’s alpha value of finalized NPQ instrument. 71 items are yielded during developmental stage of instrument to measure four dimensions of nurses’ performance. Assessment of content validity based on lay and research expert judgement resulting in elimination of 27 items (38%). Computed modified Kappa statistic further supplemented that the remaining 44 items as ‘excellent’. As for conclusion, NPQ instrument has attain acceptable criteria of content validity assessment utilized in this study and therefore proved its potential for further research
Agarwal, H. S., Saville, B. R., Slayton, J. M., Donahue, B. S., Daves, S., Christian, K. G., … Harris, Z. L. (2012). Standardized postoperative handover process improves outcomes in the intensive care unit: A model for operational sustainability and improved team performance. Critical Care Medicine, 40(7), 2109–2115. https://doi.org/10.1097/CCM.0b013e3182514bab
Al-Makhaita, H. M., Sabra, A. A., & Hafez, A. S. (2014). Job performance among nurses working in two different health care levels, Eastern Saudi Arabia: A comparative study. International Journal of Medical Science and Public Health, 3(7), 832–837.
Ali Memon, M., Ting, H., Ramayah, T., Chuah, F., & Cheah, J.-H. (2017). Editorial - A Review of the Methodological Misconceptions and Guidelines Related to the Application of Structural Equation Modelling. Journal of Applied Structural Equation Modeling, 1(1).
Ali, N., Tretiakov, A., & Whiddett, D. (2014). A Content Validity Study for a Knowledge Management Systems Success Model in Healthcare. Jitta, 15(2), 21–36.
Beckstead, J. W. (2009). Content validity is naught. International Journal of Nursing Studies, 46(9), 1274–1283. https://doi.org/10.1016/j.ijnurstu.2009.04.014
Blazun, H., Kokol, P., & Vosner, J. (2015). Survey on specific nursing competences: Students’ perceptions. Nurse Education in Practice, 15(5), 359–365. https://doi.org/10.1016/j.nepr.2015.02.002
Bogaert, P. Van, Peremans, L., Heusden, D. Van, Verspuy, M., Kureckova, V., Van De Cruys, Z., & Franck, E. (2017). Predictors of burnout, work engagement and nurse reported job outcomes and quality of care: a mixed method study. BMC Nursing, 16(5), 1–14. https://doi.org/10.1186/s12912-016-0200-4
Bramley, L., & Matiti, M. (2014). How does it really feel to be in my shoes? Patients’ experiences of compassion within nursing care and their perceptions of developing compassionate nurses. Journal of Clinical Nursing, 23(19–20), 2790–2799. https://doi.org/10.1111/jocn.12537
Brooten, D., Youngblut, J. M., & Youngblut, J. M. (2006). Nurse dose as a concept.(patient nurses ). Journal of Nursing Scholarship, 38(1), 94.
Cha, E. S., Kim, K. H., & Erlen, J. A. (2007). Translation of scales in cross-cultural research: Issues and techniques. Journal of Advanced Nursing, 58(4), 386–395. https://doi.org/10.1111/j.1365-2648.2007.04242.x
Chan, E. A., Jones, A., Fung, S., & Wu, S. C. (2012). Nurses’ perception of time availability in patient communication in Hong Kong. Journal of Clinical Nursing, 21(7–8), 1168–1177. https://doi.org/10.1111/j.1365-2702.2011.03841.x
Chan, J. (2014). Modelling The gynecologic Oncology Workforce Using System Dynamics. University of Toronto.
Cho, E., & Kim, S. (2015). Cronbach’s Coefficient Alpha. Organizational Research Methods, 18(2), 207–230. https://doi.org/10.1177/1094428114555994
Ciconelli, R. M., de Soárez, P. C., Kowalski, C. C. G., & Ferraz, M. B. (2006). The Brazilian Portuguese version of the Work Productivity and Activity Impairment - General Health (WPAI-GH) Questionnaire. Sao Paulo Medical Journal, 124(6), 325–332. https://doi.org/10.1590/S1516-31802006000600005
Cimiotti, J. P., Aiken, L. H., Sloane, D. M., & Wu, E. S. (2012). Nurse staffing, burnout, and health care-associated infection. American Journal of Infection Control, 40(6), 486–490. https://doi.org/10.1038/jid.2014.371
Coatsworth, K., Hurley, J., & Miller-Rosser, K. (2015). A phenomenological study of student nurses volunteering in Nepal: Have their experiences altered their understanding of nursing? Collegian. https://doi.org/10.1016/j.colegn.2016.07.003
Dagne, T., Beyene, W., & Berhanu, N. (2015). Motivation and Factors Affecting it… Motivation and Factors Affecting It among Health Professionals in the Public Hospitals, Central Ethiopia. Ethopian Journal of Health Science, 25(3), 231–242. https://doi.org/10.4314/ejhs.v25i3.6
de Vet, H. C. W., Terwee, C. B., Knol, D. L., & Bouter, L. M. (2006). When to use agreement versus reliability measures. Journal of Clinical Epidemiology, 59(10), 1033–1039. https://doi.org/10.1016/j.jclinepi.2005.10.015
DeVon, H. a., Block, M. E., Moyle-Wright, P., Ernst, D. M., Hayden, S. J., Lazzara, D. J., … Kostas-Polston, E. (2007). A psychometric toolbox for testing validity and reliability [Electronic Version]. Journal of Nursing Scholarship, 39(2), 155–164. https://doi.org/10.1111/j.1547-5069.2007.00161.x
Dubois, C.-A., D’Amour, D., Pomey, M.-P., Girard, F., & Brault, I. (2013). Conceptualizing performance of nursing care as a prerequisite for better measurement: a systematic and interpretive review. BMC Nursing, 12(1), 7. https://doi.org/10.1186/1472-6955-12-7
Epstein, J., Osborne, R. H., Elsworth, G. R., Beaton, D. E., & Guillemin, F. (2015). Cross-cultural adaptation of the Health Education Impact Questionnaire: Experimental study showed expert committee, not back-translation, added value. Journal of Clinical Epidemiology, 68(4), 360–369. https://doi.org/10.1016/j.jclinepi.2013.07.013
Ferrer, M., Alonso, J., Prieto, L., Plaza, V., Monsó, E., Marrades, R., … Antó, J. M. (1996). Validity and reliability of the St George’s respiratory questionnaire after adaptation to a different language and culture: The Spanish example. European Respiratory Journal, 9(6), 1160–1166. https://doi.org/10.1183/09031936.96.09061160
Fleiszer, A. R., Semenic, S. E., Ritchie, J. A., Richer, M. C., & Denis, J. L. (2015). An organizational perspective on the long-term sustainability of a nursing best practice guidelines program: A case study. BMC Health Services Research, 15(1), 1–16. https://doi.org/10.1186/s12913-015-1192-6
Flinkman, M., Leino-Kilpi, H., Numminen, O., Jeon, Y., Kuokkanen, L., & Meretoja, R. (2017). Nurse Competence Scale: a systematic and psychometric review. Journal of Advanced Nursing, 73(5), 1035–1050. https://doi.org/10.1111/jan.13183
Gisev, N., Bell, J. S., & Chen, T. F. (2013). Interrater agreement and interrater reliability: Key concepts, approaches, and applications. Research in Social and Administrative Pharmacy, 9(3), 330–338. https://doi.org/10.1016/j.sapharm.2012.04.004
Guyatt, G., Walter, S., & Norman, G. (1987). Measuring Change Over Time- Aseessing the Usefulness of Evaluative Instruments. Journal of Chronic Diseases, 40(2), 171–178.
Horgas, A. L., Yoon, S. L., Nichols, A. L., & Marsiske, M. (2008). Is the CVI an Acceptable Indicator of Content Validity? Appraisal and Recommendations. Research in Nursing & Health, 31(4), 341–354. https://doi.org/10.1002/nur
Huicho, L., Dieleman, M., Campbell, J., Codjia, L., Balabanova, D., Dussault, G., & Dolea, C. (2010). Increasing access to health workers in underserved areas: A conceptual framework for measuring results. Bulletin of the World Health Organization, 88(5), 357–363. https://doi.org/10.2471/BLT.09.070920
Jaakkimainen, L., Glazier, R., Barnsley, J., Salkeld, E., Lu, H., & Tu, K. (2014). Waiting to see the specialist: patient and provider characteristics of wait times from primary to specialty care. BMC Family Practice, 15(1), 16. https://doi.org/10.1186/1471-2296-15-16
Johns, G. (2010). Presenteeism in the workplace: a review and research agenda. Journal of Organizational Behavior, 31, 519–542. https://doi.org/10.1002/job.630
Kalisch, J. B., Lee, H., & Salas, E. (2012). The development and testing of nursing teamwork survey. In F. D. Polit & C. T. Beck (Eds.), Resource Manual for Nursing Research: Generating and Assessing Evidence for Nursing Practice (9th ed., pp. 338–351). Lippincott Williams & Wilkins.
Kanchanachitra, C., Lindelow, M., Johnston, T., Hanvoravongchai, P., Lorenzo, F. M., Huong, N. L., … Dela Rosa, J. F. (2011). Human resources for health in southeast Asia: Shortages, distributional challenges, and international trade in health services. The Lancet, 377(9767), 769–781. https://doi.org/10.1016/S0140-6736(10)62035-1
Kassa, H., Murugan, R., Zedwu, F., Hailu, M., & Woldeyohannes, D. (2014). Assessment of knowledge , attitude and practice and associated factors towards palliative care among nurses working in selected hospitals , Addis Ababa , Ethiopia. BMC Palliative Care, 13(4), 1–11. https://doi.org/10.1186/1472-684X-13-6
Kitchenham, B., & Pfleeger, S. L. (2002). Principles of survey research part 3: Constructing a survey instrument. ACM SIGSOFT Software Engineering Notes, 27(2), 20. https://doi.org/10.1145/638574.638580
Kneafsey, R., Clifford, C., & Greenfield, S. (2013). What is the nursing team involvement in maintaining and promoting the mobility of older adults in hospital? A grounded theory study. International Journal of Nursing Studies, 50(12), 1617–1629. https://doi.org/10.1016/j.ijnurstu.2013.04.007
Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28(4), 563–575. https://doi.org/10.1111/J.1744-6570.1975.TB01393.X
Leach, L. S., & Mayo, A. M. (2013). Rapid Response Teams : qualitative analysis of their effectiveness. American Journal of Critical Care, 22(3), 198–210.
Lemieux-Charles, L., & McGuire, W. L. (2006). What Do We Know about Health Care Team Effectiveness? A Review of the Literature. Medical Care Research and Review, 63(3), 263–300. https://doi.org/10.1177/1077558706287003
Lutwama, G. (2011). The performance of health workers in decentralised services in Uganda. Retrieved from http://umkn-dsp01.unisa.ac.za/handle/10500/4866
Lynn, M. R. (1986). Determination and quantification of content validity. Nursing Reserach, 35(6), 382–385. https://doi.org/http://dx.doi.org/10.1097/00006199-198611000-00017
Mackenzie, S. B., & Podsakoff, P. M. (2012). Commentary on " Common Method Bias : Nature , Causes , and Procedural Remedies ". Journal of Retailing, 88(January), 542–555. https://doi.org/10.1016/j.jretai.2012.08.001
Mackenzie, S. B., Podsakoff, P. M., & Podsakoff, N. P. (2011). Construct measurement and validation procedures in MIS and Behavioral Research : Integrating New and Existing Techniques. MIS Quarterly, 35(2), 293–334.
Makai, P., Cramm, J. M., van Grotel, M., & Nieboer, A. P. (2014). Labor productivity, perceived effectiveness, and sustainability of innovative projects. Journal for Healthcare Quality : Official Publication of the National Association for Healthcare Quality, 36(2), 14–24. https://doi.org/10.1111/j.1945-1474.2012.00209.x
Manojlovich, M., & Sidani, S. (2008). Nurse dose: What’s in a concept? Research in Nursing and Health, 31(4), 310–319. https://doi.org/10.1002/nur.20265
McElroy, C., & Esterhuizen, P. (2017). Compassionate communication in acute healthcare: establishing the face and content validity of a questionnaire. Journal of Research in Nursing, 22(1–2), 72–88. https://doi.org/10.1177/1744987116678903
Mehmet, T. (2013). Organizational variables on nurses’ job performance in Turkey: Nursing Assesments. Iranian Journal of Public Health, 42(3), 261–271. https://doi.org/10.2337/db06-1182.J.-W.Y.
Meretoja, R., Isoaho, H., & Leino-Kilpi, H. (2004). Nurse Competence Scale: development and psychometric testing. Journal of Advanced Nursing, 47(2), 124–133. https://doi.org/10.1111/j.1365-2648.2004.03071.x
Meretoja, R., & Koponen, L. (2012). A systematic model to compare nurses’ optimal and actual competencies in the clinical setting. Journal of Advanced Nursing, 68(2), 414–422. https://doi.org/10.1111/j.1365-2648.2011.05754.x
Mert, T., & Ekici, D. (2015). Development of an Assessment Model for Evaluating the Performance of Nursing Services. International Journal of Hospital Research, 4(1), 9–14.
Meyer, M. A., & Booker, J. M. (2001). Eliciting and Analysing Expert Judgement A practical Guide. American Statistical Association and the Society for Industrial and Applied Mathematics.
Needleman, J., Kurtzman, E. T., & Kizer, K. W. (2007). Performance measurement of nursing care : State of the science and the current consensus. Medical Care Research and Review, 64(2), 10S–43S. https://doi.org/10.1177/1077558707299260
Netemeyer, R., Bearden, W., & Sharma, S. (2003). Scaling Procedures Issues and Applications. Thousand Oak, London: SAGE Publications. https://doi.org/10.4135/9781412985772
North, N., & Hughes, F. (2012). A systems perspective on nursing productivity. Journal of Health Organization and Management, 26(2), 192–214. https://doi.org/10.1108/14777261211230772
Numminen, O., Leino-Kilpi, H., Isoaho, H., & Meretoja, R. (2015). Newly Graduated Nurses’ Competence and Individual and Organizational Factors: A Multivariate Analysis. Journal of Nursing Scholarship, 47(5), 446–457. https://doi.org/10.1111/jnu.12153
Numminen, O., Meretoja, R., Isoaho, H., & Leino-Kilpi, H. (2013). Professional competence of practising nurses. Journal of Clinical Nursing, 22(9–10), 1411–1423. https://doi.org/10.1111/j.1365-2702.2012.04334.x
Nunnally, J. C., & Bernstein, I. (1994). Psychometric Theory (3rd Editio). McGraw-Hill, Inc.
Olsson, C., Forsberg, A., & Bjersa, K. (2016). Safety climate and readiness for implementation of evidence and person centered practice - A national study of registered nurses in general surgical care at Swedish university hospitals. BMC Nursing, 15(1), 54. https://doi.org/10.1186/s12912-016-0174-2
Pillay, M. S., Johari, R., Hazilah, M., Asaari, A., Azman, B., Salikin, F., … Ismefariana, I. W. (2011). Hospital waiting time : the forgotten premise of healthcare service delivery ? International Journal of Healthcare, 24(7), 506–522. https://doi.org/10.1108/09526861111160553
Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2016). Recommendations for Creating Better Concept Definitions in the Organizational, Behavioral, and Social Sciences. Organizational Research Methods, 19(2), 159–203. https://doi.org/10.1177/1094428115624965
Polit, D. F., & Beck, C. T. (2006). The content Validity Index: Are You Sure You Know What’s Being Reported? Critique and Recommendations. Research in Nursing & Health, 29, 489–497. https://doi.org/10.1002/nur
Polit, D. F., Beck, C. T., & Owen, S. V. (2007). Is the CVI an Acceptable Indicator of Content Validity? Appraisal and Recommendations. Research in Nursing & Health, 30, 459–467. https://doi.org/10.1002/nur.20199
Rattray, J., & Jones, M. C. (2007). Essential elements of questionnaire design and development. Journal of Clinical Nursing, 16(2), 234–243. https://doi.org/10.1111/j.1365-2702.2006.01573.x
Raykov, T. (2008). “Alpha if Deleted” and Loss in Criterion Validity Appeared in. British Journal of Mathematical and Statistical Psychology, 61(2), 275–285.
Rowe, A. K., De Savigny, D., Lanata, C. F., & Victora, C. G. (2005). How can we achieve and maintain high-quality performance of health workers in low-resource settings? Lancet, 366(9490), 1026–1035. https://doi.org/10.1016/S0140-6736(05)67028-6
Sand-Jecklin, K., & Sherman, J. (2014). A quantitative assessment of patient and nurse outcomes of bedside nursing report implementation. Journal of Clinical Nursing, 23(19–20), 2854–2863. https://doi.org/10.1111/jocn.12575
Scott, T., Mannion, R., Davies, H., & Marshall, M. (2003). Methods The Quantitative Measurement of Organizational Culture in Health Care : A Review of the Available Instruments. Health Services Research, 38(3), 923–945.
Sheaff, R., Pickard, S., & Smith, K. (2002). Public service responsiveness to users’ demands and needs: theory, practice and primary healthcare in England. Public Administration, 80(3), 435–452. https://doi.org/10.1111/1467-9299.00312
Smith, S. (2012). Nurse Competence : A Concept Analysis. International Journal of Nursing Knowledge, 23(3), 172–182.
Sonnentag, S., Volmer, J., & Spychala, A. (2008). Job Performance. The SAGE Handbook of Organizational Behavior, 1, 427–447. https://doi.org/10.4135/9781849200448
Stolt, M., Charalambous, A., Radwin, L., Adam, C., Katajisto, J., Lemonidou, C., … Suhonen, R. (2016). Measuring trust in nurses ??? Psychometric properties of the Trust in Nurses Scale in four countries. European Journal of Oncology Nursing, 25, 46–54. https://doi.org/10.1016/j.ejon.2016.09.006
Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2, 53–55. https://doi.org/10.5116/ijme.4dfb.8dfd
Topf, M. (1986). Three estimates of interrater reliability for nominal data. Methodology Corner, 35(4), 253–255.
Umann, J. ., Guido, L. A. ., & Grazziano, E. S. . (2012). Presenteeism in hospital nurses [Presenteísmo em enfermeiros hospitalares]. Revista Latino-Americana de Enfermagem, 20(1), 159–166. https://doi.org/10.1590/S0104-11692012000100021
Vanessa, A., Rodrigues, D., Vituri, D. W., Louren, C., Terezinha, M., Vannuchi, O., & Tiago, W. (2012). Nursing responsiveness the client ’ s view. Rev Esc Enferm USP, 46(6), 1446–1452.
Viswesvaran, C., & Ones, D. S. (2000). Perspectives on Models of Job Performance. International Journal of Selection and Assessment, 8(4), 216–226. https://doi.org/10.1111/1468-2389.00151
WHO. (2006). Working together For Health, The WHO Health Report 2006. World Health (Vol. 19). https://doi.org/10.1186/1471-2458-5-67
WHO. (2010). Increasing Access To Health Workers In Remote And Rural Areas Through Improved Retention: Global Policy Recommendations. WHO. WHO Press.
Wynd, C. A., Schmidt, B., & Schaefer, M. A. (2003). Two quantitative approaches for estimating content validity. Western Journal of Nursing Research, 25(5), 508–518. https://doi.org/10.1177/0193945903252998
Yakusheva, O., & Weiss, M. (2017). Rankings matter: nurse graduates from higher-ranked institutions have higher productivity. BMC Health Services Research, 17(1), 134. https://doi.org/10.1186/s12913-017-2074-x
Zamanzadeh, T and Nemati, N. (2014). Details of content validity and objectifying it in instrument development. Nursing Practice Today, 1(3), 163–171.
This work is licensed under a Creative Commons Attribution 4.0 International License.