https://orcid.org/0000-0001-6387-2496
Abstract
Background
Job satisfaction is one of the main factors creating and bringing about work motivation, productivity, and efficiency as well as decreasing job-hopping and job turnover. No previous studies have been conducted to assess job satisfaction for community pharmacists in Vietnam.
Objective
This research was conducted to develop and validate an instrument used to measure community pharmacists’ job satisfaction in the context of the COVID-19 pandemic in Vietnam.
Methods and results
A total of 351 pharmacists participated in this survey. Data were analyzed using R software version 4.2.0. The final instrument (VIJS) has 34 items divided into six factors which were determined via a parallel analysis (including physical working conditions, work nature, income and other benefits, management policies and managers, relationships with coworkers and customers, and learning and advancement opportunities). VIJS’s internal consistency was excellent (Cronbach’s alpha = 0.97, Omega total = 0.98, split-half reliability = 0.985, and composite reliability>0.8). Two-week test-retest reliability results (intraclass correlation coefficient for the overall instrument: 0.97, for six factors: 0.865–0.938) demonstrated the consistency of the VIJS when the same test was repeated on the same sample (62 pharmacists) at different points in time. The Confirmatory Factor Analysis was employed to assess the construct validity. The VIJS was a good fit to a six-factor model (Chisq/df = 2.352, Comparative Fit Index = 0.937, Tucker-Lewis Index = 0.929, Standardized Root Mean Square Residual = 0.042, and Root Mean Square Error of Approximation = 0.062). VIJS’s good convergent and discriminant validity was demonstrated via Average Variance Extrated>0.5 and the Heterotrait-Monotrait ratio of correlations<0.85.
Figures
Citation: Nguyen TTP, Truong GTH, Nguyen HTT, Nguyen CTT, Dinh DX, Nguyen BT (2022) An instrument for measuring job satisfaction (VIJS): A validation study for community pharmacists in the context of the COVID-19 pandemic in Vietnam. PLoS ONE 17(11): e0276918. https://doi.org/10.1371/journal.pone.0276918
Editor: Jahanpour Alipour, Zahedan University of Medical Sciences, ISLAMIC REPUBLIC OF IRAN
Received: June 8, 2022; Accepted: October 15, 2022; Published: November 4, 2022
Copyright: © 2022 Nguyen et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
In this modern world, a plethora of occupations have arisen and therefore people have numerous opportunities to select the career they like to work. Job satisfaction is one of the main factors affecting job performance and employee turnover. A high level of job satisfaction can help to create and bring about work motivation, productivity, and efficiency as well as decrease job-hopping and job turnover. For professional jobs such as health workers and community pharmacists, job satisfaction also affects patient care [1, 2].
The magnitude of job satisfaction of pharmacists around the world varies considerably. The satisfaction rate of Indian pharmacists was only 17.5% [3]. In Baghdad (Iraq), community pharmacists were extremely dissatisfied with their colleagues (86.0%), recognition (72.2%), and public respect (42.0%) [4]. In Arizona (the United States), 25% of chain community pharmacists were dissatisfied with the recognition they got for good work [5]. A moderate level of job satisfaction was witnessed among Malaysian community pharmacists. Among 13 factors used to measure job satisfaction, perceived societal status, professional responsibility, job security, and work demand were the job characteristics that pharmacists were least satisfied with [6].
Many facets contributing to the job satisfaction of pharmacists were reported, including stress, workload, supervisors/managers, coworkers, advancement opportunities, and income, to mention but a few [2, 7–9]. The most common reasons for pharmacists’ job quitting and leaving intentions were dissatisfaction with their working environment, low remuneration, and a lack of promotion opportunities [10]. In comparison with pharmacists working in hospitals and primary care sectors, community pharmacists were less satisfied with their work [11, 12]. Remuneration ranked first in the aspects of pharmacists’ work that they felt least satisfied with, regardless of their age, sex, or the type of sector [12]. For Swedish pharmacists, a lower level of job satisfaction was associated with increasing years in the current job position [13]. Increasing workloads and losing income can be repercussions of the COVID-19 pandemic and affect the job satisfaction of not only pharmacists but also a myriad of other occupations [12, 14].
In Vietnam, up to the year 2021, only one study was carried out to measure job satisfaction for clinical pharmacists in hospitals [15]. No previous studies have been conducted to assess job satisfaction for community pharmacists in Vietnam. There are approximately 82,000 community pharmacists who are working in medicine outlets all over Vietnam. The outbreak of the COVID-19 pandemic has given rise to myriad negative impacts on people’s lives (especially for health workers and community pharmacists). This research was conducted to develop and validate a self-administered instrument to measure the job satisfaction of community pharmacists in the context of the COVID-19 pandemic in Vietnam.
Methods
Setting and participants
This cross-sectional study was conducted in Hanoi, Vietnam after approved by the Institutional Review Board and Ethical Committee of Hanoi University of Pharmacy (reference number 125/QĐ-DHN, 22-06/PCT-HĐĐĐ). The study population was community pharmacists working in medicine outlets in the Hanoi capital (roughly 12.000 pharmacists). The inclusion criteria for selecting participants included community pharmacists licensed to sell medicines (medicine sellers), being Vietnamese natives, at least 18 years old, and working in medicine outlets opened on the days of data collection. By reason of the COVID-19 outbreak, pharmacists were recruited using a non-probability convenience sampling technique.
For studies using Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA), as per the rules of thumb, a suggested sample size for one item/question is from 3 to 20 people. A sample of 300 people is adequate for factor analysis [16–21]. Among 450 pharmacists approached, 351 pharmacists (response rate: 78.0%) voluntarily answered a questionnaire used to measure their job satisfaction and informed consent was obtained from all of them. Two weeks after the first survey, 62 invited pharmacists agreed to answer this questionnaire one more time. The minimum sample size for evaluating the test-retest reliability was 51 people (with an expected intraclass correlation coefficient (ICC): 0.75, a type 1 error rate: 5%, a 95% confidence interval width: 0.2) [22].
Instrument development and data collection
A pool of items was developed by reviewing numerous articles and scales/instruments involving job satisfaction [3–6, 12, 23–26]. After this initial review, seven main factors which can affect the job satisfaction of community pharmacists were selected (including physical working conditions, work nature, income and other benefits, management policies and managers, relationships with coworkers, relationships with patients/customers, and learning and advancement opportunities). The number of items for these factors was 6, 11, 6, 12, 6, 5, and 7 items, respectively. The initial instrument included 53 items involving job satisfaction.
A Vietnamese self-administered questionnaire was designed (including a short introduction about this survey, a confirmation of voluntary participation in this research, and the two following main parts). The first part included eight questions involving the background information of pharmacists and their medicine outlets (including sex, year of birth, marital status, level of education, working experience, time of working per day, salary per month, and the average number of customers per day of outlets). The second part was the initial instrument including 53 items with 5-point Likert scale responses ((1) Strongly disagree, (2) Disagree, (3) Normal, (4) Agree, and (5) Strongly agree). A higher score represented a higher level of job satisfaction. This questionnaire was designed on Google Forms. From March to April 2022, data collectors paid a visit to medicine outlets and invited pharmacists to directly answer this questionnaire on an Ipad. After the process of data analysis, nineteen items were removed because their ICCs were low (<0.5) (7 items), item loadings (EFA) were low (<0.5) (8 items), and/or Cronbach’s alphas (α) for factors increased if these items were dropped (6 items). The final instrument (VIJS) includes 34 items (Table 1).
Data analysis
After collected, data were extracted into a Microsoft Excel file and analyzed using R software version 4.2.0 (packages tidyverse, psych, irr, EFAtools, GPArotation, performance, lavaan, semTools, and semPlot). The mean (standard deviation) and number (percentage) were used to report information for continuous variables and categorical variables, respectively.
Regarding the reliability of the VIJS, internal consistency was assessed through α, split-half reliability, composite reliability (CR), and McDonald’s Omega (Omega Total ωt and Omega Hierarchical ωh) with the higher values indicating the better consistency. α≥0.7 and ≥0.8 indicate acceptable and good internal consistency, respectively [20, 27]. CR>0.70 is interpreted a good reliability [20, 28]. To evaluate split-half reliability, items were randomly divided into two parts and the correlation between these two parts was computed. ωh and ωt which should be over 0.70 can be used to present the proportion of the variance of the total scores due to the general factor and due to all factors, respectively [28–30]. ICC was used to evaluate the test-retest reliability—the reliability of a test measured at different points in time (<0.5: poor, 0.50–0.75: moderate, and >0.75: good reliability) [31].
The construct validity of the VIJS was assessed using EFA and CFA (with principal axis factoring analysis, promax rotation). CFA, a type of structural equation modeling (SEM), can be used to test how well the measured items represent a set of theoretical latent factors. For studies with more than 250 people and instruments having 30 items or more, the main fit indices used for evaluating the CFA/SEM model include the p-value expected for the Chi-squared test (<0.001), Comparative Fit Index (CFI>0.92), Tucker-Lewis Index (TLI>0.92), Standardized Root Mean Square Residual (SRMR<0.08), and Root Mean Square Error of Approximation (RMSEA<0.07) [20]. Other fit indices which can be used include the Goodness-of-Fit statistic and the Adjusted Goodness-of-Fit statistic (GFI, AGFI>0.9: good, 0.8–09: acceptable); Root Mean Square Residual (RMR<0.08); Incremental Fit Indices (IFI>0.9); the Parsimony Goodness-of-Fit Index and the Parsimonious Normed Fit Index (PGFI, PNFI>0.5) [32].
The external validity of the VIJS was evaluated via convergent and discriminant validity. The Average Variance Extracted (AVE) which is used to measure convergent validity should be at least 0.50. The Heterotrait-Monotrait ratio of correlations (HTMT—the ratio of the between-trait correlations to the within-trait correlations) should be lower than 0.85 [20].
Results
Participants’ characteristics
A total of 351 community pharmacists participated in this survey. More than four-fifths of participants were women and about two-fifths of them were married. Most pharmacists were aged from 21 to 35 years old (84.33%). Their average age was 28.85±7.24 years old. About two-fifths of the participants were university graduates. Pharmacists’ working experience in medicine outlets was mostly less than 10 years (82.05%). In medicine outlets, pharmacists mainly worked eight hours a day and earned a monthly salary of four million Vietnam dongs or more. On average, the number of customers of medicine outlets ranged from 30 to 100 people per day (71.79%) (Table 2).
Construct validity (EFA and CFA)
The final VIJS included 34 items which were divided into six factors: Factor A: Physical working conditions: 5 items, Factor B: Work nature: 5 items, Factor C: Income and other benefits: 4 items, Factor D: Management policies and managers: 7 items, Factor E: Relationships with coworkers and customers (7 items), and Factor F: Learning and advancement opportunities: 6 items (Table 1). To assess the factorability of the data, Bartlett’s test of sphericity was statistically significant (with Chisq = 10,876.8, p-value<0.001) and the Kaiser-Meyer-Olkin measure of sampling adequacy (KMO) = 0.954>0.5. The number of factors to extract (n = 6 factors) was determined using a parallel analysis (Fig 1). Item loadings (cutoff = 0.5) from a principal axis factoring analysis with promax rotation were presented in Table 3. The communality values of all 34 items were higher than 0.46. For six extracted factors, the cumulative percentage of variance accounted for was 69.25>50.0.
EFA: Exploratory Factor Analysis.
Results from CFA showed that the final six-factor model had a good fit with CMIN/df (Chisq/df) = 2.352 (<3), p-value = 0.000, CFI = 0.937 (>0.92), TLI = 0.929 (>0.92), GFI = 0.841 (>0.8), AGFI = 0.812 (>0.8), NNFI = 0.929 (>0.9), NFI = 0.896 (>0.8), IFI = 0.937 (>0.9), RNI = 0.937 (>0.9), RMR = 0.023 (<0.05), SRMR = 0.042 (<0.05), RMSEA = 0.062 (<0.07), (95%CI: 0.057–0.067, p-value = 0.000), PNFI = 0.801 (>0.5), and PGFI = 0.710 (>0.5). In addition, six factors had moderate/strong correlations with one another (correlation coefficients: from 0.56 to 0.84). Each factor provided an acceptable explanation of variation found among its items with standardized loadings from 0.66 to 0.94 and error variances from 0.12 to 0.57 (Fig 2). Overall, the data indicated that the VIJS has good construct validity.
Factor A: Physical working conditions, Factor B: Work nature, Factor C: Income and other benefits, Factor D: Management policies and managers, Factor E: Relationships with coworkers and customers, Factor F: Learning and advancement opportunities.
The reliability
The good internal consistency of the VIJS was demonstrated through the high values of split-half reliability (0.985), α (0.97), ωt (0.98), and ωh (0.83) for the overall instrument with 34 items. The Spearman-Brown split-half reliability coefficients and α for all six factors were higher than 0.85. CRs for all six factors were also higher than 0.8. The item-total correlation coefficients of all 34 items ranged from 0.74 to 0.95. Furthermore, the VIJS possessed a high level of two-week test-retest reliability with an ICC of 0.974 for the overall 34-item instrument and from 0.896 to 0.938 for six factors (p-values<0.001) (Tables 1 and 4).
The external validity
For all six factors, AVEs were higher than 0.50. The HTMTs of all pairs of factors were lower than 0.85 (Table 4). As a result, the convergent and discriminant validity of the VIJS was good.
Discussion
Job satisfaction is an important factor that positively affects people’s working motivation, commitment, performance, and productivity; negatively links to complaints, tardiness/absenteeism, job-hopping, and job turnover. In Vietnam, there are no previous studies conducted to assess job satisfaction for Vietnamese community pharmacists. In this research, the authors strived to develop and validate a instrument (VIJS) that can be used to measure job satisfaction for community pharmacists. Results from this survey with the participation of 351 pharmacists demonstrated that this instrument possesses good reliability and validity.
The VIJS possesses good internal consistency and test-retest reliability. α for six factors (from 4 to 7 items per factor) ranged from 0.87 to 0.94, consistent with results from several studies using the Warr-Cook-Wall 10-item scale: α = 0.90 (for pharmacists in the United Kingdom) [25] and α = 0.87 (for community pharmacists in Baghdad, Iraq) [4]; JS-Q (34 items, 8 domains, α from 0.751 to 0.930) [33] but higher than the result of a study for pharmacists in Malaysia (15-item scale, α = 0.78) [34]. Although α is the most widely used method for evaluating internal consistency, the omega coefficient is a better choice than α [35]. High values of omega coefficients (ωt and ωh≥0.8), the split-half reliability coefficients (>0.86), and CRs>0.8 were additional evidence for the good reliability of the VIJS. The item-total correlation coefficient of an item (ranging from 0 to 1) indicates whether the response of this item is consistent with its factor. The item-total correlation coefficients of all 34 items (from 0.74 to 0.95) showed that VIJS’s items measuring different factors delivered consistent scores of job satisfaction. In addition, the results of a two-week test-retest reliability (ICC for the overall instrument: 0.97, ICCs for each factor: from 0.865 to 0.938, ICCs for each item: >0.5) showed the consistency of this instrument when the same test was repeated on the same sample (62 people) at different points in time (two weeks after the first survey).
In CFA, the Chi-squared statistic is one of the absolute fit indices used to measure how well the model reproduces the observed data and the p-value is commonly higher than 0.05. However, numerous factors can impact this test, including the number of observations (participants) and the number of items of instruments/scales. In this study, the number of participants was 351 (>250) and the number of items of the VIJS was 34 (>30). The Chi-squared test had a p-value<0.001 (significant p-value), consistent with information from a book from Hair [20]. Other absolute fit indices (GFI>0.8, RMSEA<0.08, RMR<0.05, SRMR<0.05), incremental fit indices (NFI>0.89, NNFI>0.9, TLI>0.92, CFI>0.92, RNI>0.9), and parsimony fit indices (AGFI>0.8, PNFI>0.5, PGFI>0.5) revealed that the VIJS was a good fit to a six-factor model.
Besides good construct validity, the good convergent and discriminant validity of the VIJS can be demonstrated via AVEs>0.5 and HTMTs<0.85. AVE is the mean variance extracted for items loading on a construct/factor. AVE>0.5 represents that a latent variable/factor can explain more than 50% of the variance of its observed variables/items [20]. To assess the discriminant validity, the HTMT was used in this study. Numerous previous studies used the Fornell-Lacker criterion and the examination of cross-loadings. However, a simulation study revealed that in common study situations, the aforementioned criterion and testing did not reliably detect the lack of discriminant validity and authors recommended the use of a new criterion (HTMT) to assess the discriminant validity in variance-based structural equation modeling [36].
Overall, the final instrument includes 34 items divided into six factors: (1) Physical working conditions: 5 items, (2) Work nature: 5 items, (3) Income and other benefits: 4 items, (4) Management policies and managers: 7 items, (5) Relationships with coworkers and customers: 7 items, and (6) Learning and advancement opportunities: 6 items. Results from CFA and test-retest reliability demonstrated the good reliability and validity of the VIJS. However, this research has several limitations. Firstly, participants were selected using a convenience sampling strategy in the Hanoi capital. As a consequence, the study sample cannot be representative of Vietnamese community pharmacists. Furthermore, because of the length of the initial instrument, the authors did not use other job satisfaction scales/instruments to measure job satisfaction and evaluate the VIJS’s concurrent validity.
Conclusions
Results from this study demonstrated that the VIJS (34 items, six factors) is an instrument possessing good reliability and validity. Future studies can use this instrument to measure the job satisfaction of community pharmacists in other locations or countries.
Acknowledgments
The authors would like to thank Quynh Diem Dieu and Anh Minh Hoang, two students from the Hanoi University of Pharmacy, who assisted us in data collection. We also appreciate all 351 community pharmacists who enthusiastically participated in this research.
References
- 1.
Spector PE. Job satisfaction: application, assessment, causes and consequences. 1st ed. Sage Publications; 1997.
- 2. Carvajal MJ, Popovici I. Gender, age, and pharmacists’ job satisfaction. Pharm Pract (Granada). 2018;16:1396. pmid:30637036
- 3. Ahmad A, Khan MU, Elkalmi RM, Jamshed SQ, Nagappa AN, Patel I, et al. Job satisfaction among Indian pharmacists: An exploration of affecting variables and suggestions for improvement in pharmacist role. Indian Journal of Pharmaceutical Education and Research. 2016;50:9–16.
- 4. Ibrahim IR, Ibrahim MI, Majeed IA, Alkhafaje Z. Assessment of job satisfaction among community pharmacists in Baghdad, Iraq: a cross-sectional study. Pharm Pract (Granada). 2021;19:2190. pmid:33777262
- 5. Hincapie AL, Yandow S, Hines S, Martineau M, Warholak T. Job satisfaction among chain community pharmacists: results from a pilot study. Pharm Pract (Granada). 2012;10:227–33. pmid:24155841
- 6. Teong WW, Ng YK, Paraidathathu T, Chong WW. Job satisfaction and stress levels among community pharmacists in Malaysia. J Pharm Pract Res. 2019;49:9–17.
- 7. Berassa MS, Chiro TA, Fanta S. Assessment of job satisfaction among pharmacy professionals. J Pharm Policy Pract. 2021;14:71. pmid:34419161
- 8. Majd M, Hashemian F, Younesi Sisi F, Jalal M, Majd Z. Quality of Life and Job Satisfaction of Dispensing Pharmacists Practicing in Tehran Private-sector Pharmacies. Iran J Pharm Res. 2012;11:1039–44. pmid:24250534
- 9. Radwan RM, Bentley JP, Patterson JA, Dixon DL, Salgado TM. Predictors of job satisfaction among pharmacists: A regional workforce survey. Explor Res Clin Soc Pharm. 2022;5:100124. pmid:35478529
- 10. Aspden TJ, Silwal PR, Marowa M, Ponton R. Why do pharmacists leave the profession? A mixed-method exploratory study. Pharm Pract (Granada). 2021;19:2332. pmid:34221201
- 11. Al Khalidi D, Wazaify M. Assessment of pharmacists’ job satisfaction and job related stress in Amman. Int J Clin Pharm. 2013;35:821–8. pmid:23807142
- 12. Seston E, Hassell K, Ferguson J, Hann M. Exploring the relationship between pharmacists’ job satisfaction, intention to quit the profession, and actual quitting. Res Social Adm Pharm. 2009;5:121–32. pmid:19524860
- 13. Gustafsson M, Mattsson S, Wallman A, Gallego G. Pharmacists’ satisfaction with their work: Analysis of an alumni survey. Res Social Adm Pharm. 2018;14:700–704. pmid:28870444
- 14. Tran BX, Nguyen HT, Le HT, Latkin CA, Pham HQ, Vu LG, et al. Impact of COVID-19 on Economic Well-Being and Quality of Life of the Vietnamese During the National Social Distancing. Front Psychol. 2020;11:565153. pmid:33041928
- 15. Nguyen-Thi HY, Nguyen-Ngoc TT, Do-Tran MT, Do DV, Pham LD, Le NDT. Job satisfaction of clinical pharmacists and clinical pharmacy activities implemented at Ho Chi Minh city, Vietnam. PLoS One. 2021;16:e0245537. pmid:33481846
- 16. MacCallum RC, Widaman KF, Zhang S, Hong S. Sample size in factor analysis. Psychol Methods. 1999;4:84–99.
- 17.
Brown Timothy A. Confirmatory Factor Analysis for Applied Research. The Guilford Press, New York; 2015.
- 18.
Tabachnick BG, Fidell LS. Using Multivariate Statistics (7th edition). Boston: Pearson Education Inc; 2019.
- 19.
Comrey AL, Lee HB. A First Course in Factor Analysis (2nd edition). Psychology Press; 1992. https://doi.org/10.4324/9781315827506
- 20.
Hair Joseph F Jr,Black William C, Babin Barry J, Anderson Rolph E. Multivariate Data Analysis. 8th ed. Hampshire, UK Cengage Learning, EMEA; 2019.
- 21.
Thompson B. Exploratory and confirmatory factor analysis: Understanding concepts and applications. American Psychological Association; 2004.
- 22. Bonett DG. Sample size requirements for estimating intraclass correlations with desired precision. Stat Med. 2002;21:1331–5. pmid:12111881
- 23. Foroughi Moghadam MJ, Peiravian F, Naderi A, Rajabzadeh A, Rasekh HR. An Analysis of Job Satisfaction among Iranian Pharmacists through Various Job Characteristics. Iran J Pharm Res. 2014;13:1087–96. pmid:25276212
- 24. Plianbangchang P. Survey of job satisfaction among pharmacists in Thailand. J Pharm Pract Res. 2003;33:117–120.
- 25. Hassell K, Seston E, Shann P. Measuring job satisfaction of UK pharmacists: a pilot study. Int J Pharm Prac. 2007;15:259–264.
- 26. Munger MA, Gordon E, Hartman J, Vincent K, Feehan M. Community pharmacists’ occupational satisfaction and stress: a profession in jeopardy? J Am Pharm Assoc. 2013;53:282–96. pmid:23699677
- 27. Cortina JM. What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology. 1993;78:98–104.
- 28.
Nunnally JC, Bernstein IH. Psychometric Theory. 3rd ed. New York: McGraw-Hill; 1994.
- 29.
McDonald RP. Test theory: A unified treatment. Psychology Press; 1999.
- 30. Green SB, Yang Y. Evaluation of dimensionality in the assessment of internal consistency reliability: Coefficient alpha and omega coefficients. Educat Measurem Iss Pract. 2015;34:14–20.
- 31. Koo TK, Li MY. A guideline of selecting and reporting Intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15:155–63. pmid:27330520
- 32. Hooper D, Coughlan J, Mullen M. Structural Equation Modelling: Guidelines for Determining Model Fit. Electronic Journal of Business Research Methods. 2008;6:53–60.
- 33. Ahmad NFD, Ren Jye AK, Zulkifli Z, Bujang MA. The Development and Validation of Job Satisfaction Questionnaire for Health Workforce. Malays J Med Sci. 2020;27:128–143. pmid:33447140
- 34. Chua GN, Yee LJ, Sim BA, Tan KH, Sin NK, Hassali MA, et al. Job satisfaction, organisation commitment and retention in the public workforce: a survey among pharmacists in Malaysia. Int J Pharm Pract. 2014;22:265–74. pmid:25121161
- 35. Trizano-Hermosilla I, Alvarado JM. Best Alternatives to Cronbach’s Alpha Reliability in Realistic Conditions: Congeneric and Asymmetrical Measurements. Front Psychol. 2016;7:769. pmid:27303333
- 36. Henseler J, Ringle CM, Sarstedt M. A new criterion for assessing discriminant validity in variance-based structural equation modeling. J of the Acad Mark Sci. 2015;43:115–135.
