Research Article: 2021 Vol: 27 Issue: 5
Angela Oktavia Suryani, Atma Jaya Catholic University
Benedicta Prihatin DWI Riyanti, Atma Jaya Catholic University
Christine Winstinindah Sandroto, Atma Jaya Catholic University
Citation Information: Suryani, A.O., Riyanti, B.P.D., & Sandroto, C.W. (2021). The construction of entrepreneurial competence test: situational judgment test model. Academy of Entrepreneurship Journal (AEJ), 27(5), 1-11.
This research aims to develop a psychological instrument measuring entrepreneurial competences in the situational judgment test model. All cases and choices of answers were collected through interviews with entrepreneurs. The experts reported the validation of the cases and scores weight of the alternative choices. The testing of the psychometric attributes involved 114 items and 191 entrepreneurs as participants from industrial cities in Indonesia. An EFA was executed with the Oblimin rotation method producing adequate and reliable three dimensions of competencies with 22 items. These three dimensions are the ability to see opportunities, influence others strategically, and the achievement ability. In the following research, psychometric testing is designed to involve external variables.
Entrepreneurial Competence Measurement, Exploratory Factor Analysis, Reliability, Situational Judgment Test, Validity.
The role of entrepreneurship is to contribute to developing countries (Indonesia etc.) and developed countries (the US and many European countries). Entrepreneurship has a significant contribution that is understood as a catalyst for the country's economic growth (Wuisang et al., 2019). Indonesian economy was mainly supported by Small Medium Enterprises (SMEs). Ministry of Cooperatives and Small and Medium Enterprises of the Republic of Indonesia data (2015) stated that in 2012, about 99.99% of the Indonesian economy is contributed by SMEs (Putra et al., 2019). Entrepreneurship has proven to improve the community's economic welfare. It may also carry out economic transformation and wealth (Solikahan & Mohammad, 2019). Especially in this turbulence situation in the industrial 4.0 era where the demands of competencies of human resources is increasing (Pontoh et al., 2021). The need for human resource competence is getting higher due to the COVID-19 pandemic. This need is also relevant for entrepreneurs. Sinha et al. (2021) stated that many studies have shown the impact of COVID-19 on business performance, one of which occurs in the food and beverage industry.
Conducting research on entrepreneurship competence is one way to provide support in developing entrepreneurship. Kula et al. (2020) proved that research in entrepreneurship has an impact on business performance. In this research we developed a tool of entrepreneur competencies to support entrepreneurship training/programs to help potential entrepreneurs gain sufficient competence to manage their businesses. Why is competence in the highlight? Competence is a skill in using or applying a collection of knowledge/thought patterns in a particular professional context (Lans et al., 2014). According to Mirza et al. (2020), skills can be improved over time with training and practicing before it becomes an ability. According to Jasson in Muniandy et al. (2020), the ability is a stable capability of people to perform in their job. Boyatzis (1982) defined competence as "the capacity of individuals who direct their behavior to meet job demands by the size (parameters) set by the organization, which will eventually achieve the expected results." Spencer & Spencer (1993); portrayed competencies as the significant characteristics of individuals associated with superior competencies or effective performance different from others. The competence covers aspects, namely skills, knowledge, attitudes, and even personal traits/characteristics (Shamuganathan, 2010). Saraswati & Putra (2020) stated that competency is inherent in someone who can predict performance level. It was further proven that competence is strongly associated with job success (Gangani et al., 2006). In entrepreneurship, entrepreneurial competence can lead people to achieve higher performance, growth, and economic development and increase welfare (Mitchelmore & Rowley, 2013).
Several studies showed a variety of competencies when explaining the entrepreneur's competency dimensions (Spencer & Spencer, 1993; Mitchelmore & Rowley, 2013; Chou et al., 2010). Many experts have explored entrepreneurial competencies, but they did not develop instruments measuring the competencies. In search of instruments for measuring entrepreneurial competencies, several instruments were found. However, they did not fully measure competence. One of those instruments is the Entrepreneurial Potential Assessment Inventory (EPAI) by (Santos et al., 2013). They used the instrument to explore the entrepreneurial potential among students and employees. Other instruments measured entrepreneurial orientation (Lampinen et al., 2018) and entrepreneurial intentions (Kusumawijaya, 2019). In Indonesia, Putri et al. (2016) constructed a competent entrepreneurial instrument. They built the instrument based on Man & Lau's concept (2001). However, this tool still has many weaknesses. Other researchers, namely Riyanti & Suwartono (2018), have also constructed a measure of entrepreneurial competence in the form of a self-report scale. We evaluated those instruments and found that they did not measure entrepreneurs' actual competence.
Considering previous studies on entrepreneurial competency measurements and instruments, we are interested in developing an instrument measuring entrepreneurial competency based on 13 dimensions of Spencer & Spencer theory (1993). The thirteen dimensions of characteristic entrepreneurs (Spencer & Spencer, 1993) are (1) Initiative, (2) Assertiveness, (3) Persistence, (4) Persuasion, (5) Concern for the high quality of work, (6) Commitment to work, (7) Efficiency orientation, (8) Systematic Planning, (9) Problem-Solving, (10) Self-Confidence, (11) Information seeking, (12) Using influence strategies, (13) Sees and acts on opportunities. The instrument was developed in the form of the Situational Judgment Test (SJT). SJT is classified as superior compared to Likert's scale model because it is more objective. In SJT, items are presented in cases (can be problems or conflicts that the participants must solve), which describe a typical situation in a particular activity or job and then followed by choices of answers related to the cases. Individuals who have good competence will choose (to give judgment) the most effective or efficient solution. The score for an effective or efficient solution is determined jointly by experts (theoretical experts) and successful individuals in the field (practitioner). That is why SJT is a form of testing that is more suitable for the work context (Whetzel & McDaniel, 2009).
In this study, two research questions related to the instrument's psychometric elements were displayed: is the instrument valid? Is the instrument reliable?
Measurement of Entrepreneurship Competency in the Form of Situational Judgment Test
Compared to the Likert scale model, the SJT test is classified as superior because it is more objective and created from real situations/problems. This scenario and choice of answers were developed based on critical incidents in behavioral indicators elicitation (through interviews, focus group discussions, surveys, and observations). The problem in the scenarios is the question that contains dilemmas or conflicts that require KSAs (knowledge, skills, abilities, and other characteristics) to solve the problem (Lievens et al., 2008; Whetzel & McDaniel, 2009; Oostrom et al., 2015).
According to Lievens et al. (2008); Waugh & Allen (2011); Oostrom et al. (2015), the stages of SJT preparation are: (1) gathering critical incidents to develop scenarios and answer of choices from the sources of the construct (entrepreneurs); (2) writing a scenario; (3) discussion with experts, discard irrelevant scenarios and edit the scenarios following the results of the discussion; (4) discussing the choices of response with experts; (5) Write several responses related to the prepared scenario; (6) discussing the review of the responses with experts and edit them; (7) asking other experts to give a score weight (scale) for each successful response written in each scenario; (8) determining the format of the response format; (9) collecting data to test the instrument; (10) using statistical and psychometric analysis techniques for testing the validity and reliability of the instrument; and (11) eliminating invalid questions.
Stages of Preparing a Measurement Instrument
In this study, the preparation included seven steps. They were first conducting interviews with entrepreneurs to create cases. The critical incident approach was implemented to collect problems related to Spencer and Spencer's competencies (1993).
Second, we wrote the items. An item consists of a stimulus and a list of 4 choices of answers. The stimulus is described by 4 to 5 sentences mentioning a dilemma or a conflict experienced by entrepreneurs. Third, entrepreneurs and academic experts were asked to evaluate the items, whether the situations and solutions were relevant to entrepreneurial competencies. They also assessed whether the items representing the theory's definitions and behaviors in real situations. Further, they checked whether the items were written logically. Fourth, the experts gave weight for each choice. The most effective solution was scored 4, the less effective were scored 3, 2, and 1 is the least effective. Based on their assessment, the researchers formulated the final weight for each choice because each judge might give a different weight for the same choice. We formulated the final weight by calculating the mean value of the weight rated by the judges. Fifth, we revised the sentence of the items according to inputs from the judges. Sixth, the questionnaires were delivered to participants. Lastly, we tested the validity and reliability of the instrument.
At first, 120 entrepreneurs were asked via semi-structured interviews about their business experiences, such as challenges, conflicts, and handling of the problems. Subsequently, we asked 47 entrepreneurs and 22 academicians in the Jakarta area to review the cases. Next, we recruited 191 SMEs in Jakarta, and West Java accorded their business characteristics governed by the Republic of Indonesia Number 20 of 2008 concerning Micro, Small, and Medium enterprises. All participants were recruited using convenience sampling methods.
Data Collection Method
This study involved a mixed-method approaches, namely qualitative and quantitative approaches in exploratory sequence design (Creswell, 2012). The qualitative data were collected prior to quantitative data to explore the concept/definition, examples, and indicators of the competencies. The qualitative data were collected via interviews with entrepreneurs to attain cases for creating the instrument's items. The FGDs with another entrepreneurs and academician also implemented to review the cases. The quantitative data were collected by distributing hardcopy and online questionnaires to participants, distributing the printed version carried out through attended a seminar, workshop, or sharing success stories' event, entrepreneurial conferences, and exhibitions. In those events, the researcher asked the entrepreneur to become participants. When distributing the online version, researchers were using networks to distribute them to the entrepreneurs.
The psychometric method in this study followed psychometric testing methods according to Crocker and Algina (1986) and Anastasi and Urbina (1997), namely content validity test, internal construct validity, and internal reliability. To test content validity, we involved experts in evaluating the cases and calculating each choice's weight. We examined the item's homogeneity and discrimination function by testing the correlation between item scores and total scores in the analysis of items. A positive and significant correlation explains the homogeneity of the construct. Here, the items which is behavior are part of the total score (construct). Simultaneously, the correlation describes the function of item discrimination. Individuals with a high score will get a high score in the total score, which means the items can distinguish individuals with high and low competencies. Researchers used exploratory factor analysis (EFA) to identify the internal validity of the dimensions. In internal reliability, the researchers applied the Cronbach's alpha test to each dimension formed.
EFA was implemented because it effectively explores new instruments' structure in the initial stage and supports construct validity (Flora & Flake, 2017; Gursuch, 2003). This method is relevant for instruments involving indigenous characteristics extracted from individual experience in a specific context (Schmitt, Prasad, Ryan, Bradburn, & Nye, 2019). Some studies showed that EFA could not be used to validate instruments with SJT model effectively. The cases and the participants' responses contain multidimensional latent variables (Sorrel et al., 2016; Weekley et al., 2006) However, in another study, EFA was found helpful. We try to test the EFA analysis on our instrument.
Form of Entrepreneurship Competency Measurement Test
This entrepreneurship competency test consists of items with one description of a situation or problem with four alternative solutions. An example of the question and choices are as follows: "Donny is a restaurant owner. One day, there was a new restaurant near Donny's restaurant. This situation makes Donny worry whether his customers will go to his competitors' restaurants. What should he do?" From this case, four answer choices are provided with graded scores ranging from effective solutions (score 4) to less effective (score 1). The answer choices for the case include (a) adding live music in the restaurant (score 1), (b) lowering prices to compete (score 2), (c) increasing innovation of the food to get added value (score 3), and (d) understand the advantages of the product, then improve the quality of service and product (score 4).
Based on interviews with 120 entrepreneurs, we develop a questionnaire of 114 items (Table 1, see stage 1). Next, each item was assessed by experts (N = 69; 47 entrepreneurs and 22 academicians) in FGDs. The experts evaluated whether the cases were relevant and inherent to the definition of the competencies. At the end of the discussion, they were asked to identify the level of the item's relevance on a scale of 1 (irrelevant) to 7 (highly relevant). They were also asked to appraise the alternative solutions on a scale from non-effective (score 1) to effective (score 4). They were also provided suggestions and recommendations to improve the cases and alternative solutions.
|Table 1 List of Dimension and Number Items in Each Stage of Item Analysis|
|Stage 1: Creating items||N items||Stage 2: Passing the item testing (homogeneity & discrimination)||N items||Stage 3: Exploratory Factor Analysis Test Results||N items|
2. Sees and acts on opportunities
4. Information seeking
5. Concern for a high quality of work
6. Commitment to work contract
7. Efficiency orientation
8. Systematic planning
12. Use of influence strategies
(+) Concern for employees
(+) Acknowledging your limitations
2. Sees and acts on opportunities
4. Information seeking
5. Concern for a high quality of work
6. Commitment to work contract
7. Efficiency orientation
11. Use of influence strategies
�(+) Concern for employees
2. Sees and acts on opportunities
4. Concern for a high quality of work
5. Commitment to work contract
6. Efficiency orientation
9. Use of influence strategies
10. Assertiveness �������������� �
At the end of the data analysis, we found the items were relevant to entrepreneurship competencies with M = 5.26 (SD = 1.25; min =1.5 max = 7). The irrelevant cases were revised based on recommendation by the experts. For example, item competence of persuasion (M = 1.5, SD = 0.71) “As the owner of a printing salon business, Rina's profits have decreased due to her customers choosing to use television to advertise their products. What should Rina do to persuade her customer to stay business with her?” It was changed to be “As the owner of a printing salon business, Rina's profits have decreased due to her customers choosing to use social media to advertise their products. What should Rina do to persuade her customer to stay business with her?”
Next, the instrument's quality was tested according to the quantitative psychometric standards by involving 220 respondents. However, only 191 data could be analyzed (response rate: 89.64%). The participants were entrepreneurs from Jakarta city and West Java province.
The homogeneity and item discrimination testing results using the total-item correlation method showed that some of the items have poor discrimination function presented by the negative coefficient and the coefficient below 0.30. From this analysis, there were 42 items selected as qualified items (see Table 1). Next, the instrument's validity was tested internally using the Exploratory Factor Analysis (EFA) method. The EFA with 42 items guided us to eliminate another 20 items. Therefore, in the final analysis, we had 22 items. From these items, the EFA suggested extracting the items into three dimensions (Table 2). The test was significant with KMO and Bartlett's test of sphericity 0.84, p<.001, and total variance explained about 40.12%. The rotation method was Oblimin because solid correlations were found between these dimensions (r >0.40).
|Table 2 Distribution Factor Loading Entrepreneurship Competency Measurement Tool|
|1||Yuli has been in the sleepwear business for three years. Observe sales for the past few years. It was found that 85% of buyers preferred sleepwear with dark color choices. There are even special requests that order dark colors. What should Yuli do? (Efficiency orientation)||0.55||-||-|
|2||Didi has an architecture business. Observe several development projects in the past six months. He found obstacles because his partner had a different perspective, which his partner's view was precarious. The partner has a significant enough stake in this business. What attitude did Didi need to show his partner? (Assertiveness)||0.41||-||-|
|3||The design consulting business founded by Mr. X had difficulty getting clients at the beginning of the establishment. What does Mr. X need to do? (Persuasion)||0.61||-||-|
|4||Bubu is ready to run a business as a snack food supplier. He also has prepared a list of stores he will visit. Bubu has been digging information about the prospective supplier's store providing the same price as him. What should he do when he has visited these shops? (Persuasion)||0.47||-||-|
|5||Mr. Roy suffered losses for his business and had to lay off some of his employees. But Mr. Roy needs time to pay his employees' severance pay. How does Mr. Roy explain to his employees? (Persuasion)||0.57||-||-|
|6||Keke is in the equipment and hospital supplies business. For these three months, one of Keke's regular customers always pays seven days after the due date. The hospital also did not pay the fine, even though Keke had given the warning letter. Keke and the hospital agreed that monthly payments were sent on the 20th, and there were fines for delays. This situation disrupted Keke company's cash flow. What should he do? (Problem-solving)||0.59||-||-|
|7||After several years running a convection business, Tono saw an opportunity to market its products overseas. What should he do? (Persistence)||0.44||-||-|
|8||As an optical business owner, Feli prioritizes hospitality when serving her customers. However, not all Feli employees are friendly when serving customers. This situation makes customers feel uncomfortable and not go back to shopping at her optics. What should Feli do? (Assertiveness)||-||0.66||-|
|9||Boni and his friend have been running Y's laundry business for six months. Boni wonders why his employees are still confused about carrying out their duties so that he and his friends must constantly supervise their work. After asking his employees, they were confused because there was a different method between Boni and his friend regarding how to wash clothes. Whereas Boni and his friend had agreed to these provisions, and he had demonstrated to his employees. What should Boni do? (Problem-solving)||-||0.59||-|
|10||Doni is a restaurant owner. Once upon a time, there was a new restaurant near Doni's restaurant. This situation makes Doni worried if his consumers will switch to his competitors' restaurants. What should he do? (Concern for a high quality of work)||-||0.61||-|
|11||Jeni is the owner of a bakery in a shopping center. One day, a new bakery opened in the shopping center. The bakery has a well-known brand and has branches in various places. What should Jeni do? (Concern for a high quality of work)||-||0.76||-|
|12||Bara is a coconut beverage business owner. During this time, all employees who work for Bara have never received a particular reward. This situation made many employees decide to resign. What should Bara do? ((Concern for a high quality of work)||-||0.68||-|
|13||Joni is a cellphone shop owner. One day, there was a theft at Joni's shop. This situation makes Joni suffer losses due to losing hundreds of mobile phones. What should Joni do? (Commitment to work contract)||-||0.50||-|
|14||Dodi as a grocery store owner provides a credit payment system to his customers. However, often customers will belong in paying and angry when billed. What should Dodi do? (Assertiveness)||-||0.42||-|
|15||James is the owner of a photo studio and wedding photography services. When opening a business, James only had a little capital. What should James do? (Efficiency orientation)||-||-||0.52|
|16||Yoyo is a fast-growing provider of agricultural products for the past two years. Yoyo's business was funded from his funds (if he has earned a profit). However, the fund is not enough to expand Yoyo's business. One day Yoyo heard that two of his friends had plans to invest the money. What should he do? (Persuasion)||-||-||0.54|
|17||Ibnu is an owner of a snack food business. Currently, Ibnu also works permanently as a Marketing Staff in a company. He works from Monday to Friday, 8 am to 5 pm. Since that time, many of his employees are confused because they feel less direction from Ibnu. What should he do? (Commitment to work contract)||-||-||0.40|
|18||Yeye wants to establish a corporation, but he realized that he has not enough funds to establish a corporation. Yeye remembered that his friend's father wants to invest money but had not yet received a business proposal that he considered suitable. Yeye did not personally know the father of his friend. What should he do? (Using of influence strategies)||-||-||0.59|
|19||As an online clothing and accessories seller, Siti realizes that the goods she sells are the same as items that have been circulated in the market. Siti has a lot of competitors, and sales have declined. What should Siti do? (Initiative)||-||-||0.68|
|20||Feli is an entrepreneur in the optical field. Feli realized that in each Islamic holy month's Ramadan, the optics would experience a sales decline. What should she do? (Sees and acts on opportunities)||-||-||0.51|
|21||David is a business owner in the communications field. At David's company, all work is done by one division. This situation does the work in the division pile up. What should David do? (Concern for a high quality of work)||-||-||0.55|
|22||Devi is a Korean restaurant owner. Devi often gets complaints from customers because sometimes the taste of the food is not standard. This situation happens because the chef doesn't realize that the food tastes are changing. What should Devi do? (Commitment to work contract)||-||-||0.52|
From the distribution of descriptions of items in each dimension, the dimensions' names can be determined. We named Dimension 1 as "Use of influence strategies," Dimension 2 as "Achievement," and Dimension 3 as "Sees and acts on the opportunity." The reliability test showed that Cronbach’s Alpha coefficients were satisfying, the Dimension of the use of influence strategies has α = 0.71 (n = 8 items), the achievement Dimension has α = 0.78 (n = 7 items), and dimension Sees and acts on opportunities have α = 0.72 (n =7 items).
This study was aimed at constructing an entrepreneurship competence instrument. The instrument was built by implementing the SJT model because of its superiority in measuring competence/ability compared to the Likert-scale model. EFA was recommended to explore the newly developed instrument's structure and test the instrument's validity; however, for the SJT model, EFA is inconclusive as to the valid method. In this study, we found that the structure revealed by EFA can be interpreted quickly and meaningful. For the subsequent study, we try to control the cases' effect while testing the instrument's internal validity. The psychometric testing results showed that the entrepreneurship competency measurement tool with the SJT model consisting of 22 items extracted into three dimensions is valid and internally reliable.
In the initial phase of constructing the instrument with the SJT model, collecting cases (problems and solutions) from real/actual experiences is the main requirement. A real case about entrepreneurship was collected via interview. The interview should involve successful and beginner entrepreneurs. Due to time constraints, some of the cases were retrieved from the results of our previous research interviews (Riyanti et al., 2018). We found two other competencies aside from the 13 competencies from Spencer and Spencer (1993), namely concern for employees and acknowledging self-limitation. The "concern for employees" entails empathy toward workers that play an essential role in effective communication, especially in a professional setting, such as entrepreneurship (Fuller, et al., 2021). Acknowledging self-limitation is an indicator of humility (Tangney, 2000). Li et al., (2020) found that humility is a critical competency of an entrepreneur to be succeed. We confirm that these competencies were empirical and relevant to our study; therefore, we added these competencies to our research. However, these two competencies were dropped in the subsequent quantitative phases because they did not achieve an adequate coefficient.
We asked 47 entrepreneurs and 22 entrepreneurial academicians to revalidate the cases from these cases. Another consideration is the characteristics of the participants involved in this study. Some experts mentioned that they have difficulty understanding the cases and the solutions because they were faced with lengthy descriptions and complex situations. They reported their tiredness completing 114 items, but we keep encouraged them to complete their responses.
We conducted HR management workshops for small businesses and attended entrepreneurship conferences or symposiums to gain enough respondents. In those events, we distributed hardcopy questionnaires and described the purpose of collecting data. We also explained how to fill the questionnaire to ensure participants understand and answer all items without missing them. When the participants answered the questions, we guided them one on one. We realize that this study's convenience sampling was relatively weak because of the population's unrepresentativeness. However, we tried to select the participants by following the definition of SMEs according to government law. We also involved the association of SMEs. Considering this weakness, we suggest that the result be reconfirmed by the subsequent study and design the sampling method's structure before collecting data.
Even though the test showed the instrument's good validity and reliability, we still need to consider its implementation in a practical context. First, the instrument is more measuring the potential aspect of the competence rather than the competence itself. The potential aspect is considered adaptive and adjusted to the context where the behavior is needed. The more precise method to measure competence is by reporting someone's own experience via behavior observation. However, this instrument can help instructors in entrepreneurship curriculum programs diagnose the program's effectiveness by comparing before and after test scores. Second, considering the various business fields involved in this study, it does not mean that the instrument helps various business fields because it did not test the instruments' predictive validity based on the business category. Third, the instrument cannot be used as a sole instrument to predict SMEs' success. In this study, we did not correlate it with any criteria representing business success. We suggest testing the prediction ability of the instrument in the subsequent research.
The measuring instrument for entrepreneurial competence is based on real cases from interviews with 120 entrepreneurs. 120 cases were evaluated by the research team and determined 114 cases with 4 answer choices as the initial draft of the measuring tool. These items were then evaluated by expert judgment (20 entrepreneurial academicians and 47 entrepreneurs) using the focus group discussion method. These experts revise the case and determine the gradation of the answer choices so that they reflect the level of competence.
The testing of the psychometric attributes involved 114 items and 191 entrepreneurs as participants. An EFA was executed with the Oblimin rotation method producing adequate and reliable three dimensions of competencies with 22 items. These three dimensions are the ability to see opportunities, influence others strategically, and achieve achievement. All items and dimensions are valid and reliable. In the following research, psychometric testing involves other variables to test the construct and predictive validity.