Journal of Entrepreneurship Education (Print ISSN: 1098-8394; Online ISSN: 1528-2651)

Research Article: 2017 Vol: 20 Issue: 1

The Relevancy of Graduates' Competencies to the Effectiveness of Entrepreneurship Education: A Case Study at SBM ITB-Indonesia

Astri Ghina

Telkom University

Togar M. Simatupang

Institut Teknologi Bandung

Aurik Gustomo

Institut Teknologi Bandung

Abstract

Entrepreneurship education has been widely recognized as having an important and positive impact on the launching of new commercial ventures. Although numerous studies have been conducted to evaluate the effectiveness of university level entrepreneurship education, the results have been largely varied. Most studies focus on an inappropriately-designed systematic approach to entrepreneurship education that incorporates curricula, specific pedagogy, and institutional support and, to a limited extent, psychological assessment. Consequently, such research focuses on evaluating the level of entrepreneurship education’s effectiveness through the application of a more systematic framework and relating this to a set of competencies expected on the part of graduates.

The expectation that most aspects of entrepreneurship education contained in the framework will prove effective leads all graduates’ demonstrable competencies to be deemed of a high standard. The study reported here, seeks to promote a more comprehensive understanding of factors contributing to successful management of entrepreneurship education at the institutional level. Moreover, the implications for practitioners include one that they closely monitor, as a whole, the integrated system proposed in the framework when managing entrepreneurship education to effectively achieve institution goals.

Introduction

Entrepreneurship has been increasingly thrust into the scholarly limelight to become an important issue in several countries. Indeed, in the ASEAN region, more than 50% of national workforces have been considering setting up a business as their desired career path (Global Entrepreneurship Monitor, 2013). Besides, a large majority of the most successful entrepreneurs attempted to establish their first company at a comparatively young age. Indeed, a survey of 685 leading entrepreneurs indicated that more than half started their first company when they were between the ages of 20 and 29 years old (Ernst & Young, 2011). For the purposes of this paper, the term ‘entrepreneur’ refers to a person who starts a business, while the creative process Including formulating ideas, developing business plans, applying skills and competencies, and overcoming barriers is generally referred to as ‘entrepreneurship’.

Despite having started out at a relatively young age, most of the entrepreneurs participating in the survey did not launch their commercial ventures immediately upon completing higher education. More than half described themselves as being in a “transitional” phase (58%), meaning that they had gained some non-entrepreneurial experience before branching out on their own. Moreover, several survey informants stated that different forms of business experience represent an arguably vital foundation increasing the chances of success in their future entrepreneurial endeavor (Ernst & Young, 2011), with many regarding any previous experience within a corporate environment as constituting crucial training. Table 1 summarizes their opinions when asked to rank ten factors contributing to their ventures’ success.

Table-1
Relative Importance of Aspects Contributing to The Success of A Venture
No Aspects Percentage of informants
1 Experience as an employee 33%
2 Higher education 30%
3 Mentors 26%
4 Family 21%
5 Co-founders 16%
6 Secondary education 13%
7 Colleagues 12%
8 C-level executive / board 11%
9 Friends 9%
10 Investor 5%

Despite the fact that higher education makes an important contribution to a commercial venture’s success (Ernst & Young, 2011), several countries encounter constraints on both formal and informal education and training in developing start-ups into established businesses (Global Entrepreneurship Monitor, 2013-2014). Entrepreneurship Education (hereafter referred to as EE) is, hence, posited as having a vital role in supporting all learners to become more entrepreneurial- minded (Hegarty, 2006). The implementation of EE within universities aims to imbue an entrepreneurial culture and spirit in students, as well as creating formally-educated entrepreneurs and new businesses (U.S. Department of Commerce, 2013). In other words, the expected outcome of EE is to produce well-educated entrepreneurs potentially leading to the creation of further job opportunities.

Unfortunately, there are several barriers faced by EE that become apparent when looking at 46 case interviews conducted at European Universities, including; EE dependence on the efforts of a limited number of people; academic staff members lack of time to engage in EE, educators’ inadequate EE-related competencies, the paucity of funding to support EE, academic staff members’ opposition to the introduction of EE, the lack of governmental support for EE, limited high-quality EE materials, questions surrounding EE’s academic credibility, the scant recognition of excellent EE, and restricted senior management support (Directorate-General for Enterprise and Industry, 2008). These barriers underlie the ineffectiveness of EE in several countries. On the other hand, a survey of over 549 company founders in the United States revealed that university education was stated as important in supporting students to become successful entrepreneurs by 70% of those polled (Wadhwa et al., 2009).

Several studies on EE were conducted to support students in becoming successful entrepreneurs. Tangible results, however, were often difficult to obtain due to a low number of established owner-managed business in several countries (Global Entrepreneurship Monitor, 2013). Co & Mitchell (2006) conducted a mapping of popular courses offered and observed existing classroom delivery techniques. Other studies carried out the mapping of EE in a higher education institution (Solomon, 2007; Varblane & Mets, 2010). Some investigations focused solely on teaching methods such as an entrepreneurial-directed approach (Heinonen & Poikkijoki, 2006) and problem-based learning (Tan & Ng, 2006). Several centred on students’ psychological aspects (Ibrahim & Soufani, 2002; Gelderen, 2010) and the importance of the student selection process (Dhliwayo, 2008). They give limited description of any evaluation of EE covering such issues as the assurance of learning, lecturers’ competencies, and ways to improve EE. Due to these reasons, it is necessary to evaluate the extent to which EE may be stated as having effectiveness in developing entrepreneurial graduates. Table 2 below contains research questions, research objectives and the methods adopted to elicit answers to research questions.

Table-2
Research Questions and Research Objectives
Research questions (RQ) Research objectives Methods used to answer RQ
Q 1 “What is the current situation of EE practices within a university context?” To provide mapping of EE practices within a university context. Conducting in-depth interviews with 12 informants regarding aspects of entrepreneurship program (EP), including: •contents provided by institution
•methods used to deliver the contents
•activities used by institutions to support EP
•most common assessment methods of evaluating existing EPs
       
Q 2 “How effective is existing EE in developing entrepreneurial graduates?” To demonstrate the effectiveness of EE practices within a university context. Analyzing interview transcripts using the effectiveness criteria developed by Ghina (2015).
       
Q 3 “How are graduates’ anticipated competencies after completing formal university studies?” To analyze effective factors supporting EE in developing graduates’ competencies to a high level within a university context. Analyzing the relevancy of graduates’ competencies to EE effectiveness.

Literature Review

From Hegarty’s perspective, education has assumed a vital role in enabling learners to be more entrepreneurial-minded (Hegarty, 2006). This position reflects that expressed in a study conducted by Kolvereid & Moen (1997), indicating that individuals graduating with a major in entrepreneurship are more likely to start new businesses and have stronger entrepreneurial ambitions than those without. However, this type of education may not lead directly to an increase in the number of start-ups, yet it may develop and be delivered under specific conditions to create expected outcomes (Jones, 2010). It raises the need for supporting environments, which are both internal HEIs, e.g. Facilities, structure, regulation, culture (Piperopoulos, 2012) and external Higher Education Institutions (HEIs), e.g. family, friends, role models in the society (Hegarty, 2006).

There is also little uniformity in program content which is commonly considered as related to the fact that entrepreneurship is an emerging field (Solomon et al., 2002 in Alberti, 2004). Most EE courses focus on either entrepreneurship or small business management as an overview of the knowledge and skill required for identifying, evaluating and exploiting opportunities in different circumstances and environments, including an understanding of decision-making in a small business environment. In contrast, few courses are offered in key disciplines such as entrepreneurial negotiation, leadership, new product development, creative thinking, technology innovation, entrepreneurial marketing and corporate entrepreneurship (Co & Mitchell, 2006; Solomon, 2007; Kabongo & Okpara, 2010).

Needless to say, there is also a need for appropriate teaching strategies to deliver course content. However, little is yet known about effective teaching techniques for entrepreneurship educators (Brockhaus, 2001 in Alberti et al., 2004). Some authors state that an entrepreneurial- directed approach is well-suited to EE teaching (Heinonen & Poikkijoki, 2006), while others argue that Experiential Learning and Problem-Based Learning approaches are effective in delivering EE content (Vincett & Farlow, 2008; Tan & Ng, 2006).

According to the findings, insufficient attention has been dedicated to measuring the overall effectiveness of EE. Consequently, the fact that it is not well-defined by any standardized means renders getting its results generally accepted difficult (Alberti, 2004). Most studies that present an evaluation as such are limited to a certain impact from either an internal perspective, e.g. intention and participants’ satisfaction, or an external one such as individuals’ careers after graduation. There is a lack of comprehensive internal evaluation regarding program planning and monitoring, and a parallel absence of external alumni evaluation regarding new start-ups (the composition of successful and unsuccessful entrepreneurs, time factors, cause and effect, quality of the company, focus of the company, employment prospects and quality, revenue, profitability). Most previous examples of research are largely descriptive, with few explanatory studies presenting hypotheses development, compounded by a lack of models and theories of EE.

As in business education, the growing EE discipline was developed around concepts such as the efficacy of different teaching techniques, the appropriateness of course content, the selection and usefulness of concepts and the difference between countries, among other factors. In general, the research findings seem limited in applying a generalized theory and research into EE may, therefore, be described as merely being in its exploratory stage. In fact, only studies dealing with learning processes via different teaching methods or teaching in a specific content area are contributing to the development of the body of knowledge in this field (Alberti et al., 2004).

Within this context, further research is required to build a systematic framework for and to conduct a comprehensive evaluation of EE. This may focus on inputs (curriculum, students), process (teaching methods, institutional supports), output or outcomes (competencies, alumni achievement, and students’ /alumni perceptions of their learning). Additionally, it may challenge academics to conduct evaluations into either internal (university faculty members) or external (graduates) perspectives on how to promote improved learning at a HEI. In fact, a recent piece of research has proposed a systematic framework for EE within a university context (Ghina et al., 2015) covering students, lecturers, and institutional priorities including an assurance of learning, lecturers’ competencies, and means of improving an EE. The framework is taken as a guideline to promote the effectiveness of EE within university settings. According to the research, a future study is required to correlate the effectiveness of EE in a HEI with the institution’s expectations regarding the competencies of its own graduates.

Conceptual Framework

EE has the primary objective of producing entrepreneurial graduates who can apply the competencies developed to become successful entrepreneurs. Thus, institutional goals can be posited at the input stage as a trigger to the development of such competencies. In order to support its goals, the HEI creates course content and develops this into curricula related to said goals and identified target audiences. An appropriate learning approach is required to ensure effective delivery. By way of example, project partners organized by Herrmann et al. (2008) addressed effective learning and institutional support for EE to develop entrepreneurial graduates within a university context. An entrepreneurial graduate is characterized as an individual demonstrating a set of clearly-defined entrepreneurial competencies (Ghina et al., 2015). The work of Hermann’s group proposed a framework for EE strategy according to a set of guiding principles informed by the members of an international panel of experts. The framework can be categorized as systematic in nature because it is both well-organized and includes all aspects of important concepts, values and best practices for universities to develop entrepreneurial graduates.

In fact, the framework suggests the need for an enabling institutional environment, the engagement of internal and external key stakeholders, and the development of entrepreneurial pedagogic approaches in teaching and learning as well as support practices. In detail, the need for an enabling institutional environment suggests the establishment of an appropriate milieu that would inspire and motivate individuals to find opportunities, acquire resources, and take actions in a variety of contexts which are relevant to their lives and aspirations. In such an environment, there should be a clear measure of entrepreneurial outcomes, an aligned position of the outcomes and appropriate ways of learning, and the learning activities required to take place. Moreover, the engagement of key stakeholders should ensure that entrepreneurship does not take place in a manner isolated from its broader environment, with continuous learning being sustained through relationships with stakeholders and other interested parties. Indeed, successful entrepreneurship is more likely to occur in a situation where stakeholders provide learning opportunities and facilitate the creation and exchange of tacit knowledge. The development of entrepreneurial pedagogic approaches to teaching, learning and support practices suggests that the delivery of any desired entrepreneurial outcome challenges institutions and educators to review and reflect on what needs to be taught and learnt. This includes how appropriate learning environments and approaches can be created. Practices, as such, should be clearly aligned with existing goals, outcomes and assessment processes (Herrmann et al., 2008).

To achieve goals effectively, three key actors are involved within a university setting; students, lecturers and institution. Each of them has its own attributes in an educative process, including ability, opportunity and incentive (Piper, 1993). This framework is taken as a guideline for effective learning at university. Besides, three conditions are necessary for students to perform satisfactorily, namely; they require the ability to learn if they are to follow their particular course of study effectively, they must have an opportunity to learn in order to conduct their study satisfactorily, and they need an incentive to learn in order to engage meaningfully in study. More specifically, the ability to learn includes knowledge and skills by means of which individuals undertake study. The accompanying mechanisms are focused on student recruitment/selection. Furthermore, the opportunity to learn refers to a learning environment and the underlying context provided by institutions to support students in undertaking their study satisfactorily, i.e.educational aspects such as curriculum and equipment provided by their respective HEIs. Incentives to learn, including grants and the grading scheme are intended to provide students with a motivational boost in conducting their study.

Important factors affecting lecturers’ capacity to teach effectively include how they can improve their students’ ability to learn, opportunities to conduct study satisfactorily, and desire to learn which underpins their willingness to study. The phrase ‘improving ability to learn’ refers to a review of academic progress in the form of, for example, a learning evaluation. Further, improving the opportunity to learn refers to related equipment, e.g. teaching methods or aids and social environment including lecturers and staff members, while enhancing the incentive to learn refers to rewards as an integral element in the grading of students’ participation or performance.

In fact, at the very least, three pre-conditions must exist in order for lecturers to teach satisfactorily. An institution must improve the ability to teach, the opportunity to teach, and the incentive to teach. Improving the ability to teach entails an institutional effort to achieve a far higher quality of either a natural or acquired talent enabling an individual to teach a particular subject competently or to successfully complete tasks. Initiatives undertaken in this regard may include improved recruitment and selection of lecturers, meeting salary and safety needs, and implementing meaningful training and performance appraisal systems. Moreover, improving the opportunity to teach refers to any institutional effort to create the highest quality conditions in which teaching activities are executed and, therefore, most likely to achieve their goals. Such efforts may include; distributed workloads, knowledge sharing, freedom of pedagogical approach, learning material support, and allocated funds. Improving incentives and rewards for innovative teaching form part of any institutional drive to achieve a positive teaching motivation.

Against this background, the framework by Herrmann et al. (2008) can be used as an initial guideline for effective learning to develop entrepreneurial graduates by first discovering the research gap through reference to EE literature (Ghina et al., 2015). Still, it remains unclear whether the structured responsibility of key stakeholders within a university (students, lecturers and institution) relates to all important aspects of concepts, values, and best practices as put forward by expert international panel members. In fact, there appears to be neither pattern of interaction among its key stakeholders nor any assurance of learning within the proposed framework.

On the other hand, research conducted by Piper (1993) applied a general framework of management in education within a university context. It involves all key stakeholders such as students, lecturers and the institution supporting management in education. Each has important responsibilities, including; ability, opportunity and incentive. The framework satisfied all sets of guiding principles highlighted by Herrmann et al. (2008), in being well-organized. In fact, the clearly-structured responsibility regarding key stakeholders within a university relates to all concepts, values and best practices that are critical for developing entrepreneurial graduates. It proved to have clear patterns of interaction among those key stakeholders. Based on the analyses above, the framework proposed by Piper (1993) can be used as a systematic guideline to promote effective learning within a university. In parallel, the main attributes for components in an EE context are intentionally equipped from the framework proposed by Herrmann et al. (2008). Since none of the frameworks referred to have any Assurance of Learning (hereafter, AoL) component, a systematic framework is required to arrive at a better understanding that fills the research gap in the existing literature. The framework for EE can be seen in Figure 1.

Figure 1: The framework of effective learning for entrepreneurship education (ghina et al., 2015)

Conceptually, AoL refers to a process of consistently and reliably maintaining learning standards by applying a set of success criteria in the form of a program (Mishra, 2007). In practice, enhancing students’ learning outcomes involves using a continuous improvement cycle, akin to a Plan-Do-Check-Action (PDCA) cycle. The first loop depicts students’ competencies after completing a program. It is guided by institutional vision, mission and values which, in turn, inform the learning goals and objectives of the program. The second loop shows opportunities provided by an institution. Through elements such as curriculum design, mapping to course- learning objectives, and the subsequent delivery of courses students are provided with opportunities to acquire essential knowledge, skills and values included in program-learning goals, program-learning objectives, and course-learning objectives. The third loop consists of an assessment to investigate whether students have met the desired learning objectives, and to collect evidence and identify any existing gaps. The final loop involves analyzing and interpreting evidence and also identifying potentially required adjustments to program elements or teaching methods for improving student learning outcomes (Mabin & Marshall, 2011).

Learning perceived by either internal (students, lecturers) or external (graduates) stakeholders can be used as an evaluation to pursue continuous improvement. In fact, high quality outputs and outcomes occur when the characteristics, competencies and career choices of graduates run parallel with institutional goals and objectives. The successful implementation of AoL in this particular systematic framework is, hence, supported by allocated resources such as students, lecturers and institutions in which each has its respective role in supporting AoL.

A major challenge which emerged after the proposing of a framework for the effective learning of EE was that of adapting it to a systematic and logical framework. This is a synthesis of several concepts and is reintegrated to build the systematic framework for developing successful entrepreneurs. The systematic framework for EE is depicted in Figure 2.

Figure 2: The systematic framework for entrepreneurship education in developing entrepreneurial graduates (ghina et al., 2015)

The primary output of EE within HEIs includes entrepreneurial graduates who have exhibited relevant expected competencies, for instance; identifying and evaluating business opportunities, analyzing and solving problems, decision-making, networking, oral communication skills, and innovative thinking (Izquierdo et al., 2005) which are indicative of an ability to launch new ventures (start-up businesses). Moreover, they are deemed to be capable of sustaining post-launch business growth to become successful entrepreneurs. In practical terms, the criteria for being a successful entrepreneur include; the capacity to offer quality jobs (Hytti & Kuopusjarvi, 2004), to achieve high profitability, to realize significant business growth, to develop unique business innovation, to contribute to society, to have a sense of pride in oneself, to have satisfied stakeholders, to enjoy an equitable work-life balance, to have a publicly respected business and to produce valuable and useful products or services for consumers (Gorgievski et al., 2011). The systematic framework (Ghina et al., 2015) is applied to evaluate the level of EE effectiveness in the case of developing entrepreneurial graduates. It has attempted to evaluate learning effectiveness relating only to the roles of key stakeholders (students, lecturers and institution). The characteristics of successful entrepreneurs in the framework are, therefore beyond the scope of the research reported here.

Research Method

A paradigm is a set of assumptions and perceptual orientations shared by members of a research community (Donmoyer, 2008). The research paradigm applied in this study is post- positivist in nature, given the following objectives:

a. This research emphases the creation and meaning of new knowledge to support committed socialmovements, including those seeking to effect change in theworld and contribute towards social justice (Ryan, 2006).

b. To maintain objectivity, theory and practice cannot be kept separate in this research. It cannot afford to ignore theory for the sake of just reporting facts (Ryan, 2006: 12).

c. This investigation was conducted by researchers immersing themselves among the target group and learning with its members, rather than conducting a given research project over them (Ryan, 2006: 18).

d. This research project began with problem-setting and formulating appropriate questions, meaning that it was not conducted without some idea of the focus of the investigation (Ryan, 2006: 19).

e. The research process was directed toward the developing of generalized and demonstrable propositions as well as theory across different settings (Eisenhardt, 1989: 546).

The research followed a deductive path, starting with abstract thinking before logically connecting theories to establish evidence, and finally culminating with an analysis of ideas from an individual case to extrapolate general conclusions. The current study used a qualitative approach in the expectation of arriving at more robust explanations of the phenomenon. Consequently, this investigation combined case studymethodologywith single case design. Figure 3 illustrates the research methods.

Figure 3: Research Methods

The unit of analysis incorporated in this research was that of a Study Program which incorporated a purposive sampling strategy, wherein informants were selected based on their potential contributions to the model and the demands of the research objectives. A numerically large sample of informants was preferred to help overcome any potential response bias and to uncover multiple perspectives through the use of triangulation (Yin, 2003). The criteria applied to choose the Study Program as the subject of this case included a focus on Public Universities in Indonesia, undergraduate-level institutions in the province of West Java, institutional visions/missions in producing entrepreneurial graduates or entrepreneurs, at least three batches of previous graduates, and the institution’s comparatively high profile in the mass media. Following a review of these criteria, the School of Business and Management at the Bandung Institute of Technology (SBM-ITB) in Indonesia, was selected as the subject for the case study.

Furthermore, a key approach to selecting informants was the use of numerous and highly knowledgeable individuals indicating an interest in viewing the central phenomenon fromdiverse perspectives. Ideally, such informants would include organizational actors from different hierarchical levels, functional areas and groups. The research also applied an embedded design by usingmultiple levels of analysis, focusing on three levels, namely; senior management of the study program (Dean, Vice Dean and Head of Study Program; 3 informants), staff members (lecturers; 3 informants), students (3 informants) and graduates (3 informants). On the other hand, five sources were used, including; (1) semi-structured interviews with top management of program study such as the Dean, Vice Dean and Head of Study Program, (2) semi-structured interviews with lecturers, (3) semi-structured interviews with students/graduates, (4) observations, and (5) secondary data. The duration of each in-depth interview was around 90 minutes with qualitative data from informant interviews being recorded, manually transcribed, coded and analyzed.

One key step consisted of the within-case analysis, significant due to the common reality of case study research which produced an impressive volume of data. Within-case analysis typically involves detailed case study write-ups for its site. These write-ups were often simply descriptive in nature, yet occupied a central position in generating insights (Eisenhardt, 1989) by helping researchers to cope earlier with analysis of the frequently enormous volume of data. However, there was no standardized format for such an analysis. Rather, the overall objective was to become intimately familiar with each case as a stand-alone entity, a process allowing unique patterns of the case to emerge before investigators attempted to generalize patterns across cases (Eisenhardt, 1989). Various means of evaluating the quality of the research included the following:

a. Construct Validity

1) Triangulation

The research described here employed the triangulation of sources to assess research quality - an approach utilizing various forms of in-depth interview, observation, and/or secondary data.

a) The multiple informants involved consisted of three representatives of each level (senior manager, lecturer, student, alumni) who responded to the same set of interview questions. In fact, twelve informants involved in a single case produced an extensive array of possible responses to the questions posed. Subsequently, data saturation from in-depth interviews was validated by conducting observations and relating the results to secondary data.

b) Observations were conducted during the data-gathering process with the in-depth informant interviews. Field notes from the observations were then taken to validate data saturation from the in-depth interviews and secondary data.

c) Secondary data was gathered from administrative staff members and official online sources (electronic reports and physical artifacts such as documents provided via the institution’s website). The secondary data was used to validate the findings from in- depth interviews, field notes and observations.

2) Expert Validation

Key informants (professors) regularly reviewed case study report drafts providing valuable feedback to improve the content quality of this research. The process was repeatedseveral times during the research.

b. Internal Validity

1) Conducting pattern matching Recorded interviews were transcribed before being coded manually. Empirical evidence contained in the transcripts was matched with variables from the systematic framework.

2) Conducting explanation building The matching empirical evidence can be said to have provided descriptive explanations. To enhance their internal validity, these explanationswere supported with direct quotations from the informants.

c. External Validity. Efforts to achieve this involved the use of theories in prior single-case studies. Relevant literature content was used to support the findings from this case, thereby enhancing its external validity.

d. Reliability

1) Using a case-study protocol The protocol was developed before the data collection phase to include a table of expected outcomes in an effort to elicit consistent interview responses from the informants. It was regularly reviewed by professors in order to obtain valuable feedback.

2) Developing a case-study database Each interview transcript was complemented with its corresponding informant’s profile, including name, code of informant, institution, job position, and duration and place of interview. Every informant confirmed, completed and signed their respective attendance form.

Research Findings and Discussion

As mentioned above, the effectiveness of EE is evaluated on the basis of four aspects, those being; student, lecturer, institution, and AoL. The criteria employed to assess these various aspects utilized an existing measurement tool developed by Ghina (2015). The results are as follows:

Student Aspect

This includes an ability to learn (recruitment/selection of student), an opportunity to learn (curriculum, institutional support), and incentives to learn (grading evaluation, grants). Table 3 shows the level of effectiveness of these aspects based on the content of in-depth interviews with 12 informants.

Table-3
The Effectiveness of Student Aspect
ASPECTS OF MEASUREMENT ANALYSIS OF INTERVIEW RESULTS LEVEL OF EFFECTIVENESS
     Student Aspect  Ability to Learn Recruitment and Selection of Student Institution administers IQ and psychological tests after selection process. The psychological test result is used for reactive action. Low
  Opportunity to Learn  Curriculum Institution has multidisciplinary subjects in curriculum, in each semester, sequential; and an alignment to support learning goals with the systematic process. High
 Institutional Support Institution provides all critical facilities, (1) entrepreneurship center
(2) internal/external funding for start- ups (3) community service (4) guest lectures. These activities are well- managed and routinely held.
Average
 Incentives to Learn  Grading Evaluation Measuring aspects of cognitive, effective, and psychomotor ability. It is administered to all teaching team members. High
Grants Providing internal/external grants consistently. High

The institution had already administered an IQ test. However, a psychological test was conducted after potential entrants had been selected in order that the profiling of new students might take place. The score for this aspect was, thus, low. The psychological test was critical to selecting potential students before they commenced their education at a university (Dhliwayo, 2008). An interviewee (SBM_TM1) stated,

“…starting from talent concept…each individual has his/her own talent, hence to achieve learning goals successfully we have to develop from certain criteria. I believe that to create the best graduates, we have to select the students based on certain criteria in accordance to the needs of certain fields …”

The curriculum followed at an institution contains multidisciplinary subjects each semester in sequential order. It is aligned to develop expected competencies through a systematic process. In fact, extant literature has suggested entrepreneurial learning to be the main vehicle for competency development (Markowska, 2011).Moreover, multidisciplinary subjects are a complex aspect of entrepreneurial learning criteria (Lackeus et al., 2013). They are also relevant to curriculum standards according to AACSB (2013). The score of this aspect was, as a result, high. An interviewee (SBM_L1) observed,

“…since entrepreneurship, leadership and ethics and managerial skills are our institutional flavor, hence the learning at this institution is designed to create those skills…the lecturers are encouraged to link any subjects with entrepreneurship…”

The institution provided facilities to support students in becoming entrepreneurial graduates. While boasting an entrepreneurship center, external/internal funding for start-ups, competitions, community service and guest lectures, it lacked a business incubator. Nevertheless, its activities are regular, well-managed and consistent with both common indicators of successful university-based entrepreneurial ecosystems and good entrepreneurship support practice (Cross-Border Virtual Incubation, 2013; Hofer & Potter, 2009; Hofer & Potter, 2010). The score for this aspect was, therefore, average. An interviewee (SBM_L1) commented,

“…starting from our learning pattern, all facilities actually already provided by institution bundling with the curriculum, hence it is support the students to learn optimal…”

Grading evaluation at the institution has already measured three domains, i.e. cognitive, effective and psychomotor. Furthermore, they utilize peer-reviews as an evaluative tool to measure student performance in the field, involving their soft skills as part of the grading evaluation. This measurement is a standard for all subjects within the curriculum. The practice is consistent with previous literature which argued that a complete assessment must cover all three domains of educational objectives (Eshun &Mensah, 2013). The score for this aspect was, consequently, high. One interviewee (SBM_TM1) said that

“…learning at SBM focuses on human aspect, hence, we have to observe their behavior…”

Another (SBM_L1) opined,

“…the grading evaluation depends on the subjects; if the subject is focusing on students’ behavior …it must be evaluated besides the knowledge aspect…”

The institution provides both internal and external grants for its students. Internal grants are provided for high-achievers or those lacking the necessary funds for tuition fees. Meanwhile, external sources include the Directorate General of Higher Education Indonesia for those unable to finance their studies. The practice is referred to in the earlier literature which posited that an institution should provide much-needed funds for either high-achieving individuals or those lacking funds for tuition fees to enable them to realize their learning goals (Burd, 2015). The score for this aspect was, consequently, high. Scholarships are critical for, as an interviewee (SBM_S2) stated,

“…the scholarship could encourage students’ motivation and open opportunity especially for those who lack of money to pay tuition fee…”

Lecturer Aspect

This aspect includes improving the ability to learn (evaluation method), improving the opportunity to learn (teaching method, lecturer’s role), and improving the incentives to learn (participation, rewards). Table 4 shows its level of effectiveness according to the results of in- depth interviews with 12 informants.

Table-4
The Effectiveness of Lecturer Aspect
ASPECTS OF MEASUREMENT                 ANALYSIS OF INTERVIEW RESULTS LEVEL OF EFFECTIVENESS
   Lecturer Aspect Improving Ability to Learn Evaluation Method The institution uses quantitative and qualitative evaluations consistently across all semesters. High
 Improving Opportunity to Learn  Teaching Method The institution uses an experiential learning method simultaneously in all classes and consistently across all semesters. High
 Lecturer's Role The institution consistently provides teaching and mentoring inside and outside class. High
 Improving Incentives to Learn  Participation The institution consistently uses peer- review to record participations inside and outside class. High
Rewards The institution consistently provides non-financial rewards for students. High

The institution uses a combination of quantitative and qualitative assessments of learning evaluation, which is conducted consistently during all semesters. Quantitative assessment is objective in nature, as with exam scores, assignment scores, attendance records, participation scores and competitions, while qualitative assessment is subjective, for example, peer-review reports, observation reports, satisfaction statements, and developmental feedback. From a global perspective, the standards and metrics used to measure EE’s effectiveness must be derived from concurrence among key stakeholders on a general purpose of education (Kearney, 2014). According to the literature, assessment must be relevant to learning goals, covering not only knowledge (cognitive aspect) but also effective and psychomotor aspects. While the cognitive aspect is evaluated through a quantitative assessment, the effective and psychomotor aspects can be gauged by a qualitative assessment. The score for this aspect was, consequently, high since learning at the institution focuses on human or behavioral aspects as outlined by two interviewees (SBM_L1 and SBM_TM1). For these reasons, it is very important to use a qualitative assessment alongside the quantitative one.

The institution uses experiential learning as the main method of delivering course materials. It is conducted simultaneously in all classes and consistently across all semesters. In fact, the practice is relevant according to the existing literature. There is a consensus among scholars that becoming entrepreneurial is achieved through direct experiences, i.e. learning-by- doing or direct observation (Lackeus, 2013). The score for this aspect was, therefore, high. An interviewee (SBM_L1) emphasized that,

“…one of the flavors from our institution is entrepreneurship, hence the students are encouraged to do many real projects during their learning…”

The institution provides teaching and mentoring both within and outside the classroom. The recording of students’ participation in both contexts was conducted consistently during each semester. The literature on this area highlights the fact that EE should include students’ and lecturers’ emotions in order to reflect meaningful experiences (Lackeus, 2013). The institution has already been using active learning internal and external to the classroom and provides regular tutorials for individuals studying business consulting practice (commitment-based). The literature has also argued that mentoring by professors/lecturers and entrepreneurs should be offered to support good EE practice within a university context (Hofer & Potter, 2009) and to foster student achievement (for example; counseling, career advice, mentoring, etc.) (Henard & Roseveare, 2012). The score for this aspect was high since one interviewee (SBM_TM4) stated,

“…we have so many team work for students in the curriculum, hence it need to observe, coach, and evaluated by lecturers…”

The institution has already used non-financial rewards to encourage students to become entrepreneurs which are linked to activities such as project-based learning. This approach is rooted in previous research suggesting that a well-designed mix of financial and non-financial rewards is effective (Monitor, 2014). The institution should give students a clear role in fostering quality teaching by rewarding those individuals who promote it through, for example, the awarding of extra credits. (Henard & Roseveare, 2012). Hence, the score for this aspect is average. These rewards are critical, as an interviewee (SBM_S3) emphasized,

“… I agree with rewards because it can encourage our motivation to give our best performance …”

Institution Aspect

This includes enhancing lecturers’ ability to teach (recruitment/selection procedures, meeting pay and safety needs, training provision, performance appraisal), promoting their opportunity to teach (workload, knowledge sharing, freedom in teaching, learning material support, funds allocation), improving the incentives for them to teach (incentive schema, rewards for innovative teaching). Table 5 shows the effectiveness level for this aspect based on the results of in-depth interviews with 12 informants.

Table-5
The Effectiveness of The Institutional Aspect
ASPECTS OF MEASUREMENT ANALYSIS OF INTERVIEW RESULTS LEVEL OF EFFECTIVENESS
Institutional Aspect Improving Ability to Teach Recruitment and Selection of Lecturer The institution has several steps for selecting faculty members: (1) Application form; (2) Interview; (3) References from interest group or teaching team; (4) Micro teaching; (5) Pre-employment test (Skills, Personality, Medical Check).
Academic assistantship represents a trial period for a faculty member before being confirmed as a full-time lecturer.
High
Pay and Safety Needs The institution provides a fixed salary with supplements for additional activities and availability for safety needs. High
 
     
     Training The institution provides training related to learning needs which is conducting only sporadically. Average
Performance Appraisal The institution conducts faculty members’ appraisals based on several criteria: (1) Workload; (2) Peer review; (3) Summary of students or clinical evaluation; (4) Classroom Assessment Techniques (CAT). Such evaluations are conducted routinely each semester during a calendar year. Average
Improving Opportunity to Teach  Workload Lecturer-student ratio at the institution is 1:29. Each faculty member works up to a maximum of 40 hours per week. High
Knowledge Sharing Institution has a well-organized and consistently conducted knowledge sharing routine. High
 Freedom in Teaching Lecturers at the institution enjoy freedom in methods of delivering course materials; it is the same for all classes of the same course (teaching team) High
 Learning Material Supports The institution employs innovative and pertinent teaching materials (complete, free access). Students and all faculty members need only a simple bureaucracy to access it. High
 Funds Allocation The institution provides financial support for research, community service and external/internal training, all of which are conducted consistently. High
 Improving Incentive to Teach  Incentive Schema The institution routinely provides incentives based on performance and they provide old-age benefits for employees. High
Rewards for Innovative Teaching There is a non-financial reward for innovative teaching. However, it was awarded only sporadically. Average

The institution follows several common steps for selecting faculty members, namely; application form, interview, pre-employment test (skill, personality, and medical check) and micro-teaching. They have included reference checks from interested parties in addition to the commonly-followed steps. The institution has used micro-teaching to assess the ability to teach a group of potential lecturers before accepting them as full- or part-time lecturers. In the previous literature, a selection process must include reference-checks alongside common steps (Gusdorf, 2009; Maloney, n.d.). Additional tests should involve micro teaching which is relevant to the preparation of interview sessions (Howe, 2014). The score for this aspect is, hence, high. The selection of lecturers is important, as an interviewee (SBM_TM4) states,

“…three aspects that are important to be owned by lecturers are motivation, self-development, and improvement, hencewe have to select it…”

The institution provides a fixed salary, with supplements for additional activities, pension fund contributions and health and safety cover. Prior literature asserts that all personnel should enjoy competitive salaries while outstanding performers be eligible for a larger payment and significant cash incentive awards (Greene & Ronza, n.d.). Consequently, the score for this aspect was high.

The institution provides training for lecturers in parallel with their learning needs, although it is conducted only sporadically. According to the literature, an effective human resource strategy for staff development consists of internal training programs augmented by periodic formal education at external institutions. Internally-provided management training uses planned continuing development in which staff members would be internally and externally trained depending on their needs (Greene & Ronza, n.d.). Given this combination of factors, the effectiveness level score was average for this aspect. An interviewee (SBM_L1) stated,

“…we have case-by-case training for lecturers, in accordance with the urgent needs, if we need training of teaching method … the institution will provide it …”

The institution has applied appraisal of faculty members in several areas, including; workload, peer-review, summary of student feedback or clinical evaluation, and Classroom Assessment Techniques (CAT). It is conducted routinely in each semester during the calendar year which is important as an interviewee (SBM_TM3) stressed,

“…our lecturers are evaluated by their own interest group…it is conducted to give some feedback for improvement…”

Another (SBM_L1) also said that

“…if the result of QA is not good, the lecturer will be down-graded (cannot teach in the next semester) …”

The score of effectiveness level was, thus, average in this aspect. The institution has not provided any evaluation based on teaching-team coordinator observations. Rather, a classroom observation must be completed by an Associate Dean or program coordinator for all full- and part-time faculties during an academic year (Mercy College, n.d.).

The lecturer-student ratio at the institution was 1:29. Each faculty member works a total of 40 hours per week. In keeping with Directorate General of Higher Education Indonesia rules, the maximum lecturer-student ratio for a social science school is 1:35, and a full-time lecturer’s maximum workload is 40 hours per week. Thus, the institution has succeeded in meeting the minimum ratio. Previous literature has pointed out that any new teaching and learning paradigm in HEI should re-consider the teaching load for effective learning. Therefore, the institution should quantify different elements affecting the workload and its contribution to effective teaching and learning (Henard & Roseveare, 2012). Against this background, the score for effectiveness level is high for this institutional aspect.

The institution has provided knowledge sharing in activities. such as meetings, workshops, general classes and journals which are well-organized and held regularly. The literature emphasizes that fostering capacity-building among academic staff through a range of various programs, is important. Such activities include; conferences with experts; workshops on practical applications and case studies, the creation of communities of practice for teaching material development and experimentation in the organization of education, discussions with students and the academic community on teaching and study practices, orientation of students, tutoring and mentoring, curriculum development and pedagogical coordination, as well as the use of the Blackboard system and other ICTs. (Henard & Roseveare, 2012). The score of effectiveness level was, therefore, high. One interviewee (SBM_L1) observed,

“…we have knowledge-sharing activity such as knowledge cafe, lecturer’s meeting at Tuesday twice a month, invite guest lecturer, etc…it is conducted routinely”

Lecturers at the institution enjoy freedom in their methods of delivering course material, which is uniform across all classes of the same course (teaching team). The score of effectiveness level is, thus, high in this aspect. This pedagogical approach is relevant to process-oriented entrepreneurial learning which capitalizes on the complexity and heterogeneity of human nature for value creation purposes and perceives learning as social interaction. It sees reality as a social construction in which EE acts as a discipline based on know-who and know-how. Entrepreneurship is regarded as a dynamic process that should involve emotional aspects during learning by lecturers who use repetitive learning techniques to deliver subject content (Lackeus et al., 2013).

The institution provides innovative, pertinent, complete, and freely accessible teaching materials for all faculty members. Students and all faculty members need only a simple bureaucracy to be able to access them. It reflects the suggestion contained in earlier that an institution provide innovative and pertinent teaching materials (case studies, videos, games, course contents, syllabus, etc.), and organize regular events using online services targeted at different and mixed audiences to enhance communication about, and the exchange of, new and innovative approaches in EE (Hofer & Potter, 2010). Consequently, the score of effectiveness level was high. An interviewee (SBM_L1) stated,

“…we have complete teaching materials, lecturers can propose it as their teaching needs…”

The institution has provided financial support for research exercises, community services and external/internal training which is conducted consistently. Best practice from the Entrepreneurial University ofWismar (Germany) recommended the provision of an annual budget to support EE through training, coaching and mentoring free of charge (Hofer & Potter, 2009). Therefore, the score was high for this aspect. An interviewee (SBM_L1) said,

“…we support lecturers to make self-development. In 2015, we have research funding up to 1 billion rupiahs…”

The institution has, as amatter of course, provided performance-based incentives reflecting the literature on this area suggesting such an institution provide an attractive career path for those assuming leadership responsibilities and ensure appropriate compensation (financial support, diminished teaching load, promotion prospects) for supporting leadership in promoting quality teaching (Henard & Roseveare, 2012). The score for the level of effectiveness in this aspect was, therefore, high.

There is also a non-financial reward for innovative teaching at the institution, but it is allocated sporadically. An interviewee (SBM_TM4) remarked,

“…we have award for best performance of lecturer…but it is not continued to do…”

According to previously-published literature, an institution should provide clear incentives and rewards for EE lecturers, professors and researchers who actively support EE graduates (mentoring, sharing of research results, etc.) (Hofer & Potter, 2009). The score for the level of effectiveness was, consequently, only average in this aspect.

Assurance of Learning (AoL) Aspect

The AoL aspect includes establishing learning goals and objectives, aligning curricula with adopted goals, identifying assessment instruments and measures relating to learning, collecting, analyzing, and disseminating assessment information, and using the information, e.g. documentation, to promote continuous improvement of which the assessment is being conducted on a systematic and regular basis. Table 6 shows the level of effectiveness for these sub-aspects based on the results of in-depth interviews with 12 informants.

Table-6
The Effectiveness of Aol
THE ASSURANCE OF LEARNING PROCESS ANALYSIS OF FINDINGS LEVEL OF EFFECTIVENESS
STEP 1 Establishing learning goals and objectives Institutional learning goals address not only general knowledge and competences but also managerial skills. The learning goals are depth and breadth and are measurable. High
STEP 2 Aligning curriculum with adopted goals Curriculum content is clear in supporting learning goals and the pattern of arrangement is systematic. High
STEP 3 Identifying instruments and measures to assess learning The institution has two kinds of instruments to assess learning outcomes;
(1) Course-embedded measures (2) Stand- alone testing-performance. These are consistently well-documented.
Average
STEP 4 Collecting, analyzing, and disseminating assessment information The institution conducts assessment twice per semester but it does not disseminate assessment information to faculty members. Low
STEP 5 Using information such as documentation for continuous improvement as a result of which the assessment is conducted in a regular and systematic manner. The institution does not present examples of student performance in assessment measures. The assessment outcomes are not used for continuous improvement on a systematic and regular basis. Low

Besides general knowledge and skills, institutional learning goals address managerial skills. The learning goals are deep, extensive yet measurable and the process of learning systematic. The curriculum has clear content to support learning goals, while the pattern of arrangement is systematic. These practices are in line with the Association to Advance Collegiate Schools of Business (AACSB) standards (AACSB, 2003). The effectiveness level for step 1 and step 2 are high. The curriculum is designed to support students in becoming entrepreneurial graduates with many business-related projects in which they manage real businesses, thereby gaining a better understanding of knowledge and practical aspects. As one interviewee (SBM_L1) emphasized, ‘… since entrepreneurship, leadership and ethics and managerial skills are becoming institutional flavors, the lecturers are, hence, encouraged to link any subject with entrepreneurship.’

The institution operates two assessment instruments on learning outcomes, these being consistently well-documented course-embedded measures and stand-alone testing-performance. The score for this aspect was average because while the institution administers a psychological assessment before prospective students enter the program, it does not use it to assess learning outcomes. In fact, an interviewee (SBM_TM4) explained that,

“…the psychological test is just for mapping…it uses for reactive action…”

According to AACSB (2003), a learning goal can be validated through an appropriate process that provides evidence as to whether the goal has already been met prior to an individual entering the program.

The institution conducts assessment twice per semester but the results are not disseminated to faculty members. Hence, the score of this aspect was low. According to AACSB (2003), once data on student performance is gathered through an assessment process, it must be shared with and analyzed by designated faculty committees and the institution’s leadership. The institution does not present examples of student performance in assessment measures. The outcomes are not used for any continuous improvement on a systematic, ongoing basis. The score of this aspect was, consequently, low. According to AACSB (2003), an institution must show the use of assessment data to inform its senior management and faculty on the effectiveness of their educational programs. Besides, AACSB suggests that an institution should present student performance on evaluative measures and document how and where assessment outcomes have been used for the continuous improvement of curricula.

Measuring Entrepreneurial Competencies Aspect

This aspect includes the identification and evaluation of business opportunity, problem- solving, decision-making, networking, communication and innovative thinking. Table 7 shows graduates’ entrepreneurial competencies based on in-depth interviews with three graduates. The analysis is based on a carefully crafted “Entrepreneurial Competencies Rating Scale.”

Table-7
Entrepreneurial Competencies of Graduates
No Competencies of Graduates Entrepreneurial Competencies Level
1 Identify and Evaluate Business Opportunity High
2 Identify and Solve Problems High
3 Decision-Making High
4 Networking High
5 Communication High
6 Innovative Thinking High

In fact, identifying and evaluating business opportunity is at a high level. It is supported by the coding contained in informants’ responses, indicating their requests for proactive feedback from co-workers as well as customers, and its use in identifying and exploiting business opportunities to share new knowledge of professional standards with others in order to generate new ideas for their businesses, to anticipate customer needs, and to consistently seek and seize new business opportunities.

Identifying and solving problems is at high level. It is supported by the coding, implying that informants are developing highly creative and effective solutions to problems and usingsolid negotiation skills to arrive at win-win solutions even in difficult circumstances. Decision-making was assessed as being at a high level. It is supported by the coding, indicating that informants are focused on continuous improvement by exploring opportunities for enhancing, revising or modifying existing standards/methods consistently gathering all information including opinions in order to make informed decisions, identifying and anticipating possible outcomes, creating positive solutions and reducing the impact of negative attitudes.

Networking was ranked at a high level. It is supported by the coding, implying that informants seek and initiate actions to build strategic relationships when opportunities arise, overcome obstacles to the development and maintenance of work relationships and the consistent use of skills and knowledge when working with others. Communications were also rated highly, an assessment supported by informants’ responses. They have been encouraging an open exchange of ideas and different points of view, being truthful even when being so is less than welcome, delivering accurate, clear and concise messages to inform and frequently persuade audiences to take action.

Innovative thinking was at a high level supported by informants’ responses, through which they have been encouraging new ideas, motivating others to be proactive and resourceful, know their customers and contribute unique suggestions during brainstorming and problem-solving activities.

Institutions tend to use process-based, collaborative, multidisciplinary entrepreneurial learning in which the individual is active (Lackeus et al., 2013). The curriculum of an institution is multidisciplinary with an anchor subject and cross-integration between component subjects as well as across semesters. They have introduced practical aspects into their curriculum as a means of developing entrepreneurial competencies. An immersion in real-life scenarios places a practitioner at the center of the learning experience. Furthermore, the use of drama and performance methods is an essential part of the entrepreneurial learning process as many entrepreneurs are continually, to some extent, performing when executing their roles. ITB has routinely invited successful entrepreneurs to be guest lecturers to support students in identifying and evaluating business opportunities, identifying and solving problems, networking, and developing innovative thinking. Co-teaching courses with entrepreneurs and regular faculty members is known as a means of bridging theory and practice. The curriculum also features a “performance art” course that is expected to promote critical competencies such as communication skills. Consequently, the institution has provided the students with opportunities to manage real businesses with financial support from banks. It is expected to enhance their competencies in identifying and evaluating business opportunities, highlighting and solving problems and undertaking decision-making, networking, communication, and innovative thinking.

Conclusion

The primary findings of the research reported here include an evaluation of EE at SBM- ITB, Indonesia, using a systematic framework designed to reveal the learning process’s effectiveness. The efficacy of most aspects of the framework were confirmed, with only those of student recruitment/selection and AoL, deemed to be significantly ineffective. In fact, graduates’ entrepreneurial competencies were observed to be highly developed across all aspects which several findings highlighted as being due to numerous factors. For example, the institution provides numerous internal forms of support to the learning process including selective student and lecturer recruitment requiring candidates to undergo strict tests to fulfill certain criteria. Factors contributing to the high level of competencies are concluded to encompass the following:

a. The institution has already provided students and lecturers with both learning and teaching opportunities, including curricula, learning materials, and entrepreneurial support. Such support includes, in particular, provision of an entrepreneurship center, internal and external funding for start-ups, competitions, community service opportunities, academic input from guest lecturers, internal and external training for lecturers, active knowledge-sharing routines, financial support for research, community service programmes and relevant training opportunities.

b. High-achieving students and those lacking tuition fees who demonstrated satisfactory progress qualified for a range of financial incentives, for example, grants. Furthermore, the institution gives non-financial incentives such as an appropriate evaluation scheme by considering student participation in academic grading to encourage them to perform to an optimum level. A combination of well-designed financial and non-financial incentives rewards performance of a high standard.

c. Lecturers experience high job satisfaction because of the institutionally-sanctioned freedom in the classroom that they enjoy. Improvisation is approved providing it remains within the scope of the syllabus and is, in fact, well-managed within the teaching team. Moreover, lecturers receive a competitive salary and various incentives, including both health and life insurance. On the negative side, while lecturers demonstrating innovative teaching receive non-financial rewards form the institution, this policy is currently considered as only sporadically applied.

The foregoing research findings provide insights for either academics or practitioners within HEIs as a guideline for attaining an effective EE. In particular, they offer an understanding of key HEI stakeholder priorities of in developing entrepreneurial graduates. Such theoretical findings have implications for a more comprehensive understanding of factors contributing to the successful EE management at HEIs. Moreover, implications for practitioners include monitoring the integrated system proposed in the framework as a means of managing EE to effectively realize institutional goals. Such monitoring may identify opportunities for learning improvements within a HEI, including different aspects for student, lecturer and institution. This finding is supported by previous literature which asserts that higher education managers should identify key stakeholder roles, monitor them, evaluate them, and promote improvement to make learning effective (Piper, 1993; Herrmann et al., 2008). Naturally, further research may be conducted as an explanatory study through cross-case analyses. Longitudinal study may also be needed to evaluate EE effectiveness of by identifying the characteristics and competencies of past graduates who have become successful entrepreneurs.

References