Research Article: 2021 Vol: 20 Issue: 5
Fajar Wicaksono, Jakarta State University
Citation Information: Wicaksono, F. (2021). Evaluation program management of health care assurance for worker in pertamina. Academy of Strategic Management Journal, 20(5), 1-13.
Health care assurance is a basic need for worker, Pertamina as a company provide a program for worker. The program of health assurance has undergone various forms. The purpose of this research to ensure the program running well, it is necessary to conduct program evaluations. This research is a qualitative, program evaluation and the method uses a logical model framework, with Input, Process, Output, and Outcome. Research participants are people who represent stakeholders. Significancy of this evaluation are expected to provide input in determining the decision to continue to use, improve or terminate the program. The results show that the program is the realization of company's obligations law and company work agreement. Implementation model is managed care, based on experience of implementing the program has been done before. The achieving program objectives is carried out with a credential service facilities and personnel, home clinic system, who carried out starting services, standardization of drugs, and utilization review. Conclusion, Managed care as a health care assurance can be continued as long as there are no new rules or policies that conflict with the current legal.
Citation Information: Wicaksono, F. (2021). Evaluation program management of health care assurance for worker in pertamina. Academy of Strategic Management Journal, 20(5), 1-13.
Logic Model Evaluation, Health Assurance, Managed Care.
Human resource management is needed in every company. The management including keeping workers healthy. The management of health services for workers is an integral part of managing human resources as a whole. The need for health services is a basic need for every worker, they can always be physically, mentally, and socially healthy so that they can carry out all their duties and responsibilities productively.
Pertamina is a State Oil and Gas Mining Company, in this case as a company that provides assurance for the health care of services to workers including their families to keep workers healthy. The health care and service assurance program provided by Pertamina to workers is in the form of providing direct health service facilities or facilities as well as health service facilities through various collaborations with health service providers outside Pertamina. Efforts made by Pertamina promise that financing related to providing health service assurance to workers or others will be the responsibility of the company, both financing in the form of facilities provided and financing for the use of facilities outside the facilities that have been provided. All costs incurred with workers health services will be borne or will be financed by Pertamina, not borne by workers when using or receiving health services.
One of the problems in health services is the uncertainty. Uncertainty, among others, occurs due to the use of services that cannot be ascertained correctly, the type of service, the time of use and the costs required. In the implementation of health service assurance, there is a tendency for the costs to be incurred to increase and it is increasingly difficult to estimate the exact amount, in line with the rapid development of health service technology, as well as the increasing number of alternatives to utilize other health facilities outside the facilities provided by the company. Service managers still do not explicitly limit the officers who provide services which can also be the cause of the increase in the cost of health services.
The management of health assurance at Pertamina has undergone various forms of management. Various health service management includes organizing health facilities yourself, the costs incurred are the costs of providing health service facilities including personnel providing services, another model is by buying health services or commonly referred to as fees for service and/or reimburse, namely paying the cost of health services after receiving health services. Another model that has also been used is by providing health insurance to workers. Pertamina need the program health service assurance that can expected and believed to be able to control the use of health service costs that have not been easily controlled, without having to reduce the quality of health services and be able to maintain the satisfaction or comfort of health services that will be received by service providers and recipients.
The purpose of this reserch to ensure that the program has been running well, it is necessary to conduct program evaluations, so that later the results of this evaluation are expected to provide input for Pertamina in determining the decision to continue to use, improve or terminate the program.
Health efforts or efforts to maintain a healthy life, is the process of how to use all resources in the health sector as capital in maintaining health, including health facilities, health workers, availability of funds, all of these resources are used to carry out health service efforts, including promotive efforts, preventive efforts, curative efforts as well as rehabilitative efforts. Health service efforts can be carried out by individuals, government agencies, or private institutions. The health service effort as a system (Arifin, 2009; Azwar, 2011), is a process of interaction of all health resources, especially interaction between service providers and users or recipients, apart from of course also interactions with service guarantors or insurers for service fees.
Uncertainty, in health services often occurs because the process of interaction or implementation in health services is not entirely the same as the relationship between sellers and buyers of services that have agreed, what services will be provided including costs, but interactions in health services often or usually occur not because of an agreement but because others, such as due to the imbalance of information between service providers and service recipients, as well as uncertainty about the capabilities and needs of the service itself (Amelung, 2008). Uncertainty and imbalance of information between recipients and service providers will greatly allow abuse or fraud in a health care system.
Managed Care as a model of health care assurance program, actively involves all elements involved in the program, namely 1) Service insurance program participant groups, namely workers including their families and retirees who are also covered by Pertamina, 2) Managers of health service insurance funds prepared companies, namely a part or health function in Pertamina and 3) Executing health service providers, namely doctors and/or hospitals that are bound into a contract with the health service insurance fund manager. Managed Care at Pertamina is managed jointly between Pertamina as the guarantor of costs and Pertamedika which has the facilities and resources for health workers who will provide services. Pertamina pays the service provider a fixed amount or what is commonly known as the capitation model, or a fee calculated from the number of participants in the program. Thomas Badgett in his research stated that financing with the capitation system succeeded in reducing costs compared to financing using the fee-for-service model (Badgett & Rabalais, 1997; Abbey, 2002; Mukti et al., 2008; Brull, 2015; Light, 2003; Parekh et al., 2009; Wickizer & Lessler, 2002; O’Connor, 2018; Schield, et al., 2001).
The payment model in one of the studies Balkrishnan et al. ( 2002), through capitation or global payment, will be able to make services more efficient. The capitation budget is based on a fixed number of participants paid annually for all treatments delivered. The service provider will arrange the costs incurred. Another study, Prospects for Managed Health Care in Australia (Marcus, 2000), concluded that managed health care was managed through Capitation, Utilization review/Pre-authorization, Risk-sharing, Formulary, Closed panel, Disease management and succeeded in reducing costs and being able to maintain the quality of health services.
The definition of evaluation according to Weiss in Gembrowski (2016) states that evaluation is a systematic assessment of the implementation and consequences of an activity program to produce information about program performance in achieving predetermined goals. The definition of evaluation here is an activity that is carried out by comparing the results that have been achieved against a predetermined plan or the target program implementation objectives.
The definition of program evaluation from a social perspective, among others, was put forward by Shortell & Richardson in 1978, stating that evaluation can also be seen as a way to show programs or show social improvements or changes to make improvements to the community using the program. Cronbach in 1982 suggested that the evaluation comes from the insights generated based on the findings that can accelerate the learning process to generate benefits for society.
Another definition of evaluation put forward by Ovretveit (2002) states that evaluation is assessing something by collecting valid information systematically and by making comparisons, the aim of which is to help users decide what to do or to contribute to science.
From the descriptions of the opinions of several experts, the researcher summarizes the notion of program evaluation as a process to enable program managers to know the achievements or results of their programs, then based on that they will be able to make adjustments to achieve program objectives more efficiently and effectively, and evaluation is carried out not only to determine success or failure of a program but also knowing why success or failure occurred and what can be done about the results of the evaluation.
The research begins with the preparation of activities or determining the time and place of research as well as the object of research, namely the health care and service insurance program for workers and families as well as Pertamina's retirees, then understanding the overall program including how the program is implemented, this is necessary to determine the focus of research activities, identifying the main stakeholders, what data or information will be sought and the analysis model. The research ends with conducting analysis and concluding the results in the form of disclosing findings as well as answers to research questions.
The research design uses a logic model with the flow of Input, Process, Output, and Outcome, which is a description of all the components forming and supporting the program so that it becomes a single program. Input is all program resources, including preparation of human resources, program financing, formulation of strategies including regulations for achieving program objectives. Process or activity, namely the implementation of activities to provide insurances and implementation of health services. Output, namely products or results of health insurance and service activities. The outcome, namely health insurance and services are used fairly with efficient costs but the quality of service is maintained.
The evaluation logic model in the research framework is logic with the following flow (Figure 1) (Longest, 2015; Litvak & Long, 2000; Muhlestein et al., 2013; Sugiyono, 2017; Wiler et al., 2017; Rickel & Wise, 2002; D Damberg et al., 2019; Glickman & Peterson, 2009; Saver et al., 2004; Greene et al., 2016).
1. The inputs are all the resources used for the implementation of the program. Inputs include the preparation of human resources, program financing, strategy, formulation, including regulations for achieving program objectives.
2. The activities are the implementation of program activities, includes the provision of assurance and the implementation of health services.
3. The outputs are product or program results, namely the implementation of health assurance and service activities according to the plan.
4. The outcomes are that the health assurance and services are used fairly at an efficient cost but the quality of service is maintained, the program can continue well, and reach all program users.
1. Based on the identification results of stakeholders, the research sources consisted of:
2. Participants representing the program organizer or person in charge are people who in the implementation of the program have responsibility for the implementation of the program and are deemed to have competence and experience in various health care and service insurance program.
3. Participants representing the implementer in providing services are people who are responsible for providing service facilities, and people who are in charge of being the direct executors in running health service activities.
4. Participants representing service users are people who represent users and have used or received health services directly with the ongoing health care and service insurance program model or with previous insurance models that have been carried out by Pertamina.
The collection of data and information begins with studying and understanding the theory of health care insurance coverage from various kinds of literature, including some similar results from previous studies.
Primary data collections were done through structured interviews, observations of service implementation, and questionnaires. Secondary data collection was performed through reviews of supporting documents and implementation reports. The use of both modes of the collection also serves as the triangulation purpose.
The components in the input evaluation include:
The basis for its implementation is (1) Law of the Republic of Indonesia Number 13 of 2003 concerning Manpower, which states that every worker/laborer has the right to obtain protection for occupational safety and health. Furthermore, it is stated that to improve welfare for workers/labor including their families, employers are obliged to provide welfare facilities, taking into account the needs of workers/laborers and the size of the company's capacity. (2) Law of the Republic of Indonesia Number 36 of 2009 on Health, and (3) Pertamina's Collective Labor Agreement, concerning Health and Medical Facilities.
The review on human resource assets focused on knowing and understanding the extent to which stakeholders play roles in providing and preparing personnel to be able to manage and implement the health care and service insurance program at Pertamina. From the information obtained, the company or stakeholders have provided and prepared their personnel to manage and carry out program activities or services, including the provision of doctors and other health workers.
Various fields or qualifications are currently being regularly developed for program managers and implementers such as Health Management Qualifications, Managed Care, Hospital Management, Occupational Health, Pharmacy, and Health Economics. Other forms of resources to support human resources are the availability of access to knowledge such as the provision of various books, journals, or other literature, laws including their derivatives, regulations including company policies, and other electronic information.
Program resources in the form of facilities are the building of a hospital and/or polyclinic and its accessories to be used as a place for providing services. These assets are scattered in several places so that they are easily accessible by workers in using health services. Completeness at each hospital is tailored to the needs of a tiered service system. Resources in the form of facilities are facilities owned and managed by Pertamedika, as a service provider determined by the medical function. The facilities available are as follows:
Basic level polyclinics that only provide basic services, with limited service personnel and served only by general practitioners. These polyclinics spread in various locations.
1. Advanced polyclinic which is also served by several specialist doctors, not only by general practitioners.
2. Basic hospital (we call as Type C hospital), providing inpatient services, by providing at least basic specialization services.
3. Advanced hospital (we call as Type B hospital), which providing inpatient services, with basic specialization and sub-specialist areas. This hospital serves as a primary referral place for the polyclinic and other hospitals
The health care and service insurance program is budgeted by Pertamina, allocated to the Medical function. The Medical function divides costs for worker and family services, as well as fees for services for retirees and their registered spouses, participants are no longer burdened with the cost of health services used. The calculation for the determination of the amount of the budget is carried out by considering the pattern of cost realization in previous years as well as taking into account external influences such as the existence of health cost inflation, then the determination of the amount of the cost is outlined in a budget work plan approved by Pertamina.
Planning a budget for health care and service insurance program is how the process of setting a budget for health service financing. This budget is allocated together with other health budgets and governance is carried out by the health function. The amount or amount of the budget as a whole is determined with the approval of the Pertamina Finance function, then it becomes part of Pertamina's budget and is stipulated in the Costing Procedure.
Implementation strategies are the mainstay of implementing program activities to achieve predetermined goals. Reliable strategy herein is a membership strategy, tiered services, credentials, drug standardization, supervision, and budget use which can be described as follows:
Participation in this program is automatic or not by carrying out the registration process or by selecting it. The management of participants in the program is determined into the participation clinic, namely, participants are registered in one of the participating clinics by choosing according to the considerations of each participant, such as considerations for the convenience of the location of the participation clinic and the places where the participants live (Stufflebeam & Coryn, 2007; Shi et al., 2017; Pourat et al., 2005; Mboi, 2015; Kim, 2011; Paolucci, 2011; Marcinko & Hetico, 2006).
The management of the use or delivery of health services is carried out in stages or a referral system. The use of health services by insurance participants must be started by using the facilities at the first level of service by general practitioners, then if further services are needed, referrals will be made to advanced service facilities. by specialists and subspecialists in stages. The service process at the advanced level ends when it is declared complete or the implementation of the next service can be continued at the first level service place, the participant or service user will be returned to the referring first-level facility. From the observations of researchers during the research, this tiered service process makes health services used according to service needs, because participants cannot choose or request various services as desired, but services are only provided based on needs according to the officer providing the service.
Credentials are a series of activities to conduct an assessment including the selection or selection of health service facilities and implementing personnel by determining the minimum requirements that must be met by service providers to be able to become a place for providing health services. This including ensuring that the presence or location of service facilities is reached by service users or the completeness of the facilities and the staff who serve it has met the requirements as a first-level clinic or as a hospital.
The implementation of credentials is carried out by program administrators with credential material determined based on program needs and does not violate other general provisions. The credential material includes the completeness of the administrative credentials, the ability of the personnel to serve, and the appropriateness of the tools and facilities to be used. The credential ensures that the service facilities have met the requirements or standards for use.
Pertamina's standard drugs or commonly called Pertamina's standard drug list or Pertamina's drug formulary, is a collection of drugs and other health materials selected and determined from several drugs circulating in the community, to be used by program service providers to program participants.
From the information provided by implementing sources in service delivery, the purpose of drug standardization is to control the selection and use of drugs. Other benefits including:
1. Ensuring that the drugs and health materials selected and set as standards have been selected by various experts or institutions according to their fields, to ensure their benefits and authenticity, and to maintain the quality of health service provision to program users.
2. Providing convenience to doctors or other service providers in the selection and use/administration of drugs or other health materials to patients or service users.
3. Assuring the availability of the drugs needed, including distributing them to each hospital and participating clinic that requires them.
4. Ensuring that the drugs and materials to be used have economic value or competitive prices compared to other similar products that are widely circulating in the community.
Utilization Review is a series of activities to supervise or control the services that will be or have been provided to participants. The main objective of the surveillance process is to ensure that all actions for services are provided properly by maintaining and prioritizing service quality as well as efficient cost-control efforts. All health services are asked to following medical indications and procedures so that it is hoped that it will reduce various unnecessary services.
Utilization review supervises and evaluates the implementation of services, including whether or not they need to administer drugs or services including other medical interventions. The results of utilization review activities include the use of drugs according to medical needs, referrals according to medical indications, cost-controlled actions. Even though the implementation of supervisory activities is not entirely the same, there are points of activity that are relatively the same. The most common activities performed in utilization reviews are inpatient approval, concurrent reviews, discharge planning, case management, and outpatient appointments.
There are 3 (three) models of Utilization Review activities, differentiated based on the timing of their implementation, which are carried out before the provision of services, carried out while the service is running, or carried out when the service has been completed. The explanation of utilization review activities is as follows:
1. Pre Utilization Review, which is a process carried out before or when services are to be performed, aimed at planning as well as determining service activities to be provided to service users.
2. Current Utilization Review, which is a process carried out while the service is running or service activities have not yet been completed, aimed at supervising current service activities and planning further service activities
3. Post Utilization Review, which is a process carried out after the service has been completed, aimed at evaluating or assessing the services provided.
The result or product of a program is the achievement of the objectives. Data and information on program implementation are measured or assessed with a measuring instrument to determine the outcome or success of the program. Each use of measuring instruments must be equipped with guidelines or explanations for how to use or how to measure them and there is a measure or standard value of success. Clear guidelines and measures are used to make it easier to determine or provide an evaluation of the achievements and successes of a program. Measurement tools must include quantitative and qualitative measures of success
From the results during the research process, the researcher found that the health care and service insurance program at Pertamina was equipped with several tools or instruments to measure the success or achievement of the program. Measuring the achievement of program success includes tools for assessing program success quantitatively and qualitatively. Quantitatively measure related to financial measures and service measures, while qualitatively related to measures of subjectivity or satisfaction with services.
The complete success of the health care and service insurance program at Pertamina is an assessment of data and information to determine a measure of success. The measure of product or program success used includes measures of program feasibility, program reliability, measures of cost control, measures of quality control, measures of ability to prevent fraud, measures of potential risk, measures of being able to meet group needs, are described as follows:
Cost control is all efforts to ensure that service costs are incurred because health services are carried out fairly, per standard operating procedures or services. As a program, it must be able to maintain or control costs. Cost control is measured by setting cost indicators. The form of cost control in services is the cost of using materials, especially drugs, in the program it is selected and agreed upon the selected drug from the many drugs on the market with a relatively large price range. Service providers only provide drugs according to a predetermined list. Another control is the presence of cost risk, namely the potential for a shortage of fees paid based on the average or capitation amount, as a result of excessive or unexpected service usage. The cost risk can occur to Pertamina or vice versa to Pertamedika. The purpose of this potential cost risk is so that providers can control the service as needed, so there will be no excess service which results in inefficient service costs (Porter et al., 2018; Porter & Teisberg, 2006; Hwang et al., 2013; Issel, 2009; Basu, 2012; Block, 2006; Glied & Smith, 2013; Fang & Rizzo, 2008; Doerr et al., 2014; Chan et al., 2016).
The indicators used to measure the cost of health services include:
1. The amount of the capitation fee, which is the number of health costs per person per year paid to Pertamedika. This amount is referred to as the capitation fee per participant or the cost per member per year.
2. Average cost per outpatient visit, that is, the average cost of outpatient services for each visit. The calculation is that the entire outpatient service bill of workers is divided by the number of visits.
3. The average cost of hospitalization, that is, the average cost per episode of care.
4. The average cost of drug use per visit, that is, the average cost of services for workers per visit
The form of cost control in services is the cost of using materials, especially drugs, in the program a selected drug is selected and agreed upon from the many drugs on the market with a relatively large price range. Service providers only provide drugs according to a predetermined list. Another control is the presence of cost risk, which is the potential for a shortage of fees paid based on the average or capitation amount, as a result of excessive or unexpected service usage. The cost risk can occur to Pertamina or vice versa to Pertamedika. The purpose of this potential cost risk is so that providers can control the service as needed, so there will be no excess service which results in inefficient service costs.
Service control is health services provided according to medical indications and with correct procedures and using good and appropriate drugs and materials. Being able to maintain the quality of service is being able to carry out the program by the guidelines or program management, as well as being able to provide results according to the expected indicators or targets. Researchers get data and information directly from program implementation as well as secondary data and information from program implementation reports, as well as direct information from the implementer, including the person in charge of the program
From the observation process, the researcher saw firsthand that there are medical guidelines or what is commonly called medical service standards, which are used by health service providers in providing health services. Researchers see from the monthly activity reports, that the results of program implementation are by predetermined indicators or targets, this indicates that the program can maintain service quality. Another indicator that the quality of service is always maintained is the absence of complaints from service users regarding the services they receive from the executor of implementation and regulations.
The indicators used to measure the quality of health services include:
1. Morbidity rate, namely the amount of total morbidity figure compared to the number of participants.
2. Referral Number, namely the number or number of patients referred to a higher service facility.
3. Compliance with drug formularies, namely the use of drugs according to applicable standards.
4. Service satisfaction rate, namely the number or level of satisfaction with the services received by participants, as well as the number and resolution of complaints regarding incidents of dissatisfaction with the service.
Ease of Use
There are 3 (three) criteria to measure ease of use, namely procedures that are clear, easy to understand, and easy to apply.
The results or findings in the evaluation process of the practicality of the procedure were carried out through direct observation at the service facilities. It was seen that the health care insurance program could be implemented easily. The membership system with a single identity is attached to Pertamina's employee number or pension number. Guidelines for the use of service facilities are clear, have clear and systematic guidelines or procedures for using service facilities with a tiered service model, always starting from the first level of service facilities, the program is easy to implement in its implementation, the program has clear objectives and measurements, as well as monitoring or an evaluation is carried out throughout the program, easily understood by all stakeholders involved and affected.
Easy to Understand
One indicator of the feasibility of this health care and service insurance program is that the program is easily understood by all those involved or will be affected. From the information provided by service users, it is found that this program is easy to understand. This program has the aim of providing effective and efficient services, preventing excessive or unnecessary health service costs, and not necessarily improve service quality. This program is an interaction or collaboration among all stakeholders formulated into an agreement. The success or failure of the program will be felt by all stakeholders
Easy to Apply
Another indicator of the feasibility of a program is that the program is easy to implement. From the information on service managers and implementers, this program is easy to implement because the health service process is carried out in stages, so it does not have to provide complete facilities at all service facilities. The use of services that require starting from the participation clinic makes services carried out selectively without having to reduce the benefits or rights to get health services from participants. Because all stakeholders will benefit if the program is successful, all elements will be responsible and maximize in implementing the program.
Program reliability is a condition where the program can provide results according to objectives and consistently with the same results over a certain period. There are two indicators of a program that can be said to be reliable in providing services, first a measure of success and second a measure of time. The program is said to be reliable for success if the program that is implemented is trustworthy and has been able to achieve the planned or set goals, in this case, the health care and service insurance program will be able to control costs and control or provide quality health services according to applicable standards. The program is said to be reliable over time if the program in implementation runs continuously can provide the same results or has consistency according to the goals and measures that have been set.
Researchers find that the management of health care and service insurance program with a managed care model has been and continues to be carried out for a long time for more than 10 years. This program has proven to be able to continue to be implemented by providing consistent results in cost control and in controlling service quality. In terms of program participants or service users, researchers also did not find any form of rejection of the current program model, although it is known that in other places or companies, at the same time, using a health service management insurance model that is managed by various models or other methods.
Program feasibility is a condition in which the program can be carried out relatively easily and produces results according to the objectives. The health care and service insurance program is said to be feasible if the program can control costs while maintaining the quality of service. The indicator of eligibility is that the program is easy to implement because it is carried out in stages, it does not need to provide complete facilities for all service facilities. All stakeholders will be responsible for implementing it because all will feel the benefits.
Financing feasibility is an assessment of the number of costs incurred against the results expected or obtained in health services. The use of payment fees is carried out by Pertamina's Medical function as the guarantor of participant fees to the first person as the executor in providing health services for insurance participants. Cost-effectiveness is to ensure that costs are used as needed. This program is carried out with pre-effort financing, or pre-service financing.
From the information during the study, it was found that the cost of health services with the pre-managed care model was successful in reducing the cost of health services with the previous model, namely the fee-for-service financing model and the financing using health insurance services. Pertamina is almost certain to know how much it costs to be prepared, even though various services are not necessarily the same. This situation is also an effort to control cost risk as a result of services outside the plan.
Based on the research findings and discussion of the supporting theories as well as the results of previous research, the following conclusions can be drawn:
1. The implementation of the health care and service insurance program at Pertamina is the company's obligation in carrying out the mandate of the law and the Pertamina collective labor agreement and is a requirement of workers.
2. The implementation of the program in a managed care manner or model is suitable for maintaining and ensuring that health services are provided by prioritizing the quality of health services that are properly maintained per health professional service standards.
3. The health care and service insurance program with the managed care model in Pertamina can be used as a pilot for other companies
The health care and service insurance program for Pertamina workers, families, and retirees with the managed care model can be continued as long as there are no new rules or policies that conflict with the current legal basis for implementing the program.
Abbey, D.C. (2010). Healthcare payment systems: Fee schedule payment systems. CRC Press.
Amelung, V.E. (2008). Healthcare management. Postgraduate Medical Journal, 73(863).
Arifin, Z. (2019). Evaluasi Program (1st Edn.). Remaja Rosdakarya.
Azwar, A. (2011). Pengantar Administrasi Kesehatan. Binarupa.
Badgett, J.T., & Rabalais, G.P. (1997). Prepaid capitation versus fee for service reimbursement in a medicaid population. The American Journal of Managed Care, 3(2), 277-282.
Balkrishnan, R., Hall, M.A., Mehrabi, D., Chen, G.J., Feldman, S.R., & Fleischer, A.B. (2002). Capitation payment, length of visit, and preventive services: evidence from a national sample of outpatient physicians. The American Journal of Managed Care, 8(4), 332-340.
Basu, J. (2012). Preventable hospitalizations and medicare managed care: A small area analysis. The American Journal of Managed Care, 18(8), e280-90.
Block, D.J. (2006). Healthcare outcomes management: Strategies for planning and evaluation. Jones & Bartlett Learning.
Brull, J. (2015). The future of value-based care for family physicians. AJMC
CDC. (1999). CDC Evaluation working group overview of the framework for program evaluation reference card steps in evaluation practice standards for effective evaluation.
Center for Disease Control and Prevention. (1999). Framework for program evaluation in public health. MMWR. Recommendations and Reports