Academy of Accounting and Financial Studies Journal (Print ISSN: 1096-3685; Online ISSN: 1528-2635)

Research Article: 2018 Vol: 22 Issue: 4

A Review of Delphi Technique in Developing Human Capital Disclosure Index

Salawati Sahari, Universiti Malaysia Sarawak

Michael Tinggi, Universiti Malaysia Sarawak

Sharon Cheuk, Universiti Malaysia Sarawak

Nordiana Ahmad Nordin, Universiti Malaysia Sarawak


This paper reviews the Delphi technique, a primary data collection method that could be applied in an accounting study focusing on developing voluntary disclosure index. The application of Delphi technique to develop human capital disclosure index have given a new stage for an accounting study since Delphi technique were mostly applied in the scientific, technological and medical research only. This paper attempt to shows how such technique would overcome the problem associated with other primary data collection method of solely doing interview, questionnaire and focus group. The characteristics of Delphi technique initiated with the selection of panel experts, with anonymity, controlled feedback and statistical group response. The Delphi exercise could be conducted in two or more rounds until the consensus has been met where the index would finally being develop. This paper could give recommendation on how such technique could be applied in accounting study, in the future especially in developing disclosure index. Therefore, this paper attempt to show an example on how such technique could be applied in developing human capital disclosure index. The Delphi panel could be chosen from various background of human capital such as human resource manager, labor union and governmental department involves in human capital. The data collected from the panel experts than could be finalized through the measurement of consensus and developed the human capital disclosure index.


Delphi Technique, Human Capital Disclosure, Primary Data Collection Method.


The data collection method in accounting study focus more on the secondary data collection supported with common primary data collection procedure such as questionnaires distribution or/and in-depth interview. However, there were number of primary data collection procedure practices in the accounting research in order to obtain diversification and a richer quality of data through focus group. Focus group allowed the researcher to collect data from the expert panel in their research area where the panel are required to sit together face-to-face in a group and throwing their knowledge and experience, which was collected as data for the researcher. Despite the in-depths data collected through focus group, there was an issue of autonomy among the panel where there could be one who has the power to driven others’ opinion. Hence, the Delphi technique comes in. The main prominence of Delphi technique is to moderate the “power” of panel expert to drive others to follow his/her opinion whereby the Delphi technique allowed for anonymity. Landeta (2006) visualize the group of the respondents from the Delphi study as a group of working process which could be developed with experts who do not coincide in time or space and also aims to avoid the negative influence that could be exercised by factors in the individual answers in terms of personality and status of the participating experts.

The Delphi technique is a systematic procedure to evoke expert opinion where the intended outcome is to achieve a reliable consensus of opinion among a selected panel of experts (Sourani and Sohail, 2014). The Delphi technique also highlights areas of divergence of opinions where this research methodology is based on the premise that the collective opinions of expert panelists are of richer quality than the limited view of an individual (Nworie, 2011). The process of the Delphi technique can go from one to as many rounds as are necessary to yield a consensus (Nworie, 2011). The instruments used in the process are questionnaires that require feedback from the participants in a predetermined number of rounds.

The study on corporate disclosure of human capital has received growing research attention where researchers tend to address the reporting of human capital in different countries and markets and explore the antecedent and implications of reporting human capital. Despite there were number of studies on human capital disclosure, there were lack of studies mainly emphasizes on developing such disclosure index. Therefore, this study focus on the methodology of Delphi in order to determines the human capital disclosure index from the panel experts in Malaysia. Despite there were several measurement of human capital disclosure item that maybe found significant by certain corporations, there might be insignificant by others. Therefore, the researcher has been motivated to apply the Delphi technique to obtain such agreed index or measurement among the listed corporations in Malaysia.

Characteristics of Delphi Technique

Sourani and Sohail (2014) outlined the characteristics of Delphi technique through three main criteria namely anonymity, iteration with controlled feedback, and statistical group response. As characterized as anonymity, Sourani and Sohail (2014) states that the interaction within Delphi is managed in a totally anonymous way. The advantages of anonymity are that this will assist participants in changing opinions without publicly disclosing doing so, facilitates the examination of any considerations based on their value one, minimizes the negative impacts of using committees such as group pressure, status and dominancy of powerful personalities, and provides little chance to win support for certain views at the expense of reaching a valid conclusion. The second character of Delphi which is iteration with controlled feedback describe as the exchange of information between the experts is not free but is carried out by means of a study group coordinator (or the researcher), so that all irrelevant information is eliminated Landeta (2006). Sourani and Sohail (2014) mentioned that a controlled feedback enables focusing on the objectives of Delphi rather than concentrating on winning the argument by certain participants. Moreover, it could reduce the tendency of participants to reach an agreement at the expense of producing a useful opinion.

The third characteristic of Delphi technique is the statistical group response whereby all of the opinions form part of the final answer. Hence, the questions are formulated so that a quantitative and statistical treatment of the answers can be carried out (Landeta and Barrutia, 2011). Sourani and Sohail (2014) explains that the statistical group response commonly represents the group opinion and may involve an indication of the opinion variation within the group where some studies may utilize frequency distributions to report a statistical group response.

Selection of Experts in Delphi Technique

According to Weidman et al. (2011), the literature has not specified the number of experts needed for a Delphi technique. However, it is recognized that a minimum appropriate size would include seven or eight experts. Moreover, Mitchell and McGoldrick (1994) argued that the size of the panel may be as large as time and money considerations will permit but should be no less than 8 to 10 members. While Hallowell and Gambatese (2010) indicated that most studies incorporated eight to sixteen experts and suggested a minimum of eight, they argued that the specific number should be determined by:

1. The study characteristics namely the number of available experts, the desired geographic representation and the capability of the facilitator).

2. Highlighted the importance of having a sufficient number of experts at the end of the Delphi process and the need to consider this in light of the possibility of drop out by some experts.

Maintaining high level of response is one of the major difficulties in Delphi. Experts are expected to know more about the subject of study than do others. Hallowell and Gambatese (2010) highlighted two studies that provided guidance for qualifying experts and argued that the requirements mentioned in both studies were dissimilar and vague. Knowledge is perhaps the key criterion for selecting an expert to participate in a Delphi panel. In addition to referring to criteria such as knowledge, availability and willingness, many Delphi authors tend to choose experts from a variety of backgrounds and positions so that the key viewpoints on the topic are represented (Sourani and Sohail, 2014; Nworie, 2011; Yeung et al., 2007). The results in terms of a different perspective for each criterion enable us not only to identify the non-neutrality of decision analysis, but also to (re)think the stakeholder's participation into the context of the Law referred to.

Developing and Validating Delphi Study Instruments

An essential process of a Delphi study is the development of instruments to be used in a study. Nworie (2011) stated that an instrument could be a questionnaire design to collect demographic information and a blank page with one or more questions, or a questionnaire with multiple questions to which panelists could respond based on their expertise. They further add that starting a Delphi study with a blank page that contains a question for the panelist enables them to generate relevant ideas from their wealth of knowledge. Using an open-ended format for the first round of a Delphi study reduces the chances of excluding items that the researcher may have omitted. However, generating a questionnaire for the panelists helps to identify possible issues that the panelists might forget. Different study that applied Delphi study usually has different instruments especially on the first round. However, most of prior literature applying almost a similar instrument in the proceeding round whereby the questionnaire distribution by using a Likert-scale were commonly being used.

For an instance, Yeung et al. (2007) in their Delphi exercise, on the first round of the Delphi questionnaire, the panel were asked to select the measurement item among the listed measurement item provided by them identified from prior literature. On the second round the panels were provided with the consolidated result from Round 1 and were invited to reconsider their options to see if they would like to adjust their original choice. In the third and the fourth round the panels were requested to provide rating based on the five point Likert-scale.

As for Sourani and Sohail (2014) the instruments used in the first round of Delphi was a questionnaire which involved questions in an open-ended format to facilitate the exploration of the subject and assists in developing more representative answers of the panel’s thinking. Their responses from Round one in their study have provided a rich material for analysis such as:

1. Establishing preliminary categories within which the responses can be classified.

2. Coding the responses within the established category.

However, a study by Etxerberria et al. (2014) starting the first round of Delphi exercise by sending the experts a questionnaire asking the experts to assess the importance of their listed indicators by using a scale of 0-5, where the rating 0 meant eliminating the indicator from the proposed report and 5 meant that the indicator was essential to the report. According to Wudhikarn (2017), to design a questionnaire at the first stage, the open-ended question is specifically developed in order to prevent respondents from answering any irrelevant to the objectives and direction of the questionnaire.

Similarly, Landeta and Barrutia (2011) exercise Delphi study by using two rounds, though the first round of their study the initial questionnaire includes a section with Likert scale and another section for open ended question, which they claimed as making it possible to gather free suggestions regarding the issues of their study. The first round questionnaire distribution was conducted by qualified interviewers who were familiar with the study where the interview execution process took around 60 min each, making it possible to collect quantitative opinions and valuable arguments and comments. The second questionnaire distribution in that study was sent and collected via electronic mail. Hsu et al. (2013) on the other hand, approaching their panel experts in a different way whereby they brought all the experts together at the beginning of the study to give them an opportunity to communicate face-to-face and secure both unanimous agreement on the importance of the study and their commitment to full participation The researcher also took part in an initial discussion about the definition on certain terms and the objective of the study. The expert consensus of that study was achieved after three revision cycle whereby the item measurement was generated.

Achieving Consensus in a Delphi Study

The major objective for conducting Delphi is to obtain a reliable consensus among participating experts. Nworie (2011) stated that the Delphi technique is viewed as the most important among consensus-building methodologies because it allows the participating experts to reach consensus on the significant aspects of the issues in the study. Further, they add that Delphi researcher have different views on method of determining consensus while using the Delphi methodology since this is based on the purpose of their study the number of panel experts, and the duration of the study. Among the commonly applied method to measured consensus in a Delphi study are the followings:

1. Sourani and Sohail (2014) employed the percentage of respondents agreeing on certain answers whereby for they calculated the percentage of respondent agreeing on the ranking criterion. A criterion with a value of mean that is equal to or above 3 can be considered important (based on Likert-scale ranging from 1 to 5, where 5=extremely important). Agreement among the panelists that a certain criterion is important were established based on having 75% or more respondents agreeing on giving a ranking that is equal to or more than 3.

2. Standard deviation value which is one of the common ways to measure dispersion and express the extent to which values differ from the mean whereby that study indicates a relatively low values of standard deviation (Sourani and Sohail, 2014). Landeta (2006) also measured consensus or the stability of the results between rounds using the standard deviation method whereby the study compared the stability using standard deviation between rounds 2 and 3 which is higher than between round 1 and 2. That study by Landeta (2006) also indicates that 70%of the answers in the last round, group stability were higher than the previous round (i.e. less variation in the relative inter-quartile range).

3. Statistical test involving the calculation of the Kendall’s Coefficient of Concordance (W) where if the Concordance Coefficient is equal to 1, it means that all the experts rank the item measurement in the questionnaire identically. However, if the Concordance Coefficient is equal to 0, it means that all the experts rank the item measurement in the questionnaire differently (Yeung et al., 2007). The study by Yeung et al. (2007) indicates a significant amount of agreement among the respondents within the group of panel when the calculation Concordance Coefficient was prepared on round 3 and round.

Developing Human Capital Disclosure Index

In order to conduct the Delphi method, this paper attempt to show an example on how to develop human capital disclosure index, as part of corporate social reporting in accounting research. The human capital disclosure is voluntary disclosure practices by the listed companies, where the detailed items on what to be disclosed are still varied among companies. Hence, the Delphi panel involved members with a vast knowledge and experience on human capital in corporations, employee relations and rights, and the issue being researched. Therefore, the target respondents could be the human resource manager (as a representative of the preparers of annual report on the human capital section), financial officer handling human capital expenses and nonfinancial reporting, non-government organizations focusing on human capital, labor union (representative of employees), governmental department of human capital, and academicians with related research on human capital. Based on their current and prior positions, they are considered “experts” who possess a high degree of objectivity and rationality, and are able to apply their knowledge in a professional manner.

According to Landeta and Barrutia (2011), the Delphi method allows the most relevant people and collective to take part and contribute their best, within a methodology that channels, encourages and refines their contributions. Moreover, Delphi panel allowed the panels eliminating any negative personal influence or noise that stand in the way of the communication that is desired and divert the process from the objective of the analysis proposed. Landeta and Barrutia (2011) also stated that it is worth pointing out that the absence of direct interactions and the disassociations of the responses from the experts who gave them are particularly desirable characteristics for context in which there are marked differences in status between the experts participating, clearly defined group, class or political interests, important specificities that differentiate the individuals concerned (dominant personalities, oratorical ability, the language they usually speak, etc.), or in general, a high risk of conflict.

Before the first round of Delphi, the interview questions (or questionnaire) are collected from prior literature on the measurement of human capital disclosure practices, related regulation of employment and the Global Reporting Initiatives (GRI). The main objective is to analyze the data collected from the panel experts through the Delphi exercise in order to develop the human capital disclosure index. The result for the Delphi exercise for each rounds in terms of percentage for each human capital disclosure measurement item were calculated and it is important to obtain the panel’s justifications. Next, in order to analyze the stability of responses from the panel experts between rounds, the level of consensus was measured. The Delphi exercise than finally, develops the human capital disclosure index with the importance mean weighting obtains from the final round. It should be noted that, after developing such index, the researcher needs to evaluate the quality of Delphi exercise. Landeta (2006) used the following indicators to assess such quality namely:

1. The quality and stability of the panel of experts, which referred to the number of panel between rounds. Since there is the same panel, it is important to maintain all panels in all rounds of Delphi. If there is withdrawal of panel, the researcher should ensure that the numbers are insignificant between rounds. Landeta and Barrutia (2011) stated that it is hard to provide an acceptable figure for the level of dropouts where according to them most prior studies tend to range between 20% to 30%. Since the percentage of dropout was only 8% the quality in terms of the panel experts have been reached.

2. Time between rounds: Landeta and Barrutia (2011) explain that the excessive length of an exercise has negative consequences such as waning interests from the panel an increase in dropout rates. However, Sitlington and Coetzer (2015) stated that regular and timely communication would keep participants informed and interested. Therefore, the responses from the entire questionnaire were collected should be less than one month. During this time, the researcher should ensure no notable changes of environment occurred.

3. The quality and intensity of the participation: This is referring to how long does it takes for each panel to answer the questionnaire or an interview because the longer it took them to responds, the more likely they made qualitative contributions per question. Some of the panel chooses for an interview which could be lasted for an hour or more. This would invite a richer quality of responds. Responds on justifications could also being obtain through open-ended questions in the questionnaire.

4. Modification of the initial opinions as a consequence of repetition and feedback: This could have been measured through the level of consensus whereby the percentage of the panel responded agreeing and statistical measured by Wilcoxon signed rank test to find the stability in the responses of the panel expert from the three rounds of Delphi exercise.

Stakeholder Theory

Since this study is part of social reporting focusing on human capital disclosure, the stakeholder theory would be applied in the theoretical framework. Furthermore, the application of Delphi method in the data collection procedure involves the stakeholders directly. Therefore, the application of such theory in developing human capital disclosure index could be further explains through the identifying of stakeholders as Delphi panel and through the outside-in approach of Delphi technique.

Identifying key stakeholders involves choosing key stakeholders who represent the different agents who on the one hand could be affected by the hypothetical sustainability report and on the other hand agents who although not directly affected by such a report, are interested in this kind of sustainable information. Shareholders, employees: in this study, the human capital focus on employees for the term “human” and the measurement would be manly employees oriented. The shareholders are the owners of the firm and the scope of this study is the public listed companies where the owners are the shareholders. The employees could be given shares by corporations and they are part of the “owner” of the company. As for this study, the employees would include the industrial officer in the companies and the human resource manager. Furthermore, accountant or account executives, which are also the employees in the listed companies who involved in the preparation of annual report and decision making process to what are disclosure item related to human capital of the companies. They are also responsible in the recording procedure of monetary benefits for the employees such as salaries and other allowances.

Another stakeholder is the government representative. In this case the representative is chosen based on the issue of employee and taking into consideration that this kind of dialogue necessitates a person previously trained in sustainability or knowledgeable in the government policies related to this employee or to be able to discuss the issues at hand professionally.

The second group of social agents, with a less direct relationship but with potentially significant influence (Freeman, 1984): includes other pressure groups, such as NGOs, Unions, academics and other types of economic or social networks. This group of secondary stakeholders does not usually engage in direct transactions with the company and is not usually essential to its survival. The Unions area of interest is more closely related to defense of the workers’ rights in general for the employees in the listed companies in Malaysia. Unions also they have a close relationship with ecological movements. The representative from the labor union is not the employees of any listed company since it was an independent NGO. However, the representatives of the company’s labor union are a member of such NGO. As for academics, they are taking into account the importance of technical knowledge; academics are chosen on the basis of their current or prior research in this subject area. Additionally, academicians with teaching experience in labor issues, especially related to the relevant law with regards to the labor rights would also an interested stakeholder with this human capital disclosure.

This study is based on an ‘outside-in’ approach, which includes stakeholder dialogues, screens publicly discussed issues and reports, and informs on the corporate contribution to these issues through the methodology of Delphi. The "outside-in” as explained in a study by Etxeberria et al. (2015), also applied the “stakeholder” theory through their Delphi technique. They stated that involving stakeholders in the reporting process is often considered another means by which to improve the quality and transparency of the disclosed information.

Taking in to account this “outside-in” approach, based on the stakeholder theory, is to determine what kind of sustainable information should be disclosed by the sustainable reporting of human rights disclosure. Sustainability accounting is simultaneously a process through which information flows are organized and provided for management decision making and a product obtained by internal and external parties with an interest in corporate sustainability information. The said reporting-driven sustainability accounting development process can be started on the basis of a stakeholder or shareholder-orientated view or a multiple stakeholder engagement process, or by referring to cultural expectations (Burrit & Schaltegger, 2010). Similarly, stakeholder theory states that for social and environmental information to be of material value to users, it should consider the interests of stakeholders that affect or are affected by a company’s performance (Freeman, 1984). By doing so, the relevant stakeholders are identified and their prerequisite for the development of an appropriate and mindful set of sustainability indicators are understood.

Gallhofer et al. (2011) on the other hand, view human capital disclosure as part of voluntary disclosure which may lead to the stakeholders’ pressures, where they stated that the role of such voluntary disclosure may be played by other corporate accountings (or accounts or reports of the corporation), whether prescribed in law, taking the form of “voluntary” manifestations, whereby the word “voluntary” here in quotation marks as these disclosures may be influenced by stakeholder pressures. Furthermore, according to Gallhofer et al. (2011), voluntary, being found unenforceable, the employee rights disclosures are still not a priority concern. Business initiatives, unsurprisingly, are limited by the ‘business case’ for a corporate social reporting beyond the conventional goals (in practice itself differing between temporal and spatial contexts). The United Nation promotes human rights and emphasizes the business case legislative and quasi-legislative regulation, whereby in the end, the corporations would typically insist on minimal standards.


This paper describes the Delphi technique; a primary data collection method performs in an accounting research. This study attempts to explain the characteristics of Delphi technique along with the selection of Delphi panel. The detail steps on how to conduct this methodology was further explaining in details on how to conduct such exercise which involves on selection of Delphi panel form the stakeholders, interviewing, and questionnaire distribution for number of rounds to reach consensus which was statistically analyze to obtain the human capital disclosure index with its weighted of importance. It should be noted that that the significant contribution of applying Delphi exercise was to collect an in-depth data with fine quality, especially through interviewing on the first round. The methodology of collecting on the second round onwards was to obtain the respondent’s consistency which could not being obtain by simply distributed a questionnaire only. Moreover, the characteristics of anonymity attempt to recover the problem of focus group technique of panel lobbying others’ perception.


This research is supported by Fundamental Research Grant Scheme (F01/FRGS/1605/2017). We would like to thank you the Malaysian Ministry of Higher Education for funding this research.


  1. Birko, S., Dove, E.S., Özdemir, V., & Dalal, K. (2015). Evaluation of nine consensus indices in delphi foresight research and their dependency on delphi survey characteristics: A simulation study and debate on delphi design and interpretation. PLoS ONE, 10(8).
  2. Burrit, R.L., & Schaltegger, S. (2010). Sustainability accounting and reporting: fad or trend? Accounting Auditing and Accountability Journal, 23(7), 829-846.
  3. Chen, Z., & Pak, M. (2017). A delphi analysis on green performance evaluation indices for ports in china. Maritime Policy and Management, 44(5), 537-550.
  4. Dan, P., & Fan, L. (2017). Safety risk index system screening of civil aviation airport apron based on delphi method. Revista de la Facultad de Ingenieria, 32(2), 565-571.
  5. Etxeberriaa I.A., Garayara, A., & Sánchez, J.C. (2015). Development of sustainability reports for farming operations in the basque country using the delphi method. Spanish Accounting Review, 18(1), 44-54.
  6. Freeman, R.E. (1984). Strategic management: A stakeholder approach. Boston: Pitman.
  7. Gallhofer, S., Haslam, J., & Walt, S. (2011). Accountability and transparency in relation to human rights: A critical perspective reflecting upon accounting, corporate responsibility and ways forward in the context of globalization. Critical Perspectives of on Accounting, 22(8), 765-780.
  8. Godura, S., Aggarwal, A.K., & Bhatia, P. (2017). Municipal solid waste management index in urban areas: Delphi validated tool. International Journal of Environment and Waste Management, 20(3), 215-232.
  9. Hallowell, M.R., & Gambatese, J.A. (2010). Qualitative research: Application of the delphi method to cem research. Journal of Construction Engineering and Management, 136(1), 99-107.
  10. Henchion, M., McCarthy, M., Resconi, V.C., Berry, D.P., & McParland, S. (2016). Stakeholder involvement in establishing a milk quality sub-index in dairy cow breeding goals: A Delphi approach. Animal, 10(5), 878-891.
  11. Hsu, S., Tsai, M., & Wang, Y. (2013). Developing a decomposed customer satisfaction index: An example of the boutique motel industry. Total Quality Management & Business Excellence, 24(10), 1124-1140.
  12. Hsu. C., & Sandford, B.A. (2007). The delphi technique: Making sense of consensus. Practical Assessment, Research & Evaluation 12(10), 1-8.
  13. Landeta, J. (2006). Current validity of the delphi method in social sciences. Technological Forecasting & Social Change, 73(5), 467-482.
  14. Landeta, J., & Barrutia J. (2011). People consultation to construct the future: A delphi application. International Journal of Forecasting, 27(1), 134-151.
  15. Linstone, H.A., & Turoff, M. (1975). The delphi method: techniques and application. MA: Addison-Wesley.
  16. Mitchell, V., & McGoldrick, P.J. (1994). The role of geodemographic in segmenting and targeting consumer markets: A Delphi study. European Journal of Marketing, 28(5), 54-72.
  17. Nworie, J. (2011). Using the delphi technique in educational technique in educational technology research. TechTrends, 55(5), 24-30.
  18. Qiu, B., Zhou, J., Zheng, Z., & Shen, H. (2018). Should tractors' ergonomic evaluation index system in different life cycle stage be different? A delphi survey conducted in China. Advances in Intelligent Systems and Computing, Advances in Ergonomics in Design, 588, 145-156.
  19. Seguí-Mas, E., & Izquierdo, R.J.S. (20090. Studying the financial resources for agri-food industry and rural development: Description of human capital in credit unions through delphi analysis. Interciencia, 34 (10), 718-724.
  20. Sourani A., & Sohail, M. (2014). The delphi method: Review and use in construction management research. International Journal of Construction Education and Research, 11(1), 1-23.
  21. Sitlington, H.B., & Coetzer, A.J. (2015). Using the delphi technique to support curriculum development. Education and Training, 57(3), 306-321.
  22. Weidman, J.E., Miller, K.R., Christofferson, J.P., & Newitt, J.S. (2011). Best practices for dealing with price volatility in commercial construction. International Journal of Construction Education and Research, 7(4), 276-293.
  23. Wudhikarn, R. (2017). Determining key performance indicators of intellectual capital in logistics business using Delphi method. 2nd Joint International Conference on Digital Arts, Media and Technology 2017: Digital Economy for Sustainable Growth, ICDAMT 2017, 164-169.
  24. Yeung, J.F.Y., Chan, A.P.C., Chan, D.W.M., & Li, L.K. (2007). Development of a partnering performance index (PPI) for construction projects in Hong Kong: A Delphi study. Construction Management and Economics, 25(12), 1219-1237.
Get the App