The Delphi Method is a structured approach to collecting judgments from multiple experts, providing feedback on their responses, and repeating the assessment process to systematically examine and refine expert opinions. It is used in technology forecasting, R&D theme evaluation, public policy, healthcare, corporate strategy, risk assessment, and other fields where objective data alone may be insufficient for decision-making.
When companies undertake the development of new technologies, they must evaluate technological feasibility, development time, future market potential, and competitive conditions. However, particularly in emerging technological fields, sufficient historical evidence or statistical data may not exist.
Under such circumstances, the knowledge and experience of specialists can provide valuable information.
However, simply bringing experts together does not necessarily produce sound judgments. In conventional meetings, the opinions of influential or authoritative participants may receive disproportionate attention, while other participants may conform to the prevailing view.
The Delphi Method was developed to reduce these problems associated with group discussion.
In a conventional Delphi study, experts respond to questions individually while maintaining anonymity from other participants. Aggregated responses and the reasoning behind different judgments are then presented as feedback, allowing participants to reconsider their assessments in subsequent rounds.
Through this process, participants can review their own judgments in light of information and arguments provided by other experts.
The essential purpose of the Delphi Method is not simply to average expert opinions, but to create a structured process in which participants can reconsider their judgments after examining the reasoning behind other perspectives.
However, agreement among experts does not guarantee that their conclusions are objectively correct. The Delphi Method is a way of systematically using expert knowledge, not a substitute for experimentation or empirical investigation.
Origins and Development of the Delphi Method
The Delphi Method was developed during the 1950s at the RAND Corporation in the United States.
Two researchers particularly associated with its early development are Norman Dalkey and Olaf Helmer.
In 1963, they published “An Experimental Application of the DELPHI Method to the Use of Experts” in Management Science, describing a method for obtaining group judgments through repeated questionnaires and controlled feedback.
One of the original concerns was that conventional group discussions could be influenced by participants’ social positions, authority, assertiveness, and pressure to conform.
For example, when an expert expresses an opinion with great confidence, other participants may accept that view without examining it sufficiently.
If participants provide their judgments anonymously without meeting face to face, the influence of authority and interpersonal pressure may be reduced.
Furthermore, feedback on the judgments and reasoning of other experts can provide opportunities to reconsider initial assessments.
The method subsequently became widely used in technology forecasting and policy analysis.
In Japan, large-scale technology forecasting surveys using the Delphi Method have been conducted since the Science and Technology Agency’s first technology forecasting survey in fiscal year 1971.
In May 2025, the National Institute of Science and Technology Policy (NISTEP) released the Delphi survey results from Japan’s 12th Science and Technology Foresight Survey.
The survey examined 836 science and technology topics extending toward 2055, asking experts to assess their importance, Japan’s international competitiveness, expected realization dates, and barriers to implementation.
In March 2026, NISTEP published its comprehensive report on the 12th Science and Technology Foresight Survey, incorporating the Delphi results and other findings.
The Delphi Method has therefore been used for decades and remains an important tool for examining science and technology policy and R&D strategy.
Fundamental Principles of the Delphi Method
The classical Delphi Method generally has three defining characteristics.
Anonymity
Participants provide their judgments without being directly influenced by the identities, organizational positions, or reputations of other participants.
Anonymity is intended to reduce conformity to authority and interpersonal pressure.
However, anonymity does not necessarily mean that participants are completely anonymous to the study organizers. The important principle is generally that individual responses are not attributable to specific participants by other panel members.
Iteration and Controlled Feedback
The questionnaire is administered over multiple rounds, and results from earlier rounds are presented to participants.
Participants can examine the distribution of responses and the reasons behind different judgments before deciding whether to maintain or revise their answers.
The objective is not merely to repeat the same questionnaire.
Participants should receive meaningful information that can help them reconsider their assessments.
Statistical Group Response
Responses are organized using statistical summaries such as medians, interquartile ranges, and response percentages.
These measures provide a picture of the group’s judgments.
However, the distribution of opinions and areas of disagreement can be just as important as the central tendency.
Together, these three characteristics distinguish the Delphi Method from conventional face-to-face group decision-making.
Differences Between the Delphi Method and Conventional Surveys
Although the Delphi Method uses questionnaires, its objectives and procedures differ from those of ordinary opinion surveys.
Conventional surveys generally collect responses from a target population to understand the distribution of opinions, attitudes, or behaviors at a particular time.
The Delphi Method instead selects individuals with relevant knowledge or experience and provides repeated opportunities for judgment and reconsideration.
| Aspect | Conventional Survey | Delphi Method |
|---|---|---|
| Main objective | Measure opinions, perceptions, or behaviors | Collect, reconsider, and synthesize expert judgments |
| Participants | Population or sample appropriate to the research objective | Experts with relevant knowledge and experience |
| Number of rounds | Often one | Generally multiple |
| Feedback to participants | Not necessarily required | An important component |
| Main results | Response distributions and population characteristics | Expert judgments, degree of agreement, areas of disagreement |
| Statistical interpretation | Population inference may be possible with appropriate sampling | Generally not designed to provide population-representative estimates |
In Delphi studies, the expertise and diversity of participants are often more important than the number of responses alone.
Even a large panel may overlook important perspectives if its members come from similar disciplines or organizations.
Furthermore, the percentage of Delphi participants expressing a particular judgment should not automatically be interpreted as representing the opinion of an entire industry or population.
The Basic Delphi Process
The specific design depends on the purpose of the study, but a typical Delphi process involves the following steps.
- Define the research objective. Clarify whether the study concerns technological realization dates, R&D priorities, policy issues, risks, or other judgments.
- Select the expert panel. Identify participants with relevant knowledge and experience while considering the diversity of perspectives required.
- Design the questions. Clearly describe the technologies, issues, criteria, or conditions to be evaluated.
- Conduct the first round. Collect independent judgments and explanations from the experts.
- Analyze the responses. Calculate summary statistics and identify important differences among judgments.
- Provide controlled feedback. Present aggregated results and the principal reasons behind different views without identifying individual participants.
- Conduct subsequent rounds. Allow participants to maintain or revise their judgments after examining the feedback.
- Apply termination criteria. Conclude the study according to predefined rules concerning the number of rounds, response stability, consensus, or other criteria.
- Interpret and report the results. Present aggregated judgments, remaining disagreements, and areas of uncertainty.
Classical Delphi studies involve repeated assessment, but this does not mean that questionnaires must continue indefinitely until participants agree.
In practice, some studies are designed to conclude after two or three rounds to limit participant burden and study duration.
Termination criteria should ideally be established before the study begins.
Example: Evaluating the Importance of an R&D Theme
Consider a company evaluating several R&D themes related to next-generation industrial measurement technologies.
One candidate is:
Low-Power Edge AI Anomaly Detection Technology for Retrofitting Existing Industrial Equipment
Nine experts are asked to evaluate the importance of this R&D theme using a five-point scale.
| Score | Evaluation |
|---|---|
| 1 | Very low importance |
| 2 | Low importance |
| 3 | Moderate importance |
| 4 | High importance |
| 5 | Very high importance |
Suppose the first-round responses are:
2, 2, 2, 3, 3, 3, 4, 4, 5
The median is 3.
The responses indicate that some experts consider the theme relatively unimportant, while others regard it as highly important.
The study organizer summarizes the distribution and the principal reasons given by the participants.
For example, experts assigning high scores may argue that retrofitting existing equipment creates a large potential application market.
Experts assigning low scores may instead emphasize the difficulty of achieving sufficient generality across different types of industrial equipment.
In the second round, participants reconsider their evaluations after reviewing these arguments.
Suppose the responses become:
2, 3, 3, 3, 4, 4, 4, 4, 5
The median is now 4.
Under one commonly used quartile calculation convention, the interquartile range decreases from 2 to 1.
| Indicator | Round 1 | Round 2 |
|---|---|---|
| Median | 3 | 4 |
| First quartile (Q1) | 2 | 3 |
| Third quartile (Q3) | 4 | 4 |
| Interquartile range (IQR) | 2 | 1 |
| Participants assigning a score of 4 or 5 | 3 | 5 |
All responses are hypothetical. Quartile values can vary depending on the calculation method, so actual studies should use a consistent, explicitly defined convention.
In this example, the overall evaluation becomes more favorable, and the dispersion of responses decreases.
However, only five of the nine experts assign scores of 4 or 5 in the second round.
This does not necessarily constitute strong consensus.
For example, if the study had predefined consensus as at least 75% of participants assigning a score of 4 or 5, the result would not satisfy that criterion.
The important findings are not limited to the change in the median.
It is also necessary to understand which arguments or information influenced participants’ assessments and why some experts maintained different views.
How Should Consensus Be Measured?
Consensus is often emphasized in Delphi studies.
However, there is no single universally accepted definition of consensus.
Several approaches are commonly used.
| Indicator | Main Meaning | Limitation |
|---|---|---|
| Median | Central judgment of the group | Does not indicate the dispersion of opinions |
| Interquartile Range (IQR) | Spread of the central 50% of responses | May conceal strong minority disagreement |
| Percentage within a specified range | Proportion of participants whose evaluations fall within a defined range | Requires a predetermined consensus threshold |
| Changes between rounds | Stability or movement of judgments | Stable judgments are not necessarily accurate |
| Qualitative response analysis | Reasons for judgments and sources of disagreement | Requires interpretation and coding |
A 2014 systematic review by Diamond and colleagues found substantial variation in how consensus was defined and reported in Delphi studies.
For example, some studies define consensus by requiring a certain proportion of participants to select the same response or fall within a specified range.
However, the required percentage differs among studies.
It is therefore important to establish consensus criteria before examining the results rather than selecting a convenient threshold afterward.
Failure to achieve consensus can also be a meaningful result.
Investigating why experts disagree may reveal unresolved technical questions, conflicting assumptions, or important uncertainties.
Convergence of Opinions Is Not the Same as Forecasting Accuracy
One important limitation of the Delphi Method is that convergence of expert judgments does not necessarily indicate improved forecasting accuracy.
For example, specialists working in the same technological field may share similar theoretical assumptions or expectations.
If those assumptions are incorrect, their judgments may converge toward the same erroneous conclusion.
Even when participants revise their predictions in the same direction, this does not necessarily mean that their predictions have become more accurate.
It can also be difficult to determine whether convergence reflects better understanding or psychological influence from the majority view.
Anonymity reduces direct interpersonal pressure, but presenting aggregated responses can itself influence participants’ judgments.
Therefore, the success of a Delphi study should not be judged solely by whether opinions converge. It is also important to examine whether the reasoning supporting those judgments has improved.
In technology forecasting, expert agreement should be supplemented by evidence concerning technical feasibility, physical constraints, development progress, and other relevant conditions.
Selecting an Expert Panel
Delphi results can be strongly influenced by the selection of participants.
For example, if only researchers working directly on a particular emerging technology are invited to evaluate its feasibility, the study may obtain detailed judgments about scientific and technical possibilities.
However, it may overlook manufacturing constraints, quality assurance requirements, regulations, customer needs, or commercialization issues.
For R&D evaluation, relevant expertise may include the following areas.
| Area of Expertise | Main Evaluation Perspective |
|---|---|
| Basic research | Scientific feasibility and unresolved research questions |
| Applied research and development | Technical feasibility and required development time |
| Manufacturing and production engineering | Scalability, manufacturing costs, process constraints |
| Quality and safety | Reliability, safety, compliance with standards |
| Market and business development | Customer needs, applications, conditions for market formation |
| Policy and regulation | Institutional constraints, standardization, social implications |
However, every Delphi panel does not necessarily require equal representation from all these fields.
For example, a study examining a specific physical mechanism may appropriately emphasize relevant scientific and engineering expertise.
A study concerning commercialization or social implementation requires a broader range of perspectives.
The important consideration is to identify the knowledge required by the research objective.
Expertise should also be evaluated based on relevant research, development, and practical experience rather than relying exclusively on titles or academic credentials.
Question Design Determines the Quality of the Results
Question wording is critical in Delphi studies.
Consider the question:
“Will AI-based equipment management become widespread by 2035?”
The meaning of “widespread” is unclear.
Some experts might interpret it as adoption by a limited number of large manufacturers.
Others might interpret it as deployment across a majority of manufacturing facilities.
In this situation, differences among responses may reflect inconsistent interpretations of the question rather than genuine differences in expert judgment.
The question can be made more specific:
“By what year will at least 30% of manufacturing establishments in Japan with 300 or more employees routinely use edge AI for anomaly detection in production equipment?”
This is an illustrative question, but it defines the geographic scope, organizational population, application, and adoption threshold.
However, questions can also become too narrowly specified.
When the future configuration of a technology is still uncertain, excessive specificity may exclude important possibilities that have not yet been anticipated.
One approach is to begin with open-ended questions in the first round, use the responses to identify important themes, and develop more structured evaluation items for subsequent rounds.
The Delphi Method should not be limited to evaluating a predefined list of issues. Determining what should be evaluated can itself be an important part of the process.
Delphi Method in Technology Forecasting
Technology forecasting is one of the best-known applications of the Delphi Method.
Experts may be asked to evaluate the importance, feasibility, and expected realization dates of emerging technologies.
| Evaluation Criterion | Main Question |
|---|---|
| Technological importance | How much could the technology contribute to technological progress? |
| Social and economic importance | What impact could it have on society or industry? |
| Technological realization date | When might the required technical performance or conditions be achieved? |
| Social implementation date | When might the technology be adopted in actual products or services? |
| International competitiveness | What is the competitive position of the country or organization? |
| Barriers to realization | What technical or institutional obstacles remain? |
It is important to distinguish technological realization from commercial or social implementation.
A technology may achieve the required performance in a laboratory without being suitable for mass production.
Even after commercialization becomes technically possible, adoption may depend on costs, quality, regulation, customer investment, and other conditions.
The question “When will the technology be realized?” is therefore insufficient unless the meaning of realization is defined.
Japan’s Science and Technology Foresight Survey
A representative application of the Delphi Method is Japan’s Science and Technology Foresight Survey, conducted by the National Institute of Science and Technology Policy (NISTEP).
The survey program began in fiscal year 1971 and has been conducted approximately every five years.
Its objective is to examine medium- and long-term science and technology developments and provide information relevant to science and technology policy and R&D strategy.
The 12th survey examined a wide range of topics extending toward 2055, including health, medicine and life sciences, agriculture and biotechnology, environment and energy, AI and ICT, materials, urban systems and transportation, space, and oceans.
Experts assessed factors such as technological importance, expected realization dates, and barriers to implementation.
The survey used a repeated questionnaire approach in which participants could reconsider their judgments after reviewing first-round results.
Such studies provide information not only about technological forecasts but also about perceived challenges and conditions for realization.
However, predicted realization dates are not guarantees.
Technological breakthroughs and changes in social or economic conditions may cause actual developments to differ substantially from expert forecasts.
Furthermore, the resulting picture of the future depends on which technology topics were selected for evaluation.
Therefore, the discovery and formulation of appropriate forecasting topics are important alongside the Delphi evaluation itself.
Applications to R&D Theme Evaluation
In corporate R&D management, organizations must evaluate the importance, feasibility, and commercialization potential of multiple research themes.
The Delphi Method can help collect expert judgments systematically and reveal differences among assessments.
For example, researchers may value the technological novelty of a particular R&D theme, while manufacturing engineers emphasize difficulties associated with mass production.
Sales personnel may focus on existing customer requirements, while business development specialists may emphasize opportunities in unfamiliar markets.
In conventional meetings, such differences may not be fully examined before organizational authority or interpersonal influence determines the outcome.
Delphi studies allow experts to make independent evaluations and examine the reasoning behind different judgments.
However, Delphi results should not automatically determine R&D investment priorities.
Selecting R&D themes requires consideration of investment costs, strategic alignment, business value, technological uncertainty, resource requirements, and relationships among projects.
The Delphi Method provides a procedure for obtaining expert judgments relevant to these criteria.
The final selection and allocation of resources may require additional methods, including Multi-Criteria Decision Analysis and R&D Portfolio Analysis.
Relationship with AHP and MCDA
The Delphi Method is a procedure for collecting and reconsidering expert judgments.
The Analytic Hierarchy Process (AHP) and Multi-Criteria Decision Analysis (MCDA), by contrast, are methods for evaluating alternatives according to multiple criteria.
These methods can therefore be combined.
For example, Delphi rounds can be used to identify and refine the criteria that should be considered when evaluating R&D themes.
AHP can subsequently be used to determine the relative importance of the criteria.
MCDA can then be used to compare alternative R&D themes.
Another possibility is to ask multiple experts to complete AHP pairwise comparisons and allow them to reconsider their judgments after reviewing aggregated feedback.
However, reaching agreement through Delphi and calculating weights through AHP are different analytical operations.
The results may depend on how expert judgments are aggregated and how disagreements are handled.
Reducing all expert opinions to a single average is not always desirable.
Policy Delphi: Exploring Disagreement Rather Than Seeking Consensus
Although the Delphi Method is often associated with consensus building, agreement is not always the primary objective.
This distinction is emphasized in the Policy Delphi approach.
In 1970, Murray Turoff published “The Design of a Policy Delphi,” describing a structured approach to examining alternative positions and arguments concerning policy problems.
In policy and strategy, differences among expert opinions can themselves be valuable information.
For example, when evaluating the introduction of a new technology, engineers may emphasize technical performance, regulatory specialists may emphasize safety, and business managers may focus on costs and market potential.
If these perspectives are simply averaged, important disagreements may disappear.
Policy Delphi instead emphasizes making different positions, supporting evidence, counterarguments, and potential consequences explicit.
The same approach can be valuable in R&D strategy and new business development.
Experts do not necessarily need to reach a common conclusion.
Examining why their judgments differ can reveal technical hypotheses or commercial uncertainties that require further investigation.
Real-Time Delphi
In conventional Delphi studies, the first round is completed and analyzed before the second round begins.
This requires time to collect responses, prepare statistical summaries, and distribute feedback.
Real-Time Delphi was developed as an alternative approach.
In 2006, Theodore Gordon and Adam Pease described a method that removes the conventional separation into fixed rounds.
Participants can review continuously updated response distributions and revise their judgments through an online system.
This allows individuals to contribute at convenient times and reconsider their responses as new information becomes available.
However, Real-Time Delphi introduces additional methodological concerns.
Early responses may disproportionately influence later participants.
Participants may also make judgments after seeing different amounts of feedback.
It is therefore important to manage response histories and the information presented to participants.
Relationship with Horizon Scanning
Horizon Scanning is a method for identifying early signals of potentially important future changes.
The Delphi Method is a procedure for systematically collecting expert judgments about particular questions.
The two approaches can be combined in technology forecasting and Strategic Foresight.
For example, Horizon Scanning may identify emerging technological or social developments.
These observations can be used to formulate technology forecasting topics.
The Delphi Method can then assess their technological importance, expected realization dates, possible social impacts, and barriers to implementation.
However, a signal discovered through Horizon Scanning does not necessarily lead to an important future technology.
Likewise, a high importance rating in a Delphi study does not guarantee that a technology will be realized.
Combining the methods helps broaden the range of future possibilities considered and identify which topics deserve further investigation or R&D investment.
Relationship with Concept Exploration
The Delphi Method can also be combined with Concept Exploration, an approach proposed by Mindware Research Institute.
Concept Exploration emphasizes discovering emerging concepts and searching for unknown objects through their characteristics, structures, and patterns.
The Delphi Method, however, generally requires the questions or evaluation subjects to be specified in advance.
Regardless of how rigorously a Delphi study is conducted, technologies or R&D themes that are absent from its evaluation set cannot be assessed.
This is an important limitation in technology forecasting.
Consider a study in which recognized experts from established research fields are asked to predict realization dates for familiar technological challenges.
The experts may provide valuable assessments of developments within those fields.
However, possible combinations with technologies from unrelated domains, or research themes outside established classifications, may not have been included among the questions.
Concept Exploration can be used to discover new technological candidates and R&D hypotheses beyond conventional disciplinary boundaries.
These candidates can then be evaluated through the Delphi Method.
Conversely, expert judgments obtained through Delphi may reveal technical obstacles or unresolved questions that provide new starting points for Concept Exploration.
Concept Exploration can help discover what should be investigated and evaluated, while the Delphi Method can help deepen the evaluation of those discoveries through structured expert judgment.
However, experts themselves may lack sufficient knowledge about technologies originating in unfamiliar fields.
When evaluating results from Cross-Domain Concept Exploration, it is therefore important to include relevant specialists from the other domains involved.
The Delphi Method with AI
The development of LLMs creates new possibilities for supporting Delphi studies.
For example, an LLM can assist in drafting questionnaire items based on the research objective or organizing experts’ open-ended responses.
It can also compare the reasons behind different judgments and identify important disagreements or unresolved issues.
LLM-assisted summarization and classification may be useful when study organizers must process large amounts of qualitative feedback.
However, numerical responses should generally be analyzed using conventional statistical calculations.
Medians, interquartile ranges, response percentages, and changes between rounds can be computed accurately by mathematical engines.
The more distinctive contribution of LLMs lies in examining the reasoning that numerical scores alone cannot adequately represent.
For example, AI could compare the technical conditions emphasized by experts who rate an R&D theme highly with those emphasized by experts who rate it poorly.
However, the summarization process must not eliminate minority opinions or important technical qualifications.
Study organizers should review the summaries and retain access to the original responses.
Can LLMs Replace Expert Panels?
A more fundamental question is whether LLMs can replace human experts in Delphi studies.
For example, it is technically possible to assign different expert roles to an LLM and ask it to evaluate R&D themes from multiple perspectives.
However, the resulting responses should not be treated as equivalent to judgments obtained from independent human experts.
Even when different roles are assigned to the same LLM, the resulting assessments are not necessarily independent.
They may share the same training data, reasoning tendencies, and systematic biases.
Using different LLMs does not completely solve this problem.
Different models may still depend on overlapping information sources and exhibit similar errors.
Agreement among simulated expert personas therefore does not have the same meaning as agreement among independently selected human experts.
Nevertheless, LLMs can help organize arguments for and against a technological proposal and identify additional information that should be investigated.
Multi-perspective simulations using LLMs may be useful for preparing or supporting expert assessments, but they should be distinguished from Delphi studies involving actual expert panels.
Making this distinction explicit allows AI-assisted preliminary exploration and formal expert elicitation to be used appropriately for different purposes.
Limitations of the Delphi Method
The Delphi Method has several important limitations.
First, results depend on expert selection. If participants are concentrated in particular disciplines or organizations, their judgments may reflect common biases.
Second, expert judgments are not necessarily correct. Even highly qualified specialists may fail to anticipate technological breakthroughs or social changes.
Third, convergence does not guarantee accuracy. Experts sharing incorrect assumptions may converge toward the same erroneous conclusion.
Fourth, question design strongly influences results. Ambiguous technological conditions or evaluation criteria make responses difficult to interpret.
Fifth, the method is constrained by the selection of evaluation subjects. Important unknown possibilities may be excluded if only familiar technologies or established research problems are considered.
Sixth, repeated questionnaires create participant burden. Declining response rates and panel attrition may influence the results.
Seventh, study organizers can influence participants through the selection, summarization, and presentation of feedback.
For these reasons, Delphi studies should document the rationale for expert selection, question design, number of rounds, feedback procedures, statistical methods, and consensus criteria.
In 2017, Jünger and colleagues published the CREDES guidance, emphasizing methodological transparency and consistent reporting in Delphi studies.
The Delphi Method in ThinkNavi
ThinkNavi regards the Delphi Method as a thinking-support approach for systematically collecting expert judgments and improving decision-making through feedback and reassessment.
ThinkNavi Free could provide functionality for manually recording research themes, questions, evaluation scales, and participants’ responses.
Basic features could include multiple-round response management and the calculation of medians, interquartile ranges, response percentages, and changes between rounds.
It would also be important to record qualitative explanations alongside numerical judgments so that the reasons for differing assessments can be examined.
However, a manually operated analysis tool for individual users and a full online Delphi survey platform have different functional requirements.
Conducting a formal Delphi study with multiple participants requires participant access management, anonymous response collection, round management, feedback distribution, and response-history storage.
An appropriate initial implementation may therefore focus on supporting survey design and result analysis, with multi-participant survey administration considered separately.
An LLM-assisted version could support questionnaire development, summarization of qualitative responses, identification of disagreements, and suggestions for additional questions.
However, the system should preserve minority perspectives and avoid presenting AI-generated interpretations as if they were the participants’ original judgments.
For R&D applications, integration with the ConceptMiner product family could also be considered.
For example, ConceptMiner R&D could identify new technological hypotheses and R&D themes through Cross-Domain Concept Exploration.
Expert judgments could then be collected to evaluate those candidates.
Delphi results could help distinguish themes receiving strong expert support, themes involving substantial disagreement, and technical questions requiring further investigation.
Unresolved problems and technical constraints described in expert responses could also become inputs to subsequent rounds of Concept Exploration.
Integration with R&D Portfolio Analysis and MCDA could support the comparison of research themes using these expert judgments.
However, Delphi remains a structured procedure for collecting and reconsidering expert assessments.
It does not automatically establish the value or technical feasibility of an R&D theme.
The significance of the Delphi Method lies not only in bringing expert opinions together, but also in revealing differences in judgment, examining their underlying reasons, and identifying what knowledge or evidence is still missing.
Combining AI with statistical analysis can make it easier to collect, organize, and interpret expert assessments and apply them to R&D and strategic decision-making.
Ultimately, however, what matters is not expert consensus itself, but the reasoning supporting the judgments and the evidence through which those judgments can be tested.
Primary English Terms: Delphi Method / Delphi Technique
Related Terms: Expert Elicitation, Consensus Development, Controlled Feedback, Policy Delphi, Real-Time Delphi, e-Delphi, Technological Forecasting
Related Thinking Methods:
- Horizon Scanning | Discovering Early Signals of Change and Exploring Future Possibilities
- Concept Exploration | Discovering Emerging Concepts and Unknown Possibilities
- Cross-Domain Concept Exploration,
- Scenario Analysis | Exploring Multiple Futures and Developing Strategies Under Uncertainty
- Analytic Hierarchy Process (AHP) | Quantifying Evaluation Criteria Through Pairwise Comparisons
- Multi-Criteria Decision Analysis (MCDA) | Mathematical Approaches to Integrating Multiple Criteria and Value Judgments
- Decision Matrix | Comparing Alternatives Across Multiple Criteria and Evaluating Decision Validity
- Technology Roadmapping,
- R&D Theme Evaluation and Prioritization
- R&D Portfolio Analysis
References and Further Reading
- Norman Dalkey and Olaf Helmer, “An Experimental Application of the DELPHI Method to the Use of Experts,” Management Science, Vol. 9, No. 3, pp. 458–467, 1963. Paper
- Norman Dalkey, “An Experimental Study of Group Opinion: The Delphi Method,” Futures, Vol. 1, No. 5, pp. 408–426, 1969. Paper
- Murray Turoff, “The Design of a Policy Delphi,” Technological Forecasting and Social Change, Vol. 2, No. 2, pp. 149–171, 1970. Paper
- Theodore J. Gordon and Adam Pease, “RT Delphi: An Efficient, ‘Round-less’ Almost Real Time Delphi Method,” Technological Forecasting and Social Change, Vol. 73, No. 4, pp. 321–333, 2006. Paper
- Ivan R. Diamond et al., “Defining Consensus: A Systematic Review Recommends Methodologic Criteria for Reporting of Delphi Studies,” Journal of Clinical Epidemiology, Vol. 67, No. 4, pp. 401–409, 2014. Paper
- Saskia Jünger et al., “Guidance on Conducting and REporting DElphi Studies (CREDES) in Palliative Care: Recommendations Based on a Methodological Systematic Review,” Palliative Medicine, Vol. 31, No. 8, pp. 684–706, 2017. Paper
- National Institute of Science and Technology Policy (NISTEP), Japan, Release of the Delphi Survey Results from the 12th Science and Technology Foresight Survey, May 2025. (Japanese)
- National Institute of Science and Technology Policy (NISTEP), Japan, Publication of the Comprehensive Report on the 12th Science and Technology Foresight Survey, March 2026. (Japanese)
- NISTEP, Delphi Survey Search Database. A database for searching results from Japan’s science and technology forecasting surveys since 1971. (Japanese)