Project rationale

Common standards for training and certification were needed within Europe to move forward with the mobility of medical professionals and to guarantee patient safety. The task force, which developed the first syllabus draft, was composed of experts representing different regions of Europe as well as representatives from each of the following organisations: UEMS (European Union of Medical Specialists), EBAP (European Board for Accreditation in Pneumology), CERS (Conference of European Respiratory Societies) and the ERS Early Career Members’ Committee (ECMC). The core syllabus was the first of the consensus-based documents to be produced and published under the HERMES (Harmonised Education in Respiratory Medicine for European Specialists) initiative. Focusing on knowledge, the development of learning outcomes was the next step undertaken and was successfully fulfilled with the publication of the curriculum in 2008. Based on these documents, the ERS HERMES examination was established to standardise the certification of knowledge following the completion of national training programmes in adult respiratory medicine. The ERS HERMES summer schools were established and educational resources, such as the ERS Handbook of Respiratory Medicine, were produced to provide preparatory support for the exam. Furthermore, fundamental criteria required to provide quality training in adult respiratory medicine were defined for training centres in collaboration with EBAP. The target audience was postgraduate trainees.

This document highlights the work accomplished during 2015–2017. There have been a number of considerations including some updates and changes to the original content as well as detailed discussions on target audience. This publication is a key milestone in outlining a consensus-based document of the knowledge, skills and attitudes required of respiratory medicine specialists at trainee level. Further development will continue to consider the content required of the trained specialist for the purpose of continued professional development.

What is a needs assessment?

Needs assessment, often defined as a situational analysis, is a systematic process aimed at identifying and prioritising needs; collecting and analysing information; making informed, needs-based decisions; allocating resources; and implementing actions to resolve problems underlying important needs [4, 5]. Ury

et al. outline that needs assessment is a vital component in the infancy stages of any curriculum design, forming part of the foundational platform on which a working curriculum is built; however, it does not end after implementation. Instead, it is an ongoing process “which continues to provide important and useful information and feedback that ensures the sustainability of the curriculum”. Using a range of different methods, such as questionnaire surveys, focus groups, interviews and the notable Delphi method, a needs assessment has come to be recognised as a necessary and worthwhile endeavour in curriculum planning and development. Given the proven merits of the Delphi method, it was the intention of the ERS task force to employ this form of needs assessment to both inform and prioritise certain needs based upon the results of the situational analysis as well as ascertaining the specifics of what should be included within an Adult Respiratory Medicine training programme.

Target audiences

When assessing and reviewing the Adult Respiratory Medicine syllabus, careful consideration of the relevant target audiences was necessary. The organisation and structure of Adult Respiratory Medicine in general is complex and the role of ERS at an international level requires careful consideration of the range of different audiences that will use the syllabus in different capacities. Different target audiences include trainees in respiratory medicine, trained specialists within a range of specific fields as well as health professionals who play a key role in this specialty area. The different levels of training and education required for each target audience must also be considered along with existing national educational structures for training and certification.

The Delphi methodology

This study was conducted using a modified Delphi technique, a consensus-based methodology that is favoured by medical educationalists worldwide.

Delphi is a “group communication process” that focuses on gathering information from a panel of experts, aiming to reach a convergence of opinion regarding a particular issue or set of issues. Unlike other data collection methods, the Delphi method employs a number of iterations or rounds, with three rounds being deemed sufficient by relevant experts, until a consensus of opinion is reached. To begin with participants are supplied with a prepared questionnaire, which is followed by a second questionnaire based on the results garnered from the first. Subsequent surveys refine and define the central issues at heart of the study, taking into account the opinions provided by the participants.

Determining the level of consensus must be carefully considered when developing the Delphi rounds. There is no universal consensus proportion agreed upon for the Delphi method; different experts have advocated percentage measures ranging between 50%, 70% and 80%. For this particular study, the ERS task force opted to apply an 80% cut-off rate.

The task force

Invitations to join the task force were sent in December 2014. Gernot Rohde (the Netherlands), the ERS Education Council Chair, was chosen to lead the group. The other members were active ERS members selected to represent the eleven ERS assemblies in existence at that time and the ECMC (table 1).

Study participants

Prior to commencing round one of data collection, focus issues were ascertained and participants were identified during a plenary meeting of the ERS International Congress in Amsterdam in September 2015. Following a review, the draft syllabus containing 128 items divided over 22 modules, was finalised before being forwarded to all associated parties.

The participants involved in the Delphi process formed two separate groups, an expert group and a public group. The expert group was comprised of members of the task force, the Education Council, Assembly heads and secretaries, group chairs and secretaries and a number of national respondents (one national representative per country was nominated by CERS). The public group consisted of ERS assemblies: clinical; respiratory intensive care; basic and translational sciences; clinical physiology and sleep; airway diseases; epidemiology and environment; respiratory infections; and thoracic oncology.

Data collection

The study was performed between December 2015 and February 2017, and was designed as a three-round Delphi iterative process. The first round took place between December 2015 and January 2016, the second round between August 2016 and September 2016, and the third round between November and December 2016. The time frame for the first two rounds was 4 weeks, while the third and final round ran for 5 weeks.

Delphi Round 1

All participants were asked to determine whether certain modules were “mandatory”, “optional” or “excluded”. In addition, group members were required to vote on the inclusion of individual syllabus items using a Likert scale, with the scale ranging from 1 (strongly disagree) to 5 (strongly agree). Furthermore, they were also able to add comments or suggestions for module content.

In January 2016, feedback from Delphi round 1 was collated and discussed by task force members. From the 128 items surveyed, only 12 items failed to reach the approval rate of 80% and were therefore deemed controversial. Following extensive debate as to whether certain items should be retained, edited or excluded, the revised syllabus draft was then prepared for the second round of surveys.

Delphi Round 2

Round 2 took place in August 2016 with participants invited to rate the updated draft syllabus. Following discussion of the results, it was decided by task force members to launch a third Delphi round focusing only on the non-consensual items.

Delphi Round 3

The final round had two aims: to provide the respondents with a detailed overview of the results from the previous rounds; and to offer them a final opportunity to review and comment on the syllabus. Focusing only on the items that did not reach a consensus of above 80% in previous rounds, participants were required to vote “yes” or “no” as to whether include them within a certain module. In February 2017, the Delphi 3 results were discussed and the syllabus, comprising of 20 modules and 136 items, was finalised.

Statistical analysis

Descriptive statistics for all variables were provided, describing group responses by measures of central tendency (mean), dispersion (sd) and frequency distribution (percentages). The interquartile range, the mode and the median were also described for a more robust analysis. The first set of indices was employed to obtain an indication of consensus.

To ensure the consistency of the responses between the rounds, the Delphi round 2 questionnaire was similar to that used in Delphi round 1. The answers in the top two Likert measures (agree and strongly agree) were compared, from the respondents who participated in Delphi 1 and 2 to describe the group stability (see the online supplementary table). The McNemar test for correlated variables was applied. The percentage obtained gives an indication as to the stability of the responses. The items which scored <80% are those considered controversial, which is consistent with the study undertaken by ERS.

EBUS/EUS

Endobronchial ultrasound (EBUS) and oesophageal ultrasound (EUS) were two separate items in Delphi 1 and 2. EBUS, an endoscopic technique used to diagnose and stage lung cancer, reached the approval rate of 80% in Delphi 1 and 79% in Delphi 2. However, EUS, used to diagnose and stage oesophageal, pancreatic, gastric, colorectal and bile duct cancers, only scored 65.48% (Delphi 1) and 52.22% (Delphi 2). Feedback from the respondents suggested that these techniques were too specialised and only available in select medical centres. Additionally, EUS is not implemented in all hospitals and seen as complementary to EBUS. The task force considered these comments and decided to merge EUS and EBUS as only knowledge about the technique, the indications and the basic technical aspects will be required by a number of pulmonologists. The ERS recognises, however, the rapidly growing need for EBUS and EUS expertise and the major technical advances within the interventional area and thus assume that these techniques will be included in the syllabus in the future.

Thoracoscopy

Thoracoscopy is a procedure employed in the diagnosis and treatment of pleural diseases. This procedure permits exploring the pleural cavity and conducting biopsies of pleural lesions. Only 68.8% (Delphi 1), 67.22% (Delphi 2) and 59.04% (Delphi 3) of the respondents agreed to include this procedure within the syllabus, as this expertise is not integrated within most European training centres. However, they were in agreement that the knowledge and indications were a necessary inclusion. The task force decided to retain this item, including only knowledge of the procedure, its indications and limitations. The ERS, however, recognise and support the further development of medical thoracoscopy performed by pulmonologists.

Immunotherapy

Immunotherapy is defined as “the prevention or treatment of disease with substances that stimulate the immune response” (Oxford Dictionary). According to some respondents, immunotherapy was deemed a specialised item that should not be included in the general Adult Respiratory Medicine syllabus. However, as this item is seen as an important, up-and-coming topic, it was decided to retain this item in the syllabus.

Epidemiology, environment and lifestyle module

Reaching a consensus regarding the module on environmental diseases has proven challenging for the task force. Following Delphi 1 and 2, the title and position of the module within the syllabus was edited with “Smoking” the only unanimously accepted item to be included.

1) To what degree have the ERS international standards for training of respiratory professionals been accepted and used?

To date, there is little consensus on how widely the 2006 published European core syllabus in respiratory medicine has been used for training across countries. This is very much dependent on the training structure of each individual country and their established training programmes. The uptake of an international syllabus is fraught by local and national regulations for training, use of English as an international language versus national and regional languages, and how disease and approaches to medical action differ across countries. The use of the 2006 European core syllabus in respiratory medicine across countries will require further investigation to assess the frequency and level of use.

2) Is this study overly reliant on the active participation of those associated with ERS?

The ERS task force and national delegates involved in the development of this updated syllabus have been invited through the ERS membership database. Invitations were also sent to CERS, which is also affiliated with ERS. While the task force has tried to ensure equal geographic and professional participation from respiratory specialists, it must be noted that most of the respondents held an academic position or were a consultant in a public hospital, which may result in an overly ambitious syllabus not truly representative of the respiratory medicine community.

3) How can this document have a measurable impact on national and international training for respiratory medicine specialists?

Probably the greatest limitation of this project is the inability of an international task force to provide a full training programme for local institutions which will consider the essential components of knowledge, skills and attitudes required for practice. The scope of a programme developed and delivered through the international community begins and ends at the publication of internationally agreed upon standards with some supporting initiatives which may assess knowledge with limited assessment of skills. It must be noted that this project was never intended to replace local, regional or national training for respiratory specialists, but rather to act as an aid to support and reference current training, assessment and actualisation programmes.

Future considerations

The true sustainability of any efficient syllabus and curriculum lies in the ability of those at the helm to recognise the necessity of reviewing and subsequently adapting educational structures so as to meet the ever-changing needs of the respective learners. To ensure the efficiency of the Adult Respiratory Medicine syllabus, the undertaking of the Delphi method of needs assessment proved an extremely worthwhile endeavour. Informed by the feedback garnered from the Delphi rounds, the intention of the new, updated Adult Respiratory Medicine syllabus is to ensure that emerging fields in respiratory medicine, as well as updates in diagnosis, management and treatment practices since the initial 2006 publication, are addressed and reflected within the new publication. Comprising a simplified structure with fewer modules, the reviewed syllabus is designed to act as a guide for educational design, the planning of teaching and learning, and assessment structures as well as resource activities, supporting the ERS and other institutions using the ERS syllabus. Given the scope of this project, the needs assessment process was not without its limitations, although its merits certainly outweighed the aforementioned challenges. In reviewing and learning from past accomplishments, through carrying out a needs assessment, ERS is continuing to make strides towards a healthy future in Adult Respiratory Medicine.