Expert identification and calibration for collective forecasting tasks

The widespread use of the Internet and online forecasting systems offer unprecedented opportunities to leverage collective intelligence to produce increasingly accurate forecasts. Forecast support systems also offer the opportunity to address one of the weakest aspects of expert forecasting methods, the identification of experts. In the published literature, significant criticism is addressed to the subjectivity of expert identification methods, as different methods can lead, ceteris paribus, to significantly different results. This article introduces an approach to define objectively levels of expertise in large groups in a panel setting. This information is used to fine-tune panel members’ contribution to the compound forecast in an attempt to improve the accuracy of the aggregated forecast. Tested on prospects collected from the UN World Tourism Organization Panel of Tourism Experts – probably world’s most widely used and influential forecasts for the tourism sector – the proposed approach proves efficient in identifying experts in large groups of individuals. The results also indicate that the method is promising in leveraging their collective knowledge to return more accurate forecasts compared to simpler methods.