Lehr- und Forschungseinheit für Datenbanksysteme

Breadcrumb Navigation


Accepted short paper at WI-IAT 2020

Interpretable and Fair Comparison of Link Prediction or Entity Alignment Methods



Max Berrendorf, Evgeniy Faerman, Laurent Vermue, Volker Tresp


The 2020 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT'20)
14–17 December 2020
, Virtual


Link Prediction and Entity Alignment methods are usually evaluated in a rank-based setting. We show that there are multiple competing implementations on how to deal with the case of equal scores, only one of which is preferrable. Moreover, some of these rank-based metrics get automatically better for smaller evaluation set sizes, and we show this case occurs practially. We propose a new adjusted version of the mean rank metric normalizing for chance, which is comparable for differently sized evaluation sets.