I am glad to inform you that our paper “Explainable Inference on Sequential Data via Memory-Tracking” (Biagio La Rosa, Roberto Capobianco, Daniele Nardi) has been accepted for publication at IJCAI 2020. In this paper, we address the problem of explainable AI when dealing with sequential data. To this end, we leverage memory augmented neural networks and keep track of memory usage at inference time. While the paper will be made available soon on the IJCAI proceedings, here is the abstract of our work:
In this paper we present a novel mechanism to get explanations that allow to better understand
network predictions when dealing with sequential data. Specifically, we adopt memory-based net-
works — Differential Neural Computers — to exploit their capability of storing data in memory and
reusing it for inference. By tracking both the memory access at prediction time, and the information
stored by the network at each step of the input sequence, we can retrieve the most relevant input
steps associated to each prediction. We validate our approach (1) on a modified T-maze, which is a
non-Markovian discrete control task evaluating an algorithm’s ability to correlate events far apart in
history, and (2) on the Story Cloze Test, which is a commonsense reasoning framework for evaluat-
ing story understanding that requires a system to choose the correct ending to a four-sentence story.
Our results show that we are able to explain agent’s decisions in (1) and to reconstruct the most relevant
sentences used by the network to select the story ending in (2). Additionally, we show not only that
by removing those sentences the network prediction changes, but also that the same are sufficient to
reproduce the inference.
Details, as well as source code will be available soon at the following link: https://krlgroup.github.io/explainable-inference-on-sequential-data-via-memory-tracking/