Pair the dots: Jointly examining training history and test stimuli for model interpretability