Neural nets are practically opaque to a debug walk-through. Reruns against the same training data can yield quite different results because network layers are initialised by random weights and biases, and thus there is no guarantee that the cost function has not found misleading local minima.
Sanderson (n.d.) shows that visualisations of the inner layers of a hand-written digit recognition model bear no evolving relationships to anything a human would recognise. His simple model operates in a 13,002-dimensional space (input pixels*layer weights and biases), well beyond our ability to fathom.
Therefore, one cannot explain why or how a particular prediction was correct or incorrect.
Table 1. Frequency counts of each rating.
Approaches to validation
Indirect methods are required. First, ensure the model is internally consistent, and then compare with independent benchmarks.