Conference note: 'Validating Black Box Neural Networks'

Machine learning, driven by recent advances in neural net technology, holds much promise, but how to validate any particular model? This note looks at why validation is necessary, and describes some practical techniques for doing so.

Why validate?

Neural nets are practically opaque to a debug walk-through. Reruns against the same training data can yield quite different results because network layers are initialised by random weights and biases, and thus there is no guarantee that the cost function has not found misleading local minima.

Sanderson (n.d.) shows that visualisations of the inner layers of a hand-written digit recognition model bear no evolving relationships to anything a human would recognise. His simple model operates in a 13,002-dimensional space (input pixels*layer weights and biases), well beyond our ability to fathom.

Therefore, one cannot explain why or how a...

Not a subscriber?

Schedule your live demo with our team today

WARC helps you to plan, create and deliver more effective marketing

  • Prove your case and back-up your idea

  • Get expert guidance on strategic challenges

  • Tackle current and emerging marketing themes

We’re long-term subscribers to WARC and it’s a tool we use extensively. We use it to source case studies and best practice for the purposes of internal training, as well as for putting persuasive cases to clients. In compiling a recent case for long-term, sustained investment in brand, we were able to support key marketing principles with numerous case studies sourced from WARC. It helped bring what could have been a relatively dry deck to life with recognisable brand successes from across a broad number of categories. It’s incredibly efficient to have such a wealth of insight in one place.

Insights Team
Bray Leino

You’re in good company

We work with 80% of Forbes' most valuable brands* and 80% of the world's top top-of-the-class agencies.

* Top 10 brands