It may sound strange to fit a deep neural network to a single neuron, but single neurons are surprisingly complex in their input-output relationships, and at the time when we did this work, there was an extensive debate going on over the degree to which neurons integrate their synaptic inputs in a linear/nonlinear way (by people like Bartlett Mel, Rafa Yuste, etc). We decided to approach the question through model selection over different neural network architectures. Because this work started in 2011 (just before AlexNet), our awareness of modern techniques to fit neural networks was limited, and so we did full gradient descent (not SGD), calculated gradients by hand, etc. It was very messy. It also means our language and references relate more to the neural coding / GLM literature, and less to that of neural networks. However, in retrospect, we were really just doing model selection over fully-connected MLPs with different depths.