Unit norm constraints and ANN

Sorry for not getting to you earlier. Did you manage to make progress in the meantime?

There is a good discussion on sum to zero constraints at Test: Soft vs Hard sum-to-zero constrain + choosing the right prior for soft constrain (later in the thread there is a also a clever solution using the QR decomposition).

I am also unsure that using just one layer will prevent all the nasty computational issues that arise with neural networks.

Hope that helps at least a little