Deep learning approaches are widely used for many tasks and applications, spanning from object detection, to classification and control. Certifying or enforcing performance and stability guarantees for controllers based on deep learning is, however, challenging. This work considers the use of so called non-autonomous input-output stable deep neural networks for the control of dynamical systems. We train the neural network based on an existing controller that achieves desirable nominal closed loop system properties. Assuming the infinite layer network leads to a stable closed loop, we derive bounds for the finite number of layers of the neural network, such that stability of the nominal closed loop system under the deep network controller is guaranteed. We furthermore derive conservative conditions which can be easily integrated in the learning phase to enforce stability based on the small gain theorem. The results are underlined by a simulation study considering the control of a continuously stirred tank reactor.