Explore the role of activation functions in deep learning and how they help neural networks learn complex patterns.

Consider the multivariate nonparametric regression model. It is shown that estimators based on sparsely connected deep neural networks with ReLU activation function and properly chosen network ...