If you're interested in exploring alternative activation functions, I recommend that you try and track down a relatively obscure 1991 research paper titled, "Efficient Activation Functions for the ...

Explore 20 powerful activation functions for deep neural networks using Python! From ReLU and ELU to Sigmoid and Cosine, ...

The activation function demo. The demo program illustrates three common neural network activation functions: logistic sigmoid, hyperbolic tangent and softmax. Using the logistic sigmoid activation ...