Neuron Model Sigmoid Activation Function Based on Multi-Optional Functions Entropy Conditional Optimization Doctrine

UDC 303.725.36:159.9.015:159.964.21:519.86(045) DOI:10.18372/1990-5548.58.13518

  1. A. V. Goncharenko. Neuron Model Sigmoid Activation Function Based on Multi-Optional Functions Entropy Conditional Optimization Doctrine // Electronics and Control Systems, N 4(58) – Kyiv: NAU, 2018. – pp. 108–114.

It is made an attempt to discover an explainable plausible reason for a neuron activation function, of a sigmoid type function like logistic function, substantiation in terms of the multi-optional conditional optimality doctrine for the special hybrid-optional effectiveness functions uncertainty. In the studied case, the input-output mapping is stipulated by the entropy of the activation function conditional optimal distribution in regards with the induced local field of the neuron. It is proposed to evaluate the direction of uncertainty with the combined hybrid relative pseudo-entropy function. This is a new insight into the scientific substantiation of the well-known dependency derived in another way. The developed theoretical contemplations and mathematical derivations are verified with numerical simulation and plotted diagrams.

Index Terms—Neuron initialization; activation function; sigmoid function; multi-optional doctrine; conditional optimality; hybrid-optional effectiveness function; pseudo-entropy; variational problem.

References: 36 name.