Article ; Online: ReLU Network with Bounded Width Is a Universal Approximator in View of an Approximate Identity
Applied Sciences, Vol 11, Iss 427, p
2021 Volume 427
Abstract: ... R
Abstract | Deep neural networks have shown very successful performance in a wide range of tasks, but a theory of why they work so well is in the early stage. Recently, the expressive power of neural networks, important for understanding deep learning, has received considerable attention. Classic results, provided by Cybenko, Barron, etc., state that a network with a single hidden layer and suitable activation functions is a universal approximator. A few years ago, one started to study how width affects the expressiveness of neural networks, i.e., a universal approximation theorem for a deep neural network with a Rectified Linear Unit (ReLU) activation function and bounded width. Here, we show how any continuous function on a compact set of <math display="inline"><semantics><mrow><msup><mi mathvariant="double-struck">R</mi><msub><mi>n</mi><mrow><mi>i</mi><mi>n</mi></mrow></msub></msup><mo>,</mo><msub><mi>n</mi><mrow><mi>i</mi><mi>n</mi></mrow></msub><mo>∈</mo><mi mathvariant="double-struck">N</mi></mrow></semantics></math> can be approximated by a ReLU network having hidden layers with at most <math display="inline"><semantics><mrow><msub><mi>n</mi><mrow><mi>i</mi><mi>n</mi></mrow></msub><mo>+</mo><mn>5</mn></mrow></semantics></math> nodes in view of an approximate identity. |
---|---|
Keywords | deep neural nets ; ReLU network ; universal approximation theory ; a feed-forward neural network ; Technology ; T ; Engineering (General). Civil engineering (General) ; TA1-2040 ; Biology (General) ; QH301-705.5 ; Physics ; QC1-999 ; Chemistry ; QD1-999 |
Subject code | 511 |
Language | English |
Publishing date | 2021-01-01T00:00:00Z |
Publisher | MDPI AG |
Document type | Article ; Online |
Database | BASE - Bielefeld Academic Search Engine (life sciences selection) |
Full text online
More links
Kategorien
Inter-library loan at ZB MED
Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.