This chapter contains a description of the historical evolution of artificial neural networks since their inception, with the appearance of the first relevant learning method by Paul Werbos in 1986, which remained ignored until it was discovered simultaneously by three groups of independent researchers: LeCun (1986); Parker (1985); and Rumelhart, Hinton, and Williams (1986). The basic concepts and definitions are also described, followed by the different typologies and the most relevant learning algorithms, most notably the backpropagation algorithm. The most recent advances in artificial neural networks are also described, highlighting convolutional neural networks and deep learning.
TopIntroduction
Continuous curiosity about the human brain has led to a great deal of study into it. Since the dawn of time, researchers have spent a great deal of time and energy attempting to comprehend the human mind from both a biological and cognitive standpoint. Unfortunately, it has been a long time since these efforts have yielded concrete, tangible outcomes. Most likely two of the primary reasons for the extremely sluggish development of the sciences connected to the study of cognition and the human brain were a lack of tools and technology that could examine the brain without damaging it. A remarkable percentage of the knowledge about the human brain and its structure was discovered in the last decades, supported by the increasing advance in computer technology and electronics. These two factors have had a tremendous impact in almost all areas of research, opening new directions and allowing solutions (answers) to problems previously classified as insuperable. Researchers discovered they possessed the required and sufficient circumstances to reproduce the process behind human cognition in artificial machines as a result of the availability of such technologies, leading to the development of artificial intelligence, or AI. One of the key objectives of the scientific and commercial community is still to create artificial systems with the same capabilities as the human brain.
One of the fields of research in AI is the study of Artificial Neural Networks (ANN). An ANN is a structure of Processing Elements (PE), Processing Units (PU), or artificial neurons whose functionalities are similar to those of biological neural networks (Gurney, 1999). ANNs emerged for the purpose of modeling networks of neurons in the brain, remaining from a neurophysiological perspective as highly simplified models (Hertz et al., 1991). These simplifications assume that ANNs generically maintain the behavior of biological neuronal networks, enabling their study and understanding (Bose & Liang, 1996; Hertz et al., 1991).
There are two levels of simplification in ANNs: an elementary level, in which the artificial neuron of the ANN is a simplified mathematical formalization of the biological neuron's functionalities; and a structural level, in which the ANN is a schematic and functional representation of the biological neuron's connections and interactions. ANNs have been applied in validating theories of brain function, cognition, and learning, and thus constitute simplified models of the central nervous system with recognized merit (Reed & Marks, 1999).
It is indisputable that there is a consensus in the reviewed literature regarding the concepts and paradigms of ANNs, which originated in Neurology and Behavioral Sciences. Before describing and comparing biological neurons with artificial ones, it is important to present the characteristics that are desirable in artificial systems and that are also relevant in biological neuronal systems. Their ability to perform certain tasks in a small or even instantaneous period of time and these tasks are difficult to perform on digital computers, is the most important characteristic of biological neural systems. Examples of tasks performed naturally and without apparent brain effort are voice recognition and understanding of its content, facial recognition of people (including recognition of people previously viewed in photographs, even if those people have undergone remarkable changes such as aging), recognition of facial expressions, identification of a familiar voice in a noisy environment, and many others (Bose & Liang, 1996). In biological neural systems, their flexibility is therefore remarkable, showing an appetite to handle information from various senses and, simultaneously, to handle diffuse, probabilistic, and incomplete information (Hertz et al., 1991). Another important characteristic of biological neural systems is their fault tolerance, that is, the death of one or more neurons does not prevent the system from functioning, and the results are little affected (Bose & Liang, 1996; Fausett, 1994). This characteristic is uncommon in artificial systems, particularly in digital computers, because the failure of one component results in serious errors. Finally, it should be noted that these systems have an architecture with a high degree of parallelism and small size (Hertz et al., 1991).