俄罗斯专家Galushkin的神经网络理论著作解析

需积分: 10 174 浏览量更新于2024-08-02 收藏 16.94MB PDF 举报

身份认证购VIP最低享 7 折!

领优惠券(最高得80元）

资源详情

资源推荐

ContentsXVIII

11.4.5 Clusterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

11.4.6 Traveling Salesman Problem

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

11.4.7 Dynamic System Modelling

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

11.4.8 Conclusion

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

11.5 Typical Input Signal of Multilayer Neural Networks

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

Literature

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

12 Analysis of Closed-Loop Multilayer Neural Networks

. . . . . . . . . . . . . . . . . . . . . . . . . . . . 223

12.1 Problem Statement for the Synthesis of the Multilayer Neural Networks

Adjusted in the Closed Cycle

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223

12.2 Investigation of the Neuron Under the Multi-Modal Distribution

of the Input Signal

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224

12.2.1 One-Dimensional Case – Search Adjustment Algorithm

. . . . . . . . . . . . . . . . . 224

12.2.2 Multidimensional Case – Analytical Adjustment Algorithm

. . . . . . . . . . . . . 226

12.3 Investigation of Dynamics for the Neural Networks of Particular Form

for the Non-Stationary Pattern Recognition

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231

12.4 Dynamics of the Three-Layer Neural Network in the Learning Mode

. . . . . . . . . . . 235

12.5 Investigation of the Particular Neural Network

with Backward Connections

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239

12.6 Dynamics of One-Layer Neural Networks in the Learning Mode

. . . . . . . . . . . . . . . . 242

12.6.1 Neural Network with the Search

of the Distribution Mode Centers f(x)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242

12.6.2 Neural Network with N

Output Channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245

12.6.3 Neuron with K

Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248

12.7 Two-Layer Neural Network in the Self-Learning Mode

. . . . . . . . . . . . . . . . . . . . . . . . . . . . 250

12.8 About Some Engineering Methods for the Selection of Matrix Parameters

in the Multilayer Neural Network Closed Cycle Adjustment Algorithms

. . . . . . . . 257

12.9 Design of the Multilayer Neural Network for the Matrix Inversion Problem

. . . 258

12.10 Design of the Multilayer Neural Network for the Number Transformation

from the Binary System into the Decimal One

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261

12.11 Investigation of the Multilayer Neural Network

under the Arbitrary Teacher Qualification

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262

12.12 Analytical Methods of Investigations of the Neural Network

Closed Cycle Adjustment

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263

Literature

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272

13 Synthesis of Multilayer Neural Networks with Flexible Structure

. . . . . . . . . . . . . . 273

13.1 Sequential Learning Algorithm for the First Neuron Layer

of the Multilayer Neural Network

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273

13.2 Learning Algorithm for the First Neuron Layer of the Multilayer

Neural Network Using the Method of Random Search of Local

and Global Function Extrema

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277

13.3 Analysis of Algorithm Convergence under the Hyperplane

Number Increase

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280

13.4 Learning Algorithms for the Second Layer Neurons

of the Two-Layer Neural Network

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283

XIXContents

13.4.1 Condition of the Logical Function e(y) Realizability

Using One Neuron

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283

13.4.2 Synthesis of a Neuron by the Functional Minimization Method

. . . . . . . . . 285

13.4.3 Neuron Synthesis by the Threshold Function Tables

. . . . . . . . . . . . . . . . . . . . . . 290

13.5 Learning Algorithm for Neurons of the Second and Third Layers

in the Three-Layer Neural Network

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290

13.6 General Methods of the Multilayer Neural Network Successive Synthesis

. . . . . . . 292

13.7 Learning Method for the First-Layer Neurons of a Multilayer

Neural Network with a Feature Continuum

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292

13.8 Application of the Adjustment Algorithm of the Multilayer Neural Networks

with Flexible Structure for the Problem of Initial Condition Selection . . . . . . . . . . 293

13.9 About the Self-Learning Algorithm for Multilayer Neural Networks

with Flexible Structure

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294

Literature

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294

14 Informative Feature Selection in Multilayer Neural Networks

. . . . . . . . . . . . . . . . . . 295

14.1 Statement of the Informative Feature Selection Problem

in the Learning Mode

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295

14.2 About Structural Methods for the Informative Feature Selection

in the Multilayer Neural Networks with Fixed Structure

. . . . . . . . . . . . . . . . . . . . . . . . . . 297

14.3 Selection of the Initial Space Informative Features Using Multilayer Neural

Networks with Sequential Algorithms of the First-Layer Neuron Adjustment

. . 299

14.4 Neuron Number Minimization

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300

14.5 About the Informative Feature Selection for Multilayer Neural Networks

in the Self-Learning Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302

Literature

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302

Part IV · Neural Network Reliability and Diagnostics

. . . . . . . . . . . . . . . . . . . . . . . . . . . . 303

15 Neural Network Reliability

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305

15.1 Methods for the Neural Network Functional Reliability Investigation

. . . . . . . . . . . 305

15.2 Investigation of Functional Reliability of Restoring Organs Implemented

in the Form of Multilayer Neural Networks

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306

15.3 Investigation of Multilayer Neural Network’s Functional Reliability

. . . . . . . . . . . . . 308

15.4 Investigation of the Neural Network’s Parametrical Reliability

. . . . . . . . . . . . . . . . . . . 309

15.5 Investigation of the Multilayer Neural Network’s Functional Reliability

in the Case of Catastrophic Failures

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317

Literature

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318

16 Neural Network Diagnostics

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321

16.1 Neural Network State Graph – The Main Notions and Definitions

. . . . . . . . . . . . . . . 322

16.2 Algorithm of Failure Localization in the Neural Networks

. . . . . . . . . . . . . . . . . . . . . . . . 323

16.3 Algorithm of the Minimum Test Design for the Failures

of the Logical Constant Type at the Neuron Outputs

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331

16.4 Method of the Neural Network Adaptive Failure Diagnostics

. . . . . . . . . . . . . . . . . . . . 332

Literature

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338

ContentsXX

Part V · Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339

17 Methods of Problem Solving in the Neural Network Logical Basis

. . . . . . . . . . . . 341

17.1 Neuromathematics – A New Perspective Part of Computational Mathematics

. 341

17.2 Neural Network Theory – A Logical Basis for the Development

of the Neural Network Problem Solution Algorithms

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343

17.3 Selection of the Problems Adequate to the Neural Network Logical Basis

. . . . . . . 344

17.4 The General Structure of the Program Package for Problem Solution

in the Neural Network Logical Basis

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349

17.5 Multilayer Neural Networks with Flexible Structure

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350

17.6 Neural Network with Fixed Structure

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352

17.6.1 Generation of the Input Signal of the Neural Network

. . . . . . . . . . . . . . . . . . . . 352

17.6.2 The Multilayer Neural Network Output Signal Generation

. . . . . . . . . . . . . . . 355

17.6.3 Formation of the Primary Optimization Criteria

. . . . . . . . . . . . . . . . . . . . . . . . . . 355

17.6.4 Selection of the Open Neural Network Structure

. . . . . . . . . . . . . . . . . . . . . . . . . . 356

17.6.5 Remarks about the Selection of the Open Neural Network Structure

that is Adequate to the Class of Solution Tasks

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356

17.6.6 Remarks about the Activation Function Selection

. . . . . . . . . . . . . . . . . . . . . . . . . 358

17.6.7 Selection of the Multilayer Neural Network Structure

According to its Hardware Implementation Technology

. . . . . . . . . . . . . . . . . 359

17.6.8 Generation of the Secondary Optimization Functional

in the Multilayer Neural Networks

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360

17.6.9 Generation of the Algorithm of the Search Procedure

for the Secondary Optimization Functional Extremum . . . . . . . . . . . . . . . . . . 360

17.6.10 Formation of the Adaptation Algorithms

in the Multilayer Neural Networks

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364

17.7 Verification of the Adjusted Multilayer Neural Network

. . . . . . . . . . . . . . . . . . . . . . . . . . . 364

17.8 Elaboration of the Plan of Experiments

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365

17.9 About the Importance of the Unification of Designations in the Process

of Synthesis of the Neural Network Adjustment Algorithms

. . . . . . . . . . . . . . . . . . . . . . 367

17.10 About Myths in Neural Network Theory

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368

17.11 Conclusion

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368

References

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376

Conclusion

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377

Literature

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379

Author’s Publications on Neural Network Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381

Index

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391

I.1

Neural Computers

Neurocomputers are computers of a new class. Their appearance was determined for

two objective reasons: first, the principal stages of the development of modern elemental

base technology that mainly determines the development of computer architecture,

and second, practical requirements to solve specific problems in a faster and more

economical manner.

As far back as the 1950s, the main reason for neural computer development appeared

to be a development of the threshold logic that was permanently contradistinguished

to the classical development of the elemental base on the basis of AND, OR, NOT, etc.

This resulted in the implementation of a series of specific problem-oriented and ex-

perimental universal neural computers in the 1960s and 1970s. The terms “neural com-

puter” and “neurocomputer” are not associated with any feature or characteristic of

the human or animal nervous system. They are associated only with the conditional

name of a threshold element with the adjustable or fixed weights that implement an

elementary transfer function of the neural cell. A sharp upswing of LSI technological

development at the beginning of the 1970s, as well as the implementation primarily of

microprocessor chips with the classic computer architecture was realized on the el-

emental base on the basis of AND, OR, NOT, etc., resulting in the slowdown rather than

complete termination of the development of the computing facilities based on the

threshold logic elements. The next upswing at the beginning of the 1980s allowed one

to redefine the problem of neural computers due to the fact that VLSI technology, rather

than LSI technology, allowed one to implement in one or several chips not only a great

number of processing neuron elements, but also a whole set of connections between

them. This was not possible before. Such a possibility was provided by both electronic

as well as optical implementation methods in the middle of the 1980s.

The main idea of the neural computer construction, either a problem-oriented or

universal one, is to develop computers in the analog-digital form. In this regard, the

“fast” analog part performs multidimensional operations on the threshold basis. The

algorithms of the neural network coefficient adjustment are implemented either in the

“fast” manner in the analog form, or in the “low-speed” manner in the form of special-

ized digital circuits emulating neural algorithms, or in the “low-speed” manner in the

digital form, for example, using the universal personal computer.

The development of neural computers requires a design of the principally new

algorithms for the multidimensional solution of problems. The time for the solution

of the specific problem, on the one hand, only linearly depends upon the problem

Introduction

2 I · Introduction

dimensionality and, on the other hand, is determined by the convergence time of the

iteration process for the solution in the particular neural network.

The main formal basis for the design of the neural algorithms is a neural network

theory [I-1 to I-7]. Let us call the main set of operations that are realized in the process

of the algorithm refinement as a “logical basis of the problem”. For the majority of

problems such a basis is the basis {Σa

}. The following problems require this basis: the

problems of vector algebra, Fourier transformation and optimization problems.

A class of tasks including solutions of the ordinary differential equations, Poisson,

Euler, Navier-Stokes equations, elliptical equations and so on can be reduced to the

aforementioned problems.

A logical basis of the computer system is the main group of operations implemented

by the elements of the basic operating device. In the case of classical computers, it is a

basis of AND, OR, NOT that forms, first, the level of more complex basis (Sheffer stroke,

multiple AND, OR, NOT, etc.), and a macro level, i.e., a level of microdevices. The logical

basis of the computer system is not determined by the logical basis of the solved problems

but requires an additional application of a rather complex program development system.

In the case of neural computers, the logical basis of the computer system in the

simplest case is the basis {Σa

, sign}. This basis maximally corresponds to the logical

basis of the major solved problems.

When solving problems with the neural computers’ logical basis, the basis of the

problem is in accordance with the basis of the computer system, and there are no ar-

tificial shifts in any direction:

 one such shift corresponds to the problems with the threshold basis, whereas the

basis of the computer system is AND, OR, NOT;

 another shift corresponds to the problem with the basis that differs from the thresh-

old one, whereas the computer system is neural.

It is supposed that the accordance between the computer system basis and the basis

of the problem provides the highest productivity. This statement is trivial in the case

of problem-oriented computers that are designed for the solution of a given specific

task. However, it is not trivial for the neural computers that pretend to be called uni-

versal at the present time.

All the difficulties of multiextremal search by iteration methods using neural com-

puters are evidently preserved. But these difficulties transfer from the software imple-

mentation (von-Neumann computers, computers with SIMD and MIMD architectures)

to the software/hardware implementation. In short, an algorithmic kernel of the main

array of applied problems can be realized in hardware or software/hardware form with

maximal operation speed. The neural computer is a maximally parallelized system for

a given algorithmic kernel implementation. The number of operation cycles in the

problem solving process, i.e., the number of the adjustment cycles for optimization of

the secondary functional in the neural computer, is not determined by the subjective

intuition of the circuit designer who distributes the processing among the circuit lay-

ers consisting of the Boolean elements. This number is also not determined by the

subjective views of the programmer who organizes the interaction between layers. On

the contrary, it is determined by the physical entity and complexity of the problem.

剩余401页未读，继续阅读

Zylinkultamyyrä

粉丝: 10
资源: 103

俄罗斯专家Galushkin的神经网络理论著作解析

Pattern Recognition with Neural Networks in C++

NEURAL NETWORKS with MATLAB (Marvin L.)

循环神经网络有哪些经典书籍

rbf神经网络参考文献

交叉熵损失函数来自那篇文献

计算机毕设英文参考文献

近6年有关AES加密算法的文献

Pattern Recognition and Machine Learning-01-Preface

ASRPRO语音控制系统参考文献有哪些

关于AUC<0.5时把结果取反的文献

分别给我几篇关于循环神经网络，决策树，支持向量机的参考文献

development of multi-agent reinforcement learning

有关遗传算法和系统辨识的英文文献

布谷鸟优化算法的20个参考文献

matlab程序模拟单个正电荷电位下的电子轨迹.rar

医药生物：创新药、“AI 医疗”有望引领医药产业新发展(1).pdf

飞利浦4H.09C02.A11电源板电路图.rar

CRC校验日常学习笔记

【2024首发原创】白鲨优化算法WSO-TCN-LSTM-Multihead-Attention负荷预测Matlab实现.rar

凌日优化算法TSOA-TCN-LSTM-Multihead-Attention多变量时间序列预测Matlab实现.rar

最新资源