深度学习与生物工程系统的并行计算环境

需积分: 5 157 浏览量更新于2024-07-08 收藏 24.72MB PDF 举报

"《深度学习与并行计算环境在生物工程系统中的应用》是一本由Arun Kumar Sangaiah编辑的书籍，由Elsevier出版。该书深入探讨了深度学习和并行计算如何在生物工程领域中发挥作用，旨在提供一个理解和应用这两种技术的综合平台。" 在这本书中，读者可以了解到深度学习是现代计算机科学中的一个关键领域，它通过模拟人脑神经网络的工作方式来解决复杂问题，特别是在图像识别、自然语言处理和预测分析等方面表现突出。并行计算则是为了加速处理大数据集和提高计算效率，特别是在需要大量计算资源的深度学习任务中，如训练大规模神经网络模型。书中可能涵盖了以下几个核心知识点： 1. **深度学习基础**：介绍深度学习的基本概念，包括神经网络架构（如卷积神经网络CNN、循环神经网络RNN和生成对抗网络GAN等）、损失函数、优化算法（如梯度下降和Adam优化器）以及激活函数等。 2. **并行计算原理**：讨论并行计算的基础理论，包括数据并行、模型并行和混合并行策略，以及如何在GPU和多核CPU上实现这些策略以提高深度学习的训练速度。 3. **并行计算框架**：介绍常用的并行计算工具和平台，如CUDA、OpenMP、MPI以及TensorFlow和PyTorch等深度学习框架如何支持并行计算。 4. **生物工程应用**：结合生物工程系统的特定需求，探讨深度学习和并行计算如何用于生物信号处理、基因组学分析、医学图像识别、生物系统建模等领域。 5. **案例研究**：可能包含实际的案例研究，展示深度学习和并行计算在生物工程系统中的具体应用和取得的成果。 6. **最佳实践与挑战**：讨论实施深度学习和并行计算时可能遇到的问题，如模型训练的收敛性、计算资源的分配和管理、数据预处理和模型优化等。此书对于希望将深度学习和并行计算应用于生物工程领域的研究人员、工程师和学生来说，是一份宝贵的资源。它不仅提供了理论知识，还强调了实践应用，有助于读者提升在这个交叉学科领域的专业技能。

xviii CONTENTS

6.7.5 Output of the Model, 116

6.7.6 Training a Model

Using Multiple GPU

Cards/CUDA, 118

6.8 Conclusions, 120

References, 121

7 Efﬁcient Deep Learning Approaches

for Health Informatics,

123

T.M. Navamani, ME, PhD

7.1 Introduction, 123

Machine Learning Vs. Deep

Learning, 123

7.2 Deep Learning Approaches, 125

Deep Autoencoders, 125

Recurrent Neural Networks

(RNNs), 126

Restricted Boltzmann Machine

(RBM), 127

Deep Belief Network, 127

Deep Boltzmann Machine (DBM), 127

Convolutional Neural Network, 128

7.3 Applications, 130

Translational Bioinformatics, 130

Clinical Imaging, 130

Electronic Health Records, 130

Genomics, 131

Mobile Devices, 131

7.4 Challenges and Limitations, 131

7.5 Conclusions, 134

References, 134

8 Deep Learning and

Semi-Supervised and Transfer

Learning Algorithms for Medical

Imaging,

139

R. Mohanasundaram, PhD,

Ankit Sandeep Malhotra, BE,

R. Arun, ME,

P.S. Periasamy, PhD

8.1 Introduction, 139

8.2 Image Acquisition in the Medical

Field, 139

8.3 Deep Learning Over Machine

Learning, 140

8.4 Neural Network Architecture, 140

8.5 Deﬁning Deep Learning, 140

8.6 Deep Learning Architecture, 141

8.6.1 Convolution Neural Network

(CNN), 141

8.6.2 Recurrent Neural

Network, 141

8.6.3 Deep Neural Network, 142

8.7 Deep Learning in Medical Imaging

[5], 142

8.7.1 Diabetic Retinopathy, 142

8.7.2 Cardiac Imaging, 143

8.7.3 Tumor Classiﬁcation in

Homogeneous Breast

Tissue, 143

8.8 Developments in Deep Learning

Methods, 144

8.8.1 Black Box and Deep

Learning, 144

8.8.2 Semi-Supervised and Transfer

Learning Algorithms, 144

8.8.3 Applications of

Semi-Supervised Learning in

Medical Imaging, 145

8.8.4 Method, 145

8.9 Transfer Learning, 146

8.9.1 Transfer Learning in Image

Data, 146

8.9.2 Transfer Learning Technique

for the Detection of Breast

Cancer, 149

References, 151

9 Survey on Evaluating the

Performance of Machine Lear ning

Algorithms: Past Contributions and

Future Roadmap,

153

Syed Muzamil Basha, MTech,

Dharmendra Singh Rajput, PhD

9.1 Introduction, 153

9.2 Methodology, 154

9.3 Linear Regression, 156

9.4 Nonlinear Regression, 156

9.4.1 Support Vector Machine, 156

9.4.2 K-Nearest Neighbors, 157

9.4.3 Neural Network, 158

9.5 Nonlinear Decision Tree

Regression, 158

CONTENTS xix

9.5.1 Regression With Decision

Trees, 158

9.5.2 Random Forest, 158

9.6 Linear Classiﬁcation in R, 159

9.7 Results and Discussion, 161

9.8 Conclusions, 162

References, 163

10 Miracle of Deep Learning Using

IoT,

165

Ramgopal Kashyap, PhD

10.1 Introduction, 165

10.2 Inspiration, 166

10.3 Decisions in an Area of Deep Learning

and IoT, 167

10.3.1 IoT Reference Model, 168

10.3.2 Rudiments of Machine

Learning, 169

10.3.3 Algorithms for Efﬁcient

Training, 170

10.3.4 Secure Deep Learning, 170

10.3.5 Robust and

Resolution-Invariant

Image Classiﬁcation, 171

10.3.6 Planning With Flawless and

Noisy Images, 171

10.3.7 Smart and Fast Data

Processing, 172

10.4 Simulation Results and Performance

Analysis of Handwritten Digits

Recognition in IoT, 172

10.5 An Intelligent Trafﬁc Load

Prediction, 173

10.6 Performance of Deep Learning Based

Channel Assignment, 173

10.6.1 A Deep Learning System for

the Individual Pursuit, 174

10.6.2 Network Architecture, 175

10.6.3 Dataset, 175

10.6.4 Statistics, 175

10.6.5 Effectiveness of Online

Instance Matching, 176

10.7 Discussion, 176

10.8 Conclusion, 177

References, 177

11 Challenges in Storing and

Processing Big Data Using Hadoop

and Spark,

179

Shaik Abdul Khalandar Basha, MTech,

Syed Muzamil Basha, MTech,

Durai Raj Vincent, PhD,

Dharmendra Singh Rajput, PhD

11.1 Introduction, 179

11.2 Background and Main Focus, 179

11.2.1 Challenges of Big Data

Technologies, 180

11.2.2 Real Time Applications of Big

Data Frameworks, 181

11.3 Hadoop Architecture, 181

11.4 MapReduce Architecture, 181

11.5 Joins in MapReduce, 182

11.6 Apache Storm, 184

11.7 Apache Spark Environment, 184

11.7.1 Use Cases of Big Data

Technologies, 184

11.8 Graph Analysis With GraphX, 185

11.9 Streaming Data Analytics, 185

11.10 Futer Research Directions, 186

11.11 Conclusion, 187

References, 187

12 An Efﬁcient Biography-Based

Optimization Algorithm to Solve

the Location Routing Problem With

Intermediate Depots for Multiple

Perishable Products,

189

Erfan Babaee Tirkolaee, ME,

Alireza Goli, ME,

Mani Bakhshi, ME,

Arun Kumar Sangaiah, PhD

12.1 Introduction, 189

12.2 Model Development, 191

12.2.1 Mathematical Model, 193

12.2.2 An Illustration, 195

12.3 Solution Method, 196

12.3.1 Introduction to Biography

Based Optimization

Algorithm, 196

12.3.2 Solution Representation, 198

12.3.3 Initial Solution

Generation, 198

12.3.4 Immigration Phase, 198

xx CONTENTS

12.3.5 Mutation Phase, 199

12.3.6 Optimal Parameter

Design, 199

12.4 Computational Results, 199

12.4.1 Optimal Solutions for Instance

Problems, 201

12.4.2 Sensitivity Analysis, 202

12.5 Discussion, Concluding Remarks and

Future Research Directions, 203

References, 203

13 Evolutionary Mapping Techniques

for Systolic Computing System,

207

C. Bagavathi, MTech, O. Saraniya, ME, PhD

13.1 Introduction, 207

13.2 Systolic Arrays, 208

13.3 Evolutionary Algorithms, 209

13.4 Swarm Intelligence (SI), 211

13.5 Mapping Techniques, 213

13.6 Systolic Implementation of Texture

Analysis, 215

13.7 Results and Discussion, 216

13.7.1 Performance of EA for F8

Optimization, 216

13.7.2 Texture Analysis, 217

13.8 Conclusions, 220

List of Acronyms and Abbreviations, 220

References, 220

14 Varied Expression Analysis

of Children With ASD Using

Multimodal Deep Learning

Technique,

225

S.P. Abirami, ME, G. Kousalya, ME, PhD,

Balakrishnan, ME, PhD, R. Karthick, BOT

14.1 Introduction, 225

14.2 State-of-the-Art, 226

14.3 Methodology, 227

14.3.1 Detection of Human

Faces, 227

14.3.2 Extraction of Featur es, 230

14.3.3 Expression Classiﬁer, 231

14.3.4 Expression Identiﬁcation

Through a Convolution Neural

Network (CNN), 232

14.4 Results and Analysis, 235

14.4.1 Accuracy of the Face

Expression Analysis, 240

14.5 Conclusions, 240

14.6 Future Work, 242

References, 242

15 Parallel Machine Learning and

Deep Learning Approaches for

Bioinformatics,

245

M. Madiajagan, MS, PhD,

S. Sridhar Raj, BTech, MTech

15.1 Introduction, 245

15.1.1 Machine Learning and Deep

Learning, 245

15.1.2 Role of Parallelization in Deep

Learning, 245

15.1.3 Deep Learning Applications in

Bioinformatics, 246

15.2 Deep Learning and Parallel

Processing, 246

15.2.1 Parallel Processing, 246

15.2.2 Scalability of Parallelization

Methods, 247

15.2.3 Deep Learning Using Parallel

Algorithms, 247

15.3 Deep Learning and

Bioinformatics, 248

15.3.1 Bioinformatics

Applications, 248

15.3.2 Advantages of Using

Parallel Deep Learning

in Bioinformatics

Applications, 250

15.3.3 Challenges in Using

Parallel Deep Learning

for Bioinformatics

Applications, 250

15.4 Parallel Deep Learning in

Bioinformatics Applications With

Implementation and Real Time

Numerical Example, 250

15.5 Sample Implementation Screenshots

to Visualize the Training Process, 252

15.6 Summary, 254

References, 254

INDEX, 257

CHAPTER 1

Parallel Computing,

Graphics Processing Unit (GPU) and

New Hardware for Deep Learning in

Computational Intelligence Research

M. MADIAJAGAN, MS, PHD • S. SRIDHAR RAJ, BTECH, MTECH

1.1 INTRODUCTION

Machine learning is the competency of software to per-

form a single or a series of tasks intelligently with-

out being programmed for those activities, which is a

part of artiﬁcial intelligence (AI). Normally, the soft-

ware behaves based on the programmer’s coded instruc-

tions, while machine learning is g oing one step further

by making the software capable of accomplishing in-

tended tasks by using statistical analysis and predictive

analytics techniques. In simple words, machine learn-

ing helps the software learn by itself and act accord-

ingly.

Let us consider an example, when we like or com-

ment a friend’s picture or video on a social media site,

the related images and videos are posted earlier and

stay displayed. Same with the “people you may know”

suggestions in facebook, the system suggests us another

user’s proﬁle to add as a friend who is somehow related

to our existing friend’s list. And you wonder how the

system knows that? This is called machine learning. The

software uses statistical analysis to identify the pattern

that you, as a user, are performing, and using the pre-

dictive analytics, it populates the related news feed on

your social media site.

1.1.1 Machine and Deep Learning

Machine learning algorithms are used to automatically

understand and realize the day-to-day problems that

people are facing. The number of hidden layers in an

artiﬁcial neural network reﬂects in the type of learning.

The intent is to gain knowledge by learning through

datasets using customized methods. But, in case of big

data where the data is huge and complicated, it is difﬁ-

cult to learn and analyze [1].

Deep learning plays a very vital role in resolving this

issue of learning and analyzing big data. It learns com-

plex data structures and representations acquired from

raw datasets to derive meaningful information. Big data

also supports the nature of deep learning algorithms,

which requires large amount of training data. Training

many parameters in deep learning networks increases

the testing accuracy [2].

Some deep learning applications are in natural

language processing, video processing, recommenda-

tion systems, disease prediction, drug discovery, speech

recognition, web content ﬁltering, etc. As the scope for

learning algorithms evolves, the applications for deep

learning grows drastically.

1.1.2 Graphics Processing Unit (GPU)

Graphics Processing Unit (GPU) is a specialized circuit

which can perform high level manipulation and alter

the memory. It can perform rendering of 2D and 3D

graphics to acquire the ﬁnal display. In the beginning,

the need for a GPU was driven by the world of computer

games, and slowly the researchers realized that it has

many other applications like movement planning of a

robot, image processing, video processing, etc. The gen-

eral task of the GPUs was just expressing the algorithms

in terms of pixels, graphical representations and vectors.

NVIDIA and AMD, two giants in GPU manufacturing,

changed the perspective of GPUs by introducing a ded-

icated pipeline for rendering the graphics using multi-

core systems. CPU uses vector registers to execute the

instruction stream, whereas GPUs use hardware threads

which execute a single instruction on different datasets

[1].

GPUs, and now TPUs (tensor processing units), re-

duce the time required for training a machine learn-

ing (ML) model. For example, using a CPU approach

may take a week to train, a GPU approach on the same

problem would take a day, and a TPU approach takes

only a few hours. Also, multiple GPUs and TPUs can

be used. Multiple CPUs can be used, but network la-

Deep Learning and Parallel Computing Environment for Bioengineering Systems. https://doi.org/10.1016/B978-0-12-816718-2.00008-7

2 Deep Learning and Parallel Computing Environment for Bioengineering Systems

tency and other factors make this approach untenable.

As others have noted, GPUs are designed to handle

high-dimensional matrices, which is a feature of many

ML models. TPUs are designed speciﬁcally for ML mod-

els and don’t include the technology required for image

display.

1.1.3 Computational Intelligence

Computational intelligence deals with the automatic

adaptation and organizes accordingly with respect to

the implementation environment. By possessing the at-

tributes such a s knowledge discovery, data abstraction,

association and generalization, the system can learn and

deal with new situations in the changing environments.

Silicon-based computational intelligence comprises hy-

brids of paradigms such as artiﬁcial neural networks,

fuzzy systems and evolutionary algorithms, augmented

with knowledge elements, which are often designed to

mimic one or more aspects of carbon-based biological

intelligence [3].

1.1.4 GPU, Deep Learning and

Computational Intelligence

GPU is basically based on parallel processing in na-

ture, which helps in improving the execution time

of the deep learning algorithms. By imparting the

parallel deep learning using GPU, all the computa-

tional intelligence research applications which involves

images, videos, etc., can be trained at a very fast

rate and the entire execution time is reduced drasti-

cally.

The rest of the chapter is organized as follows. In

Section 1.2, we discuss the role and types of paralleliza-

tion in deep learning. Section 1.3 tells us the role of

GPU in parallel deep learning. Section 1.4 presents the

data ﬂow of parallelization and a numerical example

on how the parallelization works in deep learning with

a real time application. Section 1.5 shows the imple-

mentation details and screenshots, while Section 1.6

summarizes the entire contents discussed in the above

sections.

1.2 DEEP LEARNING AND

PARALLELIZATION

In this section, we will discuss what is parallel pro-

cessing and the algorithms which are suitable for deep

learning through analysis. The analysis is based on the

time and throughput of the algorithms.

1.2.1 Parallel Processing Concepts

Parallel processing concept arises to facilitate the anal-

ysis of huge data and acquire meaningful information

from it. Speech processing, medical imaging, bioinfor-

matics and many similar ﬁelds are facing the difﬁculty

of analyzing huge amounts of complex data. There are

some problems in which the run-time complexity can-

not be improved even with many processors.

Parallel algorithms are called efﬁcient when their

run-time complexity divided by the number of proces-

sors is equal to the best run-time complexity in sequen-

tial processing. Not everything should be parallelized.

User experience, for example, is a serial task. If one

thread redraws the screen when some other thread is

trying to click s omething that cannot be encouraged for

parallel processing, it has to be sequential. Sometimes

sequential processing is faster than parallel where the

latter requires gathering all the data in one place, but

the former does not have to gather data [4].

In single processor systems, a set of inputs are given

to the processor and it returns the output after pro-

cessing. The performance of the processor can be made

faster by increasing the frequency limit. But, there is a

certain limit beyond which the processor emits a huge

amount of heat. The amount of heat emitted by the elec-

trons moving through the processor is very high, hence

there is a certain frequency limit beyond which the pro-

cessor melts down.

FIG. 1.1 Single processor execution.

To rectify the issue shown in Fig. 1.1,wemovetopar-

allel processing where more than one processor is used

to process the data. This way the workload is divided

between multiple processors. See Fig. 1.2.

Parallel computing has its own disadvantage such

as dependency between processors, i.e., one processor

might wait for the results of the process running on an-

other processor. In modern computing, we address the

number of processors by using the term core. Dual-core,

multi-core, i3, i5, i7, etc., all denote the number of pro-

cessors.

剩余268页未读，继续阅读

seacsd

粉丝: 10
资源: 506

深度学习与生物工程系统的并行计算环境

"并行计算环境与Linux操作系统：基础知识、环境与应用

Algorithms and Parallel Computing - Fayez Gebali

IPython 0.11: Enhanced Interactive Python Shell and Parallel Computing

Packt.Hands-On.GPU.Computing.with.Python.1789341078.epub

MATLAB Genetic Algorithm Parallel Computing: The Secret Weapon to Unlock Computational Potential and...

MATLAB Genetic Algorithm Parallel Computing Guide: Accelerating Optimization, Saving Time

MATLAB Matrix Parallel Computing: Leveraging Multi-core Advantages to Boost Computing Speed, A Three...

MATLAB Versions and Deep Learning: Model Development Training, Version Compatibility Guide

MATLAB Versions and Cloud Computing: Cloud Deployment for Efficient Utilization

MATLAB Versions and Machine Learning: Advantages and Limitations, Exploring Different Versions

最新资源