大数据分析的计算方法详解

需积分: 9 81 浏览量更新于2024-07-17 1 收藏 19.77MB PDF 举报

《计算方法在数据分析中的应用》(De Gruyter 教科书)由耶利兹·卡拉卡(Assist.Prof.)和卡洛·卡塔尼(Prof.Dr.)合著，两位作者分别来自美国马萨诸塞医学院和意大利图斯库卢姆大学的工程学院DEIM。本书于2018年12月17日出版，是针对研究生的教材，专注于数学和统计工具在处理生物医学和经济学等领域大数据的分析。书中涵盖了诸如神经网络、马尔可夫链、统计物理工具以及小波分析等，通过这些理论发展出高效的计算算法，并利用Matlab进行实际数据处理。本书的核心内容围绕以下几个方面展开： 1. **数学与统计基础**：介绍了用于处理大规模数据的数学原理，如线性代数、概率论和统计学，这些都是计算方法的基础，为后续的复杂算法提供了坚实的基础。 2. **机器学习与神经网络**：详细讲解了神经网络模型的设计和实现，包括深度学习的概念和应用，这些技术在数据挖掘和预测分析中扮演着关键角色。 3. **马尔可夫过程**：通过马尔可夫链理论，作者展示了如何建模随机过程和状态转移，这对于序列数据的分析和时间序列预测至关重要。 4. **统计物理方法**：将物理学中的概念和模型引入到数据分析中，如熵、能量函数等，帮助理解数据的结构和复杂性，对于聚类和优化问题有独特的视角。 5. **小波分析**：作为一种时频分析工具，小波分析能有效地捕捉数据中的局部特征，对信号处理和图像分析非常有用。 6. **算法设计与实现**：通过Matlab等编程语言，作者指导读者如何将理论转化为实际代码，实现各种计算方法并应用于具体的数据集。 7. **案例研究与应用**：书中包含丰富的实际案例，如医疗数据分析、经济趋势预测等，让读者能够理解和掌握所学理论在现实场景中的应用。 8. **附录与资源**：提供必要的背景知识补充和Matlab代码示例，便于读者扩展学习和实践。《计算方法在数据分析中》是一本实用且深入的教材，适合那些希望在大数据时代掌握先进数据分析技术的研究人员、工程师和学生。随着科技的发展，理解和应用这些计算方法对于各行各业的决策者和分析师来说愈发重要，这本书无疑是他们提升专业技能的宝贵资源。

Data science involves employing methods to analyze massive amounts of data and extract the

knowledge they encompassed. We can imagine that the relationship between data science and big

data is similar to that of raw material and manufacturing plant. Data science and big data have

originated from the field of statistics and traditional data management. However, for the time being,

they are seen as distinctive disciplines.

The characteristics of big data are often referred to as the three Vs as follows [2]:

–Volume – This refers to how much data are there.

–Variety – This refers to how diverse different types of data are.

–Velocity – How fast new data are generated.

These characteristics are often supplemented with a fourth V that refers to

–Veracity, meaning how accurate is the data.

These four properties make big data different from the data existing in conventional data

management tools. As a result, the challenges brought by big data are seen in almost every aspect

from data capture to curation, storage to search as well as other aspects including but not limited to

sharing, transferring and visualization. Moreover, big data requires the employment of specialized

techniques for the extraction of the insights [3, 4].

Following a brief definition of big data and what it requires, let us now have a look at the

benefits and uses of big data and data science.

Data science and big data are terms used extensively both in commercial and noncommercial

settings. There are a number of cases with big data. The examples we offer in this book will

elucidate such possibilities. Commercial companies operating in different areas use data science

and big data so as to know more about their staff, customers, processes, goods and services. Many

companies use data science to be able to offer a better user experience to their customers. They

also use big data for customization.

Let us have a look at the following example to relate the information about aforementioned big

data, for example, NYSE (https://www.nyse.com/data/transactions-statistics-data-library) [5]. Big

dataset is an area for fundamental and technical analysis. In this link, dataset includes

fundamentals.csv file. Fundamentals file show which company has the biggest chance of going out

of business, which company is undervalued, what the return on Investment is metrics extracted

from annual sec 10K fillings (from 2012 to 2016), should suffice to derive most popular

fundamental indicators as can be seen from Table 2.2.

Download from http://finelybook.com 7450911@qq.com

status scale (EDSS) scores and other relevant medical findings of the patients. In another example,

for instance, clinical psychology dataset may consist of test results of the subjects and demographic

features of the subjects such as age, marital status and education level. Data objects, which may also

be referred to as samples, examples, objects or data points, are characteristically defined by

attributes.

Definition 2.2.2. Attributes are data fields that denote feature of a data objects. In literature, there

are instances where other terms are used for attributes depending on the relevant field of study

and research. It is possible to see some terms such as feature, variable or dimension. While

statisticians prefer to use the term variable, professionals in data mining and processing use the

term attribute.

Definition 2.2.3. Data mining is a branch of computer science and of data analysis, aiming at

discovering patterns in datasets. The investigation methods of data mining usually are based on

statistics, artificial intelligence and machine learning.

Definition 2.2.4. Data processing is also called information processing is a way to discover

meaningful information from data.

Definition 2.2.5. A collection of data depending on a given parameter is called a distribution. If the

distribution of data involves only one attribute, it is called univariate. A bivariate distribution, on

the other hand, involves two attributes. A set of possible values determine the type of an attribute.

For instance, nominal, numeric, binary, ordinal, discrete and continuous are the aspects an attribute

can refer to [7, 8].

Let us see some examples of univariate and bivariate distributions. Figure 2.4 shows the graph

of univariate distribution for net domestic credit attribute. This is one of the attributes in Italy

Economy Dataset Explanation dataset (Figure 2.4(a)). The same dataset is given in Figure 2.4(b),

with bivariate distribution graph together with deposit interest rate attributes based on years as

chosen from the dataset for two attributes (for the period between 1960 and 2015).

Figure 2.4: (a) Univariate and (b) bivariate distribution graph for Italy Economy Dataset.

The following sections provide further explanations and basic examples for these attributes.

2.2.1NOMINAL ATTRIBUTES AND NUMERIC ATTRIBUTES

Nominal refers to names. A nominal attribute, therefore, can be names of things or symbols. Each

value in a nominal attribute represents a code, state or category. For instance, healthy and patient

categories in medical research are example of nominal attributes. The education level categories

such as graduate of primary school, high school or university are other examples of nominal

attributes. In studies regarding MS, subgroups such as relapsing remitting multiple sclerosis

Download from http://finelybook.com 7450911@qq.com

剩余370页未读，继续阅读

THESUMMERE

粉丝: 23

大数据分析的计算方法详解

Gabor分析MATLAB代码详解与数据处理应用

MATLAB在数据分析计算方法中的应用

R编程入门与统计计算权威指南

Computational Methods for Next Generation Sequencing Data Analysis

Computational-Methods-For-Data-Analysis

gabor分析matlab代码-Computational-Methods-For-Data-Analysis:这是数据分析计算方法类作业的g

Data analysis

Big Data Analysis for Bioinformatics and Biomedical Discoveries.CRC(2016).pdf

Advanced Data Analysis in Neuroscience

Doing Bayesian Data Analysis: A Tutorial with R and BUGS

最新资源