没有合适的资源?快使用搜索试试~ 我知道了~
首页大数据分析:理论与应用探索
大数据分析:理论与应用探索
需积分: 10 0 下载量 49 浏览量
更新于2024-07-17
收藏 7.24MB PDF 举报
《大数据分析:经典英文图书》是一本旨在探讨大数据研究与开发的关键问题的著作。该书汇集了来自学术界和业界的作者,他们不仅参与产品的开发和定制化大数据应用,而且提供了对六大核心议题的深入洞察。这六大主题包括: 1. 大数据管理:书中详细讨论了如何有效地收集、存储、组织和处理海量数据,涉及到数据仓库设计、数据质量管理和数据治理等实践。 2. 分布式处理算法和挖掘模式:介绍分布式计算框架(如Hadoop、Spark等)中的算法和技术,以及如何通过机器学习和数据挖掘技术发现隐藏的规律和模式。 3. 安全与隐私管理:关注大数据安全,包括数据加密、访问控制、隐私保护策略,以及在合规性法规(如GDPR)下的数据处理挑战。 4. 大数据服务SLA:阐述如何为大数据服务设定服务水平协议(SLA),确保服务质量,包括响应时间、可用性和数据一致性等方面的承诺。 5. 大数据分析应用:涵盖了多个实用领域的数据分析,如商业智能、物联网、社交网络分析、金融预测等,展示了大数据在现实生活中的广泛应用。 6. 计算智能与大数据结合:探讨人工智能和大数据的交叉领域,如神经网络、进化计算、模糊系统和人工智能在大数据背景下的创新应用。 该书的目的是激发读者对大数据领域未来发展的兴趣,并鼓励他们进一步探索相关文献。书中的引言部分提供了一个重要的参考框架,列出了关键的参考资料,以便读者在深入研究时有所依据。 《大数据分析:经典英文图书》为读者提供了一个全面且深入的大数据全景,既覆盖了理论基础,又包含了实际应用,是理解和投身于大数据研究与开发的宝贵资源。对于任何希望在这个领域取得突破或保持前沿知识的人来说,这本书都是不可或缺的指南。
资源详情
资源推荐
![](https://csdnimg.cn/release/download_crawler_static/11060204/bg10.jpg)
used for years by users. Relational database technology thus was at the top for
organisational and corporate usages. But, now emergent data no longer follow a
defined structure. Variety of data comes in variety of structures. All accom modating
in a defined structure is neither possible nor prudent to do so for different usages.
Our world is now literally swamped with several digital gadgets ranging from
wide variety of sensors to cell phones, as simple as a cab has several sensors to throw
data on its performance. As soon as a radio cab is hired, it starts sending messages on
travel. GPS fitted with cars and other vehicles produce a large amount of data at
every tick of time. Scenario on roads, i.e. traffic details, is generated in regular
intervals to keep an eye on traffic management. Such scenarios constitute data of
traffic commands, vehicles, people movement, road condition and much more
related information. All these information could be in various forms ranging from
visual, audio to textual. Leave aside very big cities, in medium-sized city with few
crores of population, the emerging data could be unexpectedly large to handle for
making a decision and portraying regular traffic conditions to regular commuters.
Internet of things (IoT) is the new emerging world today. Smart home is where
gadgets exchange information among themselves for getting house in order like
sensors in a refrigerator on scanning available amount of different commodities
may make and forward a purchase list to a near by super market of choice. Smart
cities can be made intelligent by processing the data of interest collected at different
city points. For examp le, regulating city traffic in pick time such that pollution
levels at city squares do not cross a marked threshold. Such applications need
processing of a huge data that emerge at instant of time.
Conducting business today unlike before needs intelligent decision makings.
More to it, decision-making now demands instant actions as business scenario
unfolds itself at quick succession. This is so for digital connectivity that makes
business houses, enterprises, and their stakeholders across the globe so closely
connected that a change at far end instantl y gets transmitted to another end. So, the
business scenario changes in no time. For example, a glut in crude oil supply at a
distributor invites changes in status of oil transport, availability at countries
sourcing the crude oil; further, this impacts economy of these countries as the
productions of its industries are badly affected. It shows an event in a business
domain can quickly generate a cascade of events in other business domains.
A smart decis ion-making for a situation like this needs quick coll ection as well as
processing of business data that evolve around.
Internet connectivity has led to a virtual society where a person at far end of the
globe can be a person like your next-door neighbour. And number of people in
one’s friend list can out number to the real number of neighbours one actually has.
Social media such as Twitter, Facebook, Instagram and many such platforms
provide connectivity for each of its members for interaction and social exchanges.
They exchange messages, pictures, audio files, etc. They talk on various issues
ranging from politics, education, research to entertainment. Of course, unfortu-
nately such media are being used for subversive activities. Every moment millions
of people on social media exchanges enormous amount of information. At times for
different usages ranging from business promotions to security enhancement,
2 H. Mohanty
![](https://csdnimg.cn/release/download_crawler_static/11060204/bg11.jpg)
monitoring and understanding data exchanged on social media become essential.
The scale and the speed at which such data are being generated are mind bugging.
Advancement in health science and technology has been so encouraging in
today’s world that healthcare can be customised to personal needs. This requires
monitoring of personal health parameters and based on such data prescription is
made. Wearable biosensors constantly feed real-time data to healthcare system and
the system prompts to concerned physicians and healthcare professionals to make a
decision. These data can be in many formats such as X-ray images, heartbeat
sounds and temperature readings. This gives an idea for a population of a district or
a city, the size of data, a system needs to process, and physicians are required to
handle.
Research in biosciences has taken up a big problem for understanding biological
phenomena and finding solution to disorders that at times set in. The research in
system biology is poised to process huge data being generated from coding
information on genes of their structure and behaviour. Researchers across the globe
need access to each other s data as soon as such data are available. As in other cases
these data are available in many forms. And for applications like study on new virus
and its spread require fast processing of such data. Further, visualisation of folds
that happen to proteins is of importance to biologists as they understand nature has
preserved gold mine of information on life at such folds.
Likewise many applications now need to store and process data in time. In year
2000, volume of data stored in the world is of size 800,000 petabytes. It is expected
to reach 35 zettabytes by the year 2020. These figures on data are taken from book
[1]. However, the forecast will change with growing use of digi tal devices. We are
storing data of several domains ranging from agriculture, environment, house
holdings, governance, health, security, finance, meteorological and many more like.
Just storing such data is of no use unless data are processed and decisions are made
on the basis of such data. But in reality making use of such large data is a challenge
for its typical characteristics [2]. More, the issues are with data capture, data
storage, data analysis and data visualisation.
Big data looks for techniques not only for storage but also to extract information
hidden within. This becom es difficult for the very characteristics of big data. The
typical characteristics that hold it different than traditional database systems include
volume, variety, velocity and value. The term volume is misnomer for its vagueness
in quantifying the size that is fit to label as big data. Data that is not only huge but
expanding and holding patterns to show the order exist in data, is generally qual-
ifying volume of big data. Variety of big data is due to its sources of data generation
that include sensors, smartphones or social networks. The types of data emanate
from these sources include video, image, text, audio, and data logs, in either
structured or unstructured format [3]. Historical database dealing with data of past
has been studied earlier, but big data now considers data emerging ahead along the
timeline and the emergence is rapid so Velocity of data generation is of prime
concern. For example, in every second large amount of data are being generated by
social networks over internet. So in addition to volume, velocity is also a dimension
for such data [4]. Value of big data refers to the process of extracting hidden
Big Data: An Introduction 3
![](https://csdnimg.cn/release/download_crawler_static/11060204/bg12.jpg)
information from emerging data. A survey on generation of big data from mobile
applications is presented in [5].
Classification of big data from different perspectives as presented in [6 ] is pre-
sented in Fig. 1. The perspectives considered are data sources, content format, data
stores, data staging, and data processing. The sources generating data could be web
and social media on it, different sensors reading values of parameters that changes as
time passes on, internet of things, various machinery that throw data on changing
subfloor situations and transactions that are carried out in various domains such as
enterprises and organisations for governance and commercial purposes. Data staging
is about preprocessing of data that is required for processing for information
extraction. From data store perspective, here the concern is about the way data stored
for fast access. Data processing presents a systemic approach required to process big
data. We will again touch upon these two issues later in Sect. 3.
Having an introduction on big data, next we will go for usages of these data in
different domains. That gives an idea why the study on big data is important for
both business as well as academic communities.
2 Big Data as a Service
In modern days, business has been empowered by data management. In 1970s,
RDBMS (Relational Database Management System) has been successful in han-
dling large volume of data for query and repository management. The next level of
Fig. 1 Big data classification
4 H. Mohanty
![](https://csdnimg.cn/release/download_crawler_static/11060204/bg13.jpg)
data usage has been since 1990s, by making use of statistical as well as data mining
techniques. This has given rise to the first generation of Business Intelligence and
Analytics (BI&A). Major IT vendors including Microsoft, IBM, Oracle, and SAP
have developed BI platforms incorporating most of these data processing and
analytical technologies.
On advent of Web technology, organisations are putting businesses online by
making use of e-commerce platforms such as Flipkart, Amazon, eBay and are
searched for by websearch engines like Google. The technologies have enabled
direct interactions among customers and business houses. User(IP)-specific infor-
mation and interaction details being collected by web technologies (through cookies
and service logs) are being used in understanding customer’s needs and new
business opportunities. Web intelligence and web analytics make Web 2.0-based
social and crowd-sourcing systems.
Now social media analytics provi de unique opportunity for business develop-
ment. Interactions among people on social media can be traced and business
intelligence model be built for two-way business transactions directly instead of
traditional one-way transaction between business-to-customer [7]. We are need of
scalable techniques in information mining (e.g. information extraction, topic
identification, opinion mining, question-answering, event detection), web mining,
social network analysis, and spatial-temporal analysis, and these need to gel well
with existing DBMS-based techniques to come up with BI&A 2.0 systems. These
systems use a variety of data emanating from different sources in different varieties
and at different intervals. Such a collection of data is known as big data. Data in
abundance being accompanied with analytics can leverage opportunities and make
high impacts in many domain-specific applications [8]. Some such selective
domains include e-governance, e-commerce, healthcare, education, security and
many such applications that require boons of data science.
Data collected from interactions on social media can be analysed to understand
social dynamics that can help in delivering governance services to people at right
time and at right way resulting to good governance. Technology-assisted gover-
nance aims to use data services by deploying data analytics for social data analysis,
visualisation, finding events in communities, extracting as well as forecasting
emerging social changes and increase understanding of human and social processes
to promote economic growth and improved health and quality of life.
E-commerce has been greatly benefited in making use of data collected from
social media analytics for customer opinions, text analysis and sentiment analysis
techniques. Personalised recommender syste ms are now a possibility following
long-tail marketing by making use of data on social relations and choices they make
[9]. Various data processing analytics based on association rule mining, database
segmentation and clustering, anomaly detection, and graph mining techniques are
being used and developed to promote data as a service in e-commerce applications.
In healthcare domain, big data is poised to make a big impact resulting to per-
sonalisation of healthcare [10]. For this objective, healthcare systems are planning to
make use of different data the domain churns out every day in huge quantity. Two
main sources that generate a lot of data include genomic-driven study, probe-driven
Big Data: An Introduction 5
![](https://csdnimg.cn/release/download_crawler_static/11060204/bg14.jpg)
treatment and health management. Genomic-driven big data includes genotyping,
gene expression and sequenc ing data, whereas probe-driven health care includes
health-probing images, health-parameter readings and prescriptions. Health-
management data include electronic health records and insurance records. The
health big data can be used for hypothesis testing, knowledge discovery as well as
innovation. The healthcare management can have a positive impact due to healthcare
big data. A recent article [11] discusses on big data impact on host trait prediction
using meta-genomic data for gastrointestinal diseases.
Security has been a prime concern and it grows more, the more our society opens
up. Security threats emanating across boundary and even from within boundary are
required to be analysed and understood [12]. And the volume of such information
flowing from different agencies such as intelligence, security and public safety
agencies is enormous. A significant challenge in security IT research is the infor-
mation stovepipe and overload resulting from diverse data sources, multiple data
formats and large data volumes. Study on big data is expected to contribute to
success in mitigating security threats. Big data technology including such as
criminal association rule mining and clustering, criminal network analysis,
spatial-temporal analysis and visualisation, multilingual text analytics, sentiment
and affect analysis, and cyber attacks analysis and attribution should be considered
for security informatics research.
Scientific study has been increasingly collaborat ive. Particularly, sharing of sci-
entific data for research and engineering data for manufacturing has been a modern
trend, thanks to internet providing a pervading infrastructure for doing so [13]. Big
data aims to advance the core scientific and technological research by analysing,
visualising, and extracting useful information from large, diverse, distributed and
heterogeneous data sets. The research community believes this will accelerate the
progress of scientific discovery and innovation leading to new fields of enquiry that
would not otherwise be possible. Particularly, curren tly we see this happening in
fields of research in biology, physics, earth science, environmental science and many
more areas needing collaborative research of interdisciplinary nature.
The power of big data, i.e. its impact in different domains, is drawn from
analytics that extracts information from collected data and provide services to
intended users. In order to emphasise on vast scope of impending data services, let
us discover some analytics of importance. Data Analytics are designed to explore
and leverage unique data characteristics, from sequential/temporal mining and
spatial mining, to data mining for high-speed data streams and sensor data.
Analytics are formulated based on strong mathematical techniques including sta-
tistical machine learning, Bayesian networks, hidden Markov models, support
vector machine, reinforcement learning and ensemble models. Data analytics are
also looking into process mining from series of data collected in sequence of time.
Privacy security concern ed data analytics ensure anonymity as well as confidenti-
ality of a data service. Text Analytics aims at event detection, trend following,
sentiment analysis, topic modelling, question-answering and opinion mining. Other
than traditional soft computing and statistical techniques, text analytics take the
help of several well-researched natural language processing techniques in parsing
6 H. Mohanty
剩余194页未读,继续阅读
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://profile-avatar.csdnimg.cn/9c8dac576b1246488ac8b5ac867b9ec0_workerstar6.jpg!1)
workerstar6
- 粉丝: 0
- 资源: 2
上传资源 快速赚钱
我的内容管理 展开
我的资源 快来上传第一个资源
我的收益
登录查看自己的收益我的积分 登录查看自己的积分
我的C币 登录后查看C币余额
我的收藏
我的下载
下载帮助
![](https://csdnimg.cn/release/wenkucmsfe/public/img/voice.245cc511.png)
最新资源
- 新型矿用本安直流稳压电源设计:双重保护电路
- 煤矿掘进工作面安全因素研究:结构方程模型
- 利用同位素位移探测原子内部新型力
- 钻锚机钻臂动力学仿真分析与优化
- 钻孔成像技术在巷道松动圈检测与支护设计中的应用
- 极化与非极化ep碰撞中J/ψ的Sivers与cos2φ效应:理论分析与COMPASS验证
- 新疆矿区1200m深孔钻探关键技术与实践
- 建筑行业事故预防:综合动态事故致因理论的应用
- 北斗卫星监测系统在电网塔形实时监控中的应用
- 煤层气羽状水平井数值模拟:交替隐式算法的应用
- 开放字符串T对偶与双空间坐标变换
- 煤矿瓦斯抽采半径测定新方法——瓦斯储量法
- 大倾角大采高工作面设备稳定与安全控制关键技术
- 超标违规背景下的热波动影响分析
- 中国煤矿选煤设计进展与挑战:历史、现状与未来发展
- 反演技术与RBF神经网络在移动机器人控制中的应用
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
![](https://img-home.csdnimg.cn/images/20220527035711.png)
![](https://img-home.csdnimg.cn/images/20220527035711.png)
![](https://img-home.csdnimg.cn/images/20220527035111.png)
安全验证
文档复制为VIP权益,开通VIP直接复制
![](https://csdnimg.cn/release/wenkucmsfe/public/img/green-success.6a4acb44.png)