Overview of Big Data and Hive Chapter 1
[ 7 ]
Once plenty of data was collected, people started to think about how to deal with the
historical data. Then, the term data warehousing came up in the 1990s. From that time
onward, people started discussing how to evaluate current performance by reviewing the
historical data. Various data models and tools were created to help enterprises effectively
manage, transform, and analyze their historical data. Traditional relational databases also
evolved to provide more advanced aggregation and analyzed functions as well as
optimizations for data warehousing. The leading query language was still SQL, but it was
more intuitive and powerful compared to the previous versions. The data was still well-
structured and the model was normalized. As we entered the 2000s, the internet gradually
became the topmost industry for the creation of the majority of data in terms of variety and
volume. Newer technologies, such as social media analytics, web mining, and data
visualizations, helped lots of businesses and companies process massive amounts of data
for a better understanding of their customers, products, competition, and markets. The data
volume grew and the data format changed faster than ever before, which forced people to
search for new solutions, especially in the research and open source areas. As a result, big
data became a hot topic and a challenging field for many researchers and companies.
However, in every challenge there lies great opportunity. In the 2010s, Hadoop, which was
one of the big data open source projects, started to gain wide attention due to its open
source license, active communities, and power to deal with the large volumes of data. This
was one of the few times that an open source project led to the changes in technology
trends before any commercial software products. Soon after, the NoSQL database, real-time
analytics, and machine learning, as followers, quickly became important components on
top of the Hadoop big data ecosystem. Armed with these big data technologies, companies
were able to review the past, evaluate the current, and grasp the future opportunities.
Introducing big data
Big Data is not simply a big volume of data. Here, the word Big refers to the big scope of
data. A well-known saying in this domain is to describe big data with the help of three
words starting with the letter V: volume, velocity, and variety. But the analytical and data
science world has seen data varying in other dimensions in addition to the fundament three
Vs of big data, such as veracity, variability, volatility, visualization, and value. The different
Vs mentioned so far are explained as follows:
Volume: This refers to the amount of data generated in seconds. 90% of the
world's data today has been created in the last two years. Since that time, the
data in the world doubles every two years. Such big volumes of data are mainly
generated by machines, networks, social media, and sensors, including
structured, semi-structured, and unstructured data.
我的内容管理
展开
