IBM InfoSphere BigInsights 2.1 安装指南

需积分: 10 63 浏览量更新于2024-07-24 收藏 1MB PDF 举报

"IBM InfoSphere BigInsights Version 2.1 安装指南" IBM InfoSphere BigInsights 是一个强大的大数据分析平台，专为处理大规模数据集而设计。Version 2.1 的安装指南是用户成功部署和配置该产品的关键参考资料。本指南涵盖了各种重要知识点，包括产品特性、架构、主要组件以及规划安装的考虑因素。 1. **InfoSphere BigInsights 特性与架构** - IBM InfoSphere BigInsights 提供了一个全面的大数据分析环境，支持多种数据处理任务，如批处理、实时分析和交互式查询。 - 其架构基于开源的 Hadoop 平台，旨在提供企业级的稳定性和安全性。 2. **Hadoop 分布式文件系统 (HDFS)** - HDFS 是 Hadoop 的核心组成部分，它提供了高容错性和可扩展的数据存储。在 BigInsights 中，HDFS 支持大数据的分布式存储，允许高效地读写大量数据。 3. **IBM General Parallel File System (GPFS)** - GPFS 是 IBM 的并行文件系统，可以集成到 BigInsights 中，提供高性能的数据访问和共享，尤其适用于大数据密集型应用。 4. **Adaptive MapReduce** - 这是 BigInsights 的一项优化技术，可以根据工作负载动态调整 MapReduce 执行策略，以提高处理效率和资源利用率。 5. **Hadoop MapReduce** - MapReduce 是 Hadoop 的编程模型，用于处理和生成大数据集。BigInsights 支持 MapReduce，允许用户编写自定义的 Map 和 Reduce 函数来处理数据。 6. **其他 Hadoop 技术** - BigInsights 集成了其他 Hadoop 相关技术，如 YARN（资源调度器）和 HBase（NoSQL 数据库），以增强其功能和性能。 7. **文本分析** - 提供了文本分析工具，用于从非结构化文本数据中提取有价值的信息，例如情感分析和主题建模。 8. **IBM BigSQL** - BigSQL 是 BigInsights 中的一个组件，提供了一个 SQL 接口，让用户能够以传统 SQL 方式查询 Hadoop 上的数据，简化大数据分析。 9. **InfoSphere BigInsights 控制台** - 提供了一个直观的管理界面，用于监控集群状态、管理用户和资源，以及执行作业。 10. **InfoSphere BigInsights Tools for Eclipse** - 这是一个开发工具集，允许开发者在 Eclipse 环境中创建、调试和管理 BigInsights 应用程序。 11. **与其他 IBM 产品的集成** - BigInsights 可以与 IBM 的其他大数据和分析产品（如 InfoSphere DataStage、InfoSphere Warehouse 等）无缝集成，形成一个完整的企业数据解决方案。 12. **规划安装** - 在安装前，需要考虑硬件需求、网络配置、安全设置以及集群规模等关键因素，以确保顺利部署和高效运行。安装指南详细阐述了这些知识点，并提供了步骤指导和最佳实践，帮助用户成功安装和配置 IBM InfoSphere BigInsights Version 2.1，从而充分利用大数据的潜力。

The goal of Text Analysis is to read unstructured text and distill insights. This

process can involve searching for customer web browsing patterns in clickstream

log files, finding fraud indicators through email analytics, or assessing customer

sentiment from social media messages. To help facilitate text analysis, InfoSphere

BigInsights includes developer tools, a programming language, a processing

engine, and prebuilt text extractors. Built-in multilingual support for tokenization

and semantic analysis enables InfoSphere BigInsights to understand multiple

languages and different parts of speech.

A text analysis application can read a paragraph of text and derive structured

information based on various rules. These rules are defined in extractors, which are

programs that extract information from within a text field. The product of

extractors is a set of annotated text that includes specific information that is

important to your business. Extractors are written in the annotation query

language (AQL), which is the core of Text Analytics in InfoSphere BigInsights.

AQL is a fully declarative Text Analytics language, which means that any module

can be customized. This implementation results in a flexible Text Analytics

language that is both highly expressive and fast. InfoSphere BigInsights includes

an Eclipse-based development environment for creating and maintaining extractors

written in AQL. By supporting modular AQL, InfoSphere BigInsights improves the

usability and development productivity of Text Analytics applications by enabling

and reusing common extractors, dictionaries, rules, and tables.

An optimizer is also included that generates an efficient runtime plan for the

extractor specification. The runtime component receives a collection of documents

and runs the extractor on each document to extract text entities. After extraction,

you can test extractors against a subset of data to refine the precision and recall of

the extractors. To begin building your own extractors, you can use the rich library

of precompiled extractors that are included with InfoSphere BigInsights.

After you develop text analysis applications, you can deploy, run, and monitor

these applications from the InfoSphere BigInsights console. You can link text

analysis applications to other applications in the console, including integration

with BigSheets. By linking applications, you can create new applications that

include multiple analytic components.

IBM Big SQL

IBM Big SQL is a data warehouse system for Hadoop that you use to summarize,

query, and analyze data. Big SQL provides SQL access to data that is stored in

InfoSphere BigInsights by using JDBC or ODBC connections.

Big SQL provides broad SQL support that is typical of commercial databases. You

can issue JDBC queries or ODBC queries to access data that is stored in InfoSphere

BigInsights, in the same way that you access databases from your enterprise

applications.

You can use Big SQL to issue queries of any size, including concurrent queries. Big

SQL provides support for large ad-hoc queries by using MapReduce parallelism

and point queries, which are low-latency queries that return information quickly to

reduce response time and provide improved access to data.

The Big SQL server is multi-threaded, so scalability is limited only by the

performance and number of cores in the computer that runs the server. If you

10 IBM InfoSphere BigInsights Version 2.1: Installation Guide

want to issue larger queries, you can increase the hardware performance of the

server computer that Big SQL runs on, or chain multiple Big SQL servers together

to increase throughput.

InfoSphere BigInsights Console

InfoSphere BigInsights includes an integrated console that you can use to view the

health of your cluster, deploy applications, manage cluster instances, work with

BigSheets, manage files, and schedule workflows, jobs, and tasks from a single

location.

The console includes quick access to tasks for system administration, application

deployment, data analysis, and file management. You can navigate between the

tasks by using the navigation tabs and access common tasks from the Welcome

panel to begin working with InfoSphere BigInsights.

You can create dashboards to monitor the status of multiple applications services,

data services, and attributes of your cluster on a single page. In addition, you can

monitor the overall status of your cluster, including the status of individual

Hadoop components.

You can deploy and run applications from the console to make data available for

analysis in BigSheets. BigSheets is a browser-based visualization tool that you can

use to extend the scope of your business intelligence data. By using BigSheets, you

can easily integrate massive amounts of unstructured data into consumable

business contexts for specific situations.

InfoSphere BigInsights Tools for Eclipse

The InfoSphere BigInsights Tools for Eclipse add functionality to your Eclipse

environment so that you can develop programs to run with InfoSphere BigInsights.

The Welcome page of the InfoSphere BigInsights console includes information

about how to enable your Eclipse environment for developing applications by

using InfoSphere BigInsights.

You can develop and test InfoSphere BigInsights programs from your Eclipse

environment and publish applications that contain workflows, text analytics

modules, BigSheets readers and functions, and Jaql modules to your cluster. After

you deploy applications to your cluster, you can run them from the InfoSphere

BigInsights console.

You can use the InfoSphere BigInsights Tools for Eclipse to develop programs that

interact with Hadoop and the Hadoop Distributed File System (HDFS) and the

General Parallel File System (GPFS):

v Create text analytics modules that contain text extractors by using an extraction

task wizard and editor. You can then test the extractor by running it locally

against sample data. Visualize the results of the text extraction and improve the

quality of your extractor by analyzing how results were obtained.

v Create Jaql scripts or modules by using a wizard, and edit scripts with an editor

that provides content assistance and syntax highlighting. Run Jaql explain

statements in your scripts, and run the scripts locally or against the InfoSphere

BigInsights server. You can open the Jaql shell from within Eclipse to run Jaql

statements against the cluster.

v Create Pig scripts by using a wizard and edit the scripts with an editor that

provides content assistance and syntax highlighting. Run Pig explain statements

Chapter 1. Introduction to InfoSphere BigInsights 11

and illustrate statements for aliases in your scripts, and then run the Pig

scripts locally or against the InfoSphere BigInsights server. You can open the Pig

shell from within Eclipse to run Pig statements against the cluster.

v Connect to the Hive server by using the Hive JDBC driver and run Hive SQL

scripts and explore the results. Browse the navigation tree to explore the

structure and content of the tables in the Hive server.

v Use the Java editor to write programs that use MapReduce, and then run these

programs locally or against the InfoSphere BigInsights server. Open the

InfoSphere BigInsights console to monitor jobs that are created by MapReduce.

v Create templates for BigSheets readers or functions and then use the Java editor

to implement the classes.

v Write Java programs that use the HBase APIs and run them against the

InfoSphere BigInsights server. Open the HBase shell from your Eclipse

environment to run HBase statements against the cluster.

Integration with other IBM products

InfoSphere BigInsights compliments and extends existing business capabilities by

integrating with other IBM products. These integration points extend existing

technologies to encompass more comprehensive information types, enabling a

complete view of your business.

IBM Cognos

Business Intelligence

By using the InfoSphere BigInsights Big SQL JDBC driver or the Hive

JDBC driver, Cognos Business Intelligence can incorporate data from

InfoSphere BigInsights into business intelligence analysis, reports, and

statistics. You can use a business intelligence modeler to create Cognos

reports, dashboards, and workspaces while using the InfoSphere

BigInsights MapReduce capabilities.

IBM DB2

You use the IBM DB2 for Linux, UNIX, and Windows user-defined

functions to query InfoSphere BigInsights data and run jobs from DB2 for

Linux, UNIX, and Windows. InfoSphere BigInsights and IBM Netezza have

optimized JDBC connections that you can access by using Jaql database

modules. From InfoSphere BigInsights, you use a Jaql server or Jaql

module on the cluster to run SQL with DB2 for Linux, UNIX, and

Windows in parallel. You can also use a generic Jaql module to connect

with third-party databases such as Derby, Informix, Oracle, and Teradata.

You can use the Big SQL LOAD FROM statement to import data from a

DB2 for Linux, UNIX, and Windows table by using a JDBC driver into a

Big SQL table.

IBM InfoSphere Data Explorer

InfoSphere Data Explorer enables navigation of data from existing

enterprise information systems and the advanced analytics that are

provided by InfoSphere BigInsights components. InfoSphere Data Explorer

Engine servers can receive data in real time from a cluster of InfoSphere

BigInsights servers or InfoSphere Streams servers. InfoSphere Data

Explorer can also push relevant data to users of information applications

that are created by using the InfoSphere Data Explorer Application Builder.

InfoSphere Data Explorer also enables federated access to other IBM

products, such as IBM Netezza, and provides a flexible mechanism for

making data available for exploration and analysis.

IBM InfoSphere DataStage

The connector to InfoSphere DataStage (shipped with that product) enables

12 IBM InfoSphere BigInsights Version 2.1: Installation Guide

InfoSphere BigInsights to quickly exchange data with any software product

that can connect with InfoSphere DataStage. InfoSphere DataStage includes

a stage that uses InfoSphere BigInsights data within a job, enabling

powerful and flexible extract-transform-load (ETL) scenarios.

IBM InfoSphere Guardium

Integration with InfoSphere Guardium provides InfoSphere BigInsights

with enterprise-level security auditing capabilities that support many

current compliance mandates. This integration provides insights into which

users are running specific InfoSphere BigInsights requests, which

MapReduce jobs each user is running, and whether file permission

exceptions are being caused by restricted access to sensitive data.

InfoSphere Streams

The adapter for InfoSphere Streams enables streaming data to be stored

directly in your InfoSphere BigInsights cluster, and for InfoSphere Streams

applications to read data from the cluster. You can use this integration to

create an infrastructure that responds to events in real time and includes

existing data to inform the response. You can also use InfoSphere Streams

as a large-scale data engine to filter a stream of data that you want to store

in the cluster.

IBM Netezza

InfoSphere BigInsights includes a connector that enables bidirectional data

exchange between an InfoSphere BigInsights cluster and an IBM Netezza

appliance. The adapter is implemented as a Jaql module, which helps you

to use the simplicity and flexibility of Jaql in your database interactions.

You use this adapter to read from or write to DB2 for Linux, UNIX, and

Windows tables and IBM Netezza tables. You can use the Big SQL LOAD

FROM statement to import data from an IBM Netezza table by using a

JDBC driver into a Big SQL table.

Rational and Data Studio

InfoSphere BigInsights integrates with IBM Rational Application Developer

Version 8.5, Rational Team Concert, and Data Studio by installing the

InfoSphere BigInsights Tools for Eclipse into your existing Rational Data

Studio, Rational Team Concert, or Data Studio environment. This

integration provides a more collaborative application development for big

data.

Chapter 1. Introduction to InfoSphere BigInsights 13

剩余101页未读，继续阅读

飞鸿踏雪Ben

粉丝: 0
资源: 1

IBM InfoSphere BigInsights 2.1 安装指南

IBM InfoSphere BigInsights Version 3.0.0安装手册

IBM 大数据训练营资料

解读IBM InfoSphere大数据分析平台

IBM infosphere datestage and qualitystage 与 DB2的区别

ibm inforsphere cdc收费吗?

InfoSphere CDC 下载地址

apache atlas有哪些替代品？

apache atlas 有哪些替代品？

如何搭建 InfoSphere CDC

数据质量管理工具有哪些

最新资源