企业全文检索利器：Solr实战

需积分: 10 191 浏览量更新于2024-07-23 收藏 8.35MB PDF 举报

"Solr in Action" 本书《Solr in Action》深入介绍了Apache Solr，一个广泛使用的全文搜索引擎，尤其适合企业级的检索需求。Solr提供了高效、可扩展且功能丰富的搜索解决方案，支持多种特性，如多语言搜索、复杂数据操作、相关性调整等。在第一部分"MEET SOLR"中，作者会介绍Solr的基本概念和功能，帮助读者理解什么是Solr以及它如何在企业环境中发挥作用。第1章"Introduction to Solr"将概述Solr的历史、架构和主要特性。第2章"Getting to know Solr"会引导读者安装和配置Solr环境。第3章"Key Solr concepts"将深入讨论Solr的核心概念，如索引、查询和处理结果。第4章"Configuring Solr"将探讨如何定制Solr以满足特定需求，而第5章"Indexing"则专注于数据的索引过程。第6章"Text analysis"讲解了文本分析在Solr中的重要性，包括分词和词干提取。第二部分"CORE SOLR CAPABILITIES"详细阐述了Solr的核心搜索功能。第7章"Performing queries & handling results"涵盖查询语法和结果处理。第8章"Faceted search"介绍了如何实现分面搜索以增强用户体验。第9章"Hits highlighting"讲解了高亮显示搜索结果的相关技术。第10章"Search suggestions"涉及自动补全和建议功能。第11章"Result Grouping/Field Collapsing"讲述了如何对结果进行分组或折叠，以改善搜索结果的组织。第12章"Taking Solr to production"提供了将Solr部署到生产环境的最佳实践。第三部分"TAKING SOLR TO THE NEXT LEVEL"探讨了更高级的主题。第13章"Scaling Solr/SolrCloud"讲述了如何扩展Solr，包括SolrCloud分布式搜索的实现。第14章"Multi-lingual Search"介绍了多语言搜索的支持和挑战。第15章"Complex data operations"展示了处理不同类型和结构数据的方法。第16章"Relevancy tuning"涵盖了调整相关性评分以优化搜索结果的技巧。第17章"Thinking outside the box"鼓励读者探索Solr的创新用法，挖掘其潜在功能。附录部分包括了如何从源代码构建Solr（附录A）以及如何参与和利用Solr社区资源（附录B），对于开发者和维护者来说非常有价值。《Solr in Action》是一本全面的指南，旨在帮助读者理解和掌握Solr的各个方面，从基础到高级，从理论到实践，以提升全文检索系统的性能和用户体验。

©Manning Publications Co. Please post comments or corrections to the Author Online forum:

http://www.manning-sandbox.com/forum.jspa?forumID=828

Admittedly, figure 1.3 is a little overwhelming at first glance. Take a moment to scan over

the diagram to get a feel for some of the terminology. Don’t worry if you’re not familiar with

all of the terms and concepts represented in the diagram--after reading this book, you

should have a strong understanding of the all concepts presented in figure 1.3.

As we mentioned in the introduction to this chapter, the Solr designers recognized that

Solr fits better as a complementary technology that works within existing architectures. In

fact, you’ll be hard-pressed to find an environment where Solr doesn't drop right in. As we'll

see in chapter 2, you can start the example Solr server in a couple of minutes after you

finish the download.

To achieve the goal of easy integration, Solr’s core services need to be accessible from

many different applications and languages. Solr provides simple REST-like services based on

proven standards of XML, JSON, and HTTP. As a brief aside, we avoid the “RESTful” label for

Solr’s HTTP-based API as it doesn’t strictly adhere to all REST (representational state

transfer) principles. For instance, in Solr you use HTTP POST to delete documents instead of

HTTP DELETE.

A REST-like interface is nice as a foundation, but often times, developers like to have

access to a client library in their language of choice to abstract away some of the boilerplate

machinery of invoking a web service and processing the response. The good news here is

that most popular languages have a Solr client library including Python, PHP, Java, .NET, and

Ruby.

1.2.4 Multiple indexes in one server

One hallmark of modern application architectures is the need for flexibility in the face of

rapidly changing requirements. One of the ways Solr helps this situation is that you don’t

have to do all things in Solr with one index, because Solr supports running multiple cores in

a single engine. In figure 1.3, we’ve depicted multiple cores as separate layers all running in

the same Java web application environment.

Think of each core as a separate index and configuration and there can be many cores in

a single Solr instance. This allows you to manage multiple cores from one server so that you

can share server resources and administration tasks like monitoring and maintenance. Solr

provides an API for creating and managing multiple cores.

One use of Solr’s multicore support is data partitioning, such as having one core for

recent documents and another core for older documents, known as chronological sharding.

Another use of Solr’s multicore support is to support multitenant applications.

In our real estate application, we might use multiple cores to manage different types of

listings that are different enough to justify having different indexes for each. Consider real

estate listings for rural land instead of homes. Buying rural land is a different process than

buying a home in a city, so it stands to reason that we might want to manage our land

listings in a separate core.

Licensed to Jiwen Zhang <kevinzhang29@gmail.com>

©Manning Publications Co. Please post comments or corrections to the Author Online forum:

http://www.manning-sandbox.com/forum.jspa?forumID=828

1.2.5 Extendable (plug-ins)

Figure 1.3 depicts three main sub-systems in Solr: document management, query

processing, and text analysis. Of course, these are high-level abstractions for complex sub-

systems in Solr; we’ll learn about each later in the book. Each of these systems is composed

of a modular “pipeline” that allows you to plug-in new functionality. This means that instead

of overriding the entire query-processing engine in Solr, you plug-in a new search

component into an existing pipeline. This makes the core Solr functionality easy to extend

and customize to meet your specific application needs.

1.2.6 Scalable

Lucene is an extremely fast search library and Solr takes full advantage of Lucene’s speed.

But regardless of how fast Lucene is, a single server will reach its limits in terms of how

many concurrent queries from different users it can handle due to CPU and I/O constraints.

As a first pass to scalability, Solr provides flexible cache management features that help

your server avoid recomputing expensive operations. Specifically, Solr comes preconfigured

with a number of caches to save expensive recomputations, such as caching the results of a

query filter. We’ll learn about Solr’s cache management features in chapter 4.

Caching only gets you so far so at some point, you’re going to need to scale out your

capacity to handle more documents and higher query throughput by adding more servers.

For now let’s focus on the two most common dimensions of scalability in Solr. First is query

throughput, which is the number of queries your engine can support per second. Even

though Lucene can execute each query quickly, it’s limited in terms of how many concurrent

requests a single server can handle. For higher query throughput, you add replicas of your

index so that more servers can handle more requests. This means if your index is replicated

across three servers, then you can handle roughly three times the number of queries per

second because each server handles one-third of the query traffic. In practice, it’s rare to

achieve perfect linear scalability so adding three servers may only allow you to handle two

and one half times the query volume of one server.

The other dimension of scalability is the number of documents indexed. If you’re dealing

with large volumes of documents, then you’ll likely reach a point where you have too many

documents in a single instance and query performance will suffer. To handle more

documents, you split the index into smaller chunks called “shards” and then distribute the

searches across the shards.

Scaling-out with virtualized commodity hardware

One trend in modern computing is building software architectures that can scale

horizontally using virtualized commodity hardware. Put simply, add more commodity

servers to handle more traffic. Fueling this trend towards using virtualized commodity

hardware are cloud-computing providers such as Amazon EC2. Although Solr will run on

virtualized hardware, you should be aware that search is I/O and memory intensive.

Licensed to Jiwen Zhang <kevinzhang29@gmail.com>

©Manning Publications Co. Please post comments or corrections to the Author Online forum:

http://www.manning-sandbox.com/forum.jspa?forumID=828

Therefore, if search performance is a top priority for your organization, then you should

consider deploying Solr on higher end hardware with high-performance disks, ideally

solid-state drives (SSD). Hardware considerations for deploying Solr are discussed in

chapter 13.

Scalability is important, but ability to survive failures is also important for a modern

system. In the next section, we discuss how Solr handles software and hardware failures.

1.2.7 Fault tolerant

Beyond scalability, you need to consider what happens if one or more of your servers fails,

particularly if you’re planning to deploy Solr on virtualized hardware or commodity hardware.

The bottom line is that you must plan for failures. Even the best architectures and high-end

hardware will experience failures.

Let’s assume you have four shards for your index and the server hosting shard #2 loses

power. At this point, Solr can’t continue indexing documents and can’t service queries, so

your search engine is effectively “down.” To avoid this situation, you can add replicas of each

shard. In this case, when shard #2 fails, Solr reroutes indexing and query traffic to the

replica and your Solr cluster remains online. The impact of this failure is that indexing and

queries can still be processed but may not be as fast because you’ve one less server to

handle requests. We'll discuss failover scenarios in chapter 16.

At this point, you’ve seen that Solr has a modern, well-designed architecture that’s

scalable and fault-tolerant. Although these are important aspects to consider if you’ve

already decided to use Solr, you still might not be convinced that Solr is the right choice for

your needs. In the next section, we describe the benefits of Solr from the perspective of

different stakeholders, such as the software architect, system administrator, and CEO.

1.3 Why Solr?

In this section, we hope to provide you with some key information to help you decide if Solr

is the right technology for your organization. Let’s begin by addressing why Solr is attractive

to software architects.

1.3.1 Solr for the software architect

When evaluating new technology, software architects must consider a number of factors, but

chief among those are stability, scalability, and fault-tolerance. Solr scores high marks in all

three categories.

In terms of stability, Solr is a mature technology supported by a vibrant community and

seasoned committers. One thing that shocks new users to Solr and Lucene is that it isn’t

unheard of to deploy from source code pulled directly from the trunk rather than waiting for

an official release. We won’t advise you either way on whether this is acceptable for your

organization. We only point this out because it’s a testament to the depth and breadth of

Licensed to Jiwen Zhang <kevinzhang29@gmail.com>

http://www.manning-sandbox.com/forum.jspa?forumID=828

automated testing in Lucene and Solr. Put simply, if you have a nightly build off trunk where

all the automated tests pass, then you can be sure the core functionality is solid.

We’ve touched on Solr’s approach to scalability and fault-tolerance in sections 1.2.6 and

1.2.7. As an architect, you’re probably most curious about the limitations of Solr’s approach

to scalability and fault-tolerance. First, you should realize that the sharding and replication

features in Solr have been rewritten in Solr 4 to be robust and easier to manage. The new

approach to scaling is called SolrCloud. Under the covers, SolrCloud uses Apache Zookeeper

to distribute configuration across a cluster of Solr nodes and to keep track of cluster state.

Here are some highlights of the new SolrCloud features in Solr:

 Centralized configuration

 Distributed indexing with no Single Point of Failure (SPoF)

 Automated fail-over to a new shard leader

 Queries can be sent to any node in a cluster to trigger a full distributed search across

all shards with fail-over and load-balancing support built-in

But this isn’t to say that Solr scaling doesn’t have room for improvement. SolrCloud is

lacking in two areas. First, not all features work in distributed mode, such as joins. Second,

the number of shards for an index is a fixed value that can’t be changed without reindexing

all of the documents. We’ll get into all of the specifics of SolrCloud in chapter 16, but we

want to make sure architects are aware that Solr scaling has come a long way in the past

couple of years yet still has some more work to do.

1.3.2 Solr for the system administrator

As a system administrator, high among your priorities in adopting a new technology like Solr

is whether it fits into your existing infrastructure. The easy answer is yes it does. As Solr is

Java based, it runs on any OS platform that has a J2SE 6.x/7.x JVM. Out of the box, Solr

embeds Jetty, the open source Java servlet engine provided by Oracle. Otherwise, Solr is a

standard Java web application that deploys easily to any Java web application server like

JBoss and Oracle AS.

All access to Solr can be done via HTTP and Solr is designed to work with caching HTTP

reverse proxies like Squid and Varnish. Solr also works with JMX so you can hook it up to

your favorite monitoring application, such as Nagios.

Lastly, Solr provides a nice administration console for checking configuration settings,

statistics, issuing test queries, and monitoring the health of SolrCloud. Figure 1.4 provides a

screen shot of the Solr 4 administration console. We’ll learn more about the administration

console in chapter 2.

Licensed to Jiwen Zhang <kevinzhang29@gmail.com>

剩余262页未读，继续阅读

xp_penggang

粉丝: 0

企业全文检索利器：Solr实战

solr in action

Solr In Action 原版

Solr In Action 2013

Solr in Action_2014

Solr in action Source Code

solr in action 完整版

Solr in action.mobi

Solr in Action中文版

Solr实战-solr in action中文版高清

Mining Solr In Action源代码

最新资源