Solr实战：探索搜索引擎的核心能力

5星 · 超过95%的资源需积分: 10 97 浏览量更新于2024-07-23 收藏 8.35MB PDF 举报

"Solr in Action 英文版，由Manning Publications出版，作者在线论坛：http://www.manning-sandbox.com/forum.jspa?forumID=828" 《Solr in Action》是一本深入探讨Apache Solr搜索引擎技术的专业书籍。Solr是一款开源的企业级搜索平台，广泛用于构建高效、可扩展的全文检索应用。该书涵盖了从入门到高级的多个方面，旨在帮助读者全面了解并掌握Solr的使用。在Part 1“Meet Solr”部分，作者首先介绍了Solr的基本概念，包括其历史、核心功能以及在现代企业中的应用。第2章“Getting to know Solr”让读者熟悉Solr的架构和组件，了解如何快速搭建一个Solr实例。第3章“Key Solr concepts”深入讲解了Solr的核心概念，如索引、文档、字段和核心（core）。第4章“Configuring Solr”详细阐述了Solr的配置过程，包括配置文件解析和优化。第5章“Indexing”则讨论了数据导入和索引构建的策略。第6章“Text analysis”重点介绍了文本分析和分词，这是全文检索的关键环节。 Part 2“Core Solr capabilities”主要聚焦于Solr的核心搜索功能。第7章“Performing queries & handling results”详细解释了如何构建查询、处理结果集以及使用各种查询语法。第8章“Faceted search”讲解了如何实现分面导航，以便用户能更有效地探索大量数据。第9章“Hit highlighting”介绍了高亮显示匹配的搜索结果，以提升用户体验。第10章“Search suggestions”探讨了自动补全和建议功能的实现。第11章“Result grouping/Field collapsing”展示了如何对搜索结果进行分组和折叠，以帮助用户更好地理解和处理结果。第12章“Taking Solr to production”提供了将Solr部署到生产环境的指导，包括性能调优和监控。 Part 3“Taking Solr to the next level”进一步提升了Solr的使用技巧。第13章“Scaling Solr/SolrCloud”涵盖了Solr的扩展性，包括集群、分布式搜索和SolrCloud的使用。第14章“Multi-lingual Search”讲解了多语言支持，使Solr能够处理多种语言的搜索需求。第15章“Complex data operations”探讨了处理复杂数据类型和操作的方法。第16章“Relevancy tuning”专注于调整相关性，以优化搜索结果的排名。第17章“Thinking outside the box”鼓励读者创新，利用Solr解决非传统搜索问题。最后，附录A“Building Solr from source”介绍了如何从源代码构建Solr，而附录B“Working with the Solr community”则引导读者如何参与到Solr社区，获取帮助和贡献自己的力量。这本书对于希望深入了解和使用Solr的开发者、系统管理员和技术决策者来说，是一份宝贵的参考资料。通过学习，读者可以具备从设计、实现到维护一个高效的Solr搜索系统的全面能力。

©Manning Publications Co. Please post comments or corrections to the Author Online forum:

http://www.manning-sandbox.com/forum.jspa?forumID=828

Admittedly, figure 1.3 is a little overwhelming at first glance. Take a moment to scan over

the diagram to get a feel for some of the terminology. Don’t worry if you’re not familiar with

all of the terms and concepts represented in the diagram--after reading this book, you

should have a strong understanding of the all concepts presented in figure 1.3.

As we mentioned in the introduction to this chapter, the Solr designers recognized that

Solr fits better as a complementary technology that works within existing architectures. In

fact, you’ll be hard-pressed to find an environment where Solr doesn't drop right in. As we'll

see in chapter 2, you can start the example Solr server in a couple of minutes after you

finish the download.

To achieve the goal of easy integration, Solr’s core services need to be accessible from

many different applications and languages. Solr provides simple REST-like services based on

proven standards of XML, JSON, and HTTP. As a brief aside, we avoid the “RESTful” label for

Solr’s HTTP-based API as it doesn’t strictly adhere to all REST (representational state

transfer) principles. For instance, in Solr you use HTTP POST to delete documents instead of

HTTP DELETE.

A REST-like interface is nice as a foundation, but often times, developers like to have

access to a client library in their language of choice to abstract away some of the boilerplate

machinery of invoking a web service and processing the response. The good news here is

that most popular languages have a Solr client library including Python, PHP, Java, .NET, and

Ruby.

1.2.4 Multiple indexes in one server

One hallmark of modern application architectures is the need for flexibility in the face of

rapidly changing requirements. One of the ways Solr helps this situation is that you don’t

have to do all things in Solr with one index, because Solr supports running multiple cores in

a single engine. In figure 1.3, we’ve depicted multiple cores as separate layers all running in

the same Java web application environment.

Think of each core as a separate index and configuration and there can be many cores in

a single Solr instance. This allows you to manage multiple cores from one server so that you

can share server resources and administration tasks like monitoring and maintenance. Solr

provides an API for creating and managing multiple cores.

One use of Solr’s multicore support is data partitioning, such as having one core for

recent documents and another core for older documents, known as chronological sharding.

Another use of Solr’s multicore support is to support multitenant applications.

In our real estate application, we might use multiple cores to manage different types of

listings that are different enough to justify having different indexes for each. Consider real

estate listings for rural land instead of homes. Buying rural land is a different process than

buying a home in a city, so it stands to reason that we might want to manage our land

listings in a separate core.

Licensed to Jiwen Zhang <kevinzhang29@gmail.com>

©Manning Publications Co. Please post comments or corrections to the Author Online forum:

http://www.manning-sandbox.com/forum.jspa?forumID=828

1.2.5 Extendable (plug-ins)

Figure 1.3 depicts three main sub-systems in Solr: document management, query

processing, and text analysis. Of course, these are high-level abstractions for complex sub-

systems in Solr; we’ll learn about each later in the book. Each of these systems is composed

of a modular “pipeline” that allows you to plug-in new functionality. This means that instead

of overriding the entire query-processing engine in Solr, you plug-in a new search

component into an existing pipeline. This makes the core Solr functionality easy to extend

and customize to meet your specific application needs.

1.2.6 Scalable

Lucene is an extremely fast search library and Solr takes full advantage of Lucene’s speed.

But regardless of how fast Lucene is, a single server will reach its limits in terms of how

many concurrent queries from different users it can handle due to CPU and I/O constraints.

As a first pass to scalability, Solr provides flexible cache management features that help

your server avoid recomputing expensive operations. Specifically, Solr comes preconfigured

with a number of caches to save expensive recomputations, such as caching the results of a

query filter. We’ll learn about Solr’s cache management features in chapter 4.

Caching only gets you so far so at some point, you’re going to need to scale out your

capacity to handle more documents and higher query throughput by adding more servers.

For now let’s focus on the two most common dimensions of scalability in Solr. First is query

throughput, which is the number of queries your engine can support per second. Even

though Lucene can execute each query quickly, it’s limited in terms of how many concurrent

requests a single server can handle. For higher query throughput, you add replicas of your

index so that more servers can handle more requests. This means if your index is replicated

across three servers, then you can handle roughly three times the number of queries per

second because each server handles one-third of the query traffic. In practice, it’s rare to

achieve perfect linear scalability so adding three servers may only allow you to handle two

and one half times the query volume of one server.

The other dimension of scalability is the number of documents indexed. If you’re dealing

with large volumes of documents, then you’ll likely reach a point where you have too many

documents in a single instance and query performance will suffer. To handle more

documents, you split the index into smaller chunks called “shards” and then distribute the

searches across the shards.

Scaling-out with virtualized commodity hardware

One trend in modern computing is building software architectures that can scale

horizontally using virtualized commodity hardware. Put simply, add more commodity

servers to handle more traffic. Fueling this trend towards using virtualized commodity

hardware are cloud-computing providers such as Amazon EC2. Although Solr will run on

virtualized hardware, you should be aware that search is I/O and memory intensive.

Licensed to Jiwen Zhang <kevinzhang29@gmail.com>

©Manning Publications Co. Please post comments or corrections to the Author Online forum:

http://www.manning-sandbox.com/forum.jspa?forumID=828

Therefore, if search performance is a top priority for your organization, then you should

consider deploying Solr on higher end hardware with high-performance disks, ideally

solid-state drives (SSD). Hardware considerations for deploying Solr are discussed in

chapter 13.

Scalability is important, but ability to survive failures is also important for a modern

system. In the next section, we discuss how Solr handles software and hardware failures.

1.2.7 Fault tolerant

Beyond scalability, you need to consider what happens if one or more of your servers fails,

particularly if you’re planning to deploy Solr on virtualized hardware or commodity hardware.

The bottom line is that you must plan for failures. Even the best architectures and high-end

hardware will experience failures.

Let’s assume you have four shards for your index and the server hosting shard #2 loses

power. At this point, Solr can’t continue indexing documents and can’t service queries, so

your search engine is effectively “down.” To avoid this situation, you can add replicas of each

shard. In this case, when shard #2 fails, Solr reroutes indexing and query traffic to the

replica and your Solr cluster remains online. The impact of this failure is that indexing and

queries can still be processed but may not be as fast because you’ve one less server to

handle requests. We'll discuss failover scenarios in chapter 16.

At this point, you’ve seen that Solr has a modern, well-designed architecture that’s

scalable and fault-tolerant. Although these are important aspects to consider if you’ve

already decided to use Solr, you still might not be convinced that Solr is the right choice for

your needs. In the next section, we describe the benefits of Solr from the perspective of

different stakeholders, such as the software architect, system administrator, and CEO.

1.3 Why Solr?

In this section, we hope to provide you with some key information to help you decide if Solr

is the right technology for your organization. Let’s begin by addressing why Solr is attractive

to software architects.

1.3.1 Solr for the software architect

When evaluating new technology, software architects must consider a number of factors, but

chief among those are stability, scalability, and fault-tolerance. Solr scores high marks in all

three categories.

In terms of stability, Solr is a mature technology supported by a vibrant community and

seasoned committers. One thing that shocks new users to Solr and Lucene is that it isn’t

unheard of to deploy from source code pulled directly from the trunk rather than waiting for

an official release. We won’t advise you either way on whether this is acceptable for your

organization. We only point this out because it’s a testament to the depth and breadth of

Licensed to Jiwen Zhang <kevinzhang29@gmail.com>

http://www.manning-sandbox.com/forum.jspa?forumID=828

automated testing in Lucene and Solr. Put simply, if you have a nightly build off trunk where

all the automated tests pass, then you can be sure the core functionality is solid.

We’ve touched on Solr’s approach to scalability and fault-tolerance in sections 1.2.6 and

1.2.7. As an architect, you’re probably most curious about the limitations of Solr’s approach

to scalability and fault-tolerance. First, you should realize that the sharding and replication

features in Solr have been rewritten in Solr 4 to be robust and easier to manage. The new

approach to scaling is called SolrCloud. Under the covers, SolrCloud uses Apache Zookeeper

to distribute configuration across a cluster of Solr nodes and to keep track of cluster state.

Here are some highlights of the new SolrCloud features in Solr:

 Centralized configuration

 Distributed indexing with no Single Point of Failure (SPoF)

 Automated fail-over to a new shard leader

 Queries can be sent to any node in a cluster to trigger a full distributed search across

all shards with fail-over and load-balancing support built-in

But this isn’t to say that Solr scaling doesn’t have room for improvement. SolrCloud is

lacking in two areas. First, not all features work in distributed mode, such as joins. Second,

the number of shards for an index is a fixed value that can’t be changed without reindexing

all of the documents. We’ll get into all of the specifics of SolrCloud in chapter 16, but we

want to make sure architects are aware that Solr scaling has come a long way in the past

couple of years yet still has some more work to do.

1.3.2 Solr for the system administrator

As a system administrator, high among your priorities in adopting a new technology like Solr

is whether it fits into your existing infrastructure. The easy answer is yes it does. As Solr is

Java based, it runs on any OS platform that has a J2SE 6.x/7.x JVM. Out of the box, Solr

embeds Jetty, the open source Java servlet engine provided by Oracle. Otherwise, Solr is a

standard Java web application that deploys easily to any Java web application server like

JBoss and Oracle AS.

All access to Solr can be done via HTTP and Solr is designed to work with caching HTTP

reverse proxies like Squid and Varnish. Solr also works with JMX so you can hook it up to

your favorite monitoring application, such as Nagios.

Lastly, Solr provides a nice administration console for checking configuration settings,

statistics, issuing test queries, and monitoring the health of SolrCloud. Figure 1.4 provides a

screen shot of the Solr 4 administration console. We’ll learn more about the administration

console in chapter 2.

Licensed to Jiwen Zhang <kevinzhang29@gmail.com>

剩余262页未读，继续阅读

yurnom

粉丝: 0
资源: 5

Solr实战：探索搜索引擎的核心能力

Solr in Action: 英文完整版指南

Solr in Action实战指南：Manning出版社教程

Solr in Action全书英文版

solr in action

Solr in action

Solr In Action 2013

Solr In Action 原版

Solr in action Source Code

Solr in Action_2014

solr in action 完整版

最新资源