Solr实战：探索搜索引擎核心技术

3星 · 超过75%的资源需积分: 10 162 浏览量更新于2024-07-25 1 收藏 8.35MB PDF 举报

"Solr in Action 是一本详细讲解Solr 4.0的书籍，被誉为条理最清晰的Solr教程之一。" 《Solr in Action》全面覆盖了Apache Solr的相关知识，它是一本专为那些希望深入了解和使用Solr进行高效全文搜索和数据分析的人而准备的指南。书中详细介绍了从基础到高级的各种功能，帮助读者逐步掌握Solr的核心能力和扩展应用。 Part 1: Meet Solr 这部分是Solr的基础入门，包括： 1. Introduction to Solr：介绍Solr的基本概念、历史背景和在现代搜索引擎中的角色。 2. Getting to know Solr：介绍Solr的安装、运行环境和基本架构。 3. Key Solr concepts：深入讲解Solr的核心概念，如索引、查询、文档等。 4. Configuring Solr：详细阐述如何配置Solr以满足特定需求，包括配置文件解析和参数调整。 5. Indexing：讨论数据导入、索引构建和更新策略。 Part 2: Core Solr Capabilities 这部分深入探讨Solr的主要功能： 6. Text analysis：详述Solr的文本分析过程，包括分词器、过滤器和分析链的设置。 7. Performing queries & handling results：讲解Solr的查询语法、查询优化和结果处理。 8. Faceted search：介绍Solr的分面搜索，用于提供多维度的筛选选项。 9. Hit highlighting：讨论高亮显示匹配查询的文本，提升用户体验。 10. Search suggestions：解释自动补全和建议功能的实现方法。 11. Result Grouping/Field Collapsing：展示如何通过分组和折叠字段来组织搜索结果。 Part 3: Taking Solr to the Next Level 这部分进一步探讨Solr的进阶主题： 12. Taking Solr to production：提供将Solr部署到生产环境的最佳实践和注意事项。 13. Scaling Solr/SolrCloud：讲解如何扩展Solr，包括分布式搜索和SolrCloud的使用。 14. Multi-lingual Search：介绍支持多语言搜索的策略和技术。 15. Complex data operations：讨论处理复杂数据类型和操作，如地理位置搜索。 16. Relevancy tuning：详细解释如何调整相关性评分，以优化搜索结果的质量。 17. Thinking outside the box：鼓励读者探索Solr的创新用法，以解决特定问题。 Appendixes 1. Building Solr from source：指导读者从源代码构建Solr，以了解内部工作原理或定制功能。 2. Working with the Solr community：介绍如何参与到Solr社区，获取帮助和贡献自己的经验。通过这本书，读者可以系统地学习Solr，从搭建环境到优化搜索性能，再到实现复杂的企业级搜索解决方案。无论你是开发者、系统管理员还是数据分析师，这本书都将是你探索Solr世界的重要参考。

©Manning Publications Co. Please post comments or corrections to the Author Online forum:

http://www.manning-sandbox.com/forum.jspa?forumID=828

Admittedly, figure 1.3 is a little overwhelming at first glance. Take a moment to scan over

the diagram to get a feel for some of the terminology. Don’t worry if you’re not familiar with

all of the terms and concepts represented in the diagram--after reading this book, you

should have a strong understanding of the all concepts presented in figure 1.3.

As we mentioned in the introduction to this chapter, the Solr designers recognized that

Solr fits better as a complementary technology that works within existing architectures. In

fact, you’ll be hard-pressed to find an environment where Solr doesn't drop right in. As we'll

see in chapter 2, you can start the example Solr server in a couple of minutes after you

finish the download.

To achieve the goal of easy integration, Solr’s core services need to be accessible from

many different applications and languages. Solr provides simple REST-like services based on

proven standards of XML, JSON, and HTTP. As a brief aside, we avoid the “RESTful” label for

Solr’s HTTP-based API as it doesn’t strictly adhere to all REST (representational state

transfer) principles. For instance, in Solr you use HTTP POST to delete documents instead of

HTTP DELETE.

A REST-like interface is nice as a foundation, but often times, developers like to have

access to a client library in their language of choice to abstract away some of the boilerplate

machinery of invoking a web service and processing the response. The good news here is

that most popular languages have a Solr client library including Python, PHP, Java, .NET, and

Ruby.

1.2.4 Multiple indexes in one server

One hallmark of modern application architectures is the need for flexibility in the face of

rapidly changing requirements. One of the ways Solr helps this situation is that you don’t

have to do all things in Solr with one index, because Solr supports running multiple cores in

a single engine. In figure 1.3, we’ve depicted multiple cores as separate layers all running in

the same Java web application environment.

Think of each core as a separate index and configuration and there can be many cores in

a single Solr instance. This allows you to manage multiple cores from one server so that you

can share server resources and administration tasks like monitoring and maintenance. Solr

provides an API for creating and managing multiple cores.

One use of Solr’s multicore support is data partitioning, such as having one core for

recent documents and another core for older documents, known as chronological sharding.

Another use of Solr’s multicore support is to support multitenant applications.

In our real estate application, we might use multiple cores to manage different types of

listings that are different enough to justify having different indexes for each. Consider real

estate listings for rural land instead of homes. Buying rural land is a different process than

buying a home in a city, so it stands to reason that we might want to manage our land

listings in a separate core.

Licensed to Jiwen Zhang <kevinzhang29@gmail.com>

©Manning Publications Co. Please post comments or corrections to the Author Online forum:

http://www.manning-sandbox.com/forum.jspa?forumID=828

1.2.5 Extendable (plug-ins)

Figure 1.3 depicts three main sub-systems in Solr: document management, query

processing, and text analysis. Of course, these are high-level abstractions for complex sub-

systems in Solr; we’ll learn about each later in the book. Each of these systems is composed

of a modular “pipeline” that allows you to plug-in new functionality. This means that instead

of overriding the entire query-processing engine in Solr, you plug-in a new search

component into an existing pipeline. This makes the core Solr functionality easy to extend

and customize to meet your specific application needs.

1.2.6 Scalable

Lucene is an extremely fast search library and Solr takes full advantage of Lucene’s speed.

But regardless of how fast Lucene is, a single server will reach its limits in terms of how

many concurrent queries from different users it can handle due to CPU and I/O constraints.

As a first pass to scalability, Solr provides flexible cache management features that help

your server avoid recomputing expensive operations. Specifically, Solr comes preconfigured

with a number of caches to save expensive recomputations, such as caching the results of a

query filter. We’ll learn about Solr’s cache management features in chapter 4.

Caching only gets you so far so at some point, you’re going to need to scale out your

capacity to handle more documents and higher query throughput by adding more servers.

For now let’s focus on the two most common dimensions of scalability in Solr. First is query

throughput, which is the number of queries your engine can support per second. Even

though Lucene can execute each query quickly, it’s limited in terms of how many concurrent

requests a single server can handle. For higher query throughput, you add replicas of your

index so that more servers can handle more requests. This means if your index is replicated

across three servers, then you can handle roughly three times the number of queries per

second because each server handles one-third of the query traffic. In practice, it’s rare to

achieve perfect linear scalability so adding three servers may only allow you to handle two

and one half times the query volume of one server.

The other dimension of scalability is the number of documents indexed. If you’re dealing

with large volumes of documents, then you’ll likely reach a point where you have too many

documents in a single instance and query performance will suffer. To handle more

documents, you split the index into smaller chunks called “shards” and then distribute the

searches across the shards.

Scaling-out with virtualized commodity hardware

One trend in modern computing is building software architectures that can scale

horizontally using virtualized commodity hardware. Put simply, add more commodity

servers to handle more traffic. Fueling this trend towards using virtualized commodity

hardware are cloud-computing providers such as Amazon EC2. Although Solr will run on

virtualized hardware, you should be aware that search is I/O and memory intensive.

Licensed to Jiwen Zhang <kevinzhang29@gmail.com>

©Manning Publications Co. Please post comments or corrections to the Author Online forum:

http://www.manning-sandbox.com/forum.jspa?forumID=828

Therefore, if search performance is a top priority for your organization, then you should

consider deploying Solr on higher end hardware with high-performance disks, ideally

solid-state drives (SSD). Hardware considerations for deploying Solr are discussed in

chapter 13.

Scalability is important, but ability to survive failures is also important for a modern

system. In the next section, we discuss how Solr handles software and hardware failures.

1.2.7 Fault tolerant

Beyond scalability, you need to consider what happens if one or more of your servers fails,

particularly if you’re planning to deploy Solr on virtualized hardware or commodity hardware.

The bottom line is that you must plan for failures. Even the best architectures and high-end

hardware will experience failures.

Let’s assume you have four shards for your index and the server hosting shard #2 loses

power. At this point, Solr can’t continue indexing documents and can’t service queries, so

your search engine is effectively “down.” To avoid this situation, you can add replicas of each

shard. In this case, when shard #2 fails, Solr reroutes indexing and query traffic to the

replica and your Solr cluster remains online. The impact of this failure is that indexing and

queries can still be processed but may not be as fast because you’ve one less server to

handle requests. We'll discuss failover scenarios in chapter 16.

At this point, you’ve seen that Solr has a modern, well-designed architecture that’s

scalable and fault-tolerant. Although these are important aspects to consider if you’ve

already decided to use Solr, you still might not be convinced that Solr is the right choice for

your needs. In the next section, we describe the benefits of Solr from the perspective of

different stakeholders, such as the software architect, system administrator, and CEO.

1.3 Why Solr?

In this section, we hope to provide you with some key information to help you decide if Solr

is the right technology for your organization. Let’s begin by addressing why Solr is attractive

to software architects.

1.3.1 Solr for the software architect

When evaluating new technology, software architects must consider a number of factors, but

chief among those are stability, scalability, and fault-tolerance. Solr scores high marks in all

three categories.

In terms of stability, Solr is a mature technology supported by a vibrant community and

seasoned committers. One thing that shocks new users to Solr and Lucene is that it isn’t

unheard of to deploy from source code pulled directly from the trunk rather than waiting for

an official release. We won’t advise you either way on whether this is acceptable for your

organization. We only point this out because it’s a testament to the depth and breadth of

Licensed to Jiwen Zhang <kevinzhang29@gmail.com>

http://www.manning-sandbox.com/forum.jspa?forumID=828

automated testing in Lucene and Solr. Put simply, if you have a nightly build off trunk where

all the automated tests pass, then you can be sure the core functionality is solid.

We’ve touched on Solr’s approach to scalability and fault-tolerance in sections 1.2.6 and

1.2.7. As an architect, you’re probably most curious about the limitations of Solr’s approach

to scalability and fault-tolerance. First, you should realize that the sharding and replication

features in Solr have been rewritten in Solr 4 to be robust and easier to manage. The new

approach to scaling is called SolrCloud. Under the covers, SolrCloud uses Apache Zookeeper

to distribute configuration across a cluster of Solr nodes and to keep track of cluster state.

Here are some highlights of the new SolrCloud features in Solr:

 Centralized configuration

 Distributed indexing with no Single Point of Failure (SPoF)

 Automated fail-over to a new shard leader

 Queries can be sent to any node in a cluster to trigger a full distributed search across

all shards with fail-over and load-balancing support built-in

But this isn’t to say that Solr scaling doesn’t have room for improvement. SolrCloud is

lacking in two areas. First, not all features work in distributed mode, such as joins. Second,

the number of shards for an index is a fixed value that can’t be changed without reindexing

all of the documents. We’ll get into all of the specifics of SolrCloud in chapter 16, but we

want to make sure architects are aware that Solr scaling has come a long way in the past

couple of years yet still has some more work to do.

1.3.2 Solr for the system administrator

As a system administrator, high among your priorities in adopting a new technology like Solr

is whether it fits into your existing infrastructure. The easy answer is yes it does. As Solr is

Java based, it runs on any OS platform that has a J2SE 6.x/7.x JVM. Out of the box, Solr

embeds Jetty, the open source Java servlet engine provided by Oracle. Otherwise, Solr is a

standard Java web application that deploys easily to any Java web application server like

JBoss and Oracle AS.

All access to Solr can be done via HTTP and Solr is designed to work with caching HTTP

reverse proxies like Squid and Varnish. Solr also works with JMX so you can hook it up to

your favorite monitoring application, such as Nagios.

Lastly, Solr provides a nice administration console for checking configuration settings,

statistics, issuing test queries, and monitoring the health of SolrCloud. Figure 1.4 provides a

screen shot of the Solr 4 administration console. We’ll learn more about the administration

console in chapter 2.

Licensed to Jiwen Zhang <kevinzhang29@gmail.com>

剩余262页未读，继续阅读

zzcake

粉丝: 1
资源: 3

Solr实战：探索搜索引擎核心技术

solr in action

Solr In Action 原版

python连接到solr

infra-solr 什么配置文件查看是否开启kerberos验证

solr配置身份认证

centos7 solr启动

linux安装solr 9.2.0

solr下载安装教程

9.2.0solr数据同步

linux solr自启动

最新资源