Ceph分布式存储系统详解：从原理到实践

需积分: 9 73 浏览量更新于2024-07-16 收藏 4.52MB PDF 举报

身份认证购VIP最低享 7 折!

30元优惠券

"Ceph是一个开源的分布式存储系统，它提供了对象、块和文件存储的统一解决方案。这个文档详细介绍了Ceph的工作原理、安装、配置和优化，适用于运维工程师和开发工程师，对于想要理解分布式存储的工程师具有很好的指导价值。" Ceph作为一个全面的存储平台，其核心特性包括： 1. RESTful接口：Ceph提供了RESTful接口，使得用户可以通过HTTP协议进行操作，方便了与其他服务的集成。 2. 对象存储：Ceph支持S3和Swift兼容的API，可以创建S3风格的子域名，并提供统一的S3/Swift命名空间。 3. 用户管理与使用跟踪：系统能够进行用户管理，同时监控和追踪存储资源的使用情况。 4. 分条式对象存储：通过分条式存储技术提高读写性能，适用于大规模数据处理。 5. 云解决方案集成：Ceph可以直接作为云解决方案的后端，如KVM/libvirt的支持，以及作为云环境中的块设备。 6. 多站点部署与灾难恢复：Ceph支持多站点部署，可以实现灾难恢复策略，确保数据的安全性。 7. 薄置备：采用薄置备技术，按需分配存储空间，有效利用资源。 8. 快照与克隆：支持快照功能，可实现基于快照的克隆，便于数据备份和恢复。 9. 内存缓存：Ceph具有内存缓存机制，提升数据访问速度。 10. 可配置的条带化：用户可以根据需求调整条带化的设置，优化I/O性能。 11. 支持多种驱动：包括内核驱动和FUSE（用户空间文件系统）支持，同时也支持NFS和CIFS协议，能与多种环境无缝对接。 12. 与Hadoop的集成：Ceph可以作为Hadoop的替代方案，用于替换HDFS，提供高性能的数据存储和处理能力。 Ceph的高可靠性源于其设计，它采用了CRUSH算法进行数据分布和故障恢复，能够在节点故障时自动重新平衡数据，确保服务连续性和数据完整性。此外，Ceph还具有动态再平衡和子目录快照等高级功能，允许在不中断服务的情况下调整存储布局。 Ceph是一个强大的分布式存储系统，无论是在云计算环境还是传统数据中心，都能提供灵活、可靠和高效的存储解决方案。其全面的特性和丰富的功能使其成为应对大数据挑战的理想选择。

资源详情

资源推荐

Ceph, Release dev

1.4 Documenting Ceph

The easiest way to help the Ceph project is to contribute to the documentation. As the Ceph user base grows and

the development pace quickens, an increasing number of people are updating the documentation and adding new

information. Even small contributions like ﬁxing spelling errors or clarifying instructions will help the Ceph project

immensely.

The Ceph documentation source resides in the ceph/docs directory of the Ceph repository, and Python Sphinx

renders the source into HTML and manpages. The http://ceph.com/docs link currenly displays the master branch

by default, but you may view documentation for older branches (e.g., argonaut) or future branches (e.g., next) as

well as work-in-progress branches by substituting master with the branch name you prefer.

1.4.1 Making Contributions

Making a documentation contribution generally involves the same procedural sequence as making a code contribution,

except that you must build documentation source instead of compiling program source. The sequence includes the

following steps:

1. Get the Source

2. Select a Branch

3. Make a Change

4. Build the Source

5. Commit the Change

6. Push the Change

7. Make a Pull Request

8. Notify the Relevant Person

Get the Source

Ceph documentation lives in the Ceph repository right alongside the Ceph source code under the ceph/doc directory.

For details on github and Ceph, see Get Involved in the Ceph Community!.

The most common way to make contributions is to use the Fork and Pull approach. You must:

1. Install git locally. For Debian/Ubuntu, execute:

sudo apt-get install git

For Fedora, execute:

sudo yum install git

For CentOS/RHEL, execute:

sudo yum install git

2. Ensure your .gitconfig ﬁle has your name and email address.

[user]

email = {your-email-address}

name = {your-name}

12 Chapter 1. Intro to Ceph

Ceph, Release dev

For example:

git config --global user.name "John Doe"

git config --global user.email johndoe@example.com

3. Create a github account (if you don’t have one).

4. Fork the Ceph project. See https://github.com/ceph/ceph.

5. Clone your fork of the Ceph project to your local host.

Ceph organizes documentation into an information architecture primarily by its main components.

• Ceph Storage Cluster: The Ceph Storage Cluster documentation resides under the doc/rados directory.

• Ceph Block Device: The Ceph Block Device documentation resides under the doc/rbd directory.

• Ceph Object Storage: The Ceph Object Storage documentation resides under the doc/radosgw directory.

• Ceph Filesystem: The Ceph Filesystem documentation resides under the doc/cephfs directory.

• Installation (Quick): Quick start documentation resides under the doc/start directory.

• Installation (Manual): Manual installation documentation resides under the doc/install directory.

• Manpage: Manpage source resides under the doc/man directory.

• Developer: Developer documentation resides under the doc/dev directory.

• Images: If you include images such as JPEG or PNG ﬁles, you should store them under the doc/images

directory.

Select a Branch

When you make small changes to the documentation, such as ﬁxing typographical errors or clarifying explanations,

use the master branch (default). You should also use the master branch when making contributions to features

that are in the current release. master is the most commonly used branch.

git checkout master

When you make changes to documentation that affect an upcoming release, use the next branch. next is the second

most commonly used branch.

git checkout next

When you are making substantial contributions such as new features that are not yet in the current release; if your

contribution is related to an issue with a tracker ID; or, if you want to see your documentation rendered on the

Ceph.com website before it gets merged into the master branch, you should create a branch. To distinguish

branches that include only documentation updates, we prepend them with wip-doc by convention, following the

form wip-doc-{your-branch-name}. If the branch relates to an issue ﬁled in http://tracker.ceph.com/issues,

the branch name incorporates the issue number. For example, if a documentation branch is a ﬁx for is-

sue #4000, the branch name should be wip-doc-4000 by convention and the relevant tracker URL will be

http://tracker.ceph.com/issues/4000.

Note: Please do not mingle documentation contributions and source code contributions in a single pull request.

Editors review the documentation and engineers review source code changes. When you keep documentation pull

requests separate from source code pull requests, it simpliﬁes the process and we won’t have to ask you to resubmit

the requests separately.

Before you create your branch name, ensure that it doesn’t already exist in the local or remote repository.

1.4. Documenting Ceph 13

Ceph, Release dev

git branch -a | grep wip-doc-{your-branch-name}

If it doesn’t exist, create your branch:

git checkout -b wip-doc-{your-branch-name}

Make a Change

Modifying a document involves opening a restructuredText ﬁle, changing its contents, and saving the changes. See

Documentation Style Guide for details on syntax requirements.

Adding a document involves creating a new restructuredText ﬁle under the doc directory or its subdirectories and

saving the ﬁle with a

.rst ﬁle extension. You must also include a reference to the document: a hyperlink or a table

of contents entry. The index.rst ﬁle of a top-level directory usually contains a TOC, where you can add the new

ﬁle name. All documents must have a title. See Headings for details.

Your new document doesn’t get tracked by git automatically. When you want to add the document to the repository,

you must use git add {path-to-filename}. For example, from the top level directory of the repository,

adding an example.rst ﬁle to the rados subdirectory would look like this:

git add doc/rados/example.rst

Deleting a document involves removing it from the repository with git rm {path-to-filename}. For exam-

ple:

git rm doc/rados/example.rst

You must also remove any reference to a deleted document from other documents.

Build the Source

To build the documentation, navigate to the ceph repository directory:

cd ceph

To build the documentation on Debian/Ubuntu, execute:

admin/build-doc

To build the documentation on Fedora, execute:

admin/build-doc

To build the documentation on CentOS/RHEL, execute:

admin/build-doc

Executing admin/build-doc will create a build-doc directory under ceph. You may need to create a directory

under ceph/build-doc for output of Javadoc ﬁles.

mkdir -p output/html/api/libcephfs-java/javadoc

The build script build-doc will produce an output of errors and warnings. You MUST ﬁx errors in documents you

modiﬁed before committing a change, and you SHOULD ﬁx warnings that are related to syntax you modiﬁed.

Important: You must validate ALL HYPERLINKS. If a hyperlink is broken, it automatically breaks the build!

Once you build the documentation set, you may navigate to the source directory to view it:

14 Chapter 1. Intro to Ceph

Ceph, Release dev

• python-pip

• python-virtualenv

• python-docutils

• python-jinja2

• python-pygments

• python-sphinx

• libxml2-dev

• libxslt1-dev

• doxygen

• graphviz

• ant

Install each dependency that isn’t installed on your host. For Debian/Ubuntu distributions, execute the following:

sudo apt-get install gcc python-dev python-pip python-virtualenv libxml2-dev libxslt-dev doxygen graphviz ant ditaa

sudo apt-get install python-sphinx

For Fedora distributions, execute the following:

sudo yum install gcc python-devel python-pip python-virtualenv libxml2-devel libxslt-devel doxygen graphviz ant

sudo pip install html2text

sudo yum install python-jinja2 python-pygments python-docutils python-sphinx

sudo yum install jericho-html ditaa

For CentOS/RHEL distributions, it is recommended to have epel (Extra Packages for Enterprise Linux) repository

as it provides some extra packages which are not available in the default repository. To install epel, execute the

following:

wget http://ftp.riken.jp/Linux/fedora/epel/7/x86_64/e/epel-release-7-2.noarch.rpm

sudo yum install epel-release-7-2.noarch.rpm

For CentOS/RHEL distributions, execute the following:

sudo yum install gcc python-devel python-pip python-virtualenv libxml2-devel libxslt-devel doxygen graphviz ant

sudo pip install html2text

For CentOS/RHEL distributions, the remaining python packages are not available in the default and epel repositories.

So, use http://rpmﬁnd.net/ to ﬁnd the packages. Then, download them from a mirror and install them. For example:

wget ftp://rpmfind.net/linux/centos/7.0.1406/os/x86_64/Packages/python-jinja2-2.7.2-2.el7.noarch.rpm

sudo yum install python-jinja2-2.7.2-2.el7.noarch.rpm

wget ftp://rpmfind.net/linux/centos/7.0.1406/os/x86_64/Packages/python-pygments-1.4-9.el7.noarch.rpm

sudo yum install python-pygments-1.4-9.el7.noarch.rpm

wget ftp://rpmfind.net/linux/centos/7.0.1406/os/x86_64/Packages/python-docutils-0.11-0.2.20130715svn7687.el7.noarch.rpm

sudo yum install python-docutils-0.11-0.2.20130715svn7687.el7.noarch.rpm

wget ftp://rpmfind.net/linux/centos/7.0.1406/os/x86_64/Packages/python-sphinx-1.1.3-8.el7.noarch.rpm

sudo yum install python-sphinx-1.1.3-8.el7.noarch.rpm

Ceph documentation makes extensive use of ditaa, which isn’t presently built for CentOS/RHEL7. You must in-

stall ditaa if you are making changes to ditaa diagrams so that you can verify that they render properly before

you commit new or modiﬁed ditaa diagrams. You may retrieve compatible required packages for CentOS/RHEL

distributions and install them manually. To run ditaa on CentOS/RHEL7, following dependencies are required:

• jericho-html

16 Chapter 1. Intro to Ceph

剩余980页未读，继续阅读

xuyongsan

粉丝: 0
资源: 4

Ceph分布式存储系统详解：从原理到实践

ceph学习资料.pdf

Ceph学习笔记.pdf

CEPH中文操作文档.pdf

ceph-common14.2.21离线安装

mount: /ceph: /ceph/ceph/ceph_14.2.8_0 is not a block device.

ceph mon add /var/run/ceph/ceph-mon.cephnode03.asok

kolla-ceph官方文档

ceph-ansible 官方文档

vim /ceph-install/ceph.conf

mount: /var/www/html: /ceph/ceph/ceph_14.2.8_0 is not a block device.

如何让ceph.conf文件生效

k8s daemonset编排ceph详细步骤

Failed to restart ceph-mon.target: Unit not found.

centos7 安装ceph

cephadm的ceph.conf配置不生效

openstack ceph官方文档

ceph集群将osd踢出集群命令

ceph needs to be installed in remote host: ceph01

最新资源