使用容器化微服务实现自动化连续部署pipeline

5星 · 超过95%的资源需积分: 50 92 浏览量更新于2024-07-20 4 收藏 8.1MB PDF 举报

"DevOps 2.0 Tookit - Automating the Continuous Deployment Pipeline with Containerized Microservices" 以下是从给定文件中生成的知识点： **DevOps 2.0 Tookit 介绍** DevOps 2.0 Tookit 是一本关于自动化连续部署管道与容器化微服务的书籍，作者是 Viktor Farcic。该书讲述了如何使用微服务、容器化、自动化配置管理工具等技术来架构软件，使其更加高效、可靠和可扩展。 **微服务架构** 微服务架构是一种软件架构风格，它将应用程序拆分成多个小的独立服务，每个服务都可以独立开发、测试和部署。微服务架构的优点包括：灵活性、可扩展性、可靠性和高可用性。 **容器化** 容器化是一种软件打包技术，它可以将应用程序及其依赖项打包成一个容器，使其可以在任何环境中运行。容器化的优点包括：轻量级、可移植性、隔离性和高效性。 **自动化配置管理** 自动化配置管理是指使用工具来自动化服务器的配置和管理，使其可以快速、可靠地部署应用程序。自动化配置管理的优点包括：快速部署、可靠性和高效性。 **continuous Integration, Delivery and Deployment** continuous Integration, Delivery and Deployment（CI/CD）是指自动化软件开发、测试、部署和交付的整个生命周期。CI/CD 的优点包括：快速反馈、自动化测试、快速部署和高效性。 **Docker, Kubernetes, Ansible 等工具** Docker、Kubernetes、Ansible 等工具是实现自动化配置管理和微服务架构的重要工具。Docker 是一个容器化平台，Kubernetes 是一个容器编排平台，Ansible 是一个自动化配置管理工具。 **微服务开发和部署生命周期** 微服务开发和部署生命周期包括：设计、开发、测试、部署、监控和反馈等阶段。该生命周期的目标是快速、可靠地交付高质量的软件。 **自愈系统设计** 自愈系统设计是指设计能够自我恢复的系统，使其能够在出现故障时自动恢复。自愈系统设计的优点包括：高可用性、可靠性和快速恢复。 **集中日志和监控** 集中日志和监控是指使用工具来集中收集和监控系统的日志和性能指标，以便快速发现和解决问题。集中日志和监控的优点包括：快速发现问题、快速恢复和高效性。 DevOps 2.0 Tookit 是一本关于自动化连续部署管道与容器化微服务的书籍，涵盖了微服务架构、容器化、自动化配置管理、CI/CD 等多个方面的内容。

The DevOps Ideal 9

is overkill. Still, armed with patience, immutable servers were a good idea, but the way we used that

approach and the tools required to support it were not good enough.

Orchestration

The orchestration was the key. Puppet and Chef proved to be a big help. Programming everything

related to servers setup and deployment was a huge improvement. Not only that the time needed

to setup servers and deploy software dropped drastically, but we could, finally, accomplish a more

reliable process. Having humans (read operations department) manually running those types of

tasks was a recipe for disaster. Finally a story with a happy ending? Not really. You probably started

noticing a pattern. As soon as one improvement was accomplished, it turned out that it comes

with, often, high price. Given enough time, Puppet and Chef scripts and configurations turn into

an enormous pile of **** (I was told not to use certain words so please fill in the blanks with your

imagination). Maintaining them tends to become a nightmare in itself. Still, with orchestration tools,

we could drastically reduce the time it took to create immutable VMs. Something is better than

nothing.

The Light at the End of the Deployment Pipeline

I could go on and on describing problems we faced. Don’t take me wrong. All those initiatives were

improvements and have their place in software history. But history is the past, and we live in the

present trying to look into the future. Many, if not all of the problems we had before are now solved.

Ansible proved that orchestration does not need to be complicated to set up nor hard to maintain.

With the appearance of Docker, containers are slowly replacing VMs as the preferable way to create

immutable deployments. New operating systems are emerging and fully embracing containers as

first class citizens. Tools for service discovery are showing us new horizons. Swarm, Kubernetes and

Mesos/DCOS are opening doors into areas that were hard to imagine only a few years ago.

Microservices are slowly becoming the preferred way to build big, easy to maintain and highly

scalable systems thanks to tools like Docker, CoreOS, etcd, Consul, Fleet, Mesos, Rocket, and others.

The idea was always great, but we did not have the tools to make it work properly. Now we do!

That does not mean that all our problems are gone. It means that when we solve one problem, the

bar moves higher up, and new issues emerge.

I started by complaining about the past. That will not happen again. This book is for readers who

do not want to live in the past but present. This book is about preparations for the future. This book

is about stepping through the looking glass, about venturing into new areas and about looking at

things from a new angle.

This is your last chance. After this, there is no turning back. You take the blue pill - the story ends,

you wake up in your bed and believe whatever you want to believe. You take the red pill - you stay

in Wonderland and I show you how deep the rabbit-hole goes.

– Morpheus (Matrix)

The Implementation Breakthrough:

Continuous Deployment,

Microservices, and Containers

On the first look continuous deployment (CD), microservices (MS) and containers might seem

like three unrelated subjects. After all, DevOps movement does not stipulate that microservices

are necessary for continuous deployment, nor microservices need to be packaged into containers.

However, when those three are combined, new doors open waiting for us to step through. Recent

developments in the area of containers and the concept of immutable deployments enable us to

overcome many of the problems microservices had before. They, on the other hand, allow us to gain

flexibility and speed without which CD is not possible or cost effective.

Before we move forward with this line of thinking, we’ll try to define correctly each of those terms.

Continuous Integration

To understand continuous deployment we should first define its predecessors; continuous integration

and continuous delivery.

Integration phase of a project development tended to be one of the most painful stages in software

development life-cycle. We would spend weeks, months or even years working in separate teams

dedicated to separate applications and services. Each of those teams would have their set of

requirements and tried their best to meet them. While it wasn’t hard to periodically verify each

of those applications and services in isolation, we all dreaded the moment when team leads would

decide that the time has come to integrate them into a unique delivery. Armed with the experience

from previous projects, we knew that integration will be problematic. We knew that we will discover

problems, unmet dependencies, interfaces that do not communicate with each others correctly and

that managers will get disappointed, frustrated, and nervous. It was not uncommon to spend weeks

or even months in this phase. The worse part of all that was that a bug found during the integration

phase could mean going back and redoing days or weeks worth of work. If someone asked me how

a feel about integration I’d say that it was closest I could get to becoming permanently depressed.

Those were different times. We thought that was the “right” way to develop applications.

A lot changed since then. Extreme Programming (XP) and other agile methodologies become

familiar, automated testing become frequent, and continuous integration started to take ground.

Today we know that the way we developed software back then was wrong. The industry moved a

long way since then.

The Implementation Breakthrough: Continuous Deployment, Microservices, and Containers 12

Continuous integration (CI) usually refers to integrating, building, and testing code within the

development environment. It requires developers to integrate code into a shared repository often.

How often is often can be interpreted in many ways and it depends on the size of the team, the size of

the project and the number of hours we dedicate to coding. In most cases it means that coders either

push directly to the shared repository or merge their code with it. No matter whether we’re pushing

or merging, those actions should, in most cases, be done at least a couple of times a day. Getting

code to the shared repository is not enough and we need to have a pipeline that, as a minimum,

checks out the code and runs all the tests related, directly or indirectly, to the code corresponding

to the repository. The result of the execution of the pipeline can be either red or green. Something

failed, or everything was run without any problems. In the former case, minimum action would be

to notify the person who committed the code.

The continuous integration pipeline should run on every commit or push. Unlike continuous

delivery, continuous integration does not have a clearly defined goal of that pipeline. Saying that

one application integrates with others does not tell us a lot about its production readiness. We do

not know how much more work is required to get to the stage when the code can be delivered to

production. All we are truly striving for is the knowledge that a commit did not break any of the

existing tests. Never the less, CI is a huge improvement when done right. In many cases, it is a

very hard practice to implement, but once everyone is comfortable with it, the results are often very

impressive.

Integration tests need to be committed together with the implementation code, if not before. To gain

maximum benefits, we should write tests in test-driven development (TDD) fashion. That way, not

only that tests are ready for commit together with implementation, but we know that they are not

faulty and would not pass no matter what we do. There are many other benefits TDD brings to the

table and, if you haven’t already, I strongly recommend to adopt it. You might want to consult the

Test-Driven Development⁴ section of the Technology Conversations⁵ blog.

Tests are not the only CI prerequisite. One of the most important rules is that when the pipeline

fails, fixing the problem has higher priority than any other task. If this action is postponed, next

executions of the pipeline will fail as well. People will start ignoring the failure notifications and,

slowly, CI process will begin losing its purpose. The sooner we fix the problem discovered during the

execution of the CI pipeline, the better we are. If corrective action is taken immediately, knowledge

about the potential cause of the problem is still fresh (after all, it’s been only a few minutes between

the commit and the failure notification) and fixing it should be trivial.

So how does it work? Details depend on tools, programming language, project, and many other

factors. The most common flow is the following.

• Pushing to the code repository

• Static analysis

• Pre-deployment testing

⁴http://technologyconversations.com/category/test-driven-development/

⁵http://technologyconversations.com/

The Implementation Breakthrough: Continuous Deployment, Microservices, and Containers 13

• Packaging and deployment to the test environment

• Post-deployment testing

Pushing to the Code Repository

Developers work on features in separate branches. Once they feel comfortable that their work is

stable, the branch they’ve been working on is merged with the mainline (or trunk). More advanced

teams may skip feature branches altogether and commit directly to the mainline. The crucial point is

that the mainline branch (or trunk) needs to receive commits often (either through merges or direct

pushes). If days or weeks pass, changes accumulate and benefits of using continuous integration

diminish. In that case, there is no fast feedback since the integration with other people’s code

is postponed. On the other hand, CI tools (we’ll talk about them later) are monitoring the code

repository, and whenever a commit is detected, the code is checked out (or cloned) and the CI

pipeline is run. The pipeline itself consists of a set of automated tasks run in parallel or sequentially.

The result of the pipeline is either a failure in one of its steps or a promotion. As a minimum, failure

should result in some form of a notification sent to the developer that pushed the commit that

resulted in a failed pipeline. It should be his responsibility to fix the problem (after all, he knows

best how to fix a problem created by him only minutes ago) and do another commit to the repository

that, in turn, will trigger another execution of the pipeline. This developer should consider fixing the

problem his highest priority task so that the pipeline continues being “green” and avoid failures that

would be produced by commits from other developers. Try to keep a number of people who receive

the failure notification to a minimum. The whole process from detecting a problem until it is fixed

should be as fast as possible. The more people are involved, the more administrative work tends to

happen and the more time is spent until the fix is committed. If, on the other hand, the pipeline run

successfully throughout all its tasks, package produced throughout the process is promoted to the

next stage and, in most cases, given to testers for manual verifications. Due to the difference in speed

between the pipeline (minutes) and manual testing (hours or days), not every pipeline execution is

taken by QAs.

剩余403页未读，继续阅读

jiaoxiaogu

粉丝: 64
资源: 169

使用容器化微服务实现自动化连续部署pipeline

The DevOps 2.4 Toolkit

The DevOps 2.2 Toolkit

The-devops-2-2-toolkit 英文完整版

mysql 切割'https://gitee.web.guosen.com.cn/devops/pipeline/devops-pipeline-shared-library.git' https://gitee.web.guosen.com.cn 之后 第一个//之间的内容

.gitlab-ci.yml 中include: - project: sh-devops/eksgitlabci-library-sh ref: main file: "templates/eksjava-allbranch-manual-sh.yml"解释

从 “op.devOps.srv-test/: up (pid 48585) 1 seconds” 获取pid，即pid后面的数字，通过shell方式

DevOps 项目演示 - 2 修改 Jenkinsfile

find /home/devops/app/core-2.0/logs -type f -ctime +1 -name "*.log" |xargs rm

devops流水线部署k8s

devops 3763.7-2021

最新资源

mysql 切割'https://gitee.web.guosen.com.cn/devops/pipeline/devops-pipeline-shared-library.git' https://gitee.web.guosen.com.cn 之后第一个//之间的内容