机器学习模型部署的两种方式

There are two very different ways to

deploy ML models, here’sboth

If an ML model makes a prediction in Jupyter, is anyone around to

hear it?

Probably not. Deploying models is the key to making them useful.

It’s not only if you’re building product, in which case deployment is a

necessity — it also applies if you’re generating reports for

management. Ten years ago it was unthinkable that execs wouldn’t

because there aren’t that many people who know how to do it; seeing

that you need both data science and engineering skills. As I recently

discovered, there are two really divergent ways to deploy models: the

traditional way, and a more recent option that, I will be honest, blew

my mind.

In this article, I’ll provide you with a straightforward yet best-

practices template for both kinds of deployment. As always, for the

kinaesthetic learner, feel free to skip straight to the code here, which I

actually deployed here if you want to test it out. I know not everybody

likes to jump around when reading; it looks like this:

architecture, so let me illustrate that first. Apologies if this is

oversimplification and man-splaining! But I have seen enough “ML

model deployments” that are really just XGBoost wrapped in Flask,

that I know it’s a real problem.

The user (on the left here) is using a browser that runs only

Javascript, HTML, and CSS. That’s the frontend. It can make calls to

a backend server to get results, which it then maybe processes and

displays. The backend server should respond ASAP to the frontend’s

requests; but the backend may need to talk to databases, third party

APIs, and microservices. The backend may also produce slow jobs —

scalability. That’s why there are bubbles coming out of ‘server’ in the

diagram above; they represent ‘more of these’. So, each instance has

to remain stateless: finish handling the HTTP request and exit. Don’t

keep anything in memory between requests, because aclient’sfirst

requestmightgotooneserver,andasubsequentrequestto

another.

It’s bad if we have a long running endpoint: it would tie up one of our

servers (say… doing some ML task), leaving it unable to handle other

users’ requests. We need to keep the web server responsive and have

it hand off long running tasks, with some kind of shared persistence

predictions to the database when done.

With the library MLQ, the following is literally all you need for a

backend web server — an endpoint to enqueue a job, an endpoint to

check the progress of a job, and an endpoint to serve up a job’s result

if the job has finished.