机器学习应用案例大百科

5星 · 超过95%的资源需积分: 5 13 浏览量更新于2024-07-07 收藏 17.64MB PDF 举报

"这本大书包含了机器学习的多种应用场景，通过行业专家的技术博客，提供实际操作的案例，涉及动态时间规整、时间序列预测、循环神经网络、金融欺诈检测、数字病理学图像分析、车辆分类、地理空间数据分析等多个领域，并且附带代码示例和Jupyter笔记本，旨在帮助用户在Databricks平台上开始或提升机器学习实践。" 《机器学习应用大全》这本书深入浅出地介绍了各种机器学习技术如何应用于不同行业的具体场景，对于想在实际工作中应用机器学习的人来说，是一份宝贵的参考资料。书中涵盖的内容广泛，每一章都专注于一个特定的主题： 1. **第一章：介绍** - 提供对机器学习基本概念的概述，以及本书的目的和结构。 2. **第二章：动态时间规整与MLflow检测销售趋势** - 分两部分解释动态时间规整（DTW）的原理和如何结合MLflow工具来发现销售趋势，这对于市场营销和销售预测至关重要。 3. **第三章：大规模细粒度时间序列预测与Prophet和Apache Spark** - 展示如何使用Prophet库和Spark进行高效的时间序列预测，适用于供应链管理、能源需求预测等领域。 4. **第四章：循环神经网络进行多变量时间序列预测** - 讨论RNN在处理多个变量的时间序列数据上的应用，如天气预报和股票市场分析。 5. **第五章：基于决策树和MLflow的大规模金融欺诈检测** - 介绍如何利用机器学习模型识别欺诈交易，这对于金融风控具有重要意义。 6. **第六章：Databricks上的机器学习自动化数字病理学图像分析** - 描述了如何运用机器学习技术自动分析医学图像，对于医疗诊断和研究有深远影响。 7. **第七章：卷积神经网络在车辆分类中的实现** - 阐述CNN在自动驾驶和智能交通系统中的应用。 8. **第八章：Databricks上的大规模地理空间数据分析** - 展示如何处理和分析大规模地理数据，对城市规划、环境监测等领域非常有用。 9. **第九章：客户案例研究** - 分享了实际企业如何成功运用这些机器学习技术，为读者提供实际应用的灵感和教训。这本书通过丰富的实例和实用的代码示例，为读者展示了如何在Databricks平台上实施这些机器学习解决方案，无论你是初学者还是经验丰富的数据科学家，都能从中受益。通过阅读和实践书中的案例，你将能够更好地理解和应用机器学习技术，解决现实世界的问题。

From there, we can identify the product codes closest to the optimal sales trend

(i.e., those that have the smallest calculated DTW distance). Since we’re using

Databricks, we can easily make this selection using a SQL query. Let’s display those

that are closest.

%sql

-- Top 10 product codes closest to the optimal sales trend

select pcode, cast(dtw_dist as oat) as dtw_dist from distances order by

cast(dtw_dist as oat) limit 10

After running this query, along with the corresponding query for the product codes

that are furthest from the optimal sales trend, we were able to identify the two

products that are closest and furthest from the trend. Let’s plot both of those

products and see how they differ.

As you can see, Product #675 (shown in the orange triangles) represents the

best match to the optimal sales trend, although the absolute weekly sales are

lower than we’d like (we’ll remedy that later). This result makes sense since we’d

expect the product with the closest DTW distance to have peaks and valleys that

somewhat mirror the metric we’re comparing it to. (Of course, the exact time

index for the product would vary on a week-by-week basis due to dynamic time

warping.) Conversely, Product #716 (shown in the green stars) is the product with

the worst match, showing almost no variability.

13EBOOK: THE BIG BOOK OF MACHINE LEARNING USE CASES

Now that MLow has saved the logs of each experiment, we can go back through

and examine the results. From your Databricks notebook, select the “Runs” icon in

the upper right-hand corner to view and compare the results of each of our runs.

Not surprisingly, as we increase our “stretch factor,” our distance metric decreases.

Intuitively, this makes sense: As we give the algorithm more exibility to warp the

time indices forward or backward, it will nd a closer t for the data. In essence,

we’ve traded some bias for variance.

Logging models in MLow

MLow has the ability to not only log experiment parameters, metrics and artifacts

(like plots or CSV les), but also to log machine learning models. An MLow model

is simply a folder that is structured to conform to a consistent API, ensuring

compatibility with other MLow tools and features. This interoperability is very

powerful, allowing any Python model to be rapidly deployed to many different

types of production environments.

MLow comes preloaded with a number of common model “avors” for many

of the most popular machine learning libraries, including scikit-learn, Spark MLlib,

PyTorch, TensorFlow and others. These model avors make it trivial to log and reload

models after they are initially constructed, as demonstrated in this blog post. For

example, when using MLow with scikit-learn, logging a model is as easy as running

the following code from within an experiment:

mlow.sklearn.log_model(model=sk_model, artifact_path=”sk_model_path”)

MLow also offers a “Python function” avor, which allows you to save any

model from a third-party library (such as XGBoost or spaCy), or even a simple

Python function itself, as an MLow model. Models created using the Python

function avor live within the same ecosystem and are able to interact with other

MLow tools through the Inference API. Although it’s impossible to plan for every

use case, the Python function model avor was designed to be as universal and

exible as possible. It allows for custom processing and logic evaluation, which

can come in handy for ETL applications. Even as more “ofcial” model avors

come online, the generic Python function avor will still serve as an important

“catchall,” providing a bridge between Python code of any kind and MLow’s

robust tracking toolkit.

Logging a model using the Python function avor is a straightforward process.

Any model or function can be saved as a model, with one requirement: It

must take in a pandas DataFrame as input, and return a DataFrame or NumPy

array. Once that requirement is met, saving your function as an MLow model

involves dening a Python class that inherits from PythonModel, and overriding the

.predict() method with your custom function, as described here.

16EBOOK: THE BIG BOOK OF MACHINE LEARNING USE CASES

剩余77页未读，继续阅读

努力+努力=幸运

粉丝: 17
资源: 136

机器学习应用案例大百科

"Spark大数据机器学习规模扩展至数十亿参数

"数据结构教学课件：第19讲 哈希查找 .pdf" - 基本原理和应用总结

"基于XGBoost的信用评价特征工程研究

Big.Data.and.Business.Analytics.1466565780

Sams.Teach.Yourself.Big.Data.Analytics.with.Microsoft.HDInsight

Mastering Java Machine Learning

hands on machine learning on google cloud platform

Research Advances in Cloud Computing-Springer(2017).pdf

Machine Learning Yearning(吴恩达的书)--Andre Ng

Machine_Learning_Yearning_V01

最新资源

"数据结构教学课件：第19讲哈希查找 .pdf" - 基本原理和应用总结