黑盒模型定制化理解：哈佛MUSE框架

需积分: 0 74 浏览量更新于2024-08-03 收藏 2.32MB PDF 举报

Hima哈佛MUSE模型论文探讨了在日益依赖预测性模型支持日常决策的领域，如医学诊断（例如医生），如何增强专家对黑盒模型行为的理解和信任至关重要。该研究提出了Model Understanding through Subspace Explanations (MUSE)，一个创新的模型独立框架，旨在解释给定的黑盒模型在特定特征子空间中的行为。 MUSE框架的核心理念是提供给最终用户（如医生）定制模型解释的能力，通过允许他们输入感兴趣的特征来实现。这种方法的目的是确保解释的准确性和相关性，同时保持对原始模型的忠实度。论文设计了一个新的目标函数，它旨在同时优化解释的精确性（即与原始模型输出的一致性）和可理解性，确保用户能够深入理解模型在特定特征条件下是如何做出决策的。为了实现这一目标，论文可能涵盖了以下几个关键技术点： 1. **模型agnostic**：MUSE适用于各种类型的黑盒模型，无论是机器学习算法还是深度学习架构，因为它不依赖于模型内部的具体实现，而是关注其对外部输入的响应。 2. **特征选择**：用户可以根据他们的专业知识和问题背景选择关键特征，这有助于聚焦于影响决策的关键因素，提高解释的针对性。 3. **子空间分析**：通过分析特征子集，论文可能探讨了如何划分和理解模型在不同特征组合下的行为模式，帮助用户识别可能的模式或异常。 4. **优化算法**：论文可能介绍了设计的目标函数优化算法，可能是基于梯度下降或其他优化技术，以找到最佳的解释策略，使得模型在子空间内的表现既准确又易于理解。 5. **评估方法**：为了确保解释的有效性，论文可能讨论了评估指标，如局部区域准确性、可解释性度量以及用户满意度调查，以衡量MUSE模型的实际应用效果。 6. **应用场景**：论文可能提供了实际案例，展示MUSE在医疗诊断、金融风险评估等领域的应用，证明其在提升专家信任和决策质量上的价值。 7. **挑战与未来方向**：随着模型复杂性的增加，论文可能会提及面临的挑战，如处理高维数据、解释深度学习模型等，并讨论如何进一步改进MUSE框架以应对这些挑战。 Hima哈佛MUSE模型论文提出了一种实用的工具，让专家能够更好地理解和信任黑盒模型，从而促进它们在实际决策过程中的可靠应用。通过定制化的特征子空间解释，该框架有望推动模型解释和透明度在AI领域的进步。

Faithful and Customizable Explanations of Black Box Models

Himabindu Lakkaraju

Harvard University

hlakkaraju@hbs.edu

Ece Kamar

Microsoft Research

eckamar@microsoft.com

Rich Caruana

Microsoft Research

rcaruana@microsoft.com

Jure Leskovec

Stanford University

jure@cs.stanford.edu

ABSTRACT

As predictive models increasingly assist human experts (e.g., doc-

tors) in day-to-day decision making, it is crucial for experts to be

able to explore and understand how such models behave in dier-

ent feature subspaces in order to know if and when to trust them.

To this end, we propose Model Understanding through Subspace

Explanations (MUSE), a novel model agnostic framework which

facilitates understanding of a given black box model by explaining

how it behaves in subspaces characterized by certain features of

interest. Our framework provides end users (e.g., doctors) with the

exibility of customizing the model explanations by allowing them

to input the features of interest. The construction of explanations is

guided by a novel objective function that we propose to simultane-

ously optimize for delity to the original model, unambiguity and

interpretability of the explanation. More specically, our objective

allows us to learn, with optimality guarantees, a small number of

compact decision sets each of which captures the behavior of a given

black box model in unambiguous, well-dened regions of the fea-

ture space. Experimental evaluation with real-world datasets and

user studies demonstrate that our approach can generate customiz-

able, highly compact, easy-to-understand, yet accurate explanations

of various kinds of predictive models compared to state-of-the-art

baselines.

CCS CONCEPTS

• Computing methodologies →

Supervised learning; Cost-sensitive

learning.

KEYWORDS

Interpretable machine learning, Decision making, Black box models

ACM Reference Format:

Himabindu Lakkaraju, Ece Kamar, Rich Caruana, and Jure Leskovec. 2019.

Faithful and Customizable Explanations of Black Box Models. In AAAI/ACM

Conference on AI, Ethics, and Society (AIES ’19), January 27–28, 2019, Honolulu,

HI, USA. ACM, New York, NY, USA, 8 pages. https://doi.org/10.1145/3306618.

3314229

Permission to make digital or hard copies of all or part of this work for personal or

classroom use is granted without fee provided that copies are not made or distributed

for prot or commercial advantage and that copies bear this notice and the full citation

on the rst page. Copyrights for components of this work owned by others than ACM

must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,

to post on servers or to redistribute to lists, requires prior specic permission and/or a

fee. Request permissions from permissions@acm.org.

AIES ’19, January 27–28, 2019, Honolulu, HI, USA

ACM ISBN 978-1-4503-6324-2/19/01.. . $15.00

https://doi.org/10.1145/3306618.3314229

1 INTRODUCTION

The successful adoption of predictive models for real world decision

making hinges on how much decision makers (e.g., doctors, judges)

can understand and trust their functionality. Only if decision makers

have a clear understanding of the behavior of predictive models,

they can evaluate when and how much to depend on these models,

detect potential biases in them, and develop strategies for further

model renement. However, the increasing complexity and the

proprietary nature of predictive models employed today is making

this problem harder [

], thus, emphasizing the need for tools which

can explain these complex black boxes in a faithful and interpretable

manner.

Prior research on explaining black box models can be catego-

rized as: 1) Local explanations, which focus on explaining individual

predictions of a given black box classier [

] and 2) Global

explanations, which focus on explaining model behavior as a whole,

often by summarizing complex models using simpler, more inter-

pretable approximations such as decision sets or lists [

]. In this

paper, we focus on a new form of explanation that is designed to

help end users (e.g., decision makers such as judges, doctors) gain

deeper understanding of model behavior: a dierential explanation

that describes how the model logic varies across dierent subspaces

of interest in a faithful and interpretable fashion. To illustrate, let us

consider a scenario where a doctor is trying to understand a model

which predicts if a given patient has depression or not. The doctor

might be keen on understanding how the model makes predictions

for dierent patient subgroups (See Figure 1 left). Furthermore,

she might be interested in asking questions such as "how does

the model make predictions on patient subgroups associated with

dierent values of exercise and smoking?" and might like to see

explanations customized to her interest (See Figure 1 right). The

problem of constructing such explanations has not been studied by

previous research aimed at understanding black box models.

Here, we propose a novel framework, Model Understanding

through Subspace Explanations (MUSE), which constructs global

explanations of black box classiers which highlight their behav-

ior in subspaces characterized by features of user interest. To the

best of our knowledge, this is the rst work to study the notion of

incorporating user input when generating explanations of black

box classiers while successfully trading o notions of delity, un-

ambiguity and interpretability. Our framework takes as input a

dataset of instances with semantically meaningful or interpretable

features (e.g. age, gender), and the corresponding class labels as-

signed by the black box model. It also accepts as an optional input

a set of features that are of interest to the end user in order to

Session 2: Norms and Explanations

AIES’19, January 27–28, 2019, Honolulu, HI, USA

131

下载后可阅读完整内容，剩余7页未读，立即下载

Orangee_FaFa

粉丝: 0
资源: 1

黑盒模型定制化理解：哈佛MUSE框架

hima 系统资料 hima的系统介绍以及ELOP II的入门

OPC.rar_HIMA；OPC_OPCPackage_hima_opc

hima产品说明书中文版

matlab程序模拟单个正电荷电位下的电子轨迹.rar

医药生物：创新药、“AI 医疗”有望引领医药产业新发展(1).pdf

飞利浦4H.09C02.A11电源板电路图.rar

CRC校验日常学习笔记

【2024首发原创】白鲨优化算法WSO-TCN-LSTM-Multihead-Attention负荷预测Matlab实现.rar

凌日优化算法TSOA-TCN-LSTM-Multihead-Attention多变量时间序列预测Matlab实现.rar

言安堂研究院：“医学级”护肤品市场趋势和消费者洞察.pdf

最新资源