NIST SP800-142：实用组合测试教程：提升软件安全与效率

NIST

SP800

需积分: 8 184 浏览量更新于2024-07-16 收藏 983KB PDF 举报

身份认证购VIP最低享 7 折!

领优惠券(最高得80元）

NIST SP800-142, 《实用组合测试：信息安全实践》, 是由美国国家标准与技术研究院(National Institute of Standards and Technology, NIST)于2010年发布的，由D. Richard Kuhn、Raghu N. Kacker和Yu Lei撰写。该出版物关注的是软件实施错误如何对信息系统安全构成重大威胁，并强调了软件测试在确保系统安全中的关键作用。它特别聚焦于组合测试方法，这是一种可以降低成本并提高软件测试效率的技术，适用于多种应用场景。该文档提供了一个全面的教程，旨在帮助读者理解和应用组合测试于实际软件开发中。它首先介绍了组合测试的核心概念，包括如何通过排列和组合不同输入元素来发现潜在问题。作者还详细解释了如何使用免费的软件工具（可以从NIST网站csrc.nist.gov/acts获取）来生成组合测试用例，这些工具对于初学者来说非常友好，适合计算机科学或工程专业的本科生阅读。文章深入探讨了高级主题，如如何利用形式化的软件模型来确定每个可能输入集的预期结果，这有助于测试人员预测并验证软件行为的正确性。通过这种方式，组合测试不仅可以检测出常见的错误，还能覆盖到较少被考虑的边界条件和异常情况。 NIST SP800-142不仅关注于理论知识，还提供了丰富的参考文献，引导读者进一步研究每个话题的深入细节。它在联邦计算机系统的敏感未分类信息的安全性和隐私保护方面，为技术和管理标准的制定提供了指导，旨在促进经济和公共福利，并确保信息技术的有效、安全使用。 NIST SP800-142是软件测试领域的重要参考资料，对于软件开发团队、测试工程师以及对信息安全感兴趣的研究人员来说，掌握组合测试策略和技术是提升软件质量、降低安全风险的重要步骤。通过遵循这份报告提供的方法，组织可以在保障信息安全的同时，提高开发效率。

资源详情

资源推荐

________________________________________________________________

Practical Combinatorial Testing

2.1.2 Input Parameter Testing

Even if an application has no configuration options, some form of input will be

processed. For example, a word processing application may allow the user to select 10

ways to modify some highlighted text:

subscript, superscript, underline, bold, italic,

strikethrough, emboss, shadow, small caps

, or

all caps

. The font-processing function

within the application that receives these settings as input must process the input and

modify the text on the screen correctly. Most options can be combined, such as bold and

small caps, but some are incompatible, such as subscript and superscript.

Thorough testing requires that the font-processing function work correctly for all

valid combinations of these input settings. But with 10 binary inputs, there are 2

= 1,024

possible combinations. But the empirical analysis reported above shows that failures

appear to involve a small number of parameters, and that testing all 3-way combinations

may detect 90% or more of bugs. For a word processing application, testing that detects

better than 90% of bugs may be a cost-effective choice, but we need to ensure that all 3-

way combinations of values are tested. To do this, we create a test suite to cover all 3-way

combinations (known as a

covering array

) [12, 14, 23, 26, 30, 43, 63].

An example is given in Figure 3, which shows a 3-way

The key component

covering array for 10 variables with two values each. The

is a covering array,

interesting property of this array is that any three columns

which includes all t-

contain all eight possible values for three binary variables.

way combinations.

For example, taking columns F, G, and H, we can see that all

eight possible 3-way combinations (000, 001, 010, 011, 100,

Each column is a

101, 110, 111) occur somewhere in the three columns

parameter. Each

together. In fact, any combination of three columns chosen in

row is a test.

any order will also contain all eight possible values.

Collectively, therefore, this set of tests will exercise all 3-way combinations of input values

in only 13 tests, as compared with 1,024 for exhaustive coverage.

Figure 3. 3-way covering array

Tests

A B C D E F G H I J

_______________________________________________________

Practical Combinatorial Testing

Similar arrays can be generated to cover up to all 6-way combinations. In general, the

number of

-way combinatorial tests that will be required is proportional to

log

, for

parameters with

possible values each.

Figure 4 contrasts these two approaches. With the first approach, we may run the

same test set against all 3-way combinations of configuration options, while for the second

approach, we would construct a test suite that covers all 3-way combinations of input

transaction fields. Of course these approaches could be combined, with the combinatorial

tests (approach 2) run against all the configuration combinations (approach 1).

Use combinations of c

onfiguration

Configuration:

values

with existing test suite

Browser

DBMS

Server

Use combinations of

input

...

System

Under Test

Inputs:

Product

Amount

Quantity

Pmt method

Shipping method

values

in generating tests

Figure 4. Two ways of using combinatorial testing

2.2 The Test Oracle Problem

Even with efficient algorithms to produce covering arrays, the oracle problem

remains – testing requires both test data and results that should be expected for each data

input. High interaction strength combinatorial testing may require a large number of tests

in some cases, although not always. Approaches to solving the oracle problem for

combinatorial testing include:

Crash testing

: the easiest and least expensive approach is to simply run tests

against the system under test (SUT) to check whether any unusual combination of input

values causes a crash or other easily detectable failure. This is essentially the same

procedure used in “fuzz testing”, which sends random values against the SUT. This form

of combinatorial testing could be regarded as a disciplined form of fuzz testing [59]. It

should be noted that although pure random testing will generally cover a high percentage of

-way combinations, 100% coverage of combinations requires a random test set much

larger than a covering array. For example, all 3-way combinations of 10 parameters with 4

values each can be covered with 151 tests. Purely random generation requires over 900

tests to provide full 3-way coverage.

________________________________________________________________

Practical Combinatorial Testing

Embedded assertions

: An increasingly popular “light-weight formal methods”

technique is to embed assertions within code to ensure proper relationships between data,

for example as preconditions, postconditions, or input value checks. Tools such as the Java

Modeling language (JML) can be used to introduce very complex assertions, effectively

embedding a formal specification within the code. The embedded assertions serve as an

executable form of the specification, thus providing an oracle for the testing phase. With

embedded assertions, exercising the application with all

-way combinations can provide

reasonable assurance that the code works correctly across a very wide range of inputs.

This approach has been used successfully for testing smart cards, with embedded JML

assertions acting as an oracle for combinatorial tests [25]. Results showed that 80% - 90%

of errors could be found in this way.

Model based test generation

uses a mathematical model

of the SUT and a simulator or model checker to generate

expected results for each input [1,8,9,52,55]. If a simulator can

be used, expected results can be generated directly from the

simulation, but model checkers are widely available and can

also be used to prove properties such as liveness in parallel

processes, in addition to generating tests. Conceptually, a

Several types of

test oracle can be

used, depending on

resources and the

system under test.

model checker can be viewed as exploring all states of a system model to determine if a

property claimed in a specification statement is true. What makes a model checker

particularly valuable is that if the claim is false, the model checker not only reports this, but

also provides a “counterexample” showing how the claim can be shown false. If the claim

is false, the model checker indicates this and provides a trace of parameter input values and

states that will prove it is false. In effect this is a complete test case, i.e., a set of parameter

values and expected result. It is then simple to map these values into complete test cases in

the syntax needed for the system under test. Later chapters develop detailed procedures for

applying each of these testing approaches.

2.3 Chapter Summary

1. Empirical data suggest that software failures are caused by the interaction of relatively

few parameter values, and that the proportion of failures attributable to

-way interactions

declines very rapidly with increase in

. That is, usually single parameter values or a pair of

values are the cause of a failure, but increasingly smaller proportions are caused by 3-way,

4-way, and higher order interactions.

2. Because a small number of parameters are involved in failures, we can attain a high

degree of assurance by testing all

-way interactions, for an appropriate interaction strength

(2 to 6 usually). The number of

-way tests that will be required is proportional to

log

for

parameters with

values each.

3. Combinatorial methods can be applied to configurations or to input parameters, or in

some cases both.

4. As with all other types of testing, the oracle problem must be solved – i.e., for every

test input, the expected output must be determined in order to check if the application is

producing the correct result for each set of inputs. A variety of methods are available to

solve the oracle problem.

_______________________________________________________

Practical Combinatorial Testing

3 C

ONFIGURATION TESTING

This chapter presents worked examples illustrating development of test configurations. As

will be seen, the advantages of combinatorial testing increase with the size of the problem.

3.1 Simple Application Platform Example

Returning to the simple example introduced in Chapter 2, we illustrate development

of test configurations, and compare the size of test suites for various interaction strengths

versus testing all possible configurations. For the five configuration parameters, we have

⋅

3 = 72 configurations. The convention for describing the variables and values in

combinatorial testing is

... where the

are number of variable values and

are

number of occurrences. Thus this configuration is designated 2

. Note that at

= 5, the

number of tests is the same as exhaustive testing for this example, because there are only

five parameters. The savings as a percentage of exhaustive testing are good, but not that

impressive for this small example. With larger systems the savings can be enormous, as

will be seen in the next section.

Parameter Values

Operating system XP, OS X, RHL

Browser IE, Firefox

Protocol IPv4, IPv6

CPU Intel, AMD

DBMS MySQL, Sybase, Oracle

Table 2. Simple example configuration options.

We can now generate test configurations using the ACTS tool. For simplicity of

presentation we illustrate usage of the command line version of ACTS, but an intuitive GUI

version is available that may be more convenient. This tool is summarized in Appendix C

and a comprehensive user manual is included with the ACTS download.

The first step in creating test configurations is to specify the parameters and

possible values in a file for input to ACTS, as shown in Figure 5:

[System]

[Parameter]

OS (enum): XP,OS_X,RHL

Browser (enum): IE, Firefox

Protocol(enum): IPv4,IPv6

CPU (enum): Intel,AMD

DBMS (enum): MySQL,Sybase,Oracle

[Relation]

[Constraint]

[Misc]

Figure 5. Simple example input file for ACTS.

剩余80页未读，继续阅读

艾米的爸爸

粉丝: 760
资源: 314

NIST SP800-142：实用组合测试教程：提升软件安全与效率

NIST SP800-46.pdf

信息安全测试评估技术指南NIST SP 800-115

NIST SP800-14.pdf

NIST SP800-162.pdf

nist sp800-161下载

NIST SP800-90A

nist sp800-181

nist sp800-108

NIST SP800-131A

nist sp800-60

2017 nist sp800-181

NIST SP800-22r1a

sp800-53 工控

SP800-22标准 MATLA

什么是SP800-121

[nist 800-38b]

nist800-207

ASPEN物性库中NIST-RK、NIST-IG、NIST-HOC的区别

windows时间服务器地址

NIST.FIPS.202

最新资源