虚拟现实与多媒体3D音效指南：NASA Ames研究中心专著

需积分: 13 112 浏览量更新于2024-07-21 收藏 1.61MB PDF 举报

《3-DSound for Virtual Reality and Multimedia》是一本经典的教材，专注于虚拟现实(Virtual Reality, VR)和多媒体技术领域。它在2000年出版，由Durand R. Begault撰写，着重于介绍如何通过3D声音设计来增强虚拟环境中的用户体验，这在当时对于提升沉浸式体验至关重要。这本书不仅探讨了声学原理在VR中的应用，还涵盖了与多媒体技术的集成，如音频与视频同步、空间定位音频等，这些都是构建身临其境虚拟世界的基石。 NASA/TM-2000-000000这一编号表明，该书是美国国家航空航天局(NASA)的研究成果，反映了其对航天科学的持续关注和支持。NASA的Scientific and Technical Information (STI) Program Office在促进这项研究方面发挥着核心作用，他们运营的数据库是全球最大的航空和太空科学科技信息资源库。通过这个平台，NASA将研究成果传播到学术界和工业界，以便于进一步的研究和创新。书中涉及的NASASTI报告系列包括了 Technical Publications，这类报告详细记录了NASA已完成的研究项目或重大研究阶段，内容深入，包含了大量的数据和理论分析。它们不仅展示了NASA在3D声音处理技术方面的最新进展，也展示了其在虚拟现实技术发展中的关键贡献。这些报告旨在推动整个行业的技术进步，尤其是在视听效果优化、用户交互设计以及虚拟世界的真实感提升方面。通过阅读《3-DSound for Virtual Reality and Multimedia》，读者可以了解到如何利用先进音频技术提升虚拟现实体验，这对于游戏开发、娱乐产业、教育仿真甚至未来可能的远程协作等领域都具有重要的参考价值。同时，它还揭示了NASA如何通过STI Program Office将科研成果转化为实际应用，为未来的太空探索和地球科学研究提供强有力的技术支持。

Figure 1.2. A simple, two-dimensional model of how a listener (L) in an enclosure will hear both

direct and indirect sound paths from a sound source (S). The distribution of the indirect

paths—reverberation—yields information for perception both about the sound source S and the

environmental context E.

perceptual information presented; Chapter 5 shows example applications; and Chapter 6 lists

resources for additional information.

Source-Medium-Receiver Model:

Natural versus Virtual Spatial Hearing

A distinction must be made between experiences heard in most everyday activities, and those

experiences heard over headphones using an audio reproduction system. Natural spatial hearing

refers to how we hear sounds spatially in everyday hearing, with our ears uncovered, our head

moving, and in interaction with other sensory input. Note that spatial auditory images are not

confined to “two-ear,” or “binaural” hearing; one-ear hearing can provide spatial cues, as proven

by experience with telephone conversations. A special case of binaural hearing is virtual spatial

hearing; this refers here to the formation of synthetic spatial acoustic imagery using a 3-D sound

system and stereo headphones. Virtual or natural spatial hearing situations can both give rise to the

percepts described in Figure 1.1.

The transmission path from operator to listener can be described according to simple, or increasingly

complex, communications theory models. These models at root involve a source, medium, and

receiver (see Figure 1.3). For a psychoacoustic experiment, the purpose of such a model is to

illustrate how manipulation of a particular independent variable of sound will give rise to changes in

spatial imagery. For a 3-D sound system, the purpose is to elucidate the difference between the

operator’s specification of a particular spatial image and the listener’s spatial imagery, as a result of

the requirements of the particular application.

While the source can involve one or more vibratory sources, it is most convenient to describe the

system in terms of a single source in isolation. Natural spatial hearing seldom involves a single

source, whereas virtual spatial hearing with a 3-D sound system involves the positioning of each of a

number of sources individually. The medium involves the paths by which the source arrives at the

listener. In natural spatial hearing, this involves the environmental context (reverberation and the

effects of physical objects on the propagation of the sound); in a 3-D sound system, sound

reproduction nonlinearities, signal processing, and headphones become the major components.

Source

Medium

Receiver

1....................

1.............. 1.....................

n n

Figure 1.3. A source-medium-receiver model. Each element contains a number of physical,

neurological, or perceptual transformations (dotted line sections numbered 1 through n) in the

communication of the spatial location of a sound source. For a 3-D sound system, the question is to

what degree the first element of the source—the specified position indicated by the operator—equals

the nth element of the receiver, the ultimate spatial perception of a virtual acoustic image.

Finally, the receiver involves the listener’s physiology: The hearing system, from the ear to the final

stages of perception by the brain.

Ultimately, it is the nonlinear transformations, or distortions that occur at various “significant way

points” along the source-medium-receiver chain (each of the sections 1–n in Figure 1.3) that are

useful to identify in both natural and synthetic spatial hearing contexts. The nonlinear

transformations within a source-medium-receiver chain are analogous to the party game

“telephone,” where a verbal message is whispered secretly in a sequential manner between several

persons—the game’s amusement value is in the difference between the original message and its result

at the last person. From an engineering standpoint, a perceptually meaningful, segmented approach

for effecting synthetic binaural hearing can result from understanding natural spatial hearing. The

nonlinearities between the various source-medium-receiver stages can be described statistically via

psychoacoustic research results related to 3-D sound. Such research attempts to describe the degree

of mismatch between the operator’s intended spatial manipulation and the listener’s perception,

based on the probability of responses from the overall population for a limited set of independent

variables.

First, take the case of natural spatial hearing. Figure 1.4 depicts a schematic transformation between

sound source and listener through a medium. The sound source consists of undulations within the

elastic medium of air resulting from the vibration of a physical object, such as vocal cords within a

larynx, or of a loudspeaker cone. If the sound source propagates in a manner irrespective of

direction, it can be said to be omnidirectional, and it therefore forms a spherical field of emission. If

the sound source is within an environmental context without reflections—e.g., an anechoic

chamber—then beyond a certain distance the sound waves arrive at the front of a listener as a plane

field, meaning that the sound pressure would be constant in any plane perpendicular to the direction

of propagation. But within nonanechoic environmental contexts, sound arrives to a listener by both

direct and indirect paths, as shown in Figure 1.2. In this case, the sound source can be said to arrive

at the listener as a diffuse field, due to the effect of the environmental context. The environmental

context is therefore the

SPATIAL

HEARING

"HIGHER LEVEL"

LOCALIZATION

SOUND SOURCE

RADIATION

INTERACTION OF

SOUND SOURCE WITH

ENVIRONMENTAL

CONTEXT

AUDITORY

SYSTEM

(OUTER,

MIDDLE,

INNER EAR)

Free head

movement

within a

room

NOISE

MEMORY

IMMEDIATE

LOCALIZATION

JUDGMENTS

character

formation

Figure 1.4. A model of natural spatial hearing.

principal component of the medium in the case of natural spatial hearing. Other sound sources within

the environmental context might be considered undesirable; these are generally classified as noise

sources. The manipulation and diminution of the noise experienced by most people is often within

the domain of architectural acoustic and community noise-control specialists; sometimes their

solutions are inconspicuous, as in a standard for noise isolation for a building wall, and other times

conspicuous, as with freeway or airport sound barriers.

Next, consider the source-medium-receiver communication chain for a hypothetical 3-D sound

system (see Figure 1.5). Assume that two-channel (stereo) headphones are used to reproduce a

sound source. In this case, an actual environmental context is absent, and it must be simulated in

most cases. The source must either be transduced from acoustic to electrical energy via a

microphone, obtained from a storage medium such as a CD or a disc drive, or produced by a tone

generator—a generic phrase for samplers, synthesizers, and the like. In addition, the receiver is no

longer moving his or her head in the same environment as the sound source. Instead, a pair of

transducers, in the form of the headphones placed against the outer ears, become the actual source

and medium. This type of interaction will of course affect an incoming sound field differently than

with natural spatial hearing. In addition, if the sound source should stay in a single location in virtual

space, then its location relative to the listener’s head position will also need to be synthesized and

updated in real-time.

The “black box” that comprises the arbitrary spatial sound processor will transform the electrical

representation of sound in some way, activating the spatial hearing mechanism’s perceptual and

cognitive aspects to work together to form a particular spatial judgment. This spatial judgment could

be one similar to natural spatial hearing as shown in Figure 1.1, or could be a spatial hearing

experience only possible with the given apparatus, and not heard within natural spatial hearing. This

contrast underlies an important difference between the phrases virtual environment and virtual

reality—no correlation to reality is necessarily assumed grammatically by the first term, and we may

certainly want to create virtual experiences that have no counterpart in reality. In fact, the phrase

“naive realism” has been used to critique virtual environment designers that purport to effect

perceptual experiences that are as real as those experienced under natural conditions.

Auditory

system and

spatial

hearing

mechanism

Electrical

representation

of sound

"Black box"

3-D audio

system

Figure 1.5. A model of virtual spatial hearing.

shoulder,

torso

reflection

head

diffraction

Figure 1.6. Highly simplified, schematic overview of the auditory system. See text for explanation

of letters. The cochlea (G) is “unrolled” from its usual snail shell–like shape.

At the listener, natural and synthetic spatial hearing experiences consist of both physical and

perceptual transformations of an incoming sound field. For the moment, the basic physical system of

the ear is reviewed through the following thumbnail sketch of the auditory system (see Figure 1.6).

Sound (A) is first transformed by the pinnae (the visible portion of the outer ear) (B) and proximate

parts of the body such as the shoulder and head. Following this are the effects of the meatus (or

“ear canal”) (C) that leads to the middle ear, which consists of the eardrum (D) and the ossicles

(the small bones popularly termed the “hammer-anvil-stirrup”) (E). Sound is transformed at the

middle ear from acoustical energy at the eardrum to mechanical energy at the ossicles; the ossicles

convert the mechanical energy into fluid pressure within the inner ear (the cochlea) (G) via motion at

the oval window (F). The fluid pressure causes frequency dependent vibration patterns of the

basilar membrane (H) within the inner ear; which causes numerous fibers protruding from auditory

hair cells to bend. These in turn activate electrical action potentials within the neurons of the

auditory system, which are combined at higher levels with information from the opposite ear

(explained more in detail in Chapter 2). These neurological processes are eventually transformed

into aural perception and cognition, including the perception of spatial attributes of a sound resulting

from both monaural and binaural listening.

Application Types

Within the applications of virtual reality and multimedia lies a wealth of confusing or even

contradictory terminology that is inherent to any young technology. Virtual reality systems would

be better described as virtual environment systems, since reality is often difficult to simulate, and it’s

usually better to simulate something other than “reality.” Multimedia systems on the other hand

would be better described specifically as what kinds of media are involved in the particular

application. Both fields are similar in that they are at root multiperceptual, real-time interactive

communication media.

Like all “buzzword” concepts that engender great potential financial and scientific value, virtual

reality and multimedia are represented by a seemingly endless number of forms, applications, and

implementations. How one actually defines either area or distinguishes between them is in itself an

endless source of conferences, articles, and speculation. As time progresses, the tools and concepts of

virtual reality and multimedia increasingly overlap. Pimentel and Teixeira (1992) distinguish

between multimedia and virtual reality by using a description given by author Sandra Morris:

“ . . . virtual reality involves the creation of something new, while multimedia is about bringing old

media forms together to the computer.” The distinction is fairly useless, though, if for no other

reason than “old” media quickly results from “new” media. Hardware from virtual reality can be

integrated into a multimedia system, and virtual reality often involves older media forms.

Virtual reality and multimedia systems can be described in terms of the diagrammatic representation

shown in Figure 1.7. A hardware component definition of a virtual reality system usually involves

the following elements:

1. One or more host computers for scenario management, termed a reality engine by Teixeira and

Pimentel, that contain algorithms useful for organizing and creating visual, auditory, and haptic

illusions for one or more users according to a particular scenario (popularly termed a virtual

world).

2. Effectors, hardware devices such as helmet-mounted displays, headphones and force-feedback

devices that cause the illusion at the user to occur in response to data contained within the reality

engine.

3. Sensors, hardware such as magnetic or mechanical position trackers, six- degrees-of-freedom

trackballs, speech analyzers, and data gloves.

The reality engine can process information to and from multiple effectors and sensors for one or

several persons or machines. When controlling a machine such as a robot, tool, or vehicle-mounted

camera, the word telerobotics is used.

In contrast to a virtual reality’s system of effectors, sensors, and reality engine database, a multimedia

system can be defined simply as any actual or near real-time interactive software interface that

includes multiple effectors, including the addition of “high-quality” sound (i.e., something more

sophisticated than system “beeps”). The sound component is easily available on plug-in sound

cards, such as Creative Labs’ SoundBlaster

The multimedia system can also include advanced

graphics, animation, and/or video-based visual interfaces, and possibly advanced types of sensors such

as speech recognition. Desktop computer systems such as the Apple Macintosh

Quadra

series or the

Silicon Graphics Indy

combine high-speed graphic chips for interactive rendering of screen images

and simple but useful software for including “CD-quality” sound with images; equivalent systems

can also be assembled “from scratch” by adding sound and graphics accelerator cards to an existing

personal computer.

剩余245页未读，继续阅读

toxyz

粉丝: 5
资源: 6

虚拟现实与多媒体3D音效指南：NASA Ames研究中心专著

jdk-15.0.1_windows-x64_bin.exe + openjfx-15.0.1_windows-x64_bin-sdk

OMA-TS-MMS-CTR-V1_3-20080128-C.pdf

使用gcc-arm-10.2-2020.11-x86_64-aarch64-none-elf.tar.xz交叉编译qt4.8.7流程

D:\Qt_Project\build-Qt_myMedia-Desktop_Qt_5_14_2_MinGW_64_bit-Debug\debug\main.o:-1: In function `qMain(int, char**)':

将QT5.8图形应用程序移植到S5PV210开发板上需要怎么做，详细告诉我

libclang-dev 已安装 但是QT 编译 仍然提示 WARNING: QDoc will not be compiled, probably because libclang could not be located. This means that you cannot build the Qt documentation

mulit media

Microsoft DirectX SDK

最新资源

libclang-dev 已安装但是QT 编译仍然提示 WARNING: QDoc will not be compiled, probably because libclang could not be located. This means that you cannot build the Qt documentation