使用ThMAD的音频可视化：Ubuntu Linux实时图形渲染

需积分: 10 81 浏览量更新于2024-07-18 1 收藏 12.77MB PDF 举报

"Audio Visualization Using ThMAD - 实时图形渲染在Ubuntu Linux上的音频可视化技术" 本书《Audio Visualization Using ThMAD》由Peter Späth撰写，专注于使用ThMAD工具进行音频可视化，特别是针对Ubuntu Linux平台的实时图形渲染。音频可视化是一种将声音数据转化为视觉表现形式的技术，它能够帮助我们理解和探索音乐以及音频信号的内在结构。 ThMAD（The Mad Audio Display）是一个强大的工具，用于创建交互式的、基于时间的音频可视化应用。它允许开发者和艺术家以创新的方式呈现音频数据，通过视觉效果来增强听觉体验。ThMAD支持多种平台，包括Ubuntu Linux，这使得它在开源操作系统上具有广泛的应用潜力。书中内容可能涵盖以下几个关键知识点： 1. **音频基础**：首先，读者会接触到音频的基本概念，包括频率、振幅、波形和频谱分析等，这些是理解音频可视化的基础。 2. **ThMAD工具详解**：详细介绍了ThMAD的工作原理、功能特性以及如何安装和配置。可能包括它的API、库函数、脚本语言支持，以及如何使用它来创建自定义的音频可视化项目。 3. **实时图形渲染**：讲解了如何使用ThMAD实现音频数据的实时转换为动态图形，这涉及到图形处理器（GPU）的利用、帧率优化和性能调优等技术。 4. **可视化设计**：探讨了各种音频可视化的设计理念和技巧，如颜色理论、形状与动画的使用，以及如何有效地传达音频信息。 5. **编程实践**：提供了实际的代码示例和项目教程，帮助读者掌握ThMAD的编程接口，以及如何结合其他开源工具如FFmpeg或Jack进行音频处理。 6. **案例研究**：可能包含多个已完成的音频可视化项目，展示ThMAD在不同应用场景下的实际效果，如音乐制作、音频分析、艺术装置等。 7. **版权和许可信息**：书中提到版权信息，表明所有内容受到保护，但同时也指出，根据特定的许可证条款，可以对内容进行特定方式的使用和传播。通过学习《Audio Visualization Using ThMAD》，读者不仅可以深入理解音频可视化的基本原理，还能掌握使用ThMAD在Ubuntu Linux上创建动态视觉表现的高级技能，为音乐制作、音频分析和创意艺术提供新的视角和工具。

■ INTRODUCTION

xviii

Installation

The installation process of ThMAD is discussed in Chapter 2, “Visualization Basics”.

Conventions Used in This Text

Working with ThMAD involves using its modules, which are organized in a tree-like

structure.

Modules are usually named maths → converters → 4float_to_float4. Or in short with

fixed width font, 4float_to_float4 if the module position inside the module tree is clear

from the context.

State is the common notion for a rendering pipeline while constructing it. Finished

states are also called visuals. References to sample states, including associated code

provided with the installation, as well as informational hints in general, are highlighted

like so:

■ Note This sample is as a source available under A-3.2.1_Visualization_basics_

basic_samples_basic_2d_sample

inside the TheArtOfAudioVisualization folder.

With the “Note” label replaced with “Tip” for informational hints. By folder in this

context I mean a folder as showing up in the module lister or browser. Important notes

and pitfalls are marked as follows:

■ Caution Due to the backfeeding it might easily happen… …

… … …

Code and script snippets, as well as terminal input and output, usually show up in

monospace font like here:

apt-get install libc6 libfreetype6 libgcc1 libpulse0 libstdc++6 libglfw3

Very small code snippets appear directly inside the text. If a longer line does not fit

into a line, a trailing ┐ at each line of code signifies that while actually writing it, the ┐

must be removed and the subsequent line break must be discarded. For example,

echo "cmd [...] rectangle ┐

abc [...]"

should be entered as

echo "cmd [...] rectangle abc [...]"

P. Späth, Audio Visualization Using ThMAD, https://doi.org/10.1007/978-1-4842-3168-5_1

CHAPTER 1

Sound Input

In this chapter, we deepen the knowledge of how the computer can be prepared to capture

incoming sound or produced sound, how the sound is represented internally, and how the

data arrives at ThMAD. We distinguish between the obvious air pressure elongation versus

time representation, and the power versus frequency representation, or spectrum.

Preparing for Sound Input

After you purchase a PC or laptop with Ubuntu Linux, or after you have Ubuntu Linux

installed on your PC, you have basically two options—you can use external audio sources

or you can let the sound play from the computer. As for the latter, you could use the CD

player of your PC, you could play some files from a USB stick or your hard disk, or you

could stream audio files via the Internet or some other means.

Note that other Linux distributions might work as well. Give it a try—chances are

good that you’ll find similar programs, tools, and settings to accomplish the same thing.

If you want to use external audio sources, you need a microphone or a sound card with

a line-in to connect to. Especially for laptops, the built-in microphones are not of the highest

quality, but they might be enough for your purposes. You actually don’t want to accurately

reproduce the sound, but react to it, and for this aim, having perfectly linear input curves is

not too important. On the other hand, if you don’t want to lose important impulses from the

basses, which can happen with cheap microphones, getting yourself a decent microphone

might help you avoid surprises. Also, bear in mind that audio visualizations might be brittle

to the structure of the incoming sound under certain circumstances.

Usually you want to avoid that and the overall outcome should be interesting for

any kind of music input. This is easy enough to check with different recordings. But if,

for example, the basses never make their way through the audio hardware to a suitable

extent, because your microphone misses the basses, your rendering pipeline might lack

reactiveness to an important part of the incoming sound. If instead for external sound

input you just connect some audio source to the line-in jack of your computer, you are

automatically on the safe side.

If you want to play CDs using your computer’s CD player, or play audio files or

streamed audio contents, e.g., using your browser, chances are good you don’t have

to do anything but start suitable programs or let the operating system do it for you

automatically. For larger sound file collections, a program for administering them might

be handy. RhythmBox, which is preinstalled on Ubuntu, is a good option.

Chapter 1 ■ Sound Input

Time-Elongation Representation

On a diagram with the x-axis denoting the time and the y-axis denoting the pressure, the

time-elongation representation might, for a sine wave, look like Figure1-2.

In computer systems, we need a digital representation for this sine wave. The idea

is as follows: we divide the time into small time steps, say 44,100 steps per second, and

for each time step, we write down the current air pressure, or y-value, and save it inside

an array. This is sometimes called analog-to-digital conversion (ADC). Note that 44,100

is a widely adopted industry standard, for example, it’s used with music CDs. Because

we have two ears and like stereo, we do that conversion twice, for the left ear and the

right ear. By that means, we end up with 88,200 numbers, which digitally represent

one second of stereo sound. For the pressure or y-value representation, we use integer

values (-32,768 up to 32,767), with the lower value representing a negative pressure

offset, so maybe 0.997 bars, and the higher value representing a positive pressure offset,

say 1.003 bars.

According to a scaling we can freely define, these could be mapped to 0.997 bars →

y=-1000 and 1.003 bars → y=+1000. All the other numbers are mean pressures between

these values. Of course we could use number ranges other than -32,768 … 32,767 , but the

range we chose here is internally represented by exactly two bytes of data, and computers

like that very much. It is also a trade-off: fewer different values means less resolution and

poorer quality, and more different values means higher storage need.

Of course, in reality music is stored in lots of different formats, including MP3, Ogg

Vorbis and others, mainly for reasons of saving space. The 88,200 numbers per second

add up quite rapidly. But in case of letting an application like ThMAD access PulseAudio

data, it will receive the data in an uncompressed and untransformed, raw format. This is

nice, since then ThMAD doesn’t need extra logic to handle different sound formats.

Frequency-Power Representation

A practically less obvious representation of sound consists of writing down the frequency

distribution at each instant of time.

Consider the time range [10s;10.1s] when listening to some music. Instead of

reporting the air pressure amplitudes at each instant, e.g., 10.000s, 10.001s, 10.002s,

Figure 1-2. A sine wave

剩余359页未读，继续阅读

起明

粉丝: 0
资源: 4

使用ThMAD的音频可视化：Ubuntu Linux实时图形渲染

Advanced Audio Visualization Using ThMAD.

Audio Visualization Using ThMAD Realtime Graphics Rendering for Ubuntu epub

AudioVisualization2D.cs

Data Analysis and Visualization Using Python

Audio Visualization Playlist System v1.0.unitypackage

3D visualization using pulsed and CW digital holographic tomography techniques (Invited Paper)

audio-visualization .ipynb

audio-visualization::musical_note::woman_running_dark_skin_tone:audio-visualization 是一个通过 WEB-AUDIO-API 实现的实现的音乐可视化的库，通过本库你可以获取到当前音频的音频频谱

audio-visualization-shader-boilerplate

最新资源