Technical Information
You can read this page just for fun. You might find something interesting in this page if you are considering writing a review or something about this app.
Table of Contents
This audio effect uses a method which is based on autoregressive modeling for both the noise detection part and repair part of its processing. This method requires much calculation to solve sets of linear algebraic equations which contain many unknowns per set (e.g. 44 unknowns when creating a 1-millisecond detection filter at 44.1 kHz).
When you solve a set of linear equations numerically using a computer, you usually do it in the form of matrix. A set of N linear equations can be written using an NxN square matrix and an N-length vector. Once giving the equation set a matrix representation, you can use the LU decomposition technique or some other to solve it.
The tasks that use matrix math include filter creation for noise detection and patch sequence creation for repair. A process of transient noise removal that involves a heavy use of large matrices takes time to finish compared to other simpler approaches but it can conceal sound damages well.
The effector divides the given sound into short segments and processes each separately in a frame. Here is a rough description of the process for a frame:
- The noise detection filter is an FIR filter. We have to determine the set of coefficients which defines the filter.
- The coefficients are determined so that the filter will pass the frequencies that spread throughout the frame well and the others poorly.
- Once the values of the coefficients have been determined, we calculate the error signal, the difference between the sound signal and its filtered copy.
- If the sound signal contains a transient noise somewhere in it, the error signal rises high at that location, which indicates the presence of a transient noise or something irregular.
- We patch that irregular part with an alternative sequence.
- The patch sequence is determined so that it will mainly consist of the frequencies which the filter passes well.
This is an FFT-based audio effect. Audio editors would often have an audio effect like this. There might be some variants.
The basic idea is the following:
- We prepare a noise floor which defines the power level of each noise component in the frequency domain.
- The given sound is divided into short segments, each processed separately in a frame.
- An FFT is performed on each frame to get the frequency-domain representation.
- For each of the sound frequencies in a frame, if the frequency is lower than the noise floor, we take it as a noise component and remove it. If it is obviously higher than the noise floor, we take it as a sound component and keep it untouched. If it is not so high, we reduce its power by an appropriate degree.
- An inverse FFT is performed on each frame to take it back to the time domain.
- Successive two frames are reunited with a cross fade to make the output waveform smooth without discontinuity.
Because it is difficult for the user to make a well-defined noise floor by hand, the effector takes a time-domain noise sequence and makes an estimation for noise floor in a statistical way.
This is a kind of convolution reverb which uses FIR room impulse responses.
The execution of this reverb consists of three stages of processing.
First, using the virtual room setting the user has set, paths of sound reflections physically possible in the room are collected.
- There is one sound source and one stereo microphone set in the room. There is nothing else in it.
- A simulation is performed to find and collect valid paths. It obeys the physical law of sound reflection. This path search uses a kind of beam tracing method.
- The simulation is done in 3D space because, 2D simulation is easier though, 2D rooms cannot yield an enough number of echoes for realistic reverberation.
- Finding a valid sound reflection path equals locating the corresponding virtual sound source which has a valid path to the receiver. The path from a virtual sound source to the receiver is always a straight line, so that it can be easily translated to the arrival delay and amplitude of the echo emitted from there.
- A virtual sound source is located behind a reflection surface, at the mirrored location of its source. The source of a virtual sound source is either the original one supposed to physically exist or the virtual one already got located in the ongoing recursive search. What is actually done in the simulation is to locate valid virtual sound sources and record their locations.
Second, a pair of room impulse responses is created based on the result of the reflection simulation.
- The output sound of this reverb is always stereo, so two room impulse responses are created, one for a channel.
- A room impulse response is an aggregation of elemental impulse responses. In the case of this reverb, the elements include the direct sound, every echo collected via the reflection simulation, and nothing else.
- The echoes are rendered in a pair of memory buffers. They simply become the left and right room impulse responses once every element has been rendered.
- Each echo is rendered in the two buffers as accurately as possible at its arrival time position, given an appropriate amplitude. The arrival time position of an echo for one channel is often slightly different from that for the other channel, since the left and right reception points of the microphone set are not at the same location. The ears take this arrival time difference as a direction cue for the echo. To achieve the highest possible accuracy, the arrival time position of an echo is not rounded off to multiples of the sampling interval. Rendering in this way also keeps a room impulse response from taking a sampling-rate specific shape.
- The frequency response of an echo is determined using the echo color setting and the reflection count of that echo.
- The direct sound is rendered in the same way as echoes.
- The direct sound and all of its echoes are rendered in the most accurate way possible for both channels. This naturally means that their aggregating entity has the same level of accuracy thus coherency between channels.
Finally, the convolver convolves the input sound with the room impulse responses, producing an output in stereo.
- The input sound has to be mono to get the intended output. You have to use headphones to hear the intended coherency.
- Convolution is performed in the form of frequency-domain multiplication using FFT for fast calculation.
As an option, data-parallel processing is possible, which takes advantage of multi-core CPUs and can speed up the following tasks:
- Sound reflection simulation
- Creation of room impulse responses
- Convolution with room impulse responses
As an option, GPU-accelerated convolution is also possible.
- This option is only for 32-bit floating point convolution. This is because GPUs that ordinary computers are equipped with do not perform well with 64-bit floating point numbers.
- The GPU convolver can use up to 2048 GPU threads (shaders) concurrently.
The advantages of this FIR reverb compared to typical IIR reverbs include:
- It can produce a stereo reverberation effect which has accurately calculated binaural coherency, while IIR reverbs can only produce a pseudo stereo effect and cannot offer the same level of reality.
- Arranging echoes in a simulation-derived formation ensures smooth transition from early reflections to late reverberation, in addition to coherency between channels.
The disadvantages of this FIR reverb compared to typical IIR reverbs include:
- It requires much execution time.
- Because FIR impulse responses are always of finite length, abrupt terminations of reverberation can be heard if an enough number of distant echoes have not been gathered at the simulation stage. (In that case, you need to set the Echo Density setting to a higher value in the Reverb dialog box.)
A time-scaling procedure that processes complex polyphonic sound, or musical sound, must separately manipulate each of the frequency components of the given sound in order to prevent the pitch from changing and to retain the quality. This can be done by using the phase vocoder technique which uses FFT to manipulate sound in the frequency domain.
Note that there is another popular time-scaling technique, which might be called as the SOLA method. It simply arranges short segments of sound in the time domain. It works well with speech, but not with music.
You can get the knowledge of the phase vocoder technique online or somewhere, but in reality, you cannot make a time-scaling procedure that really works well with music by using only that technique alone. You'll need some complementary techniques of your own.
To achieve a good level of perceived similarity to the original sound, you have to preserve the following kinds of phase relations as accurately as possible:
- The phase relation of every one toward the others among the frequencies that form together each sound onset
- The phase relation of each frequency component of the sound appearing for one channel toward the same component appearing for the other
Using other expressions which might be more intuitive:
- The time-domain shape of each sound onset has to be preserved.
- The sound's stereo image has to be preserved.
Note that a sound onset is the very early part of a newly appearing sound or note. A new sound onset appears, for example, every time a pianist presses down a key, a guitarist picks a string, or a drummer beats a drum.
The author has developed his own techniques for preservation of these kinds of phase relations. The author believes these tweaks make his phase vocoder somewhat different from others.
In addition to the use in playback, you can have the app put time-scaling results on the disk by using the Pitch/Tempo audio effect.
Last Update 2026/02/25