SDL3 audio subsystem wishlist #6632
Assignees
Milestone
Comments
Originally posted by @slouken in #3519 (comment)
Originally posted by @flibitijibibo in #3519 (comment)
Originally posted by @flibitijibibo in #3519 (comment) |
This was referenced Jan 5, 2023
Merged
|
I've been working on ripping up the audio API for SDL3. This is the current state. I'm not sure how readable a diff is here, but this is still changing a lot and not ready for a pull request. Right now we're moving from opening a device to creating a "flow" (didn't want to use "sink" because it might be going either direction or "stream" because we already have SDL_AudioStream). Flows work like a device did in SDL2; they have a callback, etc, but there might be multiple flows, so you can have several unrelated things feeding into a single device. A large part of me wants to eliminate audio callbacks (and audio threads!) entirely and just say "you can attach multiple SDL_AudioStreams to hardware devices, just queue a little more audio every frame and we'll figure it all out," but that might be heretical. SDL_AudioSpec is still in there, but I'm looking to remove that, too. Most of it is gone, there's just some old tidbits still there I haven't cleaned out yet. I've removed the idea that you can open an arbitrary device name; there are only device IDs. For things that want a network address or whatever, we'll have the app open the default device and use an SDL hint to specify it in a backend-specific way. In practice, this flexibility wasn't actually useful, I suspect, and it was backend-specific magic anyhow. Feedback is welcome! This diff is both huge and obsolete, so I've hidden it behind a `details` tag. Click to expand it.diff --git a/include/SDL3/SDL_audio.h b/include/SDL3/SDL_audio.h
index 3f6262559..82ce3e5ef 100644
--- a/include/SDL3/SDL_audio.h
+++ b/include/SDL3/SDL_audio.h
@@ -27,6 +27,42 @@
* Access to the raw audio mixing buffer for the SDL library.
*/
+/**
+ * New SDL3 stuff:
+ *
+ * SDL_AudioSpec::freq changed from `int` to `Uint32`.
+ *
+ * AUDIO_U16 is gone. It wasn't super-popular, I assume, and you can't
+ * memset it to silence with a single byte value.
+ *
+ * Audio devices follow the new SDL3 convention of being individual IDs,
+ * and that list of IDs can be atomically requested; no more APIs that
+ * provide an index into a potentially-unstable list.
+ *
+ * To use the audio hardware in SDL3, you use a "flow" to play or
+ * record audio. Conceptually, this mostly works like SDL2 did, except
+ * instead of opening a device to play audio, you create a flow on a device
+ * and then use a callback or SDL_QueueAudio, like before, except on a
+ * specific flow instead of a specific device.
+ *
+ * The difference is that you can have as many flows on a single physical
+ * device as you like, which means different pieces of code don't have to
+ * worry about interfering with each other, or the system not allowing
+ * multiple device opens. This can also be useful for implementing a basic
+ * mixer.
+ *
+ * Flows can be in any format, they maintain an SDL_AudioStream behind the
+ * scenes to convert between its input and output. As such, there is no
+ * longer a "desired/obtained" format interface. The device is configured
+ * to match the first flow created on it (or an app can optionally
+ * set a default in case they need more control than that), and if it
+ * couldn't be set to the exact format, the AudioStream will handle the
+ * differences.
+ *
+ * Capture devices may also have multiple flows: each flow gets its own
+ * copy of any incoming audio.
+ */
+
#ifndef SDL_audio_h_
#define SDL_audio_h_
@@ -90,11 +126,8 @@ typedef Uint16 SDL_AudioFormat;
/* @{ */
#define AUDIO_U8 0x0008 /**< Unsigned 8-bit samples */
#define AUDIO_S8 0x8008 /**< Signed 8-bit samples */
-#define AUDIO_U16LSB 0x0010 /**< Unsigned 16-bit samples */
#define AUDIO_S16LSB 0x8010 /**< Signed 16-bit samples */
-#define AUDIO_U16MSB 0x1010 /**< As above, but big-endian byte order */
#define AUDIO_S16MSB 0x9010 /**< As above, but big-endian byte order */
-#define AUDIO_U16 AUDIO_U16LSB
#define AUDIO_S16 AUDIO_S16LSB
/* @} */
@@ -121,31 +154,16 @@ typedef Uint16 SDL_AudioFormat;
*/
/* @{ */
#if SDL_BYTEORDER == SDL_LIL_ENDIAN
-#define AUDIO_U16SYS AUDIO_U16LSB
#define AUDIO_S16SYS AUDIO_S16LSB
#define AUDIO_S32SYS AUDIO_S32LSB
#define AUDIO_F32SYS AUDIO_F32LSB
#else
-#define AUDIO_U16SYS AUDIO_U16MSB
#define AUDIO_S16SYS AUDIO_S16MSB
#define AUDIO_S32SYS AUDIO_S32MSB
#define AUDIO_F32SYS AUDIO_F32MSB
#endif
/* @} */
-/**
- * \name Allow change flags
- *
- * Which audio format changes are allowed when opening a device.
- */
-/* @{ */
-#define SDL_AUDIO_ALLOW_FREQUENCY_CHANGE 0x00000001
-#define SDL_AUDIO_ALLOW_FORMAT_CHANGE 0x00000002
-#define SDL_AUDIO_ALLOW_CHANNELS_CHANGE 0x00000004
-#define SDL_AUDIO_ALLOW_SAMPLES_CHANGE 0x00000008
-#define SDL_AUDIO_ALLOW_ANY_CHANGE (SDL_AUDIO_ALLOW_FREQUENCY_CHANGE|SDL_AUDIO_ALLOW_FORMAT_CHANGE|SDL_AUDIO_ALLOW_CHANNELS_CHANGE|SDL_AUDIO_ALLOW_SAMPLES_CHANGE)
-/* @} */
-
/* @} *//* Audio flags */
/**
@@ -179,7 +197,7 @@ typedef void (SDLCALL * SDL_AudioCallback) (void *userdata, Uint8 * stream,
*/
typedef struct SDL_AudioSpec
{
- int freq; /**< DSP frequency -- samples per second */
+ Uint32 freq; /**< DSP frequency -- samples per second */
SDL_AudioFormat format; /**< Audio data format */
Uint8 channels; /**< Number of channels: 1 mono, 2 stereo */
Uint8 silence; /**< Audio buffer silence value (calculated) */
@@ -262,97 +280,111 @@ extern DECLSPEC const char *SDLCALL SDL_GetAudioDriver(int index);
extern DECLSPEC const char *SDLCALL SDL_GetCurrentAudioDriver(void);
/**
- * SDL Audio Device IDs.
+ * SDL audio device IDs.
+ *
+ * This is a unique ID for a physical audio device for the time it is
+ * connected to the system, and is never reused for the lifetime of the
+ * application. While some devices (like the audio chip on a computer's
+ * motherboard), are always present, others (like USB headphones) may come
+ * and go at runtime. If the device is disconnected and reconnected, it
+ * will get a new ID.
+ *
+ * The ID value starts at 1 and increments from there. The value 0 is an
+ * invalid ID.
*/
typedef Uint32 SDL_AudioDeviceID;
+
/**
- * Get the number of built-in audio devices.
+ * Get a list of currently connected audio output devices.
*
* This function is only valid after successfully initializing the audio
- * subsystem.
+ * subsystem.
*
- * Note that audio capture support is not implemented as of SDL 2.0.4, so the
- * `iscapture` parameter is for future expansion and should always be zero for
- * now.
+ * This returns a list of devices that can produce sound ("output"). It
+ * won't include devices that can record audio ("capture"). If one piece of
+ * physical hardware is capable of both, it will be split into two logical
+ * SDL_AudioDeviceIDs, and only the one type is returned here.
*
- * This function will return -1 if an explicit list of devices can't be
- * determined. Returning -1 is not an error. For example, if SDL is set up to
- * talk to a remote audio server, it can't list every one available on the
- * Internet, but it will still allow a specific host to be specified in
- * SDL_OpenAudioDevice().
+ * \param count a pointer filled in with the number of devices returned
+ * \returns a 0 terminated array of audio device instance IDs which should be
+ * freed with SDL_free(), or NULL on error; call SDL_GetError() for
+ * more details.
*
- * In many common cases, when this function returns a value <= 0, it can still
- * successfully open the default device (NULL for first argument of
- * SDL_OpenAudioDevice()).
+ * \threadsafety This function is safe to call from any thread.
*
- * This function may trigger a complete redetect of available hardware. It
- * should not be called for each iteration of a loop, but rather once at the
- * start of a loop:
+ * \since This function is available since SDL 3.0.0.
*
- * ```c
- * // Don't do this:
- * for (int i = 0; i < SDL_GetNumAudioDevices(0); i++)
+ * \sa SDL_OpenAudioDevice
+ * \sa SDL_GetAudioCaptureDevices
+ */
+extern DECLSPEC SDL_AudioDeviceID *SDLCALL SDL_GetAudioOutputDevices(int *count);
+
+/**
+ * Get a list of currently connected audio capture devices.
*
- * // do this instead:
- * const int count = SDL_GetNumAudioDevices(0);
- * for (int i = 0; i < count; ++i) { do_something_here(); }
- * ```
+ * This returns a list of devices that can record audio ("capture"). It
+ * won't include devices that can produce sound ("output"). If one piece of
+ * physical hardware is capable of both, it will be split into two logical
+ * SDL_AudioDeviceIDs, and only the one type is returned here.
+ *
+ * \param count a pointer filled in with the number of devices returned
+ * \returns a 0 terminated array of audio device instance IDs which should be
+ * freed with SDL_free(), or NULL on error; call SDL_GetError() for
+ * more details.
*
- * \param iscapture zero to request playback devices, non-zero to request
- * recording devices
- * \returns the number of available devices exposed by the current driver or
- * -1 if an explicit list of devices can't be determined. A return
- * value of -1 does not necessarily mean an error condition.
+ * \threadsafety This function is safe to call from any thread.
*
* \since This function is available since SDL 3.0.0.
*
- * \sa SDL_GetAudioDeviceName
* \sa SDL_OpenAudioDevice
+ * \sa SDL_GetAudioOutputDevices
*/
-extern DECLSPEC int SDLCALL SDL_GetNumAudioDevices(int iscapture);
+extern DECLSPEC SDL_AudioDeviceID *SDLCALL SDL_GetAudioCaptureDevices(int *count);
+
/**
* Get the human-readable name of a specific audio device.
*
- * This function is only valid after successfully initializing the audio
- * subsystem. The values returned by this function reflect the latest call to
- * SDL_GetNumAudioDevices(); re-call that function to redetect available
- * hardware.
+ * Querying device zero will query the "default" device, which might not
+ * be a physical device with a real SDL_AudioDeviceID, depending on how
+ * a given platform handles this; as such, the name might be extremely
+ * generic and unlocalized.
*
- * The string returned by this function is UTF-8 encoded, read-only, and
- * managed internally. You are not to free it. If you need to keep the string
- * for any length of time, you should make your own copy of it, as it will be
- * invalid next time any of several other SDL functions are called.
+ * The string returned by this function is UTF-8 encoded, and you should free
+ * it with SDL_free() once you are done with it.
*
- * \param index the index of the audio device; valid values range from 0 to
- * SDL_GetNumAudioDevices() - 1
- * \param iscapture non-zero to query the list of recording devices, zero to
- * query the list of output devices.
- * \returns the name of the audio device at the requested index, or NULL on
- * error.
+ * Note that SDL2 guaranteed that device names were unique, appending numbers to
+ * the end of them, since these strings were used to open the audio device. This
+ * is no longer the case in SDL3, so it's possible to end up with multiple devices
+ * named "SoundBlaster Pro" if there are several of them attached to the system.
+ *
+ * \param dev The device ID to query, or zero for the default device.
+ * \returns the name of the audio device, or NULL on error.
*
* \since This function is available since SDL 3.0.0.
*
- * \sa SDL_GetNumAudioDevices
+ * \sa SDL_GetAudioOutputDevices
+ * \sa SDL_GetAudioCaptureDevices
* \sa SDL_GetDefaultAudioInfo
*/
-extern DECLSPEC const char *SDLCALL SDL_GetAudioDeviceName(int index,
- int iscapture);
+extern DECLSPEC char *SDLCALL SDL_GetAudioDeviceName(SDL_AudioDeviceID dev);
+
/**
* Get the preferred audio format of a specific audio device.
*
* This function is only valid after a successfully initializing the audio
- * subsystem. The values returned by this function reflect the latest call to
- * SDL_GetNumAudioDevices(); re-call that function to redetect available
- * hardware.
+ * subsystem.
+ *
+ * Querying device zero will query the "default" device, which might not
+ * be a physical device with a real SDL_AudioDeviceID, depending on how
+ * a given platform handles this.
*
* `spec` will be filled with the sample rate, sample format, and channel
* count.
*
- * \param index the index of the audio device; valid values range from 0 to
- * SDL_GetNumAudioDevices() - 1
+ * \param dev the audio device ID to query. Zero queries the default device.
* \param iscapture non-zero to query the list of recording devices, zero to
* query the list of output devices.
* \param spec The SDL_AudioSpec to be initialized by this function.
@@ -364,140 +396,65 @@ extern DECLSPEC const char *SDLCALL SDL_GetAudioDeviceName(int index,
* \sa SDL_GetNumAudioDevices
* \sa SDL_GetDefaultAudioInfo
*/
-extern DECLSPEC int SDLCALL SDL_GetAudioDeviceSpec(int index,
- int iscapture,
+extern DECLSPEC int SDLCALL SDL_GetAudioDeviceSpec(SDL_AudioDeviceID dev,
SDL_AudioSpec *spec);
/**
- * Get the name and preferred format of the default audio device.
+ * SDL audio flows.
*
- * Some (but not all!) platforms have an isolated mechanism to get information
- * about the "default" device. This can actually be a completely different
- * device that's not in the list you get from SDL_GetAudioDeviceSpec(). It can
- * even be a network address! (This is discussed in SDL_OpenAudioDevice().)
+ * To access audio hardware, you create a "flow" on it, which is the
+ * mechanism through which audio data literally flows. You can have
+ * as many flows on a device as you want, and SDL will mix all of them
+ * together to feed to the audio device on your behalf.
*
- * As a result, this call is not guaranteed to be performant, as it can query
- * the sound server directly every time, unlike the other query functions. You
- * should call this function sparingly!
- *
- * `spec` will be filled with the sample rate, sample format, and channel
- * count, if a default device exists on the system. If `name` is provided,
- * will be filled with either a dynamically-allocated UTF-8 string or NULL.
- *
- * \param name A pointer to be filled with the name of the default device (can
- * be NULL). Please call SDL_free() when you are done with this
- * pointer!
- * \param spec The SDL_AudioSpec to be initialized by this function.
- * \param iscapture non-zero to query the default recording device, zero to
- * query the default output device.
- * \returns 0 on success or a negative error code on failure; call
- * SDL_GetError() for more information.
- *
- * \since This function is available since SDL 3.0.0.
- *
- * \sa SDL_GetAudioDeviceName
- * \sa SDL_GetAudioDeviceSpec
- * \sa SDL_OpenAudioDevice
+ * Audio can flow in different directions. You use a flow to feed data
+ * to the device, but you also use them to receive recorded audio
+ * from a capture device.
*/
-extern DECLSPEC int SDLCALL SDL_GetDefaultAudioInfo(char **name,
- SDL_AudioSpec *spec,
- int iscapture);
-
+typedef struct SDL_AudioFlow SDL_AudioFlow; /* opaque data */
/**
- * Open a specific audio device.
+ * Create a flow on a specific audio device.
*
- * Passing in a `device` name of NULL requests the most reasonable default.
- * The `device` name is a UTF-8 string reported by SDL_GetAudioDeviceName(),
- * but some drivers allow arbitrary and driver-specific strings, such as a
- * hostname/IP address for a remote audio server, or a filename in the
- * diskaudio driver.
+ * Passing in an SDL_AudioDeviceID of 0 requests the most reasonable default.
+ * This might not result in a specific physical device, as the system
+ * might decide to migrate between devices as appropriate (user changed the
+ * system default, plugged in headphones, etc). In many cases, requesting the
+ * default device is what you want.
*
- * An opened audio device starts out paused, and should be enabled for playing
- * by calling SDL_PlayAudioDevice(devid) when you are ready for your audio
+ * A created flow starts out paused, and should be enabled for playing
+ * by calling SDL_PlayAudioFlow() when you are ready for your audio
* callback function to be called. Since the audio driver may modify the
* requested size of the audio buffer, you should allocate any local mixing
- * buffers after you open the audio device.
+ * buffers after you create the flow.
*
* The audio callback runs in a separate thread in most cases; you can prevent
* race conditions between your callback and other threads without fully
- * pausing playback with SDL_LockAudioDevice(). For more information about the
+ * pausing playback with SDL_LockAudioFlow(). For more information about the
* callback, see SDL_AudioSpec.
*
- * Managing the audio spec via 'desired' and 'obtained':
- *
- * When filling in the desired audio spec structure:
- *
- * - `desired->freq` should be the frequency in sample-frames-per-second (Hz).
- * - `desired->format` should be the audio format (`AUDIO_S16SYS`, etc).
- * - `desired->samples` is the desired size of the audio buffer, in _sample
- * frames_ (with stereo output, two samples--left and right--would make a
- * single sample frame). This number should be a power of two, and may be
- * adjusted by the audio driver to a value more suitable for the hardware.
- * Good values seem to range between 512 and 8096 inclusive, depending on
- * the application and CPU speed. Smaller values reduce latency, but can
- * lead to underflow if the application is doing heavy processing and cannot
- * fill the audio buffer in time. Note that the number of sample frames is
- * directly related to time by the following formula: `ms =
- * (sampleframes*1000)/freq`
- * - `desired->size` is the size in _bytes_ of the audio buffer, and is
- * calculated by SDL_OpenAudioDevice(). You don't initialize this.
- * - `desired->silence` is the value used to set the buffer to silence, and is
- * calculated by SDL_OpenAudioDevice(). You don't initialize this.
- * - `desired->callback` should be set to a function that will be called when
- * the audio device is ready for more data. It is passed a pointer to the
- * audio buffer, and the length in bytes of the audio buffer. This function
- * usually runs in a separate thread, and so you should protect data
- * structures that it accesses by calling SDL_LockAudioDevice() and
- * SDL_UnlockAudioDevice() in your code. Alternately, you may pass a NULL
- * pointer here, and call SDL_QueueAudio() with some frequency, to queue
- * more audio samples to be played (or for capture devices, call
- * SDL_DequeueAudio() with some frequency, to obtain audio samples).
- * - `desired->userdata` is passed as the first parameter to your callback
- * function. If you passed a NULL callback, this value is ignored.
- *
- * `allowed_changes` can have the following flags OR'd together:
- *
- * - `SDL_AUDIO_ALLOW_FREQUENCY_CHANGE`
- * - `SDL_AUDIO_ALLOW_FORMAT_CHANGE`
- * - `SDL_AUDIO_ALLOW_CHANNELS_CHANGE`
- * - `SDL_AUDIO_ALLOW_SAMPLES_CHANGE`
- * - `SDL_AUDIO_ALLOW_ANY_CHANGE`
- *
- * These flags specify how SDL should behave when a device cannot offer a
- * specific feature. If the application requests a feature that the hardware
- * doesn't offer, SDL will always try to get the closest equivalent.
- *
- * For example, if you ask for float32 audio format, but the sound card only
- * supports int16, SDL will set the hardware to int16. If you had set
- * SDL_AUDIO_ALLOW_FORMAT_CHANGE, SDL will change the format in the `obtained`
- * structure. If that flag was *not* set, SDL will prepare to convert your
- * callback's float32 audio to int16 before feeding it to the hardware and
- * will keep the originally requested format in the `obtained` structure.
- *
- * The resulting audio specs, varying depending on hardware and on what
- * changes were allowed, will then be written back to `obtained`.
- *
- * If your application can only handle one specific data format, pass a zero
- * for `allowed_changes` and let SDL transparently handle any differences.
- *
- * \param device a UTF-8 string reported by SDL_GetAudioDeviceName() or a
- * driver-specific name as appropriate. NULL requests the most
- * reasonable default device.
- * \param iscapture non-zero to specify a device should be opened for
- * recording, not playback
- * \param desired an SDL_AudioSpec structure representing the desired output
- * format
- * \param obtained an SDL_AudioSpec structure filled in with the actual output
- * format
- * \param allowed_changes 0, or one or more flags OR'd together
- * \returns a valid device ID that is > 0 on success or 0 on failure; call
+ * `callback` should be set to a function that will be called when
+ * the audio device is ready for more data. It is passed a pointer to the
+ * audio buffer, and the length in bytes of the audio buffer. This function
+ * usually runs in a separate thread, and so you should protect data
+ * structures that it accesses by calling SDL_LockAudioFlow() and
+ * SDL_UnlockAudioFlow() in your code. Alternately, you may pass a NULL
+ * pointer here, and call SDL_QueueAudio() with some frequency, to queue
+ * more audio samples to be played (or for capture flows, call
+ * SDL_DequeueAudio() with some frequency, to obtain audio samples).
+ * `userdata` is passed as the first parameter to your callback
+ * function. If you passed a NULL callback, this value is ignored.
+ *
+ * \param device A device ID of a specific physical hardware device, or 0 for
+ * the most reasonable default.
+ * \param fmt the desired audio format (`AUDIO_S16SYS`, etc).
+ * \param channels the desired number of channels (1 == mono, 2 == stereo, etc)
+ * \param freq the desired frequency in sample-frames-per-second (Hz).
+ *
+ * \returns a new SDL_AudioFlow on success or NULL on failure; call
* SDL_GetError() for more information.
*
- * For compatibility with SDL 1.2, this will never return 1, since
- * SDL reserves that ID for the legacy SDL_OpenAudio() function.
- *
* \since This function is available since SDL 3.0.0.
*
* \sa SDL_CloseAudioDevice
@@ -507,12 +464,7 @@ extern DECLSPEC int SDLCALL SDL_GetDefaultAudioInfo(char **name,
* \sa SDL_PauseAudioDevice
* \sa SDL_UnlockAudioDevice
*/
-extern DECLSPEC SDL_AudioDeviceID SDLCALL SDL_OpenAudioDevice(
- const char *device,
- int iscapture,
- const SDL_AudioSpec *desired,
- SDL_AudioSpec *obtained,
- int allowed_changes);
+extern DECLSPEC SDL_AudioFlow SDLCALL SDL_CreateAudioFlow(SDL_AudioDeviceID device, SDL_AudioFormat fmt, Uint8 channels, Uint32 freq, SDL_AudioCallback callback, void *userdata);
@@ -521,31 +473,40 @@ extern DECLSPEC SDL_AudioDeviceID SDLCALL SDL_OpenAudioDevice(
*
* Get the current audio state.
*/
-/* @{ */
typedef enum
{
+ SDL_AUDIO_INVALID = -1,
SDL_AUDIO_STOPPED = 0,
SDL_AUDIO_PLAYING,
SDL_AUDIO_PAUSED
} SDL_AudioStatus;
/**
- * Use this function to get the current audio state of an audio device.
+ * Use this function to get the current audio state of an audio flow.
*
- * \param dev the ID of an audio device previously opened with
- * SDL_OpenAudioDevice()
- * \returns the SDL_AudioStatus of the specified audio device.
+ * \param flow the SDL_AudioFlow to query.
+ * \returns the SDL_AudioStatus of the specified audio flow.
*
* \since This function is available since SDL 3.0.0.
*
- * \sa SDL_PlayAudioDevice
- * \sa SDL_PauseAudioDevice
+ * \sa SDL_PlayAudioFlow
+ * \sa SDL_PauseAudioFlow
+ */
+extern DECLSPEC SDL_AudioStatus SDLCALL SDL_GetAudioFlowStatus(SDL_AudioFlow *flow);
+
+/**
+ * Get the audio device that a flow was created on.
+ *
+ * \param flow the flow to query
+ * \returns the audio device ID associated with the flow
+ *
+ * \threadsafety This function is safe to call at any time from any thread.
*/
-extern DECLSPEC SDL_AudioStatus SDLCALL SDL_GetAudioDeviceStatus(SDL_AudioDeviceID dev);
-/* @} *//* Audio State */
+extern DECLSPEC SDL_AudioDeviceID SDLCALL SDL_GetAudioFlowDevice(SDL_AudioFlow *flow);
+
/**
- * Use this function to play audio on a specified device.
+ * Use this function to play audio on a specified flow.
*
* Newly-opened audio devices start in the paused state, so you must call this
* function after opening the specified audio device to start playing sound.
@@ -564,8 +525,7 @@ extern DECLSPEC SDL_AudioStatus SDLCALL SDL_GetAudioDeviceStatus(SDL_AudioDevice
* \sa SDL_LockAudioDevice
* \sa SDL_PauseAudioDevice
*/
-extern DECLSPEC int SDLCALL SDL_PlayAudioDevice(SDL_AudioDeviceID dev);
-
+extern DECLSPEC int SDLCALL SDL_PlayAudioFlow(SDL_AudioFlow *flow)
/**
@@ -589,7 +549,7 @@ extern DECLSPEC int SDLCALL SDL_PlayAudioDevice(SDL_AudioDeviceID dev);
* \sa SDL_LockAudioDevice
* \sa SDL_PlayAudioDevice
*/
-extern DECLSPEC int SDLCALL SDL_PauseAudioDevice(SDL_AudioDeviceID dev);
+extern DECLSPEC int SDLCALL SDL_PauseAudioFlow(SDL_AudioFlow *flow)
/**
@@ -881,13 +841,13 @@ extern DECLSPEC int SDLCALL SDL_MixAudioFormat(Uint8 * dst,
Uint32 len, int volume);
/**
- * Queue more audio on non-callback devices.
+ * Queue more audio on non-callback flows.
*
* If you are looking to retrieve queued audio from a non-callback capture
- * device, you want SDL_DequeueAudio() instead. SDL_QueueAudio() will return
- * -1 to signify an error if you use it with capture devices.
+ * flow, you want SDL_DequeueAudio() instead. SDL_QueueAudio() will return
+ * -1 to signify an error if you use it with capture flows.
*
- * SDL offers two ways to feed audio to the device: you can either supply a
+ * SDL offers two ways to feed audio to a flow: you can either supply a
* callback that SDL triggers with some frequency to obtain more audio (pull
* method), or you can supply no callback, and then SDL will expect you to
* supply data at regular intervals (push method) with this function.
@@ -900,9 +860,7 @@ extern DECLSPEC int SDLCALL SDL_MixAudioFormat(Uint8 * dst,
* aren't routinely queueing sufficient data.
*
* This function copies the supplied data, so you are safe to free it when the
- * function returns. This function is thread-safe, but queueing to the same
- * device from two threads at once does not promise which buffer will be
- * queued first.
+ * function returns.
*
* You may not queue audio on a device that is using an application-supplied
* callback; doing so returns an error. You have to use the audio callback or
@@ -915,71 +873,76 @@ extern DECLSPEC int SDLCALL SDL_MixAudioFormat(Uint8 * dst,
* planar audio formats into a non-planar one (see SDL_AudioFormat) before
* queuing audio.
*
- * \param dev the device ID to which we will queue audio
+ * \param flow the flow to which we will queue audio
* \param data the data to queue to the device for later playback
* \param len the number of bytes (not samples!) to which `data` points
* \returns 0 on success or a negative error code on failure; call
* SDL_GetError() for more information.
*
+ * \threadsafety This function is thread-safe, but queueing to the same
+ * flow from two threads at once does not promise which buffer
+ * will be queued first.
+ *
* \since This function is available since SDL 3.0.0.
*
* \sa SDL_ClearQueuedAudio
* \sa SDL_GetQueuedAudioSize
*/
-extern DECLSPEC int SDLCALL SDL_QueueAudio(SDL_AudioDeviceID dev, const void *data, Uint32 len);
+extern DECLSPEC int SDLCALL SDL_QueueAudio(SDL_AudioFlow *flow, const void *data, Uint32 len);
/**
- * Dequeue more audio on non-callback devices.
+ * Dequeue more audio on non-callback flows.
*
* If you are looking to queue audio for output on a non-callback playback
- * device, you want SDL_QueueAudio() instead. SDL_DequeueAudio() will always
+ * flow, you want SDL_QueueAudio() instead. SDL_DequeueAudio() will always
* return 0 if you use it with playback devices.
*
- * SDL offers two ways to retrieve audio from a capture device: you can either
- * supply a callback that SDL triggers with some frequency as the device
+ * SDL offers two ways to retrieve audio from a capture flows: you can either
+ * supply a callback that SDL triggers with some frequency as the flow
* records more audio data, (push method), or you can supply no callback, and
* then SDL will expect you to retrieve data at regular intervals (pull
* method) with this function.
*
* There are no limits on the amount of data you can queue, short of
- * exhaustion of address space. Data from the device will keep queuing as
+ * exhaustion of address space. Data from the flow will keep queuing as
* necessary without further intervention from you. This means you will
* eventually run out of memory if you aren't routinely dequeueing data.
*
- * Capture devices will not queue data when paused; if you are expecting to
- * not need captured audio for some length of time, use SDL_PauseAudioDevice()
+ * Capture flows will not queue data when paused; if you are expecting to
+ * not need captured audio for some length of time, use SDL_PauseAudioFlow()
* to stop the capture device from queueing more data. This can be useful
- * during, say, level loading times. When unpaused, capture devices will start
+ * during, say, level loading times. When unpaused, capture flows will start
* queueing data from that point, having flushed any capturable data available
* while paused.
*
- * This function is thread-safe, but dequeueing from the same device from two
- * threads at once does not promise which thread will dequeue data first.
- *
- * You may not dequeue audio from a device that is using an
+ * You may not dequeue audio from a flow that is using an
* application-supplied callback; doing so returns an error. You have to use
* the audio callback, or dequeue audio with this function, but not both.
*
- * You should not call SDL_LockAudio() on the device before dequeueing; SDL
+ * You should not call SDL_LockAudioFlow() on the flow before dequeueing; SDL
* handles locking internally for this function.
*
- * \param dev the device ID from which we will dequeue audio
+ * \param flow the flow from which we will dequeue audio
* \param data a pointer into where audio data should be copied
* \param len the number of bytes (not samples!) to which (data) points
* \returns the number of bytes dequeued, which could be less than requested;
* call SDL_GetError() for more information.
*
+ * \threadsafety This function is thread-safe, but dequeueing from the same
+ * flow from two threads at once does not promise which
+ * thread will dequeue data first.
+ *
* \since This function is available since SDL 3.0.0.
*
* \sa SDL_ClearQueuedAudio
* \sa SDL_GetQueuedAudioSize
*/
-extern DECLSPEC Uint32 SDLCALL SDL_DequeueAudio(SDL_AudioDeviceID dev, void *data, Uint32 len);
+extern DECLSPEC Uint32 SDLCALL SDL_DequeueAudio(SDL_AudioFlow *flow, void *data, Uint32 len);
/**
* Get the number of bytes of still-queued audio.
*
- * For playback devices: this is the number of bytes that have been queued for
+ * For playback flows: this is the number of bytes that have been queued for
* playback with SDL_QueueAudio(), but have not yet been sent to the hardware.
*
* Once we've sent it to the hardware, this function can not decide the exact
@@ -987,19 +950,19 @@ extern DECLSPEC Uint32 SDLCALL SDL_DequeueAudio(SDL_AudioDeviceID dev, void *dat
* hardware several kilobytes right before you called this function, but it
* hasn't played any of it yet, or maybe half of it, etc.
*
- * For capture devices, this is the number of bytes that have been captured by
- * the device and are waiting for you to dequeue. This number may grow at any
+ * For capture flows, this is the number of bytes that have been captured by
+ * the flow and are waiting for you to dequeue. This number may grow at any
* time, so this only informs of the lower-bound of available data.
*
- * You may not queue or dequeue audio on a device that is using an
- * application-supplied callback; calling this function on such a device
+ * You may not queue or dequeue audio on a flow that is using an
+ * application-supplied callback; calling this function on such a flow
* always returns 0. You have to use the audio callback or queue audio, but
* not both.
*
- * You should not call SDL_LockAudio() on the device before querying; SDL
+ * You should not call SDL_LockAudioFlow() on the flow before querying; SDL
* handles locking internally for this function.
*
- * \param dev the device ID of which we will query queued audio size
+ * \param flow the flow for which we will query queued audio size
* \returns the number of bytes (not samples!) of queued audio.
*
* \since This function is available since SDL 3.0.0.
@@ -1008,15 +971,15 @@ extern DECLSPEC Uint32 SDLCALL SDL_DequeueAudio(SDL_AudioDeviceID dev, void *dat
* \sa SDL_QueueAudio
* \sa SDL_DequeueAudio
*/
-extern DECLSPEC Uint32 SDLCALL SDL_GetQueuedAudioSize(SDL_AudioDeviceID dev);
+extern DECLSPEC Uint32 SDLCALL SDL_GetQueuedAudioSize(SDL_AudioFlow *flow);
/**
* Drop any queued audio data waiting to be sent to the hardware.
*
* Immediately after this call, SDL_GetQueuedAudioSize() will return 0. For
- * output devices, the hardware will start playing silence if more audio isn't
- * queued. For capture devices, the hardware will start filling the empty
- * queue with new data if the capture device isn't paused.
+ * output flows, the hardware will start playing silence if more audio isn't
+ * queued. For capture flows, the hardware will start filling the empty
+ * queue with new data if the capture flow isn't paused.
*
* This will not prevent playback of queued audio that's already been sent to
* the hardware, as we can not undo that, so expect there to be some fraction
@@ -1024,17 +987,17 @@ extern DECLSPEC Uint32 SDLCALL SDL_GetQueuedAudioSize(SDL_AudioDeviceID dev);
* want to, say, drop any pending music or any unprocessed microphone input
* during a level change in your game.
*
- * You may not queue or dequeue audio on a device that is using an
- * application-supplied callback; calling this function on such a device
+ * You may not queue or dequeue audio on a flow that is using an
+ * application-supplied callback; calling this function on such a flow
* always returns 0. You have to use the audio callback or queue audio, but
* not both.
*
- * You should not call SDL_LockAudio() on the device before clearing the
+ * You should not call SDL_LockAudioFlow() on the flow before clearing the
* queue; SDL handles locking internally for this function.
*
* This function always succeeds and thus returns void.
*
- * \param dev the device ID of which to clear the audio queue
+ * \param flow the flow of which to clear the audio queue
* \returns 0 on success or a negative error code on failure; call
* SDL_GetError() for more information.
*
@@ -1044,7 +1007,7 @@ extern DECLSPEC Uint32 SDLCALL SDL_GetQueuedAudioSize(SDL_AudioDeviceID dev);
* \sa SDL_QueueAudio
* \sa SDL_DequeueAudio
*/
-extern DECLSPEC int SDLCALL SDL_ClearQueuedAudio(SDL_AudioDeviceID dev);
+extern DECLSPEC int SDLCALL SDL_ClearQueuedAudio(SDL_AudioFlow *flow);
/**
@@ -1096,7 +1059,7 @@ extern DECLSPEC int SDLCALL SDL_ClearQueuedAudio(SDL_AudioDeviceID dev);
*
* \sa SDL_UnlockAudioDevice
*/
-extern DECLSPEC int SDLCALL SDL_LockAudioDevice(SDL_AudioDeviceID dev);
+extern DECLSPEC int SDLCALL SDL_LockAudioFlow(SDL_AudioFlow *flow);
/**
* Use this function to unlock the audio callback function for a specified
@@ -1110,32 +1073,34 @@ extern DECLSPEC int SDLCALL SDL_LockAudioDevice(SDL_AudioDeviceID dev);
*
* \sa SDL_LockAudioDevice
*/
-extern DECLSPEC void SDLCALL SDL_UnlockAudioDevice(SDL_AudioDeviceID dev);
+extern DECLSPEC void SDLCALL SDL_UnlockAudioFlow(SDL_AudioFlow *flow);
/* @} *//* Audio lock functions */
/**
- * Use this function to shut down audio processing and close the audio device.
+ * Use this function to clean up resources from a flow.
*
- * The application should close open audio devices once they are no longer
- * needed. Calling this function will wait until the device's audio callback
- * is not running, release the audio hardware and then clean up internal
- * state. No further audio will play from this device once this function
- * returns.
+ * The application should destroy audio flows once they are no longer
+ * needed. Calling this function will wait until the flow's audio callback
+ * is not running and then clean up any allocated resources. If this was
+ * the only flow on an audio device, any device-specific resources will be
+ * released No further audio will play from this flow once this function
+ * returns, and its pointer will be invalid for further use.
*
* This function may block briefly while pending audio data is played by the
* hardware, so that applications don't drop the last buffer of data they
* supplied.
*
- * The device ID is invalid as soon as the device is closed, and is eligible
- * for reuse in a new SDL_OpenAudioDevice() call immediately.
+ * \param flow a flow previously created with SDL_CreateAudioFlow()
*
- * \param dev an audio device previously opened with SDL_OpenAudioDevice()
+ * \threadsafety It is safe to call this function from any thread, but once
+ * this function is called, the flow is invalid and should not
+ * be accessed from any thread thereafter.
*
* \since This function is available since SDL 3.0.0.
*
* \sa SDL_OpenAudioDevice
*/
-extern DECLSPEC void SDLCALL SDL_CloseAudioDevice(SDL_AudioDeviceID dev);
+extern DECLSPEC void SDLCALL SDL_DestroyAudioFlow(SDL_AudioFlow *flow);
/**
* Convert some audio data of one format to another format.
@@ -1169,6 +1134,24 @@ extern DECLSPEC int SDLCALL SDL_ConvertAudioSamples(SDL_AudioFormat src_format,
Uint8 **dst_data,
int *dst_len);
+
+/**
+ * Determine the value that equals silence for a given audio format.
+ *
+ * This value can be passed to SDL_memset() to silence a buffer of samples
+ * in a given format.
+ *
+ * \param fmt The audio format to query
+ * \returns A byte value that can be used to memset a buffer. Will guess 0 for unknown formats.
+ *
+ * \threadsafety This function can be called from any thread at any time.
+ *
+ * \since This function is available since SDL 3.0.0.
+ *
+ * \sa SDL_CreateAudioStream
+ */
+extern DECLSPEC Uint8 SDLCALL SDL_GetSilenceValue(SDL_AudioFormat fmt);
+
/* Ends C function definitions when using C++ */
#ifdef __cplusplus
} |
|
Indeed, IDs as now more conventional with SDL3 So 'flows' are somehow DataStream attached to a device. So the device has to do some mixing when there are multiples flow ? SDL_{Lock,Unlock}AudioFlow -> {Pause/Start/Stop}AudioFlow (?) Should we need also a Pause/Start/Stop for AudioDevice ? ( wondering if we shouldn't have a struct for { SDL_AudioFormat fmt, Uint8 channels, Uint32 freq } ) |
|
Maybe this could really be a an AudioStream like: Eventually, Bind/Unbind API to attach/detach AudioStream to DeviceID Could be an idea type things as src or sink interface like gstreamer... (could it be compatible with gstreamer ? seems difficult...) |
|
Ok, so thinking about this more, the problem is that SDL_AudioStream doesn't let you change output formats once you create it, but that's because we convert data as it is input. But this is silly, we should be queuing it up and converting it as we need output. This would let us do a couple of things:
Make that change, and we don't need "flows" or "sinks" or whatever. Just bind a stream (or multiple streams) to a device, and they'll convert as necessary to device format. The device gets opened behind the scenes in whatever format the first stream attached to it uses for convenience (or the app can specify, in case a low-quality thing is the first stream), and closes when the last stream is unbound. We REMOVE the callback API. If someone needs it, we provide a single-header library to feed an audiostream from a thread. This lets us remove the device locking APIs, too (audiostreams will still be threadsafe for queuing and dequeuing data). Behind the scenes, we'll probably still have a thread to feed the audio device when appropriate, but we can probably have a single thread that serves all the open devices. |
I'm thinking maybe we don't need these at all; unbinding a stream would "pause" it with whatever data was still in it untouched, clearing a stream's contents (and optionally unbinding it) is the same as "stopping" as an empty stream will just not add anything to the device's output. |
Ooh, I like that. |
|
The original reason why we converted on input was because on some devices when the audio interrupt fired, we didn't have enough time to do on the fly conversion of the audio data before the system needed it for playback. This was especially problematic on older iOS devices when using libsamplerate for high quality (slow) conversion. |
This was referenced Feb 28, 2023
Closed
|
another thought: |
Originally posted by @icculus in #7704 (comment) (Just copying this in here so it stays with the rest of the wishlist discussion.) |
|
I think iOS has a similar issue for playback devices, but it's been a while since I've actually looked at this so FAudio's comments are all I've got: https://github.com/FNA-XNA/FAudio/blob/master/src/FAudio_platform_sdl2.c#L162 So audio-specific suspend/resume events may be useful to handle too (I think OpenAL Soft has an ALC extension for this, now that I think about it). |
|
This isn't a permission request thing, this is just iOS not allowing you to access audio from the background unless you're specifically entitled as a background music player. |
icculus
added a commit
that referenced
this issue
Aug 4, 2023
This rips up the entire SDL audio subsystem! While we still feed the audio device from a separate thread, the audio callback into the app is now gone a totally optional alternative. Now the app will bind an SDL_AudioStream to a given device and feed data to it. As many streams as one likes can be bound to a device; SDL will mix them all into a single buffer and feed the device from there. So not only does this function as a basic mixer, it also means that multiple device opens are handled seamlessly (so if you want to open the device for your game, but you also link to a library that provides VoIP and it wants to open the device separately, you don't have to worry about stepping on each other, or that the OS will fail to allow multiple opens of the same device, etc). Merged from pull request #7704. Fixes #7379. Reference Issue #6889. Reference Issue #6632.
|
I think this is good to go at this point? The audio subsystem has had a massive redesign and looks like it's in pretty good shape, so I'm closing this. Any other pending concerns should get new issues instead of a wishlist megathread at this point. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
(Just thinking out loud, this isn't locked down at all yet.)
Big Audio changes:
Allow SDL_AudioStream to change its sample rate on the fly (for pitch shifting).
Originally posted by @icculus in #3519 (comment)
The text was updated successfully, but these errors were encountered: