summaryrefslogtreecommitdiffstats
path: root/pica-client/audiovideocallcontroller.h
diff options
context:
space:
mode:
authorAnton <anton@picapica.im>2026-08-24 05:06:23 +0400
committerAnton <anton@picapica.im>2026-08-24 05:06:23 +0400
commita4ddbc49c82e31ee980ea1096cc737db438c012e (patch)
tree045ae5a4031f928dedb772a8998b220f0d5aa3e2 /pica-client/audiovideocallcontroller.h
parent5c5fe2b41da2f4f1da32b12909a9b982e5dec70a (diff)
downloadpica-pica-a4ddbc49c82e31ee980ea1096cc737db438c012e.tar.gz
pica-pica-a4ddbc49c82e31ee980ea1096cc737db438c012e.zip
pica-client: echo cancellation, and native audio on Windows and macOS
PlatformDriverName() now takes a direction, and answers with one of two kinds of name. A bare one - "alsa", "pulse", "oss" - is an FFmpeg libavdevice driver opened the way everything else here is opened. One prefixed "platform-" is not an FFmpeg driver at all: what follows names a platform audio API the caller drives itself through NativeAudioBackend. Windows and macOS take the second route, because both provide acoustic echo cancellation but only to streams that declare themselves part of a voice call - AudioCategory_Communications on WASAPI, the VoiceProcessingIO audio unit on CoreAudio - and libavdevice's wrappers have no way to ask for either. Doing our own cancelling on platforms that would have done it better below us is not a trade worth making. Where the platform will not do it, speexdsp does: one EchoCanceller per call, shared by both directions through a QSharedPointer, since the call is torn down without joining either audio thread. Playback feeds it the far end as it hands it to the device; capture subtracts before encoding. Alignment is left to the FIFO - in steady state its backlog settles at the sound card's buffer depth, which is also how far ahead of the speaker the playback side runs, leaving only the room for the filter tail to model. Measured 27 dB ERLE on a 60 ms echo with 1.6 dB of loss on speech when nothing is playing. On macOS the canceller is a single audio unit that renders and captures at once, so both directions share one refcounted instance. Ringtones deliberately get a plain HALOutput unit instead - opening a voice unit to play a bell would switch the microphone on and prompt for permission. Linux gains a choice of alsa or pulse, with PulseAudio device enumeration through libpulse when configure finds it. Info.plist gains NSMicrophoneUsageDescription and NSCameraUsageDescription. Without them macOS 10.14 and later kills the process on first use of the microphone or camera rather than asking, so calls did not merely fail, they took the application with them. Two things fixed along the way that were not about echo at all: the settings dialog's microphone test paced playback by counting timer ticks, which on Windows meant a 20 ms packet every ~31 ms because the system timer granularity is 15.6 ms - the recording came out slowed down and chopped up. It now paces against a QElapsedTimer. And chooseEncoderSampleFmt() uses avcodec_get_supported_config() where available, the AVCodec format arrays it replaced having been removed in FFmpeg 9. Only the Linux paths are verified here; the Windows and macOS backends compile to nothing on this machine and have been reviewed by hand. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CBqDgvq3Ty6C6BjLFH16bw
Diffstat (limited to 'pica-client/audiovideocallcontroller.h')
-rw-r--r--pica-client/audiovideocallcontroller.h5
1 files changed, 5 insertions, 0 deletions
diff --git a/pica-client/audiovideocallcontroller.h b/pica-client/audiovideocallcontroller.h
index 2a9a0fa..1dd9f20 100644
--- a/pica-client/audiovideocallcontroller.h
+++ b/pica-client/audiovideocallcontroller.h
@@ -73,6 +73,11 @@ private:
QThread cam_thread;
QThread remotevideo_thread;
+ // Acoustic echo cancellation for the current call, shared by microphone
+ // and output. Null when it is switched off in the settings, when
+ // speexdsp could not start one, or between calls.
+ EchoCancellerPtr m_echoCanceller;
+
// Sequence number / capture-clock timestamp stamped onto outgoing
// 0x76 audio packets (see PICA_PROTO_CALL_PACKET_HDRSIZE).
quint16 m_audioSeq;