Commit Graph

308 Commits

Author SHA1 Message Date
Hoan Luu Huu
44e6a3513d support speech aws polly by role_arn (#729)
* support speech aws polly by role_arn

* support aws stt assume role

* wip

* update speech utils version
2024-05-02 07:59:21 -04:00
Dave Horton
59cf02bd04 wait for session:reconnect ack to send queued msgs (#723) (#732)
Co-authored-by: Hoan Luu Huu <110280845+xquanluu@users.noreply.github.com>
2024-04-25 11:22:15 -04:00
Hoan Luu Huu
7da11df88e default DEEPGRAM_SPEECH_UTTERANCE_END_MS is 1000 (#719) 2024-04-14 19:39:07 -04:00
Hoan Luu Huu
9eab81268b support mod_rimelabs_tts (#716)
* support mod_rimelabs_tts

* update speech utils
2024-04-12 07:28:45 -04:00
Dave Horton
ecf3d140d6 fix #714 (#715) 2024-04-10 16:23:22 -04:00
Hoan Luu Huu
4a52be9171 support mod_playht_tts (#711)
* support mod_playht_tts

* update speech utils version
2024-04-08 10:21:54 -04:00
Dave Horton
8999c85a71 Fixes/ws testing dh (#704)
* fixes from testing with translator app

* more updates

* linting

* update gh actions to node 20

* add support for google v2 preconfigured recognizer

* add support for google voice activity events

* update to speech-utils@0.0.45

* update speech-utils to support caching azure tts

* transcribe must buffer transcripts for channel 1 and 2 separately

* further fix for accumulating transcripts

* linting

* deepgram sends transcripts with empty alternatives array

* fix deepgram returning an empty array
2024-04-03 14:30:49 -04:00
Hoan Luu Huu
72147a8110 support google v2 enableVoiceActivityEvents (#703)
* support google v2 enableVoiceActivityEvents

* support google v2 enableVoiceActivityEvents
2024-04-02 10:14:54 -04:00
Hoan Luu Huu
93d0e41e31 support google version 2 (#699)
* support google version 2

* update new parameters for google v2
2024-04-02 07:33:22 -04:00
Hoan Luu Huu
5b1d8a8ff3 Feat/ambient sounds (#678)
* initial support for coaching mode in conference

* wip

* wip

* add support for answer verb

* wip

* wip

* wip

* wip

* wip

* updates to rename option to dub

* wip

* wip

* wip

* update verb-specs

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* add option to boost audio signal in main channel

* wip

* wip

* wip

* wip

* wip

* wip

* for now, bypass use of streaming apis when generating tts audio for dub tracks

* add nested dub to dial

* wip

* add support for filler noise

* kill filler noise when gather killed

* wip

* wip

* while using sayOnTrack, we have to enclose the say command in double quotes

* disableTtsStreaming = false

* allow transcribe of b leg only on dial verb

* dub.say can either be text or object like say verb with text and synthesizer

* remove loop for sayOnTrack

* update speech-utils

* fixes for testing transcribe verb and support for dub and boostAudioSignal in lcc commands

* add dial.boostAudioSignal

* fix bug where session-level recognizer settings incorrectly overwrite verb-level settings

* update verb specs

* update dial to support array of dub verbs

* fix bug setting gain

* lint

* wip

* update speech-utils

* use new endpoint methods for mod_dub

---------

Co-authored-by: Dave Horton <daveh@beachdognet.com>
2024-03-23 16:23:57 -04:00
Dave Horton
ec58232b61 Fix/replace application issue (#692)
* fix scenario where ws replace application from gather while awaiting command and no tasks on execution stack

* lint

* remove some debug logging
2024-03-23 16:14:16 -04:00
Hoan Luu Huu
b80d39d205 fix asrtimer always return vendor=deepgram (#682) 2024-03-13 12:57:55 -04:00
Hoan Luu Huu
1914b88af9 support azure language id mode (#674) 2024-03-12 08:35:01 -04:00
Dave Horton
fc3655c9bd fixes for confirm session (#663)
* fixes for confirm session

* allow empty dialconfirm array
2024-02-22 12:33:35 -05:00
Anton Voylenko
5c070597cf tag outdial session (#643) 2024-02-12 13:16:43 -05:00
Markus Frindt
ef2306e558 Improve Deepgram default modely by language (#641)
Co-authored-by: Markus Frindt <m.frindt@cognigy.com>
2024-02-12 09:53:14 -05:00
Dave Horton
9f9a9ec598 initial changes for deepgram on-prem (#636)
* initial changes for deepgram on-prem

* typo

* fixes for selecting deepgram model

* update some property names

* wip

* wip

* wip
2024-02-07 14:21:05 -05:00
Hoan Luu Huu
a55f81676b Tts/elevenlabs streaming (#629)
* update to fsmrf with fix

* changes to support elevenlabs tts streaming

* say: add vendor data to span

* bug: tts spans must include cached property

* add env for JAMBONES_USE_FREESWITCH_TIMER_FD

* fix bug in prev commit

* wip

* linting

* wip - caching files generating by streaming tts

* wip caching

* cleanup some logs

* handle tts streaming failure, write alert

* update node version dependency

* set timerfd on outbound call scenarios

* default model to nova-2-phonecall when using deepgram

---------

Co-authored-by: Dave Horton <daveh@beachdognet.com>
2024-02-07 08:49:36 -05:00
Hoan Luu Huu
74ede31cd3 fix ws reconnect does not send verb:hook data (#633) 2024-01-31 07:20:57 -08:00
Dave Horton
0491c5ce25 minor logging changes 2024-01-27 12:59:23 -05:00
Hoan Luu Huu
b867395d87 fix aldulting call does not send status callback when hhangup (#623) 2024-01-23 07:12:43 -05:00
Hoan Luu Huu
b423a51638 feat: allow update azure endpoint ID from recognizer property (#612) 2024-01-17 07:34:02 -05:00
Hoan Luu Huu
b4ff2ea702 fix onholdHOok (#540)
* fix onholdHOok

* wip

* wip

* wip

* wip

* adding more debug log

* wip

* wip

* wip
2024-01-15 08:34:45 -05:00
Dave Horton
f22d66dfd6 set default deepgram model by language and task (gather vs transcribe) (#610)
* set default deepgram model by language and task (gather vs transcribe)

* wip
2024-01-14 10:38:14 -05:00
Hoan Luu Huu
29bbcf1be0 add user-agent to http and ws requestor (#602)
* add user-agent to http and ws requestor

* wip

* fix review comment
2024-01-10 08:54:46 -05:00
Hoan Luu Huu
2e69630544 fix siprec to remap sdp base on participant label (#587)
* fix siprec to remap sdp base on participant label

* fix
2024-01-03 11:10:31 -05:00
Dave Horton
2c48083c26 fix to be more precise about removing custom event handlers so that w… (#580)
* fix to be more precise about removing custom event handlers so that when we stop a gather we dont also inadvertently stop a background transcribe as well

* test fixes

* fix: endpointing=false was being ignored for Deepgram
2023-12-28 11:00:27 -05:00
Hoan Luu Huu
9d8291f892 Transcribe background task (#576)
* first draft

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* update verb-specification

* fix comment reviews

* provide bugname when stopping transcription, otherwise it will continue

---------

Co-authored-by: Dave Horton <daveh@beachdognet.com>
2023-12-26 21:50:51 -05:00
Hoan Luu Huu
9eb315ecd6 fix config and stt task for altLanguages (#575)
* fix config and stt task for altLanguages

* clear freeswitch channel var when altLanguages is empty list
2023-12-25 22:21:34 -05:00
Dave Horton
47fbc1a4a4 allow custom speech with no auth token (#571) 2023-12-18 14:51:34 -05:00
Hoan Luu Huu
30977b309c punctuation for microsoft (#566)
* punctuation for microsoft

* wip
2023-12-18 08:38:05 -05:00
Dave Horton
e5b60ca9b0 trying to pass redis client to speech utils causing restart (#557) 2023-12-01 18:26:33 -05:00
Hoan Luu Huu
0a0cbd57ba support elevenlabs options (#553)
* support elevenlabs options

* elevenlabs options from synthezier

* wip

* fix
2023-11-30 09:28:12 -05:00
Hoan Luu Huu
eb2d90ffaa fix release freeswitch media properly (#550)
* fix release freeswitch media properly

* if a leg is opus, modify b leg offer opus first

* if a leg is opus, modify b leg offer opus first

* wip

* wip

* fix review comments

* fix review comments

* fix review comments
2023-11-29 10:17:15 -05:00
Hoan Luu Huu
1cdd0cf611 feat Audult call session should have its own requestor (#535)
* feat Audult call session should have its own requestor

* fix eslint

* fix eslint

* if user authenticate from http server instead of databse

* wip

* fix custom STT

* fix custom STT

* fix custom STT
2023-11-16 08:22:35 -05:00
Dave Horton
7c36a08852 Feat/dial via sip proxy (#532)
* add support for dialing via proxy with target.proxy

* update verb-specifications with support for target.proxy
2023-11-14 09:06:43 -05:00
Hoan Luu Huu
633237da1b let realtimedb-help build configuration from env vars (#526)
* let realtimedb-help build configuration from env vars

* update speech-utils version
2023-11-14 08:57:50 -05:00
Hoan Luu Huu
87632c549e feat support Whisper TTS (#523)
* feat support openai

* update speech utils version
2023-11-09 09:51:20 -05:00
Hoan Luu Huu
31559cbb3b user restriction (#520) 2023-11-08 12:39:56 -05:00
Dave Horton
1156bae2de fix for #521 - allow pause in confirmHook applications (#522) 2023-11-07 09:46:41 -05:00
Dave Horton
6d625d87ad Feat/assemblyai testing (#516)
* handle errors from assemblyai

* wip

* fix alert

* normalizeAssemblyai
2023-11-02 17:05:28 -04:00
Hoan Luu Huu
7fee2ba2dc feat assembly (#515)
* fix

* wip

* wip

* wip

* wip

* fix review comments
2023-11-02 09:25:04 -04:00
Dave Horton
6b9f6a7d90 if dial.confirmHook returns empty array do not create a confirmSession (#513) 2023-11-01 08:47:41 -04:00
Hoan Luu Huu
e73bef4af0 google custom voice (#506)
* google custom voice

* fixed

* wip

* wip

* wip

* wip
2023-10-30 20:10:30 -04:00
Dave Horton
f43a5c1491 deepgram: rework continuous asr, and resolve on speech_final not is_f… (#501)
* deepgram: rework continuous asr, and resolve on speech_final not is_final (wip)

* wip

* deepgram: empty final transcript should trigger resolve with speech if we have buffered transcripts

* wip

* fixes for deepgram compiling multiple transcripts

* test deepgram utteranceEndMs

* more handling of utteranceEndMs

* wip

* better handling of digit strings collected over multiple deepgram responses

* wip

* add support for deepgramOptions.shortUtterance which triggers off of is_final instead of speech_final

* apply deepgram fixes to transcribe

* cleanup continnuous asr

* more continuous asr fixes for deepgram

* update to verb-specifications for handling SttTask properties

* set log level for tests back to error
2023-10-30 13:57:25 -04:00
Dave Horton
67f8f7181a #508 - add support for azureOptions.speechSegmentationSilenceTimeoutMs (#509)
* #508 - add support for azureOptions.speechSegmentationSilenceTimeoutMs

* update verb specs
2023-10-30 13:10:31 -04:00
Dave Horton
916d988dbd add support for deepgram smart_format option (#500)
* add support for deepgram smart_format option

* handle nonexistent hints
2023-10-25 14:29:38 -04:00
Hoan Luu Huu
24fccbdae5 Feat/devices call other (#491)
* calls between clients on same domain

* wip

* wip

* wip

* wip
2023-10-20 13:45:13 +02:00
Hoan Luu Huu
7992bc6ca0 feat add http proxy to undicy (#487)
* feat add http proxy to undicy

* fix typo

* fix typo

* add http proxy testcase
2023-10-20 08:36:24 +02:00
Hoan Luu Huu
320baf4ac8 update speech version (#485)
* update speech version

* create elevenlabs credential for synthesis

* create elevenlabs credential for synthesis
2023-10-14 20:51:19 +02:00