Commit Graph

94 Commits

Author SHA1 Message Date
Hoan Luu Huu
98b845f489 fix say verb does not close streaming when finish say (#1412)
* fix say verb does not close streaming when finish say

* wip

* wip

* ttsStreamingBuffer reset eventHandlerCount after remove listeners

* only send tokens to module if connected

* wip

* sent stream_open when successfully connected to vendor
2025-11-17 08:56:09 -05:00
Hoan Luu Huu
2de24af169 fixed gather does not start timeout on bargin (#1421)
* fixed gather does not start timeout on bargin

* with previous change, no need to emit playDone since no where in the code are we listening for it

---------

Co-authored-by: Dave Horton <daveh@beachdognet.com>
2025-11-03 13:11:59 -05:00
Ed Robbins
5acb19225b If an error occurs during initial TTS request, propagate the error (#1369)
* If an error occurs during initial TTS request, propagate the error

* fix missing semicolon

* fix jslint error

* add null check to r.playbackMilliseconds
2025-10-01 00:02:51 -04:00
Dave Horton
f2fe7c4d24 Fix/playback race by fs generates playback (#1331)
* update to speech-utils that generates playback id

* modify tts and say task to track current playback id and match against start and stop events

* bump speech utils

* wip

* wip

* fix race condition where say with playbackId gets stop event from previous play from cache file

* logging

* wip

* fix comparison when playing cached files

* logging
2025-08-26 09:39:25 -04:00
Hoan Luu Huu
44f69fa76d Support resemble tts (#1322)
* support resemble tts

* update speech utils version
2025-08-13 08:15:29 -04:00
Dave Horton
2a42ccb0e1 fix for #1285 (#1286)
* fix for #1285

* typo
2025-07-15 11:38:48 -04:00
Dave Horton
e415420150 Fix/cached audio race (#1279)
* fix for race condition when play-stop event from earlier command received

* wip

* say verb should not cache if disableTtsCache = true

* logging

* modify race condition logic to validate playback id in playback-stopped matches that from playback-start

* logging

---------

Co-authored-by: Quan HL <quan.luuhoang8@gmail.com>
2025-07-13 10:56:41 -04:00
Hoan Luu Huu
c82a835e70 FD_1079: tts modules response_code = 0 should make say fail to exec. (#1269)
* FD_1079: tts modules response_code = 0 should make say fail to exec.

* fixed tts azure does not have response code

* fixed tts error does not raise alarm

* wip

* fixed
2025-07-07 07:59:29 -04:00
Hoan Luu Huu
ab83b21979 support inworld tts (#1262)
* support inworld tts

* wip

* wip
2025-06-27 10:05:18 -04:00
Hoan Luu Huu
b0b74871e7 support say stream with text (#1227)
* support say stream with text

* wip

* wip

* wip

* wip

* update verb  specification
2025-06-10 16:56:44 +02:00
Hoan Luu Huu
9585018147 support whisper instructions (#1198)
* support whisper instructions

* wip

* update speech utils and verb specification
2025-05-13 09:44:00 -04:00
rammohan-y
e4bb00b382 Send stop-playback event (#1186)
* Send stop-playback event

https://github.com/jambonz/jambonz-feature-server/issues/1185

* check if not notified in playback-stop, ensure that the stop-playback is sent when kill-playback is sent
2025-05-07 08:59:59 -04:00
Hoan Luu Huu
ed92cb2632 update speech utils 0.2.7 (#1177)
* update speech utils 0.2.7

* wip
2025-04-29 08:26:09 -04:00
Hoan Luu Huu
7e2fe72b6c fix say verb cannot failover if tts_response-code != 2xx (#1174) 2025-04-28 08:46:08 -04:00
rammohan-y
87195b6444 Stop tts streaming, when bargeIn is enabled for gather verb and input is detected (#1154)
https://github.com/jambonz/jambonz-feature-server/issues/1153
2025-04-14 09:18:50 -04:00
Hoan Luu Huu
33bca8e67c tts stream should save tts.response_time metric (#1086)
* tts stream should save tts.response_time metric

* wip
2025-02-18 08:45:21 -05:00
Hoan Luu Huu
7105453d81 support caching tts audio with model/model_id (#1062)
* support caching tts audio with model/model_id

* update speech utils version
2025-02-03 08:47:44 -05:00
Hoan Luu Huu
17250f8386 support cartesia tts (#1008)
* support cartesia tts

* update speech util version

* update speech utils version
2024-12-19 07:35:47 -05:00
Dave Horton
ba3f46df64 Feat/tts streaming (#994)
* wip

* add TtsStreamingBuffer class to abstract handling of streaming tokens

* wip

* add throttling support

* support background ttsStream (#995)

* wip

* add TtsStreamingBuffer class to abstract handling of streaming tokens

* wip

* support background ttsStream

* wip

---------

Co-authored-by: Dave Horton <daveh@beachdognet.com>

* wip

* dont send if we have nothing to send

* initial testing with cartesia

* wip

---------

Co-authored-by: Hoan Luu Huu <110280845+xquanluu@users.noreply.github.com>
2024-12-18 14:44:37 -05:00
rammohan-y
4b5df855e1 feat/952: removed unnecessary condition which is not logging the target_sid (#953) 2024-10-29 07:34:49 -04:00
Hoan Luu Huu
96b3b0fe07 Allow Say, Gather, Transcribe is able to finished if there is error for speech credential (#910)
* allow move to next task if say verb is failed because of speech credential

* allow move to next task if say verb is failed because of speech credential

* allow move to next task if say verb is failed because of speech credential

* wip

* wip
2024-10-01 13:40:41 -04:00
Hoan Luu Huu
e69afc4be4 fix recognizer/synthesizer label wrongly select between verb and app (#881)
* fix recognizer/synthesizer label wrongly select between verb and application

* fix jslint

* fix ASR cannot fallback

* update tts fallback does not send notification
2024-09-11 09:34:52 +01:00
Dave Horton
c0ab936b76 wip (#830)
* wip

* wip

* wip

* wip

* update deps

* update test to use latest freeswitch image
2024-08-29 15:23:49 -04:00
rammohan-y
f95d8639be Feat/868: Use global synthesizer config properties for say verb (#869)
* feat/868: Use the properties from global config in verb for TTS

* feat/868: setting this.options to combination of cs.synthesizer.options and this.options

* feat/868: Move the logic of copying cs properties to parent class tts-task.js

* feat/868: add empty line that was removed, say.js restored to original version

* feat/868: moved _synthesizeWithSpecificVendor to tts-task.js

---------

Co-authored-by: Rammohan Yadavalli <rammohan.yadavalli@kore.com>
2024-08-20 08:31:44 -04:00
Hoan Luu Huu
12174359f2 fix support precache audio with tts stream (#855)
* fix support precache audio with tts stream

* update speech util
2024-08-15 08:22:00 -04:00
rammohan-kore
e4e37d5697 feat/836: capturing callSid for STT and TTS alerts (#838)
* feat/836: capturing callSid for STT and TTS alerts

* feat/836: corrected assignment of callSid and added target_sid at few more alerts

* update github action

---------

Co-authored-by: Quan HL <quan.luuhoang8@gmail.com>
2024-08-05 12:14:08 -04:00
Dave Horton
edd8f20642 fix for #826 race condition in say (#827)
* fix for #826 race condition in say

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip
2024-07-24 12:56:03 -04:00
Hoan Luu Huu
5ccea65b7f stt/tts label can be empty, should not assign application level label… (#804)
* stt/tts label can be empty, should not assign application level label as default value

* wip
2024-07-10 08:36:00 -06:00
Vinod Dharashive
5e1b826da4 Aws polly engine fix (#789)
* Aws polly engine fix  

engine parameter was  not able to change using synthesizer

* WIP

code correction and set default engine to Neural

* WIP

* WIP

Updated  tts-task.js

* WIP
2024-06-25 13:29:28 -04:00
Dave Horton
d9fd82fa60 major refactor and simplification of actionHookDelay feature (#771)
* major refactor and simplification of actionHookDelay feature

* wip for #765

* wip

* testing

* wip

* added validity checks for actionHookDelay properties

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* fix bug where config happens before endpoint is established

* wip

* hangup and clear ws connection if nogiveuptimer expires

* wip

* wip

* wip
2024-06-14 09:24:26 -04:00
Hoan Luu Huu
c5e19bf775 support verbio speech (#757)
* support verbio speech

* wip

* wip

* wip

* update speech utils

* update verb specification
2024-05-29 07:55:46 -04:00
Hoan Luu Huu
d7c3a4a632 support mod_custom_tts (#731) 2024-05-28 12:30:25 -04:00
Hoan Luu Huu
6be3fd9b64 say verb should not print speech credentials in log when tts stream API is used (#756) 2024-05-21 08:38:18 -04:00
Hoan Luu Huu
5e97847a2f fix fs keep looping forever if there is no fallback TTS (#749) 2024-05-09 06:15:57 -04:00
Hoan Luu Huu
e7bc0b0737 fix dead lock in say verb while waiting playback-stop and say verb is killed (#742) 2024-05-05 08:12:29 -04:00
Hoan Luu Huu
7bcbab5b74 feat tts stream fallback (#736)
* feat tts stream fallback

* wip

* wip

* wip

* wip

* wip

* wip

* fix review comment
2024-05-02 08:43:41 -04:00
Dave Horton
9cf9d4f587 Fix/0.8.5 cherries (#724)
* kill play task if bot responds verbs while actionHook delay is enabled (#712)

* kill play task if bot responds verbs while actionHook delay is enabled

* fix actionHook delay continues even the bot already responded verbs

* wip

* wip

* wip

* gather is hang if listenDuringPrompt = false and say/play task throw exception (#717)

* merge fix for Support ASR TTS fallback (#713)

---------

Co-authored-by: Hoan Luu Huu <110280845+xquanluu@users.noreply.github.com>
2024-04-17 11:01:21 -04:00
Hoan Luu Huu
09cf1345f6 tts span for whisper (#718)
* tts span for whisper

* support deepgram tts span

* support playht tts span

* support rimelabs tts span

* wip
2024-04-14 09:14:49 -04:00
Hoan Luu Huu
5b1d8a8ff3 Feat/ambient sounds (#678)
* initial support for coaching mode in conference

* wip

* wip

* add support for answer verb

* wip

* wip

* wip

* wip

* wip

* updates to rename option to dub

* wip

* wip

* wip

* update verb-specs

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* add option to boost audio signal in main channel

* wip

* wip

* wip

* wip

* wip

* wip

* for now, bypass use of streaming apis when generating tts audio for dub tracks

* add nested dub to dial

* wip

* add support for filler noise

* kill filler noise when gather killed

* wip

* wip

* while using sayOnTrack, we have to enclose the say command in double quotes

* disableTtsStreaming = false

* allow transcribe of b leg only on dial verb

* dub.say can either be text or object like say verb with text and synthesizer

* remove loop for sayOnTrack

* update speech-utils

* fixes for testing transcribe verb and support for dub and boostAudioSignal in lcc commands

* add dial.boostAudioSignal

* fix bug where session-level recognizer settings incorrectly overwrite verb-level settings

* update verb specs

* update dial to support array of dub verbs

* fix bug setting gain

* lint

* wip

* update speech-utils

* use new endpoint methods for mod_dub

---------

Co-authored-by: Dave Horton <daveh@beachdognet.com>
2024-03-23 16:23:57 -04:00
Hoan Luu Huu
06b066a3f2 update speech util to support whisper stream (#657)
* update speech util to support whisper stream

* minor editing of span attributes

* more span attrs cleanup

---------

Co-authored-by: Dave Horton <daveh@beachdognet.com>
2024-02-22 14:17:29 -05:00
Dave Horton
d0ebe3f99f fix possible undefined reference in precache audio (#662)
* fix possible undefined reference in precache audio

* fix parsing of JAMBONES_EAGERLY_PRE_CACHE_AUDIO
2024-02-22 07:58:41 -05:00
Dave Horton
9c33a790bd update to latest speech-utils (#639) 2024-02-08 15:54:45 -05:00
Hoan Luu Huu
a55f81676b Tts/elevenlabs streaming (#629)
* update to fsmrf with fix

* changes to support elevenlabs tts streaming

* say: add vendor data to span

* bug: tts spans must include cached property

* add env for JAMBONES_USE_FREESWITCH_TIMER_FD

* fix bug in prev commit

* wip

* linting

* wip - caching files generating by streaming tts

* wip caching

* cleanup some logs

* handle tts streaming failure, write alert

* update node version dependency

* set timerfd on outbound call scenarios

* default model to nova-2-phonecall when using deepgram

---------

Co-authored-by: Dave Horton <daveh@beachdognet.com>
2024-02-07 08:49:36 -05:00
Dave Horton
09a83e3a31 Feature/precache audio (#609)
* wip

* fix for establishing vendor etc

* more fixes

* avoid a pre-caching attempt if synth settings change
2024-01-13 12:51:25 -05:00
Hoan Luu Huu
3e8474867f support deepgram (#579)
* support deepgram

* update speech utils
2023-12-26 07:46:35 -05:00
Markus Frindt
86a14daf79 Update say task and add possibility to use elevenlabs options (#556)
* Update say task and add possibility to use elevenlabs options from synthesizer

* revert ms change

* fix contdition for alerting

---------

Co-authored-by: Markus Frindt <m.frindt@cognigy.com>
2023-12-01 09:51:22 -05:00
Hoan Luu Huu
0a0cbd57ba support elevenlabs options (#553)
* support elevenlabs options

* elevenlabs options from synthezier

* wip

* fix
2023-11-30 09:28:12 -05:00
Hoan Luu Huu
4b7b0e309b say verb support play audio in array (#482)
* say verb support play audio in array

* fixed review comments

* added separated testcase
2023-10-20 08:14:57 +02:00
Hoan Luu Huu
b1ecf069bf Feat/fallback speech 02 (#429)
* feat fallback speech

* feat fallback speech

* feat fallback speech

* gather verb

* fix

* wip

* wip

* wip

* wip

* wip

* transcribe

* transcribe

* fix

* wip

* wip

* wip

* wip

* fix

* wip

* fix
2023-08-22 09:22:01 -04:00
Hoan Luu Huu
f8c5abe9e9 feat: multi speech credential diff labels but same vendor (#426)
* feat: multi speech credential diff labels but same vendor

* update sql

* fix

* fix

* fix jslint

* fix review comment

* update verb spec version
2023-08-15 08:57:49 -04:00