Commit Graph

69 Commits

Author SHA1 Message Date
rammohan-kore
e4e37d5697 feat/836: capturing callSid for STT and TTS alerts (#838)
* feat/836: capturing callSid for STT and TTS alerts

* feat/836: corrected assignment of callSid and added target_sid at few more alerts

* update github action

---------

Co-authored-by: Quan HL <quan.luuhoang8@gmail.com>
2024-08-05 12:14:08 -04:00
Dave Horton
edd8f20642 fix for #826 race condition in say (#827)
* fix for #826 race condition in say

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip
2024-07-24 12:56:03 -04:00
Hoan Luu Huu
5ccea65b7f stt/tts label can be empty, should not assign application level label… (#804)
* stt/tts label can be empty, should not assign application level label as default value

* wip
2024-07-10 08:36:00 -06:00
Vinod Dharashive
5e1b826da4 Aws polly engine fix (#789)
* Aws polly engine fix  

engine parameter was  not able to change using synthesizer

* WIP

code correction and set default engine to Neural

* WIP

* WIP

Updated  tts-task.js

* WIP
2024-06-25 13:29:28 -04:00
Dave Horton
d9fd82fa60 major refactor and simplification of actionHookDelay feature (#771)
* major refactor and simplification of actionHookDelay feature

* wip for #765

* wip

* testing

* wip

* added validity checks for actionHookDelay properties

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* fix bug where config happens before endpoint is established

* wip

* hangup and clear ws connection if nogiveuptimer expires

* wip

* wip

* wip
2024-06-14 09:24:26 -04:00
Hoan Luu Huu
c5e19bf775 support verbio speech (#757)
* support verbio speech

* wip

* wip

* wip

* update speech utils

* update verb specification
2024-05-29 07:55:46 -04:00
Hoan Luu Huu
d7c3a4a632 support mod_custom_tts (#731) 2024-05-28 12:30:25 -04:00
Hoan Luu Huu
6be3fd9b64 say verb should not print speech credentials in log when tts stream API is used (#756) 2024-05-21 08:38:18 -04:00
Hoan Luu Huu
5e97847a2f fix fs keep looping forever if there is no fallback TTS (#749) 2024-05-09 06:15:57 -04:00
Hoan Luu Huu
e7bc0b0737 fix dead lock in say verb while waiting playback-stop and say verb is killed (#742) 2024-05-05 08:12:29 -04:00
Hoan Luu Huu
7bcbab5b74 feat tts stream fallback (#736)
* feat tts stream fallback

* wip

* wip

* wip

* wip

* wip

* wip

* fix review comment
2024-05-02 08:43:41 -04:00
Dave Horton
9cf9d4f587 Fix/0.8.5 cherries (#724)
* kill play task if bot responds verbs while actionHook delay is enabled (#712)

* kill play task if bot responds verbs while actionHook delay is enabled

* fix actionHook delay continues even the bot already responded verbs

* wip

* wip

* wip

* gather is hang if listenDuringPrompt = false and say/play task throw exception (#717)

* merge fix for Support ASR TTS fallback (#713)

---------

Co-authored-by: Hoan Luu Huu <110280845+xquanluu@users.noreply.github.com>
2024-04-17 11:01:21 -04:00
Hoan Luu Huu
09cf1345f6 tts span for whisper (#718)
* tts span for whisper

* support deepgram tts span

* support playht tts span

* support rimelabs tts span

* wip
2024-04-14 09:14:49 -04:00
Hoan Luu Huu
5b1d8a8ff3 Feat/ambient sounds (#678)
* initial support for coaching mode in conference

* wip

* wip

* add support for answer verb

* wip

* wip

* wip

* wip

* wip

* updates to rename option to dub

* wip

* wip

* wip

* update verb-specs

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* add option to boost audio signal in main channel

* wip

* wip

* wip

* wip

* wip

* wip

* for now, bypass use of streaming apis when generating tts audio for dub tracks

* add nested dub to dial

* wip

* add support for filler noise

* kill filler noise when gather killed

* wip

* wip

* while using sayOnTrack, we have to enclose the say command in double quotes

* disableTtsStreaming = false

* allow transcribe of b leg only on dial verb

* dub.say can either be text or object like say verb with text and synthesizer

* remove loop for sayOnTrack

* update speech-utils

* fixes for testing transcribe verb and support for dub and boostAudioSignal in lcc commands

* add dial.boostAudioSignal

* fix bug where session-level recognizer settings incorrectly overwrite verb-level settings

* update verb specs

* update dial to support array of dub verbs

* fix bug setting gain

* lint

* wip

* update speech-utils

* use new endpoint methods for mod_dub

---------

Co-authored-by: Dave Horton <daveh@beachdognet.com>
2024-03-23 16:23:57 -04:00
Hoan Luu Huu
06b066a3f2 update speech util to support whisper stream (#657)
* update speech util to support whisper stream

* minor editing of span attributes

* more span attrs cleanup

---------

Co-authored-by: Dave Horton <daveh@beachdognet.com>
2024-02-22 14:17:29 -05:00
Dave Horton
d0ebe3f99f fix possible undefined reference in precache audio (#662)
* fix possible undefined reference in precache audio

* fix parsing of JAMBONES_EAGERLY_PRE_CACHE_AUDIO
2024-02-22 07:58:41 -05:00
Dave Horton
9c33a790bd update to latest speech-utils (#639) 2024-02-08 15:54:45 -05:00
Hoan Luu Huu
a55f81676b Tts/elevenlabs streaming (#629)
* update to fsmrf with fix

* changes to support elevenlabs tts streaming

* say: add vendor data to span

* bug: tts spans must include cached property

* add env for JAMBONES_USE_FREESWITCH_TIMER_FD

* fix bug in prev commit

* wip

* linting

* wip - caching files generating by streaming tts

* wip caching

* cleanup some logs

* handle tts streaming failure, write alert

* update node version dependency

* set timerfd on outbound call scenarios

* default model to nova-2-phonecall when using deepgram

---------

Co-authored-by: Dave Horton <daveh@beachdognet.com>
2024-02-07 08:49:36 -05:00
Dave Horton
09a83e3a31 Feature/precache audio (#609)
* wip

* fix for establishing vendor etc

* more fixes

* avoid a pre-caching attempt if synth settings change
2024-01-13 12:51:25 -05:00
Hoan Luu Huu
3e8474867f support deepgram (#579)
* support deepgram

* update speech utils
2023-12-26 07:46:35 -05:00
Markus Frindt
86a14daf79 Update say task and add possibility to use elevenlabs options (#556)
* Update say task and add possibility to use elevenlabs options from synthesizer

* revert ms change

* fix contdition for alerting

---------

Co-authored-by: Markus Frindt <m.frindt@cognigy.com>
2023-12-01 09:51:22 -05:00
Hoan Luu Huu
0a0cbd57ba support elevenlabs options (#553)
* support elevenlabs options

* elevenlabs options from synthezier

* wip

* fix
2023-11-30 09:28:12 -05:00
Hoan Luu Huu
4b7b0e309b say verb support play audio in array (#482)
* say verb support play audio in array

* fixed review comments

* added separated testcase
2023-10-20 08:14:57 +02:00
Hoan Luu Huu
b1ecf069bf Feat/fallback speech 02 (#429)
* feat fallback speech

* feat fallback speech

* feat fallback speech

* gather verb

* fix

* wip

* wip

* wip

* wip

* wip

* transcribe

* transcribe

* fix

* wip

* wip

* wip

* wip

* fix

* wip

* fix
2023-08-22 09:22:01 -04:00
Hoan Luu Huu
f8c5abe9e9 feat: multi speech credential diff labels but same vendor (#426)
* feat: multi speech credential diff labels but same vendor

* update sql

* fix

* fix

* fix jslint

* fix review comment

* update verb spec version
2023-08-15 08:57:49 -04:00
Hoan Luu Huu
d96712a8d6 feat: add tts on account level (#366) 2023-06-02 09:31:28 -04:00
Dave Horton
feccc0fca7 add support for azure custom voices on a per-say basis (#346) 2023-05-09 13:25:43 -04:00
Dave Horton
690deed89d prune unused logging 2023-03-19 12:04:02 -04:00
Hoan Luu Huu
5ab24337b2 fix: use TTS_FAILURE alert type for synthAudio (#278)
Co-authored-by: Quan HL <quanluuhoang8@gmail.com>
2023-03-08 07:42:06 -05:00
Hoan Luu Huu
088316d266 fix: split ssml to correct chunks (#225)
* fix: split ssml to correct chunks

* fix: split ssml to correct chunks

* fixed: eslint

* fixed: eslint

* fixed: add comment to testcase

* fixed: review comments

* fixed: review comments

* fixed: review comments

* fixed: review comments

* fixed: review comments

Co-authored-by: Quan HL <quanluuhoang8@gmail.com>
2023-01-24 09:48:31 -05:00
Dave Horton
05a4665f87 Feature/force tts generation (#208)
* Feature: add option synthesizer.forceTtsGeneration #198

* Feature: add option synthesizer.forceTtsGeneration #198

* minor cleanup

* minor

Co-authored-by: Michal Tesar <michal@irevolution.group>
2023-01-04 15:42:48 -05:00
Dave Horton
a60c6a4740 add support for ws verb:status event notifications (#196) 2022-12-09 21:11:47 -05:00
Dave Horton
39fb762a15 ibm speech fix 2022-12-04 11:28:02 -05:00
Dave Horton
509bb065bb Feature/nuance stt (#185)
* initial changes to gather to support nuance stt

* updateSpeechCredentialLastUsed could be called without a speech_credential_sid if credentials are passed in the flow

* fix bugname

* typo

* added handlers for nuance

* logging

* major refactor of parsing transcriptions

* initial support for nuance in transcribe verb

* updates from testing

* cleanup some tests

* update action

* typo

* gather: start nuance timers after say/play completes

* update drachtio-fsrmf

* refactor some code

* typo

* log nuance error detail

* timeout handling

* typo

* handle nuance 413 response when recognition times out

* typo in specs.json

* add support for nuance resources

* fixes and tests for transcribe

* remove logging from test

* initial support for kryptonEndpoint

* try getting access token even when using krypton

* typo in kryptonEndpoint property

* add support for Nuance tts

* parse nuance voice and model for tts

* use nuance credentials from db

* update to db-helpers@0.7.0 with caching option

* add support for azure audio logging in gather/transcribe

* sync package-lock.json
2022-11-01 12:23:49 -04:00
Dave Horton
bd49dacac4 Say length text (#165)
* typo for media bug name in azure and punctuation fix

* say: split very long text intelligently

* more fixes from testing

* update to latest synthAudio
2022-09-14 17:17:29 +02:00
Dave Horton
3298918322 Feature/siprec server (#143)
* fixes from testing

* modify Task#exec to take resources as an object rather than argument list

* pass 2 endpoints to Transcribe when invoked in a SipRec call session

* logging

* change siprec invite to sendrecv just so freeswitch does not try to reinvite (TODO: block outgoing media at rtpengine)

* Config: when enabling recording, block until siprec dialog is established

* missed play verb in commit 031c79d

* linting

* bugfix: get final transcript in siprec call
2022-08-09 15:23:55 +02:00
Dave Horton
13ea559cb1 send error notification over websocket if tts fails 2022-04-21 14:33:49 -04:00
Dave Horton
698d12a95f clean up error handling in say verb 2022-04-21 10:27:33 -04:00
Dave Horton
a950f9f738 Feature/trace propagation (#96)
* add b3 header for trace propagation on initial webhook

* logging

* add tracing context to all webhooks

* Add span parameter to Task.getTracingPropagation. Pass proper span to getTracingPropagation calls in Task methods to propagate the proper spanId (#91)

* some tracing cleanup

* bugfix: azure stt results need to be ordered by confidence level before processing (#92)

* fix assertion

* bugfix: vad was not enabled on config verb, restart STT on empty transcript in gather

* gather: dont send webhook if call is gone

* rest outdial: handle 302 redirect so we can later cancel request if needed (#95)

* gather: restart if we get an empty transcript (looking at you, Azure)

Co-authored-by: javibookline <98887695+javibookline@users.noreply.github.com>
2022-04-01 14:48:27 -04:00
Dave Horton
6abfdafe05 Feature/opentelemetry (#89)
* initial adds for otel tracing

* initial basic testing

* basic tracing for incoming calls

* linting

* add traceId to the webhook params

* trace webhook calls

* tracing: add new commands as tags when receiving async commands over websocket

* tracing new commands

* add summary for config verb

* trace async commands

* bugfix: undefined ref

* tracing: give time for final webhooks before closing root span

* tracing bugfix: span for background gather was not ended

* tracing - minor tag changes

* tracing - add span atttribute for reason call ended

* trace call status webhooks, add app version to trace output

* config: add support for automatically re-enabling

* env var to customize service name in tracing UI

* config: change to use 'sticky' attribute to re-enable bargein automatically

* fix warnings

* when adulting create a new root span

* when background gather triggers bargein via vad clear queue of tasks

* additional trace attributes for dial and refer

* fix dial tracing

* add better summary for dial

* fix prev commit

* add exponential backoff to WsRequestor reconnection logic

* add calling number to log metadata, as this will be frequently the key data given for troubleshooting

* add accountSid to log metadata

* make handshake timeout for ws connections configurable with default 1.5 secs

* rename env var

* fix bug prev checkin

* logging fixes

* consistent env naming
2022-03-28 15:38:28 -04:00
Dave Horton
172dc1aaa7 Feature/config verb (#77)
* remove cognigy verb

* initial implementation of config verb

* further updates to config

* Bot mode alex (#75)

* do not use default as value for TTS/STT

* fix gather listener if no say or play provided

Co-authored-by: akirilyuk <a.kirilyuk@cognigy.com>

* gather: listenDuringPrompt requires a nested play/say

* fix exception

* say: fix exception where caller hangs up during say

* bugfix: sip refer was not ending if caller hungup during refer

* add support for sip:request to ws commands

* gather: when bargein is set and minBargeinWordCount is zero, kill audio on endOfUtterrance

* gather/transcribe: add support for google boost and azure custom endpoints

* minor logging changes

* lint error

Co-authored-by: akirilyuk <45361199+akirilyuk@users.noreply.github.com>
Co-authored-by: akirilyuk <a.kirilyuk@cognigy.com>
2022-03-06 15:09:45 -05:00
Dave Horton
f317fbaa45 Feature/gather enhancements (#73)
* add bargein support to gather

* bugfix: gather handles interim results from azure

* gather: support for min/max digits and interdigit timeout

* add task summary to some log messages

* logging improvements
2022-02-27 13:38:02 -05:00
Dave Horton
3c5d392407 Feature/ws api (#72)
initial changes to support websockets as an alternative to webhooks
2022-02-26 14:06:52 -05:00
Dave Horton
b27f349fc6 linting 2022-01-21 10:15:33 -05:00
Dave Horton
e1a023c21e bugfix: aws property is engine not platform 2022-01-21 09:57:58 -05:00
Dave Horton
8acb4d1a24 #58 - add support for platform (standard, or neural) when using aws tts 2022-01-19 19:46:24 -05:00
Dave Horton
12c34de15c changes for azure tts 2021-11-19 18:28:42 +00:00
Dave Horton
dcf27ba5d3 trim sensitive info from logs 2021-11-03 14:37:57 -04:00
Dave Horton
72345f83c1 Feature/minimal media anchoring (#36)
* initial WIP to remove freeswitch from media path when not recording or transcribing dial calls

* implement release-media and anchor-media operations

* mute/unmute now handled by rtpengine

* Dial: dtmf detection now based on SIP INFO events from sbcs and rtpengine

* add reason to gather action, bugfixes for transcribe and say
2021-10-21 11:59:45 -04:00
Dave Horton
a9e789f466 add support for autoscaling SBC SIP servers; bugfix: synthAudio calls must past stats obj 2021-10-02 12:40:56 -04:00