Commit Graph

85 Commits

Author SHA1 Message Date
rammohan-kore
e4e37d5697 feat/836: capturing callSid for STT and TTS alerts (#838)
* feat/836: capturing callSid for STT and TTS alerts

* feat/836: corrected assignment of callSid and added target_sid at few more alerts

* update github action

---------

Co-authored-by: Quan HL <quan.luuhoang8@gmail.com>
2024-08-05 12:14:08 -04:00
Dave Horton
ec6d7b3f42 persistent connection for custom stt vendors in transcribe (#794) 2024-06-28 08:33:10 -04:00
Dave Horton
76a3aa7f42 send end of utterance events if using deepgram, interim events are enabled, and utterance_end_ms option is set (#772) (#782) 2024-06-13 13:18:32 -04:00
Dave Horton
c0b56d4fc6 per email from microsoft, do not restart STT connection when we get a no audio event (#754) 2024-05-17 11:19:01 -04:00
Dave Horton
8d2b60c284 minor 2024-04-21 09:51:05 -04:00
Dave Horton
9cf9d4f587 Fix/0.8.5 cherries (#724)
* kill play task if bot responds verbs while actionHook delay is enabled (#712)

* kill play task if bot responds verbs while actionHook delay is enabled

* fix actionHook delay continues even the bot already responded verbs

* wip

* wip

* wip

* gather is hang if listenDuringPrompt = false and say/play task throw exception (#717)

* merge fix for Support ASR TTS fallback (#713)

---------

Co-authored-by: Hoan Luu Huu <110280845+xquanluu@users.noreply.github.com>
2024-04-17 11:01:21 -04:00
Dave Horton
bd002ede48 ignore google errors with error_code 0 2024-04-16 20:06:26 -04:00
Dave Horton
8999c85a71 Fixes/ws testing dh (#704)
* fixes from testing with translator app

* more updates

* linting

* update gh actions to node 20

* add support for google v2 preconfigured recognizer

* add support for google voice activity events

* update to speech-utils@0.0.45

* update speech-utils to support caching azure tts

* transcribe must buffer transcripts for channel 1 and 2 separately

* further fix for accumulating transcripts

* linting

* deepgram sends transcripts with empty alternatives array

* fix deepgram returning an empty array
2024-04-03 14:30:49 -04:00
Hoan Luu Huu
5b1d8a8ff3 Feat/ambient sounds (#678)
* initial support for coaching mode in conference

* wip

* wip

* add support for answer verb

* wip

* wip

* wip

* wip

* wip

* updates to rename option to dub

* wip

* wip

* wip

* update verb-specs

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* add option to boost audio signal in main channel

* wip

* wip

* wip

* wip

* wip

* wip

* for now, bypass use of streaming apis when generating tts audio for dub tracks

* add nested dub to dial

* wip

* add support for filler noise

* kill filler noise when gather killed

* wip

* wip

* while using sayOnTrack, we have to enclose the say command in double quotes

* disableTtsStreaming = false

* allow transcribe of b leg only on dial verb

* dub.say can either be text or object like say verb with text and synthesizer

* remove loop for sayOnTrack

* update speech-utils

* fixes for testing transcribe verb and support for dub and boostAudioSignal in lcc commands

* add dial.boostAudioSignal

* fix bug where session-level recognizer settings incorrectly overwrite verb-level settings

* update verb specs

* update dial to support array of dub verbs

* fix bug setting gain

* lint

* wip

* update speech-utils

* use new endpoint methods for mod_dub

---------

Co-authored-by: Dave Horton <daveh@beachdognet.com>
2024-03-23 16:23:57 -04:00
Hoan Luu Huu
b80d39d205 fix asrtimer always return vendor=deepgram (#682) 2024-03-13 12:57:55 -04:00
Dave Horton
dbd3b59901 fix #666 2024-02-26 09:39:49 -05:00
Dave Horton
565478cc0a #573 address race condition in pause/resume recording (#584) 2024-02-12 10:26:34 -05:00
Hoan Luu Huu
1a80910f91 fix pause transcribe cannot close transcription on 2nd leg (#621) 2024-01-18 11:21:25 -05:00
Dave Horton
f22d66dfd6 set default deepgram model by language and task (gather vs transcribe) (#610)
* set default deepgram model by language and task (gather vs transcribe)

* wip
2024-01-14 10:38:14 -05:00
Dave Horton
6f6d7a06b0 Revert "Fix/dual legs transcribe race condition (#593)" (#600)
This reverts commit 9d70ed96a1.
2024-01-09 08:23:13 -05:00
Hoan Luu Huu
9d70ed96a1 Fix/dual legs transcribe race condition (#593)
* fs only stop one of bugname when transcribe is used for dual legs

* wip

* fix review comment

* wip

* wip
2024-01-06 19:12:51 -05:00
Dave Horton
2c48083c26 fix to be more precise about removing custom event handlers so that w… (#580)
* fix to be more precise about removing custom event handlers so that when we stop a gather we dont also inadvertently stop a background transcribe as well

* test fixes

* fix: endpointing=false was being ignored for Deepgram
2023-12-28 11:00:27 -05:00
Hoan Luu Huu
9d8291f892 Transcribe background task (#576)
* first draft

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* update verb-specification

* fix comment reviews

* provide bugname when stopping transcription, otherwise it will continue

---------

Co-authored-by: Dave Horton <daveh@beachdognet.com>
2023-12-26 21:50:51 -05:00
Hoan Luu Huu
9eb315ecd6 fix config and stt task for altLanguages (#575)
* fix config and stt task for altLanguages

* clear freeswitch channel var when altLanguages is empty list
2023-12-25 22:21:34 -05:00
Hoan Luu Huu
2ec1460b4e nuance transcribe should have utteranceDetectionMode=multiple (#574)
* nuance transcribe should have utteranceDetectionMode=multiple

* nuance transcribe should have utteranceDetectionMode=multiple
2023-12-20 18:44:08 -05:00
Hoan Luu Huu
e30782ea7b fix riva transcribe issue (#570) 2023-12-20 10:25:17 -05:00
Hoan Luu Huu
30977b309c punctuation for microsoft (#566)
* punctuation for microsoft

* wip
2023-12-18 08:38:05 -05:00
Hoan Luu Huu
59248c7638 fix config verb can override recognizer *Options (#558)
* fix config verb can override recognizer *Options

* wip

* clean up gather and transcribe

* fix failing testcase

* fix failing testcase

* fix failing testcase
2023-12-05 20:27:55 -05:00
Hoan Luu Huu
1cdd0cf611 feat Audult call session should have its own requestor (#535)
* feat Audult call session should have its own requestor

* fix eslint

* fix eslint

* if user authenticate from http server instead of databse

* wip

* fix custom STT

* fix custom STT

* fix custom STT
2023-11-16 08:22:35 -05:00
Dave Horton
6d625d87ad Feat/assemblyai testing (#516)
* handle errors from assemblyai

* wip

* fix alert

* normalizeAssemblyai
2023-11-02 17:05:28 -04:00
Hoan Luu Huu
7fee2ba2dc feat assembly (#515)
* fix

* wip

* wip

* wip

* wip

* fix review comments
2023-11-02 09:25:04 -04:00
Dave Horton
f43a5c1491 deepgram: rework continuous asr, and resolve on speech_final not is_f… (#501)
* deepgram: rework continuous asr, and resolve on speech_final not is_final (wip)

* wip

* deepgram: empty final transcript should trigger resolve with speech if we have buffered transcripts

* wip

* fixes for deepgram compiling multiple transcripts

* test deepgram utteranceEndMs

* more handling of utteranceEndMs

* wip

* better handling of digit strings collected over multiple deepgram responses

* wip

* add support for deepgramOptions.shortUtterance which triggers off of is_final instead of speech_final

* apply deepgram fixes to transcribe

* cleanup continnuous asr

* more continuous asr fixes for deepgram

* update to verb-specifications for handling SttTask properties

* set log level for tests back to error
2023-10-30 13:57:25 -04:00
Hoan Luu Huu
2285ec5329 fix transcribe credential (#480) 2023-10-10 21:06:46 -04:00
Hoan Luu Huu
09ae083c9a fix: transcribe 2 channels (#479)
* fix: transcribe 2 channels

* fix

* fix
2023-10-10 08:46:50 -04:00
Hoan Luu Huu
ae9386791f fix cobalt model (#469) 2023-09-26 07:41:43 -04:00
Dave Horton
6a8a2aa955 update to provide hostport and model on the command line not via env (#465) 2023-09-13 13:53:00 -04:00
Dave Horton
a1793ac359 Speech vendor/cobalt (#463)
* initial changes for cobalt speech

* wip

* wip

* update to drachtio-fsmrf that supports cobalt

* update to verb-specifications with cobalt speech support

* more wip

* lint

* use node 18 with gh actions

* support for compiling cobalt hints

* fix bug in uuid_cobalt_compile_context

* update verb-specifications

* remove repeated code

* cobalt support for transcribe

* update to verb specs
2023-09-13 09:47:30 -04:00
Hoan Luu Huu
1477605e66 fix custom stt transcribe (#457) 2023-09-08 08:09:03 -04:00
Hoan Luu Huu
2935574440 feat pause resume transcribe (#438)
* feat pause resume transcribe

* wip

* fix jslint

* update fsmrf
2023-08-30 22:43:50 -04:00
Hoan Luu Huu
b1ecf069bf Feat/fallback speech 02 (#429)
* feat fallback speech

* feat fallback speech

* feat fallback speech

* gather verb

* fix

* wip

* wip

* wip

* wip

* wip

* transcribe

* transcribe

* fix

* wip

* wip

* wip

* wip

* fix

* wip

* fix
2023-08-22 09:22:01 -04:00
Hoan Luu Huu
f8c5abe9e9 feat: multi speech credential diff labels but same vendor (#426)
* feat: multi speech credential diff labels but same vendor

* update sql

* fix

* fix

* fix jslint

* fix review comment

* update verb spec version
2023-08-15 08:57:49 -04:00
Hoan Luu Huu
d7dcdb1d0c Continuos ASR for transcribe (#398)
* asrTimeout

* fix jslint

* change log

* fix interrim
2023-08-03 09:49:44 -04:00
Dave Horton
fbd0782258 #388 - support custom speech vendor in transcribe verb (#414)
Co-authored-by: Hoan Luu Huu <110280845+xquanluu@users.noreply.github.com>
2023-08-02 19:06:31 -04:00
Dave Horton
a7d07ce7ae add channel to transcribe, gather, and dtmf spans (#376) 2023-06-13 09:12:26 -04:00
Dave Horton
30beb9c093 transcribe: default hints and altLanguages (#371) 2023-06-06 13:41:31 -04:00
Dave Horton
01260ad054 transcribe: create otel child spans for each stt result that is returned in a long-running transcribe (#368) 2023-06-02 14:25:32 -04:00
Dave Horton
bd911c88f9 in verb transcribe, recognizer should be optional (default to application default) but transcriptionHook mudt be mandatory (#367) 2023-06-02 13:15:32 -04:00
Dave Horton
5131d524ce bugfix: allow for empty transcripts that nuance returns 2023-03-27 14:13:50 -04:00
Hoan Luu Huu
7be3c64116 feat: update speech-ultil version 1.0.1 (#275)
* feat: update speech-ultil version 1.0.1

* feat: update speech-ultil version 1.0.1

* more fixes for custom stt

* more fixes

* fixes

* update drachtio-fsmrf

* pass url to mod_jambonz_transcribe

* transcription utils: handle custom results

* handle custom speech vendor errors

* add support for hints to custom speech

* change to custom speech options

* send hints as an array for custom speech

* update latest speech-utils

* transcribe: changes to support soniox

* bugfix: soniox transcribe

---------

Co-authored-by: Quan HL <quanluuhoang8@gmail.com>
Co-authored-by: Dave Horton <daveh@beachdognet.com>
2023-03-12 19:38:36 -04:00
Dave Horton
1c683f1142 initial changes for soniox (#270)
* initial changes for soniox

* changes to gather for soniox

* parse soniox stt results

* handle <end> token for soniox

* soniox: handle empty array of words

* support for soniox hints

* add soniox storage options

* update to verb specs

* add support for transcribe

* compile soniox transcripts

* gather: kill no input timer for soniox when we get interim results

* fix buffering of soniox transcripts

* fix for compiling soniox transcript

* another fix for compiling soniox transcript

* another fix

* handling of <end> token

* fix soniox bug

* gather: fixes for soniox continous asr

* fix undefined variable reference

* fix prev commit

* bugfix: allow verb_status requests

* gather: for soniox no need to restart transcription after final transcription received

* update verb specs

* update verb specs, fixes for continuous asr:
2023-03-03 13:37:55 -05:00
Hoan Luu Huu
c09425fa89 feat: use verb-specifications (#262)
* feat: use verb-specifications

* feat: use verb-specifications

* fix: verb specification v2

* remove irrelevant tests

* fix: verb-scpecification

* update to use @jambonz/verb-specifications

---------

Co-authored-by: Quan HL <quanluuhoang8@gmail.com>
Co-authored-by: Dave Horton <daveh@beachdognet.com>
2023-02-15 09:56:23 -05:00
Dave Horton
0fdcb3a6d6 Feature/nvidia speech (#261)
* initial changes for nvidia speech

* allow nvidia speech credentials to be set at runtime

* update drachtio-fsmrf

* fix handling of nvidia-specific options

* fix nvidia custom config

* fix nvidia word time offsets

* fix nvidia custom configuration

* normalize nvidia transcripts

* update to @jambonz/realtime-dbhelpers with nvidia tts support
2023-02-12 14:06:01 -05:00
Dave Horton
567b03fd36 bugfix: transcribe/gather using default as vendor 2023-01-11 15:31:24 -05:00
Dave Horton
d5c04d2133 transcribe and gather: silently discard listening events from ibm stt 2023-01-11 14:59:15 -05:00
Dave Horton
71a2435c63 Feature/ibm watson (#193)
* initial changes to support ibm watson

* update specs.json for ibm

* update to drachtio-fsmrf with support for ibm

* bugfix: set access token for ibm stt, not api_key

* fix name of api_key

* normalize ibm transcription results

* rework ibm credentials

* bugfix setting runtime speech creds

* bugfix: ibm region

* typo

* changes to transcribe for ibm watson

* implement connect handler

* bugfix: bind error

* proper use of result_index

* ibm error handling
2022-11-21 22:09:37 -05:00