103 Commits

Author SHA1 Message Date
Hoan Luu Huu
a5fa5fce5b Fixed transcribe 2 legs cannot fallback (#1451)
* fixed transcribe cannot fallback for specific endpoint

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip
2025-11-28 21:43:05 -05:00
Hoan Luu Huu
89cc39f726 support gladia stt (#1397)
* support gladia stt

* wip

* wip

* update verb specification
2025-10-20 04:56:39 -04:00
Hoan Luu Huu
a00703a067 support houndify stt (#1364)
* support houndify stt

* wip

* wip

* wip

* update houndify stt parameters

* wip

* wip
2025-10-14 00:55:21 -04:00
Hoan Luu Huu
854c26db11 support deepgramflux (#1373)
* support deepgramflux

* wip

* wip

* wip

* wip

* update verb scpecification
2025-10-03 10:38:39 -04:00
Dave Horton
1d6f84c2d7 add event handler for when deepgram closes with an error (#1372) 2025-09-28 14:18:56 -04:00
Hoan Luu Huu
eb449e9169 support deepgram river (#1273)
* support deepgram river

* wip

* rebase

* fix review comment

---------

Co-authored-by: Dave Horton <daveh@beachdognet.com>
2025-07-29 13:49:43 -04:00
Hoan Luu Huu
158d9d7d25 support stt latency metrics (#1252)
* support stt latency metrics

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* enable stt latency calculator by config verb

* wip

* wip

* wip

* fix jslint

* fixed gather timeout does not have latency calculation

* upadte verb specification to use notifySttLatency

* move stt latency metric from call session to stt-latency calculator

* wip
2025-07-29 09:56:37 -04:00
Hoan Luu Huu
337c1cded0 fixed transcription is not received when call is terminated (#1259)
* fixed transcription is not received when call is terminated

* wip

* fixed failing testcases

* wip

* wip

* wip

* wip

* should not do gracefulshutdown on stopAmd
2025-07-09 10:20:09 -04:00
Hoan Luu Huu
5be6c54339 support mod_cartesia_transcribe (#1245) 2025-06-17 20:54:26 +02:00
Hoan Luu Huu
6205959f53 fix microsoft stt max client buffer size error for transcribe verb (#1173) 2025-04-29 09:41:24 -04:00
rammohan-y
0a541e089d Fix for https://github.com/jambonz/jambonz-feature-server/issues/1138 (#1139) 2025-04-04 09:02:18 -04:00
Dave Horton
fcaf2e59e7 initial changes for openai stt (#1127)
* initial changes for openai stt

* wip

* wip

* wip

* wip

* wip

* make minBargeinWordCount work for openai

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wipp

* wip

* wip

* wip

* openai stt: support for prompt templates

* lint

* wip

* support openai semantic_vad

* wip

* transcribe supports openai stt

* sip

* wip

* wip

* refactor list of stt vendors that dont need to be restarted after a final transcript

* cleanup

* wip

* cleanup

* wip

* wip

* wip

* remove credentials from log

* comment
2025-03-28 13:14:58 -04:00
Hoan Luu Huu
dbdc1cd43d support voxist stt (#1066)
* support voxist stt

* wip
2025-02-05 08:33:35 -05:00
Dave Horton
c1330d4651 fix transcribe fixes for speechmatics (#978)
* fix transcribe fixes for speechmatics

* update to verb-specs with fixes for speechmatics

* add support for speechmatics translation

* add handlers for receiving translations

* call translation hookd

* gather: no need to restart speechmatics after a final transcript during continuous asr

* graceful shutdown

* wip

* wip

* wip

* wip

* wip
2024-11-16 10:21:04 -05:00
rammohan-y
7911459c8c feat/940 stopped calling updateSpeechCredentialLastUsed (#944) 2024-11-05 15:19:08 -05:00
Dave Horton
f84b3793e1 Feat/speechmatics (#932)
* wip

* initial working version of speechmatics

* linting
2024-10-12 18:42:53 -04:00
Hoan Luu Huu
96b3b0fe07 Allow Say, Gather, Transcribe is able to finished if there is error for speech credential (#910)
* allow move to next task if say verb is failed because of speech credential

* allow move to next task if say verb is failed because of speech credential

* allow move to next task if say verb is failed because of speech credential

* wip

* wip
2024-10-01 13:40:41 -04:00
Markus Frindt
402adc2098 add label to tts stt spans (#909)
Co-authored-by: Markus Frindt <m.frindt@cognigy.com>
2024-09-25 16:44:15 -04:00
rammohan-kore
e4e37d5697 feat/836: capturing callSid for STT and TTS alerts (#838)
* feat/836: capturing callSid for STT and TTS alerts

* feat/836: corrected assignment of callSid and added target_sid at few more alerts

* update github action

---------

Co-authored-by: Quan HL <quan.luuhoang8@gmail.com>
2024-08-05 12:14:08 -04:00
Dave Horton
ec6d7b3f42 persistent connection for custom stt vendors in transcribe (#794) 2024-06-28 08:33:10 -04:00
Dave Horton
76a3aa7f42 send end of utterance events if using deepgram, interim events are enabled, and utterance_end_ms option is set (#772) (#782) 2024-06-13 13:18:32 -04:00
Dave Horton
c0b56d4fc6 per email from microsoft, do not restart STT connection when we get a no audio event (#754) 2024-05-17 11:19:01 -04:00
Dave Horton
8d2b60c284 minor 2024-04-21 09:51:05 -04:00
Dave Horton
9cf9d4f587 Fix/0.8.5 cherries (#724)
* kill play task if bot responds verbs while actionHook delay is enabled (#712)

* kill play task if bot responds verbs while actionHook delay is enabled

* fix actionHook delay continues even the bot already responded verbs

* wip

* wip

* wip

* gather is hang if listenDuringPrompt = false and say/play task throw exception (#717)

* merge fix for Support ASR TTS fallback (#713)

---------

Co-authored-by: Hoan Luu Huu <110280845+xquanluu@users.noreply.github.com>
2024-04-17 11:01:21 -04:00
Dave Horton
bd002ede48 ignore google errors with error_code 0 2024-04-16 20:06:26 -04:00
Dave Horton
8999c85a71 Fixes/ws testing dh (#704)
* fixes from testing with translator app

* more updates

* linting

* update gh actions to node 20

* add support for google v2 preconfigured recognizer

* add support for google voice activity events

* update to speech-utils@0.0.45

* update speech-utils to support caching azure tts

* transcribe must buffer transcripts for channel 1 and 2 separately

* further fix for accumulating transcripts

* linting

* deepgram sends transcripts with empty alternatives array

* fix deepgram returning an empty array
2024-04-03 14:30:49 -04:00
Hoan Luu Huu
5b1d8a8ff3 Feat/ambient sounds (#678)
* initial support for coaching mode in conference

* wip

* wip

* add support for answer verb

* wip

* wip

* wip

* wip

* wip

* updates to rename option to dub

* wip

* wip

* wip

* update verb-specs

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* add option to boost audio signal in main channel

* wip

* wip

* wip

* wip

* wip

* wip

* for now, bypass use of streaming apis when generating tts audio for dub tracks

* add nested dub to dial

* wip

* add support for filler noise

* kill filler noise when gather killed

* wip

* wip

* while using sayOnTrack, we have to enclose the say command in double quotes

* disableTtsStreaming = false

* allow transcribe of b leg only on dial verb

* dub.say can either be text or object like say verb with text and synthesizer

* remove loop for sayOnTrack

* update speech-utils

* fixes for testing transcribe verb and support for dub and boostAudioSignal in lcc commands

* add dial.boostAudioSignal

* fix bug where session-level recognizer settings incorrectly overwrite verb-level settings

* update verb specs

* update dial to support array of dub verbs

* fix bug setting gain

* lint

* wip

* update speech-utils

* use new endpoint methods for mod_dub

---------

Co-authored-by: Dave Horton <daveh@beachdognet.com>
2024-03-23 16:23:57 -04:00
Hoan Luu Huu
b80d39d205 fix asrtimer always return vendor=deepgram (#682) 2024-03-13 12:57:55 -04:00
Dave Horton
dbd3b59901 fix #666 2024-02-26 09:39:49 -05:00
Dave Horton
565478cc0a #573 address race condition in pause/resume recording (#584) 2024-02-12 10:26:34 -05:00
Hoan Luu Huu
1a80910f91 fix pause transcribe cannot close transcription on 2nd leg (#621) 2024-01-18 11:21:25 -05:00
Dave Horton
f22d66dfd6 set default deepgram model by language and task (gather vs transcribe) (#610)
* set default deepgram model by language and task (gather vs transcribe)

* wip
2024-01-14 10:38:14 -05:00
Dave Horton
6f6d7a06b0 Revert "Fix/dual legs transcribe race condition (#593)" (#600)
This reverts commit 9d70ed96a1.
2024-01-09 08:23:13 -05:00
Hoan Luu Huu
9d70ed96a1 Fix/dual legs transcribe race condition (#593)
* fs only stop one of bugname when transcribe is used for dual legs

* wip

* fix review comment

* wip

* wip
2024-01-06 19:12:51 -05:00
Dave Horton
2c48083c26 fix to be more precise about removing custom event handlers so that w… (#580)
* fix to be more precise about removing custom event handlers so that when we stop a gather we dont also inadvertently stop a background transcribe as well

* test fixes

* fix: endpointing=false was being ignored for Deepgram
2023-12-28 11:00:27 -05:00
Hoan Luu Huu
9d8291f892 Transcribe background task (#576)
* first draft

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* update verb-specification

* fix comment reviews

* provide bugname when stopping transcription, otherwise it will continue

---------

Co-authored-by: Dave Horton <daveh@beachdognet.com>
2023-12-26 21:50:51 -05:00
Hoan Luu Huu
9eb315ecd6 fix config and stt task for altLanguages (#575)
* fix config and stt task for altLanguages

* clear freeswitch channel var when altLanguages is empty list
2023-12-25 22:21:34 -05:00
Hoan Luu Huu
2ec1460b4e nuance transcribe should have utteranceDetectionMode=multiple (#574)
* nuance transcribe should have utteranceDetectionMode=multiple

* nuance transcribe should have utteranceDetectionMode=multiple
2023-12-20 18:44:08 -05:00
Hoan Luu Huu
e30782ea7b fix riva transcribe issue (#570) 2023-12-20 10:25:17 -05:00
Hoan Luu Huu
30977b309c punctuation for microsoft (#566)
* punctuation for microsoft

* wip
2023-12-18 08:38:05 -05:00
Hoan Luu Huu
59248c7638 fix config verb can override recognizer *Options (#558)
* fix config verb can override recognizer *Options

* wip

* clean up gather and transcribe

* fix failing testcase

* fix failing testcase

* fix failing testcase
2023-12-05 20:27:55 -05:00
Hoan Luu Huu
1cdd0cf611 feat Audult call session should have its own requestor (#535)
* feat Audult call session should have its own requestor

* fix eslint

* fix eslint

* if user authenticate from http server instead of databse

* wip

* fix custom STT

* fix custom STT

* fix custom STT
2023-11-16 08:22:35 -05:00
Dave Horton
6d625d87ad Feat/assemblyai testing (#516)
* handle errors from assemblyai

* wip

* fix alert

* normalizeAssemblyai
2023-11-02 17:05:28 -04:00
Hoan Luu Huu
7fee2ba2dc feat assembly (#515)
* fix

* wip

* wip

* wip

* wip

* fix review comments
2023-11-02 09:25:04 -04:00
Dave Horton
f43a5c1491 deepgram: rework continuous asr, and resolve on speech_final not is_f… (#501)
* deepgram: rework continuous asr, and resolve on speech_final not is_final (wip)

* wip

* deepgram: empty final transcript should trigger resolve with speech if we have buffered transcripts

* wip

* fixes for deepgram compiling multiple transcripts

* test deepgram utteranceEndMs

* more handling of utteranceEndMs

* wip

* better handling of digit strings collected over multiple deepgram responses

* wip

* add support for deepgramOptions.shortUtterance which triggers off of is_final instead of speech_final

* apply deepgram fixes to transcribe

* cleanup continnuous asr

* more continuous asr fixes for deepgram

* update to verb-specifications for handling SttTask properties

* set log level for tests back to error
2023-10-30 13:57:25 -04:00
Hoan Luu Huu
2285ec5329 fix transcribe credential (#480) 2023-10-10 21:06:46 -04:00
Hoan Luu Huu
09ae083c9a fix: transcribe 2 channels (#479)
* fix: transcribe 2 channels

* fix

* fix
2023-10-10 08:46:50 -04:00
Hoan Luu Huu
ae9386791f fix cobalt model (#469) 2023-09-26 07:41:43 -04:00
Dave Horton
6a8a2aa955 update to provide hostport and model on the command line not via env (#465) 2023-09-13 13:53:00 -04:00
Dave Horton
a1793ac359 Speech vendor/cobalt (#463)
* initial changes for cobalt speech

* wip

* wip

* update to drachtio-fsmrf that supports cobalt

* update to verb-specifications with cobalt speech support

* more wip

* lint

* use node 18 with gh actions

* support for compiling cobalt hints

* fix bug in uuid_cobalt_compile_context

* update verb-specifications

* remove repeated code

* cobalt support for transcribe

* update to verb specs
2023-09-13 09:47:30 -04:00