Commit Graph
190 Commits
Author SHA1 Message Date
Dave HortonandClaude Opus 5.5 baf97dabc2 fix(gather): clear a finalized deepgram interim word so UtteranceEnd is not deferred forever (#1588)
* fix(gather): clear a finalized deepgram interim word so UtteranceEnd is not deferred forever

The #1088 check defers UtteranceEnd while an interim word newer than
last_word_end is pending. The marker was only cleared by a non-empty
is_final, so an interim word that Deepgram later dropped (e.g. line
noise, followed by an empty final) left it set: the gather waited for
the caller to speak again and merged both utterances.

Clear the marker on any Deepgram is_final result whose window
(start + duration) covers it. A word that Deepgram carries into the
next segment still defers UtteranceEnd.

Port of jambonz/feature-server#223.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(gather): return the buffer when a deferred deepgram UtteranceEnd is satisfied

Clearing the unprocessed-word marker only helps if the covering final
arrives before UtteranceEnd. If it arrives after, UtteranceEnd has
already been deferred and Deepgram sends no further UtteranceEnd until
new speech, so the next utterance is still merged in.

Remember a deferred UtteranceEnd, and when a later final clears the
marker, return the buffered transcript once that final has been
handled (setImmediate, since _onTranscription has early returns).

Port of jambonz/feature-server#225.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(gather): only an empty final ends a gather early after a deferred UtteranceEnd

A final with words means the caller resumed talking after the pause;
Deepgram will send a new UtteranceEnd after those words, so keep the
existing flow instead of cutting the caller off mid-sentence (or
skipping the asrTimeout wait). Only a final that drops the pending word
(empty) leaves nothing else to end the gather.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 17:45:27 -04:00
Sam Machin 521a89e5f3 Update gather.js (#1557)
* Update gather.js

* use spread
2026-06-23 07:38:20 -04:00
Hoan Luu Huu 44fbd91053 fixed listern verb terminated due to background task listen failure,teardown wrong handler as well (#1554) 2026-06-05 07:48:44 -04:00
Sam MachinandDave Horton 315eb98d86 add sp_sid to alerts (#1533)
* add sp_sid to alerts

* bump time-series

---------

Co-authored-by: Dave Horton <daveh@beachdognet.com>
2026-03-29 16:07:08 -04:00
Sam Machin 1f5fa8d49e Update gather.js (#1526) 2026-03-19 15:29:03 -04:00
Hoan Luu Huu e4ec0025c3 Fix/gladia multi sessions (#1487)
* support gladia transcribe multi channels

* wip
2026-01-07 10:46:33 -05:00
Hoan Luu Huu c5cd488fdf fixed gather should ignore transcription if task is killed/resolved. (#1465)
* fixed gather should ignore transcription if task is killed/resolved.

* wip
2025-12-12 09:03:08 -05:00
Dave Horton e396b6aa98 fix #1466: (#1467)
* fix #1466:

* do not send tts streaming events when we are not doing tts streaming
2025-12-09 09:43:53 -05:00
Hoan Luu Huu a5fa5fce5b Fixed transcribe 2 legs cannot fallback (#1451)
* fixed transcribe cannot fallback for specific endpoint

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip
2025-11-28 21:43:05 -05:00
Dave Horton cc1751f500 fix race condition where gather resolves with speech transcript but t… (#1449)
* fix race condition where gather resolves with speech transcript but timeout timer gets set after the resolve and is left running after gather completes

* remove unneeded line of code
2025-11-27 11:44:49 -06:00
Hoan Luu HuuandDave Horton 2de24af169 fixed gather does not start timeout on bargin (#1421)
* fixed gather does not start timeout on bargin

* with previous change, no need to emit playDone since no where in the code are we listening for it

---------

Co-authored-by: Dave Horton <daveh@beachdognet.com>
2025-11-03 13:11:59 -05:00
Hoan Luu Huu 89cc39f726 support gladia stt (#1397)
* support gladia stt

* wip

* wip

* update verb specification
2025-10-20 04:56:39 -04:00
Hoan Luu Huu a00703a067 support houndify stt (#1364)
* support houndify stt

* wip

* wip

* wip

* update houndify stt parameters

* wip

* wip
2025-10-14 00:55:21 -04:00
Dave Horton b4ed4c8c46 #1385: Gather - dont start the continuous asr timer when we first start listening if this is a background gather (#1386) 2025-10-09 08:47:51 -04:00
Dave Horton 28bf0d3477 send eager_eot events (#1382) 2025-10-06 16:50:20 -04:00
Hoan Luu Huu 854c26db11 support deepgramflux (#1373)
* support deepgramflux

* wip

* wip

* wip

* wip

* update verb scpecification
2025-10-03 10:38:39 -04:00
Dave Horton 1d6f84c2d7 add event handler for when deepgram closes with an error (#1372) 2025-09-28 14:18:56 -04:00
Sam Machin a95a6d1683 clear main timout when interdigit timeout is started, (#1351)
* clear main timout when interdigit timeout is started,

also clear the asrTimeout when dtmf has taken priority

* lint

* lint
2025-09-12 09:01:51 -04:00
Sam Machin 654ccd9d9d start timeout on bargein digits (#1312) 2025-08-19 08:34:33 -04:00
Hoan Luu HuuandDave Horton eb449e9169 support deepgram river (#1273)
* support deepgram river

* wip

* rebase

* fix review comment

---------

Co-authored-by: Dave Horton <daveh@beachdognet.com>
2025-07-29 13:49:43 -04:00
Hoan Luu Huu 158d9d7d25 support stt latency metrics (#1252)
* support stt latency metrics

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* enable stt latency calculator by config verb

* wip

* wip

* wip

* fix jslint

* fixed gather timeout does not have latency calculation

* upadte verb specification to use notifySttLatency

* move stt latency metric from call session to stt-latency calculator

* wip
2025-07-29 09:56:37 -04:00
rammohan-y 6e7521c91c Fix for issue where we should not delay from gather when JAMBONES_TRANSCRIBE_EP_DESTROY_DELAY_MS is set (#1283)
* Fix for issue https://github.com/jambonz/jambonz-feature-server/issues/1281

* Commented the reason for the change
2025-07-17 10:36:22 -04:00
Hoan Luu Huu e0e2ade289 fixed gather cannot timeout if listenDuringPrompt is true (#1276)
* fixed gather cannot timeout if listenDuringPrompt is true

* wip

* wip

* wip
2025-07-15 22:53:22 -04:00
Hoan Luu Huu 337c1cded0 fixed transcription is not received when call is terminated (#1259)
* fixed transcription is not received when call is terminated

* wip

* fixed failing testcases

* wip

* wip

* wip

* wip

* should not do gracefulshutdown on stopAmd
2025-07-09 10:20:09 -04:00
Hoan Luu Huu 8c67c05d87 fixed bargin task loop forever (#1253) 2025-06-22 06:20:24 +02:00
Hoan Luu Huu 5be6c54339 support mod_cartesia_transcribe (#1245) 2025-06-17 20:54:26 +02:00
Dave Horton 603dd482bc add a bit more info logging to gather (#1211) 2025-05-26 09:34:21 -04:00
Sam Machin 6b68d32e2c end if last_word_end is -1 (#1196)
* end if last_word_end is -1

* lint
2025-05-12 12:11:32 -04:00
Hoan Luu Huu c2666b7a09 fixed deepgram gather cannot be timeout on empty transcription with continueAsr (#1171) 2025-04-28 08:36:31 -04:00
Dave Horton fcaf2e59e7 initial changes for openai stt (#1127)
* initial changes for openai stt

* wip

* wip

* wip

* wip

* wip

* make minBargeinWordCount work for openai

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wipp

* wip

* wip

* wip

* openai stt: support for prompt templates

* lint

* wip

* support openai semantic_vad

* wip

* transcribe supports openai stt

* sip

* wip

* wip

* refactor list of stt vendors that dont need to be restarted after a final transcript

* cleanup

* wip

* cleanup

* wip

* wip

* wip

* remove credentials from log

* comment
2025-03-28 13:14:58 -04:00
Hoan Luu Huu f33f197e8d gather say support ttstream (#1128) 2025-03-27 07:19:19 -04:00
Dave Horton e09c763d3a #1088 ignore UtteranceEnd if we have unprocessed words (#1089)
* #1088 ignore UtteranceEnd if we have unprocessed words

* wip
2025-02-18 16:30:59 -05:00
Dave Horton 122d267816 better handling of flush commands (#1081)
* better handling of flush commands

* rework buffering of tokens

* gather: when returning low confidence also provide the transcript

* better error handling in tts:tokens

* special handling of asr timeout for speechmatics

* remove some logs that were excessively wordy
2025-02-18 09:31:11 -05:00
Hoan Luu Huu dbdc1cd43d support voxist stt (#1066)
* support voxist stt

* wip
2025-02-05 08:33:35 -05:00
Dave Horton f71f0ac69a Fix/speechmatics (#1042)
* add speechmatics options

* wip

* speechmatics does not do endpointing for us so we need to flip on continuousAsr

* speechmatics: continousAsr should be at least equal to max_delay, if set
2025-01-15 19:12:15 -05:00
Dave HortonandHoan Luu Huu ba3f46df64 Feat/tts streaming (#994)
* wip

* add TtsStreamingBuffer class to abstract handling of streaming tokens

* wip

* add throttling support

* support background ttsStream (#995)

* wip

* add TtsStreamingBuffer class to abstract handling of streaming tokens

* wip

* support background ttsStream

* wip

---------

Co-authored-by: Dave Horton <daveh@beachdognet.com>

* wip

* dont send if we have nothing to send

* initial testing with cartesia

* wip

---------

Co-authored-by: Hoan Luu Huu <110280845+xquanluu@users.noreply.github.com>
2024-12-18 14:44:37 -05:00
rammohan-y efdb56f0a0 feat/971 - fix to allow hints objects array (#987) 2024-11-28 07:25:10 -05:00
rammohan-y c874ab8100 feat/975: fixed continuous asr not stopping when asrDtmfTerminationDi… (#977)
* feat/975: fixed continuous asr not stopping when asrDtmfTerminationDigit is configured

* feat/975: giving first preference to asrDtmfTerminationDigit if there is already ASR happened
2024-11-26 08:23:11 -05:00
Hoan Luu Huu c8c3738ae8 custom stt vendor ws connection should not be closed in asrTimeout (#973) 2024-11-18 10:17:31 -05:00
Dave Horton c1330d4651 fix transcribe fixes for speechmatics (#978)
* fix transcribe fixes for speechmatics

* update to verb-specs with fixes for speechmatics

* add support for speechmatics translation

* add handlers for receiving translations

* call translation hookd

* gather: no need to restart speechmatics after a final transcript during continuous asr

* graceful shutdown

* wip

* wip

* wip

* wip

* wip
2024-11-16 10:21:04 -05:00
Dave Horton 851c071345 fix for #962 (#963) 2024-11-08 07:12:08 -05:00
rammohan-y 7911459c8c feat/940 stopped calling updateSpeechCredentialLastUsed (#944) 2024-11-05 15:19:08 -05:00
Dave Horton f541ff1a15 add support for aws language model name when transcribing (#890)
* add support for aws language model name when transcribing

* wip - from prev branch

* wip

* support both aws grpc and ws api - detect based on transcription payload

* update to drachtio-fsmrf@4.0.0

* fix for grpc compatibility, requires JAMBONES_AWS_TRANSCRIBE_USE_GRPC env

* back out major update to drachtio-srf and fsmrf; that should come in a separate PR

* update drachtio-srf
2024-10-12 19:46:31 -04:00
Dave Horton f84b3793e1 Feat/speechmatics (#932)
* wip

* initial working version of speechmatics

* linting
2024-10-12 18:42:53 -04:00
Hoan Luu Huu 96b3b0fe07 Allow Say, Gather, Transcribe is able to finished if there is error for speech credential (#910)
* allow move to next task if say verb is failed because of speech credential

* allow move to next task if say verb is failed because of speech credential

* allow move to next task if say verb is failed because of speech credential

* wip

* wip
2024-10-01 13:40:41 -04:00
Hoan Luu Huu b9ef00dfc7 Fixed Gather digits does not work without nested say/play (#914)
* Fixed Gather digits does not work without nested say/play

* fix review comment

* add assert to make sure we don't register dtmf twice in gather verb
2024-09-30 07:46:21 -04:00
Markus FrindtandMarkus Frindt 402adc2098 add label to tts stt spans (#909)
Co-authored-by: Markus Frindt <m.frindt@cognigy.com>
2024-09-25 16:44:15 -04:00
Dave Horton 68992bccf6 fix #866 (#867) 2024-08-16 14:54:22 -04:00
Hoan Luu Huu d3eb106d5d clear gather timeout if imterim result received (#800)
* clear gather timeout if imterim result received

* fix to reset timeout timer if receive interrim result

* fix to reset timeout timer if receive interrim result
2024-08-08 07:50:46 -04:00
c2a1819cbb feat/813 - notify speech-bargein-detected and dtmf-bargein-detected events (#823)
* feat/813 - notify speech-bargein-detected and dtmf-bargein-detected events

* fix for #826 race condition in say (#827)

* fix for #826 race condition in say

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* Update transcription-utils.js (#802)

* Check the confidence levels of a transcript  with minConfidence (#808)

* https://github.com/jambonz/jambonz-feature-server/issues/807

* feat/807: Using minConfidence from recognizer settings

* feat/807: new reason stt-min-confidence-error

* feat/807: sending stt-min-confidence instead of  stt-min-confidence-error

* feat/807: sending stt-low-confidence instead of  stt-min-confidence-error

* feat/807 - removed ? for this.data

* fix conference end is not sent when moderator leave conference (#825)

* fix conference end is not sent when moderator leave conference

* wip

* fix review comment

* feat/813: checking for playComplete before sending dtmf-bargein-detected event

* feat/813: added this.playComplete=true at the end of _killAudio method

* feat/813: removed empty line

* feat/813: removed nested if and added condition to main if

* feat/813: notifyStatus called when not playComplete

* feat/813: referring to time-series 0.2.9 version

* feat/813: generated package-lock.json

---------

Co-authored-by: Dave Horton <daveh@beachdognet.com>
Co-authored-by: Vinod Dharashive <vdharashive@gmail.com>
Co-authored-by: Hoan Luu Huu <110280845+xquanluu@users.noreply.github.com>
2024-08-06 09:34:15 -04:00