Commit Graph

43 Commits

Author SHA1 Message Date
Markus Frindt
ef2306e558 Improve Deepgram default modely by language (#641)
Co-authored-by: Markus Frindt <m.frindt@cognigy.com>
2024-02-12 09:53:14 -05:00
Dave Horton
9f9a9ec598 initial changes for deepgram on-prem (#636)
* initial changes for deepgram on-prem

* typo

* fixes for selecting deepgram model

* update some property names

* wip

* wip

* wip
2024-02-07 14:21:05 -05:00
Hoan Luu Huu
a55f81676b Tts/elevenlabs streaming (#629)
* update to fsmrf with fix

* changes to support elevenlabs tts streaming

* say: add vendor data to span

* bug: tts spans must include cached property

* add env for JAMBONES_USE_FREESWITCH_TIMER_FD

* fix bug in prev commit

* wip

* linting

* wip - caching files generating by streaming tts

* wip caching

* cleanup some logs

* handle tts streaming failure, write alert

* update node version dependency

* set timerfd on outbound call scenarios

* default model to nova-2-phonecall when using deepgram

---------

Co-authored-by: Dave Horton <daveh@beachdognet.com>
2024-02-07 08:49:36 -05:00
Hoan Luu Huu
b423a51638 feat: allow update azure endpoint ID from recognizer property (#612) 2024-01-17 07:34:02 -05:00
Dave Horton
f22d66dfd6 set default deepgram model by language and task (gather vs transcribe) (#610)
* set default deepgram model by language and task (gather vs transcribe)

* wip
2024-01-14 10:38:14 -05:00
Dave Horton
2c48083c26 fix to be more precise about removing custom event handlers so that w… (#580)
* fix to be more precise about removing custom event handlers so that when we stop a gather we dont also inadvertently stop a background transcribe as well

* test fixes

* fix: endpointing=false was being ignored for Deepgram
2023-12-28 11:00:27 -05:00
Hoan Luu Huu
9eb315ecd6 fix config and stt task for altLanguages (#575)
* fix config and stt task for altLanguages

* clear freeswitch channel var when altLanguages is empty list
2023-12-25 22:21:34 -05:00
Dave Horton
47fbc1a4a4 allow custom speech with no auth token (#571) 2023-12-18 14:51:34 -05:00
Hoan Luu Huu
30977b309c punctuation for microsoft (#566)
* punctuation for microsoft

* wip
2023-12-18 08:38:05 -05:00
Dave Horton
6d625d87ad Feat/assemblyai testing (#516)
* handle errors from assemblyai

* wip

* fix alert

* normalizeAssemblyai
2023-11-02 17:05:28 -04:00
Hoan Luu Huu
7fee2ba2dc feat assembly (#515)
* fix

* wip

* wip

* wip

* wip

* fix review comments
2023-11-02 09:25:04 -04:00
Dave Horton
f43a5c1491 deepgram: rework continuous asr, and resolve on speech_final not is_f… (#501)
* deepgram: rework continuous asr, and resolve on speech_final not is_final (wip)

* wip

* deepgram: empty final transcript should trigger resolve with speech if we have buffered transcripts

* wip

* fixes for deepgram compiling multiple transcripts

* test deepgram utteranceEndMs

* more handling of utteranceEndMs

* wip

* better handling of digit strings collected over multiple deepgram responses

* wip

* add support for deepgramOptions.shortUtterance which triggers off of is_final instead of speech_final

* apply deepgram fixes to transcribe

* cleanup continnuous asr

* more continuous asr fixes for deepgram

* update to verb-specifications for handling SttTask properties

* set log level for tests back to error
2023-10-30 13:57:25 -04:00
Dave Horton
67f8f7181a #508 - add support for azureOptions.speechSegmentationSilenceTimeoutMs (#509)
* #508 - add support for azureOptions.speechSegmentationSilenceTimeoutMs

* update verb specs
2023-10-30 13:10:31 -04:00
Dave Horton
916d988dbd add support for deepgram smart_format option (#500)
* add support for deepgram smart_format option

* handle nonexistent hints
2023-10-25 14:29:38 -04:00
Dave Horton
1aa0b07b8f fix: deepgram redact option 2023-09-22 08:13:54 -04:00
Dave Horton
a1793ac359 Speech vendor/cobalt (#463)
* initial changes for cobalt speech

* wip

* wip

* update to drachtio-fsmrf that supports cobalt

* update to verb-specifications with cobalt speech support

* more wip

* lint

* use node 18 with gh actions

* support for compiling cobalt hints

* fix bug in uuid_cobalt_compile_context

* update verb-specifications

* remove repeated code

* cobalt support for transcribe

* update to verb specs
2023-09-13 09:47:30 -04:00
Paulo Telles
7f59bba634 issue #456 error on validating hints using microsoft (#454)
* error ono validating hints using microsoft

* fix lint

* fix lint 2

* remove try catch

---------

Co-authored-by: p.souza <p.souza@cognigy.com>
2023-09-08 09:21:44 -04:00
Hoan Luu Huu
2ccd33e212 feat azure fromHost (#416)
* feat azure fromHost

* wip

* wip

* wip
2023-08-30 21:04:53 -04:00
Hoan Luu Huu
7a02c36bad fix missing speech vendor in telemetry metric (#437) 2023-08-28 07:45:00 -04:00
Hoan Luu Huu
b1ecf069bf Feat/fallback speech 02 (#429)
* feat fallback speech

* feat fallback speech

* feat fallback speech

* gather verb

* fix

* wip

* wip

* wip

* wip

* wip

* transcribe

* transcribe

* fix

* wip

* wip

* wip

* wip

* fix

* wip

* fix
2023-08-22 09:22:01 -04:00
Hoan Luu Huu
51bcb5a2d2 fix: rivauri (#345)
* fix: rivauri

* fix: rivauri
2023-05-09 10:27:15 -04:00
Dave Horton
565ee609ef based on more testing default google to command_and_search for gather and latest_long for transcribe 2023-04-07 07:34:03 -04:00
Dave Horton
410c07fae6 further fix for google model 2023-03-31 12:37:04 -04:00
Dave Horton
2ebfbfb3d8 google STT: when altLanguges are used default to a model that supports it 2023-03-31 12:31:14 -04:00
Dave Horton
afb381eec9 bugfix: setting altLanguages on Azure once left it turned on 2023-03-29 08:49:34 -04:00
Dave Horton
6e945dde9a google stt fixes, including defaulting to phone_call model based on c… (#288)
* google stt fixes, including defaulting to phone_call model based on comparative model testing

* lint error
2023-03-28 10:02:03 -04:00
Dave Horton
efdea3e514 gather defaults to multiple utterances 2023-03-27 15:53:01 -04:00
Dave Horton
f71ae83ce4 bugfix: nuance on-prem stt 2023-03-26 14:26:36 -04:00
Dave Horton
63ab554908 google STT: default to command_and_search for Gather, as latest_short seems to have issues, various other fixes (#285) 2023-03-26 12:20:03 -04:00
Dave Horton
c06a43adfa Gather: bugfix for alternate languages with Azure 2023-03-22 14:32:25 -04:00
Hoan Luu Huu
7be3c64116 feat: update speech-ultil version 1.0.1 (#275)
* feat: update speech-ultil version 1.0.1

* feat: update speech-ultil version 1.0.1

* more fixes for custom stt

* more fixes

* fixes

* update drachtio-fsmrf

* pass url to mod_jambonz_transcribe

* transcription utils: handle custom results

* handle custom speech vendor errors

* add support for hints to custom speech

* change to custom speech options

* send hints as an array for custom speech

* update latest speech-utils

* transcribe: changes to support soniox

* bugfix: soniox transcribe

---------

Co-authored-by: Quan HL <quanluuhoang8@gmail.com>
Co-authored-by: Dave Horton <daveh@beachdognet.com>
2023-03-12 19:38:36 -04:00
Dave Horton
1c683f1142 initial changes for soniox (#270)
* initial changes for soniox

* changes to gather for soniox

* parse soniox stt results

* handle <end> token for soniox

* soniox: handle empty array of words

* support for soniox hints

* add soniox storage options

* update to verb specs

* add support for transcribe

* compile soniox transcripts

* gather: kill no input timer for soniox when we get interim results

* fix buffering of soniox transcripts

* fix for compiling soniox transcript

* another fix for compiling soniox transcript

* another fix

* handling of <end> token

* fix soniox bug

* gather: fixes for soniox continous asr

* fix undefined variable reference

* fix prev commit

* bugfix: allow verb_status requests

* gather: for soniox no need to restart transcription after final transcription received

* update verb specs

* update verb specs, fixes for continuous asr:
2023-03-03 13:37:55 -05:00
Dave Horton
3a3544a5e8 remove some wordy logging 2023-02-23 12:32:41 -05:00
Dave Horton
0fdcb3a6d6 Feature/nvidia speech (#261)
* initial changes for nvidia speech

* allow nvidia speech credentials to be set at runtime

* update drachtio-fsmrf

* fix handling of nvidia-specific options

* fix nvidia custom config

* fix nvidia word time offsets

* fix nvidia custom configuration

* normalize nvidia transcripts

* update to @jambonz/realtime-dbhelpers with nvidia tts support
2023-02-12 14:06:01 -05:00
Dave Horton
4894a85569 bugfix: recognizer.punctuation not working on nuance 2023-02-06 12:07:28 -05:00
Dave Horton
f2ff5250b0 reset variables like hints so that previous hints do not automatically carry over (#243) 2023-02-03 13:36:14 -05:00
Dave Horton
c37fba541f for backwards compatibility in gather/transcribe return only 1 alternative at top level (#240) 2023-02-02 12:58:22 -05:00
Dave Horton
2e3fb60e72 support google hints as an array of objects containing both hint phra… (#209)
* support google hints as an array of objects containing both hint phrase and boost value

* handle structured hints for non-google STT (#205)
2023-01-06 08:00:17 -05:00
Dave Horton
bf19d2ae6d fixes for runtime speech credentials 2022-12-09 20:25:39 -05:00
Dave Horton
71a2435c63 Feature/ibm watson (#193)
* initial changes to support ibm watson

* update specs.json for ibm

* update to drachtio-fsmrf with support for ibm

* bugfix: set access token for ibm stt, not api_key

* fix name of api_key

* normalize ibm transcription results

* rework ibm credentials

* bugfix setting runtime speech creds

* bugfix: ibm region

* typo

* changes to transcribe for ibm watson

* implement connect handler

* bugfix: bind error

* proper use of result_index

* ibm error handling
2022-11-21 22:09:37 -05:00
Dave Horton
8686348454 Feature/deepgram stt (#190)
* initial changes to support deepgram stt

* fixes for normalizing vendor-specific transcriptions

* update to latest drachtio-fsmrf with support for deepgram stt

* deepgram parsing error

* hints support for deepgram

* handling deepgram errors

* ignore late arriving transcripts for deepgram

* handling of empty transcripts

* transcribe changes

* allow deepgram stt credentials to be provided at run time

* bind channel in transcription handler

* fixes for transcribe when handling empty transcripts

* more empty transcript fixes

* update tests to latest modules

* add test cases for deepgram speech recognition
2022-11-12 19:48:59 -05:00
Dave Horton
509bb065bb Feature/nuance stt (#185)
* initial changes to gather to support nuance stt

* updateSpeechCredentialLastUsed could be called without a speech_credential_sid if credentials are passed in the flow

* fix bugname

* typo

* added handlers for nuance

* logging

* major refactor of parsing transcriptions

* initial support for nuance in transcribe verb

* updates from testing

* cleanup some tests

* update action

* typo

* gather: start nuance timers after say/play completes

* update drachtio-fsrmf

* refactor some code

* typo

* log nuance error detail

* timeout handling

* typo

* handle nuance 413 response when recognition times out

* typo in specs.json

* add support for nuance resources

* fixes and tests for transcribe

* remove logging from test

* initial support for kryptonEndpoint

* try getting access token even when using krypton

* typo in kryptonEndpoint property

* add support for Nuance tts

* parse nuance voice and model for tts

* use nuance credentials from db

* update to db-helpers@0.7.0 with caching option

* add support for azure audio logging in gather/transcribe

* sync package-lock.json
2022-11-01 12:23:49 -04:00
Dave Horton
2882fa2d0a Feature/vm detection (#137)
* initial changes for amd

* wip

* fix bug where transcripts were discarded

* a bit of refactoring, and adding support for avmd in config verb

* bug fixes
2022-07-27 17:46:52 +01:00