Commit Graph

91 Commits

Author SHA1 Message Date
Dave Horton
1c683f1142 initial changes for soniox (#270)
* initial changes for soniox

* changes to gather for soniox

* parse soniox stt results

* handle <end> token for soniox

* soniox: handle empty array of words

* support for soniox hints

* add soniox storage options

* update to verb specs

* add support for transcribe

* compile soniox transcripts

* gather: kill no input timer for soniox when we get interim results

* fix buffering of soniox transcripts

* fix for compiling soniox transcript

* another fix for compiling soniox transcript

* another fix

* handling of <end> token

* fix soniox bug

* gather: fixes for soniox continous asr

* fix undefined variable reference

* fix prev commit

* bugfix: allow verb_status requests

* gather: for soniox no need to restart transcription after final transcription received

* update verb specs

* update verb specs, fixes for continuous asr:
2023-03-03 13:37:55 -05:00
Hoan Luu Huu
c09425fa89 feat: use verb-specifications (#262)
* feat: use verb-specifications

* feat: use verb-specifications

* fix: verb specification v2

* remove irrelevant tests

* fix: verb-scpecification

* update to use @jambonz/verb-specifications

---------

Co-authored-by: Quan HL <quanluuhoang8@gmail.com>
Co-authored-by: Dave Horton <daveh@beachdognet.com>
2023-02-15 09:56:23 -05:00
Dave Horton
0fdcb3a6d6 Feature/nvidia speech (#261)
* initial changes for nvidia speech

* allow nvidia speech credentials to be set at runtime

* update drachtio-fsmrf

* fix handling of nvidia-specific options

* fix nvidia custom config

* fix nvidia word time offsets

* fix nvidia custom configuration

* normalize nvidia transcripts

* update to @jambonz/realtime-dbhelpers with nvidia tts support
2023-02-12 14:06:01 -05:00
Dave Horton
567b03fd36 bugfix: transcribe/gather using default as vendor 2023-01-11 15:31:24 -05:00
Dave Horton
d5c04d2133 transcribe and gather: silently discard listening events from ibm stt 2023-01-11 14:59:15 -05:00
Dave Horton
71a2435c63 Feature/ibm watson (#193)
* initial changes to support ibm watson

* update specs.json for ibm

* update to drachtio-fsmrf with support for ibm

* bugfix: set access token for ibm stt, not api_key

* fix name of api_key

* normalize ibm transcription results

* rework ibm credentials

* bugfix setting runtime speech creds

* bugfix: ibm region

* typo

* changes to transcribe for ibm watson

* implement connect handler

* bugfix: bind error

* proper use of result_index

* ibm error handling
2022-11-21 22:09:37 -05:00
Dave Horton
8686348454 Feature/deepgram stt (#190)
* initial changes to support deepgram stt

* fixes for normalizing vendor-specific transcriptions

* update to latest drachtio-fsmrf with support for deepgram stt

* deepgram parsing error

* hints support for deepgram

* handling deepgram errors

* ignore late arriving transcripts for deepgram

* handling of empty transcripts

* transcribe changes

* allow deepgram stt credentials to be provided at run time

* bind channel in transcription handler

* fixes for transcribe when handling empty transcripts

* more empty transcript fixes

* update tests to latest modules

* add test cases for deepgram speech recognition
2022-11-12 19:48:59 -05:00
Dave Horton
706cd4b94b bugfix: handle gather/transcribe where vendor not explicitly specified #187 2022-11-07 09:31:51 -05:00
Dave Horton
509bb065bb Feature/nuance stt (#185)
* initial changes to gather to support nuance stt

* updateSpeechCredentialLastUsed could be called without a speech_credential_sid if credentials are passed in the flow

* fix bugname

* typo

* added handlers for nuance

* logging

* major refactor of parsing transcriptions

* initial support for nuance in transcribe verb

* updates from testing

* cleanup some tests

* update action

* typo

* gather: start nuance timers after say/play completes

* update drachtio-fsrmf

* refactor some code

* typo

* log nuance error detail

* timeout handling

* typo

* handle nuance 413 response when recognition times out

* typo in specs.json

* add support for nuance resources

* fixes and tests for transcribe

* remove logging from test

* initial support for kryptonEndpoint

* try getting access token even when using krypton

* typo in kryptonEndpoint property

* add support for Nuance tts

* parse nuance voice and model for tts

* use nuance credentials from db

* update to db-helpers@0.7.0 with caching option

* add support for azure audio logging in gather/transcribe

* sync package-lock.json
2022-11-01 12:23:49 -04:00
Dave Horton
b25f92e17a Feature/azure custom stt (#171)
* gather/transcribe: support for azure custom speech models (endpoint id)

* allow azure stt custom speech endpoint id to be passed as property in recognizer

* fix to add custom stt endpoint to session speech credentials object
2022-10-07 09:46:25 +01:00
Dave Horton
90cb5e1348 bugfix: typo in bugname was causing transcripts to be ignored 2022-10-04 12:59:58 +01:00
Dave Horton
bd49dacac4 Say length text (#165)
* typo for media bug name in azure and punctuation fix

* say: split very long text intelligently

* more fixes from testing

* update to latest synthAudio
2022-09-14 17:17:29 +02:00
Dave Horton
c88163fe11 Bugfix/config stt punctuation (#164)
* support recognizer.punctuation in config verb (#163)

* fixes from testing
2022-09-13 11:45:36 +02:00
Dave Horton
887c6243e2 handle altLanguages set at the session level via config verb; fix azure stt race condition with final transcripts from stopped recognition 2022-08-25 22:43:38 +02:00
Dave Horton
6346954e7a session-level speech hints, strip trailing punctuation on continuous asr (#151) 2022-08-18 23:18:24 +02:00
Dave Horton
3298918322 Feature/siprec server (#143)
* fixes from testing

* modify Task#exec to take resources as an object rather than argument list

* pass 2 endpoints to Transcribe when invoked in a SipRec call session

* logging

* change siprec invite to sendrecv just so freeswitch does not try to reinvite (TODO: block outgoing media at rtpengine)

* Config: when enabling recording, block until siprec dialog is established

* missed play verb in commit 031c79d

* linting

* bugfix: get final transcript in siprec call
2022-08-09 15:23:55 +02:00
Dave Horton
2882fa2d0a Feature/vm detection (#137)
* initial changes for amd

* wip

* fix bug where transcripts were discarded

* a bit of refactoring, and adding support for avmd in config verb

* bug fixes
2022-07-27 17:46:52 +01:00
Dave Horton
c3e5ffa52d bugfix: transcribe of a dialed call can now occur on both legs 2022-05-15 13:45:55 -04:00
Dave Horton
6d34850dc6 bugfix: transcribe Azure interim transcripts were missing 2022-05-11 19:22:14 -04:00
Dave Horton
182ad8c716 expose model and singleUtterance to gather/transcribe when using google 2022-05-08 12:29:55 -04:00
Dave Horton
b37881a059 bugfix: second part of outbound dial fix over wss 2022-05-07 11:52:29 -04:00
Dave Horton
72aaf80335 add support for multiple languages when using Azure STT 2022-04-26 15:07:55 -04:00
Dave Horton
359cb82d80 per recommendation from microsoft, do NOT sort transcripts by confidence: first transcript in the returned list is 'best' 2022-04-17 17:53:16 -04:00
Dave Horton
a950f9f738 Feature/trace propagation (#96)
* add b3 header for trace propagation on initial webhook

* logging

* add tracing context to all webhooks

* Add span parameter to Task.getTracingPropagation. Pass proper span to getTracingPropagation calls in Task methods to propagate the proper spanId (#91)

* some tracing cleanup

* bugfix: azure stt results need to be ordered by confidence level before processing (#92)

* fix assertion

* bugfix: vad was not enabled on config verb, restart STT on empty transcript in gather

* gather: dont send webhook if call is gone

* rest outdial: handle 302 redirect so we can later cancel request if needed (#95)

* gather: restart if we get an empty transcript (looking at you, Azure)

Co-authored-by: javibookline <98887695+javibookline@users.noreply.github.com>
2022-04-01 14:48:27 -04:00
Dave Horton
172dc1aaa7 Feature/config verb (#77)
* remove cognigy verb

* initial implementation of config verb

* further updates to config

* Bot mode alex (#75)

* do not use default as value for TTS/STT

* fix gather listener if no say or play provided

Co-authored-by: akirilyuk <a.kirilyuk@cognigy.com>

* gather: listenDuringPrompt requires a nested play/say

* fix exception

* say: fix exception where caller hangs up during say

* bugfix: sip refer was not ending if caller hungup during refer

* add support for sip:request to ws commands

* gather: when bargein is set and minBargeinWordCount is zero, kill audio on endOfUtterrance

* gather/transcribe: add support for google boost and azure custom endpoints

* minor logging changes

* lint error

Co-authored-by: akirilyuk <45361199+akirilyuk@users.noreply.github.com>
Co-authored-by: akirilyuk <a.kirilyuk@cognigy.com>
2022-03-06 15:09:45 -05:00
Dave Horton
3c5d392407 Feature/ws api (#72)
initial changes to support websockets as an alternative to webhooks
2022-02-26 14:06:52 -05:00
Dave Horton
30ed5b6a02 add support for vad to gather and transcribe (#67) 2022-02-10 08:45:16 -05:00
Dave Horton
752eed428f cognigy: when use azuyre tts, request detailed output format 2022-01-14 08:48:55 -05:00
Dave Horton
afb874aabc minor logging change 2022-01-14 07:56:11 -05:00
Dave Horton
3bf1984854 K8s changes (#55)
* K8S: dont send OPTIONS pings

* fix missing ref

* k8s pre-stop hook added

* k8s pre-stop hook changes

* chmod +x utility

* more k8s pre-stop changes

* pre stop

* fix healthcheck

* k8s pre-stop working

* add readiness probe

* fix bug in pre-stop

* logging

* revamp k8s pre-stop a bit

* initial support for cognigy bot

* more cognigy changes

* switch to use transcribe for cognigy

* #54 include callInfo in dialogflow event payload
2022-01-06 12:41:14 -05:00
Dave Horton
1e93973419 Feature/azure recognition (#46)
* add support for microsoft speech recognition

* update to drachtio-fsmrf that support microsoft stt

* gather and transcribe now support microsoft
2021-11-26 16:40:25 -06:00
Dave Horton
72345f83c1 Feature/minimal media anchoring (#36)
* initial WIP to remove freeswitch from media path when not recording or transcribing dial calls

* implement release-media and anchor-media operations

* mute/unmute now handled by rtpengine

* Dial: dtmf detection now based on SIP INFO events from sbcs and rtpengine

* add reason to gather action, bugfixes for transcribe and say
2021-10-21 11:59:45 -04:00
Dave Horton
9b59d08dcf merge features from hosted branch (#32)
major merge of features from the hosted branch that was created temporarily during the initial launch of jambonz.org
2021-06-17 16:25:50 -04:00
Dave Horton
8eb0cd1520 bugfix: speech to text was ignoring language and setting to en-US always 2021-04-07 18:40:14 -04:00
Dave Horton
873729edb1 gather now supports aws for transcribe as well as google 2021-02-01 10:21:52 -05:00
Dave Horton
756db59671 update transcribe to support google v1p1beta1 and aws 2021-01-31 15:49:19 -05:00
Dave Horton
8ee590172b added support for conference verb 2020-04-27 11:25:39 -04:00
Dave Horton
446000ee97 major revamp of http client functionalit 2020-02-14 12:45:28 -05:00
Dave Horton
03e8727c8b fixes for listen and transcribe 2020-01-25 16:39:37 -05:00
Dave Horton
4a1ea4e091 major refactoring 2020-01-25 11:47:33 -05:00
Dave Horton
0d4c1d9d8c wip: implemented listen, transcribe, play 2020-01-17 09:15:23 -05:00