Compare commits

..

10 Commits

Author SHA1 Message Date
Dave Horton
1c118b171c further refinement 2023-04-10 09:27:28 -04:00
Dave Horton
0cde92798d improve deprecated syntax 2023-04-10 09:20:49 -04:00
Dave Horton
45045fd39b improve deprecated syntax 2023-04-10 09:12:41 -04:00
Dave Horton
92da5518eb try 5 2023-04-10 09:08:54 -04:00
Dave Horton
1126ea827f try 4 2023-04-10 09:02:39 -04:00
Dave Horton
fcc4c0a5ce try 3 2023-04-10 09:00:19 -04:00
Dave Horton
ba3ff42987 try 2 2023-04-10 08:56:44 -04:00
Dave Horton
765311c491 interpolate tags 2023-04-10 08:53:57 -04:00
Dave Horton
dcd8b378b2 use docker/build-push-action@v4 2023-04-10 08:49:44 -04:00
Quan HL
552a4e9fd1 feat: upload docker image to docker hub 2023-04-10 15:12:50 +07:00
110 changed files with 12424 additions and 17788 deletions

View File

@@ -6,17 +6,12 @@ jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
with:
node-version: 20
node-version: 16
- run: npm ci
- run: npm run jslint
- name: Install Docker Compose
run: |
sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
docker-compose --version
- run: docker pull drachtio/sipp
- run: npm test
env:

View File

@@ -2,8 +2,6 @@ name: Docker
on:
push:
branches:
- main
tags:
- '*'
@@ -20,7 +18,7 @@ jobs:
- name: prepare tag
id: prepare_tag
run: |
IMAGE_ID=jambonz/feature-server
IMAGE_ID=$GITHUB_REPOSITORY
# Strip git ref prefix from version
VERSION=$(echo "${{ github.ref }}" | sed -e 's,.*/\(.*\),\1,')

2
.gitignore vendored
View File

@@ -42,5 +42,3 @@ ecosystem.config.js
test/credentials/*.json
run-tests.sh
run-coverage.sh
.vscode
.env

17
.vscode/launch.json vendored Normal file
View File

@@ -0,0 +1,17 @@
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"type": "node",
"request": "launch",
"name": "Launch Program",
"program": "${workspaceFolder}/test/index.js",
"env": {
"NODE_ENV": "test"
}
}
]
}

View File

@@ -1,4 +1,4 @@
FROM --platform=linux/amd64 node:20-alpine as base
FROM --platform=linux/amd64 node:18.14.1-alpine3.16 as base
RUN apk --update --no-cache add --virtual .builds-deps build-base python3

View File

@@ -1,6 +1,6 @@
MIT License
Copyright (c) 2018-2024 FirstFive8, Inc.
Copyright (c) 2021 Drachtio Communications Services, LLC
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal

View File

@@ -1,4 +1,4 @@
# jambonz-feature-server [![CI](https://github.com/jambonz/jambonz-feature-server/actions/workflows/build.yml/badge.svg)](https://github.com/jambonz/jambonz-feature-server/actions/workflows/build.yml)
# jambones-feature-server ![Build Status](https://github.com/jambonz/jambonz-feature-server/workflows/CI/badge.svg)
This application implements the core feature server of the jambones platform.
@@ -13,7 +13,7 @@ Configuration is provided via environment variables:
|AWS_ACCESS_KEY_ID| aws access key id, used for TTS/STT as well SNS notifications|no|
|AWS_REGION| aws region| no|
|AWS_SECRET_ACCESS_KEY| aws secret access key, used per above|no|
|AWS_SNS_TOPIC_ARN| aws sns topic arn that scale-in lifecycle notifications will be published to|no|
|AWS_SNS_TOPIC_ARM| aws sns topic arn that scale-in lifecycle notifications will be published to|no|
|DRACHTIO_HOST| ip address of drachtio server (typically '127.0.0.1')|yes|
|DRACHTIO_PORT| listening port of drachtio server for control connections (typically 9022)|yes|
|DRACHTIO_SECRET| shared secret|yes|
@@ -21,7 +21,6 @@ Configuration is provided via environment variables:
|ENCRYPTION_SECRET| secret for credential encryption(JWT_SECRET is deprecated) |yes|
|GOOGLE_APPLICATION_CREDENTIALS| path to gcp service key file|yes|
|HTTP_PORT| tcp port to listen on for API requests from jambonz-api-server|yes|
|HTTP_IP| IP Address for API requests from jambonz-api-server |no|
|JAMBONES_GATHER_EARLY_HINTS_MATCH| if true and hints are provided, gather will opportunistically review interim transcripts if possible to reduce ASR latency |no|
|JAMBONES_FREESWITCH| IP:port:secret for Freeswitch server (e.g. '127.0.0.1:8021:JambonzR0ck$'|yes|
|JAMBONES_LOGLEVEL| log level for application, 'info' or 'debug'|no|
@@ -38,11 +37,6 @@ Configuration is provided via environment variables:
|STATS_PORT| listening port for metrics host|no|
|STATS_PROTOCOL| 'tcp' or 'udp'|no|
|STATS_TELEGRAF| if 1, metrics will be generated in telegraf format|no|
|JAMBONZ_RECORD_WS_BASE_URL| recording websocket URL to send the recording audio|no|
|JAMBONZ_RECORD_WS_USERNAME| recording websocket username|no|
|JAMBONZ_RECORD_WS_PASSWORD| recording websocket password|no|
|ANCHOR_MEDIA_ALWAYS| keep media on media server|no|
|JAMBONZ_DISABLE_DIAL_PAI_HEADER| control P-Asserted-Identity header on B-Leg|no|
### running under pm2
Typically, this application runs under [pm2](https://pm2.io) using an [ecosystem.config.js](https://pm2.keymetrics.io/docs/usage/application-declaration/) file similar to this:
@@ -72,7 +66,7 @@ module.exports = {
STATS_PORT: 8125,
STATS_PROTOCOL: 'tcp',
STATS_TELEGRAF: 1,
AWS_SNS_TOPIC_ARN: 'arn:aws:sns:us-west-1:xxxxxxxxxxx:terraform-20201107200347128600000002',
AWS_SNS_TOPIC_ARM: 'arn:aws:sns:us-west-1:xxxxxxxxxxx:terraform-20201107200347128600000002',
JAMBONES_NETWORK_CIDR: '172.31.0.0/16',
JAMBONES_MYSQL_HOST: 'aurora-cluster-jambonz.cluster-yyyyyyyyyyy.us-west-1.rds.amazonaws.com',
JAMBONES_MYSQL_USER: 'admin',
@@ -95,4 +89,4 @@ module.exports = {
#### Running the test suite
Please [see this](./docs/contributing.md#run-the-regression-test-suite).
Please [see this]](./docs/contributing.md#run-the-regression-test-suite).

186
app.js
View File

@@ -1,98 +1,27 @@
const {
DRACHTIO_PORT,
DRACHTIO_HOST,
DRACHTIO_SECRET,
JAMBONES_OTEL_SERVICE_NAME,
JAMBONES_LOGLEVEL,
JAMBONES_CLUSTER_ID,
JAMBONZ_CLEANUP_INTERVAL_MINS,
getCleanupIntervalMins,
K8S,
NODE_ENV,
checkEnvs,
} = require('./lib/config');
checkEnvs();
const assert = require('assert');
assert.ok(process.env.JAMBONES_MYSQL_HOST &&
process.env.JAMBONES_MYSQL_USER &&
process.env.JAMBONES_MYSQL_PASSWORD &&
process.env.JAMBONES_MYSQL_DATABASE, 'missing JAMBONES_MYSQL_XXX env vars');
assert.ok(process.env.DRACHTIO_PORT || process.env.DRACHTIO_HOST, 'missing DRACHTIO_PORT env var');
assert.ok(process.env.DRACHTIO_SECRET, 'missing DRACHTIO_SECRET env var');
assert.ok(process.env.JAMBONES_FREESWITCH, 'missing JAMBONES_FREESWITCH env var');
assert.ok(process.env.JAMBONES_REDIS_HOST, 'missing JAMBONES_REDIS_HOST env var');
assert.ok(process.env.JAMBONES_NETWORK_CIDR || process.env.K8S, 'missing JAMBONES_SUBNET env var');
assert.ok(process.env.ENCRYPTION_SECRET || process.env.JWT_SECRET, 'missing ENCRYPTION_SECRET env var');
const Srf = require('drachtio-srf');
const srf = new Srf();
const tracer = require('./tracer')(JAMBONES_OTEL_SERVICE_NAME);
const tracer = require('./tracer')(process.env.JAMBONES_OTEL_SERVICE_NAME || 'jambonz-feature-server');
const api = require('@opentelemetry/api');
srf.locals = {...srf.locals, otel: {tracer, api}};
const opts = {
level: JAMBONES_LOGLEVEL
};
const opts = {level: process.env.JAMBONES_LOGLEVEL || 'info'};
const pino = require('pino');
const logger = pino(opts, pino.destination({sync: false}));
const {LifeCycleEvents, FS_UUID_SET_NAME, SystemState, FEATURE_SERVER} = require('./lib/utils/constants');
const {LifeCycleEvents, FS_UUID_SET_NAME} = require('./lib/utils/constants');
const installSrfLocals = require('./lib/utils/install-srf-locals');
const createHttpListener = require('./lib/utils/http-listener');
const healthCheck = require('@jambonz/http-health-check');
logger.on('level-change', (lvl, _val, prevLvl, _prevVal, instance) => {
if (logger !== instance) {
return;
}
logger.info('system log level %s was changed to %s', prevLvl, lvl);
});
// Install the srf locals
installSrfLocals(srf, logger, {
onFreeswitchConnect: (wraper) => {
// Only connect to drachtio if freeswitch is connected
logger.info(`connected to freeswitch at ${wraper.ms.address}, start drachtio server`);
if (DRACHTIO_HOST) {
srf.connect({host: DRACHTIO_HOST, port: DRACHTIO_PORT, secret: DRACHTIO_SECRET });
srf.on('connect', (err, hp) => {
const arr = /^(.*)\/(.*)$/.exec(hp.split(',').pop());
srf.locals.localSipAddress = `${arr[2]}`;
logger.info(`connected to drachtio listening on ${hp}, local sip address is ${srf.locals.localSipAddress}`);
});
}
else {
logger.info(`listening for drachtio requests on port ${DRACHTIO_PORT}`);
srf.listen({port: DRACHTIO_PORT, secret: DRACHTIO_SECRET});
}
// Start Http server
createHttpListener(logger, srf)
.then(({server, app}) => {
httpServer = server;
healthCheck({app, logger, path: '/', fn: getCount});
return {server, app};
})
.catch((err) => {
logger.error(err, 'Error creating http listener');
});
},
onFreeswitchDisconnect: (wraper) => {
// check if all freeswitch connections are lost, disconnect drachtio server
logger.info(`lost connection to freeswitch at ${wraper.ms.address}`);
const ms = srf.locals.getFreeswitch();
if (!ms) {
logger.info('no freeswitch connections, stopping drachtio server');
disconnect();
}
}
});
if (NODE_ENV === 'test') {
srf.on('error', (err) => {
logger.info(err, 'Error connecting to drachtio');
});
}
// Init services
const writeSystemAlerts = srf.locals?.writeSystemAlerts;
if (writeSystemAlerts) {
writeSystemAlerts({
system_component: FEATURE_SERVER,
state : SystemState.Online,
fields : {
detail: `feature-server with process_id ${process.pid} started`,
host: srf.locals?.ipv4
}
});
}
installSrfLocals(srf, logger);
const {
initLocals,
@@ -107,6 +36,24 @@ const {
const InboundCallSession = require('./lib/session/inbound-call-session');
const SipRecCallSession = require('./lib/session/siprec-call-session');
if (process.env.DRACHTIO_HOST) {
srf.connect({host: process.env.DRACHTIO_HOST, port: process.env.DRACHTIO_PORT, secret: process.env.DRACHTIO_SECRET });
srf.on('connect', (err, hp) => {
const arr = /^(.*)\/(.*)$/.exec(hp.split(',').pop());
srf.locals.localSipAddress = `${arr[2]}`;
logger.info(`connected to drachtio listening on ${hp}, local sip address is ${srf.locals.localSipAddress}`);
});
}
else {
logger.info(`listening for drachtio requests on port ${process.env.DRACHTIO_PORT}`);
srf.listen({port: process.env.DRACHTIO_PORT, secret: process.env.DRACHTIO_SECRET});
}
if (process.env.NODE_ENV === 'test') {
srf.on('error', (err) => {
logger.info(err, 'Error connecting to drachtio');
});
}
srf.use('invite', [
initLocals,
createRootSpan,
@@ -132,28 +79,23 @@ sessionTracker.on('idle', () => {
}
});
const getCount = () => sessionTracker.count;
const healthCheck = require('@jambonz/http-health-check');
let httpServer;
const monInterval = setInterval(async() => {
const createHttpListener = require('./lib/utils/http-listener');
createHttpListener(logger, srf)
.then(({server, app}) => {
httpServer = server;
healthCheck({app, logger, path: '/', fn: getCount});
return {server, app};
})
.catch((err) => {
logger.error(err, 'Error creating http listener');
});
setInterval(() => {
srf.locals.stats.gauge('fs.sip.calls.count', sessionTracker.count);
try {
const systemInformation = await srf.locals.dbHelpers.lookupSystemInformation();
if (systemInformation && systemInformation.log_level) {
const envLogLevel = logger.levels.values[JAMBONES_LOGLEVEL.toLowerCase()];
const dbLogLevel = logger.levels.values[systemInformation.log_level];
const appliedLogLevel = Math.min(envLogLevel, dbLogLevel);
if (logger.levelVal !== appliedLogLevel) {
logger.level = logger.levels.labels[Math.min(envLogLevel, dbLogLevel)];
}
}
} catch (err) {
if (process.env.NODE_ENV === 'test') {
clearInterval(monInterval);
logger.error('all tests complete');
}
else logger.error({err}, 'Error checking system log level in database');
}
}, 20000);
const disconnect = () => {
@@ -161,41 +103,23 @@ const disconnect = () => {
httpServer?.on('close', resolve);
httpServer?.close();
srf.disconnect();
srf.removeAllListeners();
srf.locals.mediaservers?.forEach((ms) => ms.disconnect());
srf.locals.mediaservers.forEach((ms) => ms.disconnect());
});
};
process.on('SIGTERM', handle);
process.on('SIGINT', handle);
async function handle(signal) {
process.on('SIGTERM', handle);
function handle(signal) {
const {removeFromSet} = srf.locals.dbHelpers;
srf.locals.disabled = true;
logger.info(`got signal ${signal}`);
const writeSystemAlerts = srf.locals?.writeSystemAlerts;
if (writeSystemAlerts) {
// it has to be synchronous call, or else by the time system saves the app terminates
await writeSystemAlerts({
system_component: FEATURE_SERVER,
state : SystemState.Offline,
fields : {
detail: `feature-server with process_id ${process.pid} stopped, signal ${signal}`,
host: srf.locals?.ipv4
}
});
}
const setName = `${(JAMBONES_CLUSTER_ID || 'default')}:active-fs`;
const fsServiceUrlSetName = `${(JAMBONES_CLUSTER_ID || 'default')}:fs-service-url`;
const setName = `${(process.env.JAMBONES_CLUSTER_ID || 'default')}:active-fs`;
if (setName && srf.locals.localSipAddress) {
logger.info(`got signal ${signal}, removing ${srf.locals.localSipAddress} from set ${setName}`);
removeFromSet(setName, srf.locals.localSipAddress);
}
if (fsServiceUrlSetName && srf.locals.serviceUrl) {
logger.info(`got signal ${signal}, removing ${srf.locals.serviceUrl} from set ${fsServiceUrlSetName}`);
removeFromSet(fsServiceUrlSetName, srf.locals.serviceUrl);
}
removeFromSet(FS_UUID_SET_NAME, srf.locals.fsUUID);
if (K8S) {
if (process.env.K8S) {
srf.locals.lifecycleEmitter.operationalState = LifeCycleEvents.ScaleIn;
}
if (getCount() === 0) {
@@ -204,7 +128,7 @@ async function handle(signal) {
}
}
if (JAMBONZ_CLEANUP_INTERVAL_MINS) {
if (process.env.JAMBONZ_CLEANUP_INTERVAL_MINS) {
const {clearFiles} = require('./lib/utils/cron-jobs');
/* cleanup orphaned files or channels every so often */
@@ -214,7 +138,7 @@ if (JAMBONZ_CLEANUP_INTERVAL_MINS) {
} catch (err) {
logger.error({err}, 'app.js: error clearing files');
}
}, getCleanupIntervalMins());
}, 1000 * 60 * (process.env.JAMBONZ_CLEANUP_INTERVAL_MINS || 60));
}
module.exports = {srf, logger, disconnect};

2
bin/k8s-pre-stop-hook.js Normal file → Executable file
View File

@@ -1,7 +1,7 @@
#!/usr/bin/env node
const bent = require('bent');
const getJSON = bent('json');
const {PORT} = require('../lib/config')
const PORT = process.env.HTTP_PORT || 3000;
const sleep = (ms) => {
return new Promise((resolve) => setTimeout(resolve, ms));

View File

@@ -5,116 +5,14 @@
"at the tone",
"leave a message",
"leave me a message",
"not available",
"not available right now",
"not available to take your call",
"can't take your call",
"will get back to you",
"I will get back to you",
"I'll get back to you",
"we will get back to you",
"we are unable",
"Unable to take your call now",
"I'll reply soon",
"I'll call back",
"I'll reach out to you as soon as possible",
"Leave a message",
"Away from phone",
"Not available now",
"I'll return call",
"On another call",
"Currently on another call",
"I will return call later",
"Busy please leave message",
"Message will be returned promptly",
"Currently unavailable to answer",
"Planning to return your call soon",
"Apologies for missing your call",
"Not by the phone at the moment",
"Expecting to return your call",
"Currently not accessible",
"Intend to call back",
"Appreciate your patience!",
"Engaged in another conversation",
"I Will respond promptly",
"Kindly leave a message",
"Currently occupied leave a message",
"Unfortunately unable to answer right now",
"Occupied at the moment",
"Not present leave a message",
"Regrettably unavailable kindly leave a message",
"Will ensure a prompt response to your message",
"Currently engaged",
"Will return your call at the earliest opportunity",
"Your message will receive my prompt attention",
"I'll respond as soon as I can",
"Your message is important please leave it after the beep",
"Away from the phone at the moment",
"Unable to answer right now",
"Engaged in another task",
"Not by the phone presently",
"I'll respond at my earliest convenience",
"Away from the phone momentarily",
"I'll return your call shortly",
"Currently not able to answer",
"Your message is important please leave it after the tone",
"I'm unable to take your call right now",
"Please leave your message for me",
"I'll get back to you soon",
"Your call has been missed",
"Please leave a detailed message for me to respond to",
"Leave a message I'll make sure to respond",
"Feel free to leave a message",
"Your call is important to me",
"I'll get back to you shortly",
"Your message will be attended to promptly",
"Not available at the moment",
"I'll be sure to get back to you",
"I'll call you back soon",
"I'll ensure a prompt response",
"Sorry for the inconvenience",
"I'll return your call",
"I'll make sure to get back to you",
"I'll call you back shortly",
"I'll return your call as soon as possible",
"Apologies for the inconvenience leave your message",
"Your call is appreciated",
"I'm unavailable to answer",
"I'm currently away",
"I'll return your call as soon as I can",
"I'm away from the phone",
"I'm currently unavailable to take your call",
"Sorry for missing your call",
"I'll ensure it receives my immediate attention",
"I'm away from the phone momentarily",
"I'll reach out to you shortly",
"Apologies for the inconvenience",
"Currently occupied",
"Unable to answer your call at the moment",
"I'll make sure to follow up with you",
"Sorry for not being available",
"I'll reach out to you as soon as I can",
"I'm currently engaged",
"I'm currently busy",
"I'm currently unavailable",
"I'll respond to you at my earliest convenience",
"Your message is appreciated",
"I'll get back to you promptly",
"I'll get back to you without delay",
"Currently away from the phone",
"I'll return your call at my earliest opportunity",
"Sorry for the missed call",
"I'll make sure to address your concerns",
"Please provide your details for a callback",
"I'll make every effort to respond promptly",
"I'll ensure it's attended to promptly",
"Away from the phone temporarily",
"I'll get back to you as soon as I return",
"Currently not in a position to answer your call",
"Your call cannot be answered at the moment",
"I'll ensure to respond as soon as I'm able",
"Your call is important please leave a message",
"Unable to answer right now please leave your message",
"Currently not accessible intending to return your call",
"I'll respond promptly to your message",
"leave a memo",
"please leave a memo"
"we are not available"
],
"es-ES": [
"le pasamos la llamada",
@@ -150,18 +48,5 @@
"ens posarem en contacto",
"ara no estem disponibles",
"no hi som"
],
"de-DE": [
"nicht erreichbar",
"nnruf wurde weitergeleitet",
"beim piepsen",
"am ton",
"eine nachricht hinterlassen",
"hinterlasse mir eine Nachricht",
"nicht verfügbar",
"kann ihren anruf nicht entgegennehmen",
"wird sich bei Ihnen melden",
"ich melde mich bei dir",
"wir können nicht"
]
}

View File

@@ -45,7 +45,7 @@ The GCP credential is the JSON service key in stringified format.
#### Install Docker
The test suite also requires [Docker](https://www.docker.com/) and docker-compose to be installed on your laptop. Docker is used to set up a network with all of the elements required to test the jambonz-feature-server in a black-box type of fashion.
The test suite ralso equires [Docker](https://www.docker.com/) and docker-compose to be installed on your laptop. Docker is used to set up a network with all of the elements required to test the jambonz-feature-server in a black-box type of fashion.
Once you have docker installed, you can optionally make sure everything Docker-wise is working properly by running this command from the project folder:

View File

@@ -1,235 +0,0 @@
const assert = require('assert');
const checkEnvs = () => {
assert.ok(process.env.JAMBONES_MYSQL_HOST &&
process.env.JAMBONES_MYSQL_USER &&
process.env.JAMBONES_MYSQL_PASSWORD &&
process.env.JAMBONES_MYSQL_DATABASE, 'missing JAMBONES_MYSQL_XXX env vars');
assert.ok(process.env.DRACHTIO_PORT || process.env.DRACHTIO_HOST, 'missing DRACHTIO_PORT env var');
assert.ok(process.env.DRACHTIO_SECRET, 'missing DRACHTIO_SECRET env var');
assert.ok(process.env.JAMBONES_FREESWITCH, 'missing JAMBONES_FREESWITCH env var');
if (process.env.JAMBONES_REDIS_SENTINELS) {
assert.ok(process.env.JAMBONES_REDIS_SENTINEL_MASTER_NAME,
'missing JAMBONES_REDIS_SENTINEL_MASTER_NAME env var, JAMBONES_REDIS_SENTINEL_PASSWORD env var is optional');
} else {
assert.ok(process.env.JAMBONES_REDIS_HOST, 'missing JAMBONES_REDIS_HOST env var');
}
assert.ok(process.env.JAMBONES_NETWORK_CIDR || process.env.K8S, 'missing JAMBONES_SUBNET env var');
};
const NODE_ENV = process.env.NODE_ENV;
/* database mySQL */
const JAMBONES_MYSQL_HOST = process.env.JAMBONES_MYSQL_HOST;
const JAMBONES_MYSQL_USER = process.env.JAMBONES_MYSQL_USER;
const JAMBONES_MYSQL_PASSWORD = process.env.JAMBONES_MYSQL_PASSWORD;
const JAMBONES_MYSQL_DATABASE = process.env.JAMBONES_MYSQL_DATABASE;
const JAMBONES_MYSQL_PORT = parseInt(process.env.JAMBONES_MYSQL_PORT, 10) || 3306;
const JAMBONES_MYSQL_REFRESH_TTL = parseInt(process.env.JAMBONES_MYSQL_REFRESH_TTL, 10) || 0;
const JAMBONES_MYSQL_CONNECTION_LIMIT = parseInt(process.env.JAMBONES_MYSQL_CONNECTION_LIMIT, 10) || 10;
/* gather and hints */
const JAMBONES_GATHER_EARLY_HINTS_MATCH = process.env.JAMBONES_GATHER_EARLY_HINTS_MATCH;
const JAMBONZ_GATHER_EARLY_HINTS_MATCH = process.env.JAMBONZ_GATHER_EARLY_HINTS_MATCH;
const JAMBONES_GATHER_CLEAR_GLOBAL_HINTS_ON_EMPTY_HINTS = process.env.JAMBONES_GATHER_CLEAR_GLOBAL_HINTS_ON_EMPTY_HINTS;
const SMPP_URL = process.env.SMPP_URL;
/* drachtio */
const DRACHTIO_PORT = process.env.DRACHTIO_PORT;
const DRACHTIO_HOST = process.env.DRACHTIO_HOST;
const DRACHTIO_SECRET = process.env.DRACHTIO_SECRET;
/* freeswitch */
const JAMBONES_API_BASE_URL = process.env.JAMBONES_API_BASE_URL;
const JAMBONES_FREESWITCH = process.env.JAMBONES_FREESWITCH;
const JAMBONES_FREESWITCH_MAX_CALL_DURATION_MINS = parseInt(process.env.JAMBONES_FREESWITCH_MAX_CALL_DURATION_MINS, 10)
|| 180;
const JAMBONES_SBCS = process.env.JAMBONES_SBCS;
/* websockets */
const JAMBONES_WS_HANDSHAKE_TIMEOUT_MS = parseInt(process.env.JAMBONES_WS_HANDSHAKE_TIMEOUT_MS, 10) || 1500;
const JAMBONES_WS_MAX_PAYLOAD = parseInt(process.env.JAMBONES_WS_MAX_PAYLOAD, 10) || 24 * 1024;
const JAMBONES_WS_PING_INTERVAL_MS = parseInt(process.env.JAMBONES_WS_PING_INTERVAL_MS, 10) || 0;
const MAX_RECONNECTS = 5;
const RESPONSE_TIMEOUT_MS = parseInt(process.env.JAMBONES_WS_API_MSG_RESPONSE_TIMEOUT, 10) || 5000;
const JAMBONES_NETWORK_CIDR = process.env.JAMBONES_NETWORK_CIDR;
const JAMBONES_TIME_SERIES_HOST = process.env.JAMBONES_TIME_SERIES_HOST;
const JAMBONES_CLUSTER_ID = process.env.JAMBONES_CLUSTER_ID || 'default';
const JAMBONES_ESL_LISTEN_ADDRESS = process.env.JAMBONES_ESL_LISTEN_ADDRESS;
/* tracing */
const JAMBONES_OTEL_ENABLED = process.env.JAMBONES_OTEL_ENABLED;
const JAMBONES_OTEL_SERVICE_NAME = process.env.JAMBONES_OTEL_SERVICE_NAME || 'jambonz-feature-server';
const OTEL_EXPORTER_JAEGER_AGENT_HOST = process.env.OTEL_EXPORTER_JAEGER_AGENT_HOST;
const OTEL_EXPORTER_JAEGER_ENDPOINT = process.env.OTEL_EXPORTER_JAEGER_ENDPOINT;
const OTEL_EXPORTER_ZIPKIN_URL = process.env.OTEL_EXPORTER_ZIPKIN_URL;
const OTEL_EXPORTER_COLLECTOR_URL = process.env.OTEL_EXPORTER_COLLECTOR_URL;
const JAMBONES_LOGLEVEL = process.env.JAMBONES_LOGLEVEL || 'info';
const JAMBONES_INJECT_CONTENT = process.env.JAMBONES_INJECT_CONTENT;
const PORT = parseInt(process.env.HTTP_PORT, 10) || 3000;
const HTTP_IP = process.env.HTTP_IP;
const HTTP_PORT_MAX = parseInt(process.env.HTTP_PORT_MAX, 10);
const K8S = process.env.K8S;
const K8S_SBC_SIP_SERVICE_NAME = process.env.K8S_SBC_SIP_SERVICE_NAME;
const JAMBONES_SUBNET = process.env.JAMBONES_SUBNET;
/* clean up */
const JAMBONZ_CLEANUP_INTERVAL_MINS = process.env.JAMBONZ_CLEANUP_INTERVAL_MINS;
const getCleanupIntervalMins = () => {
const interval = parseInt(JAMBONZ_CLEANUP_INTERVAL_MINS, 10) || 60;
return 1000 * 60 * interval;
};
/* speech vendors */
const AWS_REGION = process.env.AWS_REGION;
const AWS_ACCESS_KEY_ID = process.env.AWS_ACCESS_KEY_ID;
const AWS_SECRET_ACCESS_KEY = process.env.AWS_SECRET_ACCESS_KEY;
const AWS_SNS_PORT = parseInt(process.env.AWS_SNS_PORT, 10) || 3001;
const AWS_SNS_TOPIC_ARN = process.env.AWS_SNS_TOPIC_ARN;
const AWS_SNS_PORT_MAX = parseInt(process.env.AWS_SNS_PORT_MAX, 10) || 3005;
const GCP_JSON_KEY = process.env.GCP_JSON_KEY;
const MICROSOFT_REGION = process.env.MICROSOFT_REGION;
const MICROSOFT_API_KEY = process.env.MICROSOFT_API_KEY;
const SONIOX_API_KEY = process.env.SONIOX_API_KEY;
const DEEPGRAM_API_KEY = process.env.DEEPGRAM_API_KEY;
const ANCHOR_MEDIA_ALWAYS = process.env.ANCHOR_MEDIA_ALWAYS;
const VMD_HINTS_FILE = process.env.VMD_HINTS_FILE;
const JAMBONES_AWS_TRANSCRIBE_USE_GRPC = process.env.JAMBONES_AWS_TRANSCRIBE_USE_GRPC;
/* security, secrets */
const LEGACY_CRYPTO = !!process.env.LEGACY_CRYPTO;
const JWT_SECRET = process.env.JWT_SECRET;
const ENCRYPTION_SECRET = process.env.ENCRYPTION_SECRET;
/* HTTP/1 pool dispatcher */
const HTTP_POOL = process.env.HTTP_POOL && parseInt(process.env.HTTP_POOL);
const HTTP_POOLSIZE = parseInt(process.env.HTTP_POOLSIZE, 10) || 10;
const HTTP_PIPELINING = parseInt(process.env.HTTP_PIPELINING, 10) || 1;
const HTTP_TIMEOUT = 10000;
const HTTP_PROXY_IP = process.env.JAMBONES_HTTP_PROXY_IP;
const HTTP_PROXY_PORT = process.env.JAMBONES_HTTP_PROXY_PORT;
const HTTP_PROXY_PROTOCOL = process.env.JAMBONES_HTTP_PROXY_PROTOCOL || 'http';
const HTTP_USER_AGENT_HEADER = process.env.JAMBONES_HTTP_USER_AGENT_HEADER || 'jambonz';
const OPTIONS_PING_INTERVAL = parseInt(process.env.OPTIONS_PING_INTERVAL, 10) || 30000;
const JAMBONZ_RECORD_WS_BASE_URL = process.env.JAMBONZ_RECORD_WS_BASE_URL || process.env.JAMBONES_RECORD_WS_BASE_URL;
const JAMBONZ_RECORD_WS_USERNAME = process.env.JAMBONZ_RECORD_WS_USERNAME || process.env.JAMBONES_RECORD_WS_USERNAME;
const JAMBONZ_RECORD_WS_PASSWORD = process.env.JAMBONZ_RECORD_WS_PASSWORD || process.env.JAMBONES_RECORD_WS_PASSWORD;
const JAMBONZ_DISABLE_DIAL_PAI_HEADER = process.env.JAMBONZ_DISABLE_DIAL_PAI_HEADER || false;
const JAMBONES_DISABLE_DIRECT_P2P_CALL = process.env.JAMBONES_DISABLE_DIRECT_P2P_CALL || false;
const JAMBONES_EAGERLY_PRE_CACHE_AUDIO = parseInt(process.env.JAMBONES_EAGERLY_PRE_CACHE_AUDIO, 10) || 0;
const JAMBONES_USE_FREESWITCH_TIMER_FD = process.env.JAMBONES_USE_FREESWITCH_TIMER_FD;
const JAMBONES_DIAL_SBC_FOR_REGISTERED_USER = process.env.JAMBONES_DIAL_SBC_FOR_REGISTERED_USER || false;
const JAMBONES_MEDIA_TIMEOUT_MS = process.env.JAMBONES_MEDIA_TIMEOUT_MS || 0;
const JAMBONES_MEDIA_HOLD_TIMEOUT_MS = process.env.JAMBONES_MEDIA_HOLD_TIMEOUT_MS || 0;
// jambonz
const JAMBONES_TRANSCRIBE_EP_DESTROY_DELAY_MS =
process.env.JAMBONES_TRANSCRIBE_EP_DESTROY_DELAY_MS;
module.exports = {
JAMBONES_MYSQL_HOST,
JAMBONES_MYSQL_USER,
JAMBONES_MYSQL_PASSWORD,
JAMBONES_MYSQL_DATABASE,
JAMBONES_MYSQL_REFRESH_TTL,
JAMBONES_MYSQL_CONNECTION_LIMIT,
JAMBONES_MYSQL_PORT,
DRACHTIO_PORT,
DRACHTIO_HOST,
DRACHTIO_SECRET,
JAMBONES_GATHER_EARLY_HINTS_MATCH,
JAMBONZ_GATHER_EARLY_HINTS_MATCH,
JAMBONES_GATHER_CLEAR_GLOBAL_HINTS_ON_EMPTY_HINTS,
JAMBONES_FREESWITCH,
SMPP_URL,
JAMBONES_NETWORK_CIDR,
JAMBONES_API_BASE_URL,
JAMBONES_TIME_SERIES_HOST,
JAMBONES_INJECT_CONTENT,
JAMBONES_EAGERLY_PRE_CACHE_AUDIO,
JAMBONES_ESL_LISTEN_ADDRESS,
JAMBONES_SBCS,
JAMBONES_OTEL_ENABLED,
JAMBONES_OTEL_SERVICE_NAME,
OTEL_EXPORTER_JAEGER_AGENT_HOST,
OTEL_EXPORTER_JAEGER_ENDPOINT,
OTEL_EXPORTER_ZIPKIN_URL,
OTEL_EXPORTER_COLLECTOR_URL,
JAMBONES_LOGLEVEL,
JAMBONES_CLUSTER_ID,
PORT,
HTTP_PORT_MAX,
HTTP_IP,
K8S,
K8S_SBC_SIP_SERVICE_NAME,
JAMBONES_SUBNET,
NODE_ENV,
JAMBONZ_CLEANUP_INTERVAL_MINS,
getCleanupIntervalMins,
checkEnvs,
AWS_REGION,
AWS_ACCESS_KEY_ID,
AWS_SECRET_ACCESS_KEY,
AWS_SNS_PORT,
AWS_SNS_TOPIC_ARN,
AWS_SNS_PORT_MAX,
ANCHOR_MEDIA_ALWAYS,
VMD_HINTS_FILE,
JAMBONES_FREESWITCH_MAX_CALL_DURATION_MINS,
JAMBONES_AWS_TRANSCRIBE_USE_GRPC,
LEGACY_CRYPTO,
JWT_SECRET,
ENCRYPTION_SECRET,
HTTP_POOL,
HTTP_POOLSIZE,
HTTP_PIPELINING,
HTTP_TIMEOUT,
HTTP_PROXY_IP,
HTTP_PROXY_PORT,
HTTP_PROXY_PROTOCOL,
HTTP_USER_AGENT_HEADER,
OPTIONS_PING_INTERVAL,
RESPONSE_TIMEOUT_MS,
JAMBONES_WS_HANDSHAKE_TIMEOUT_MS,
JAMBONES_WS_MAX_PAYLOAD,
JAMBONES_WS_PING_INTERVAL_MS,
MAX_RECONNECTS,
GCP_JSON_KEY,
MICROSOFT_REGION,
MICROSOFT_API_KEY,
SONIOX_API_KEY,
DEEPGRAM_API_KEY,
JAMBONZ_RECORD_WS_BASE_URL,
JAMBONZ_RECORD_WS_USERNAME,
JAMBONZ_RECORD_WS_PASSWORD,
JAMBONZ_DISABLE_DIAL_PAI_HEADER,
JAMBONES_DISABLE_DIRECT_P2P_CALL,
JAMBONES_USE_FREESWITCH_TIMER_FD,
JAMBONES_DIAL_SBC_FOR_REGISTERED_USER,
JAMBONES_MEDIA_TIMEOUT_MS,
JAMBONES_MEDIA_HOLD_TIMEOUT_MS,
JAMBONES_TRANSCRIBE_EP_DESTROY_DELAY_MS
};

View File

@@ -1,69 +0,0 @@
const appsMap = {
queue: {
// Dummy hook to follow later feature server logic.
call_hook: {
url: 'https://jambonz.org',
method: 'GET'
},
account_sid: '',
app_json: [{
verb: 'dequeue',
name: '',
timeout: 5
}]
},
user: {
// Dummy hook to follow later feature server logic.
call_hook: {
url: 'https://jambonz.org',
method: 'GET'
},
account_sid: '',
app_json: [{
verb: 'dial',
callerId: '',
answerOnBridge: true,
target: [
{
type: 'user',
name: ''
}
]
}]
},
conference: {
// Dummy hook to follow later feature server logic.
call_hook: {
url: 'https://jambonz.org',
method: 'GET'
},
account_sid: '',
app_json: [{
verb: 'conference',
name: '',
beep: false,
startConferenceOnEnter: true
}]
}
};
const createJambonzApp = (type, {account_sid, name, caller_id}) => {
const app = {...appsMap[type]};
app.account_sid = account_sid;
switch (type) {
case 'queue':
case 'conference':
app.app_json[0].name = name;
break;
case 'user':
app.app_json[0].callerId = caller_id;
app.app_json[0].target[0].name = name;
break;
}
app.app_json = JSON.stringify(app.app_json);
return app;
};
module.exports = {
createJambonzApp
};

View File

@@ -5,358 +5,247 @@ const CallInfo = require('../../session/call-info');
const {CallDirection, CallStatus} = require('../../utils/constants');
const uuidv4 = require('uuid-random');
const SipError = require('drachtio-srf').SipError;
const { validationResult, body } = require('express-validator');
const { validate } = require('@jambonz/verb-specifications');
const sysError = require('./error');
const HttpRequestor = require('../../utils/http-requestor');
const WsRequestor = require('../../utils/ws-requestor');
const RootSpan = require('../../utils/call-tracer');
const dbUtils = require('../../utils/db-utils');
const { mergeSdpMedia, extractSdpMedia } = require('../../utils/sdp-utils');
const { createCallSchema, customSanitizeFunction } = require('../schemas/create-call');
const { selectHostPort } = require('../../utils/network');
const { JAMBONES_DIAL_SBC_FOR_REGISTERED_USER } = require('../../config');
const removeNullProperties = (obj) => (Object.keys(obj).forEach((key) => obj[key] === null && delete obj[key]), obj);
const removeNulls = (req, res, next) => {
req.body = removeNullProperties(req.body);
next();
};
router.post('/', async(req, res) => {
const {logger} = req.app.locals;
const accountSid = req.body.account_sid;
const {srf} = require('../../..');
router.post('/',
removeNulls,
createCallSchema,
body('tag').custom((value) => {
if (value) {
customSanitizeFunction(value);
}
return true;
}),
async(req, res) => {
const {logger} = req.app.locals;
const errors = validationResult(req);
if (!errors.isEmpty()) {
logger.info({errors: errors.array()}, 'POST /Calls: validation errors');
return res.status(400).json({ errors: errors.array() });
}
const accountSid = req.body.account_sid;
const {srf} = require('../../..');
logger.debug({body: req.body}, 'got createCall request');
try {
let uri, cs, to;
const restDial = makeTask(logger, {'rest:dial': req.body});
const {lookupAccountDetails} = dbUtils(logger, srf);
const {getSBC, getFreeswitch} = srf.locals;
const sbcAddress = getSBC();
if (!sbcAddress) throw new Error('no available SBCs for outbound call creation');
const target = restDial.to;
const opts = {
callingNumber: restDial.from,
headers: req.body.headers || {}
};
const app_json = req.body['app_json'];
try {
// app_json is created only by api-server.
if (app_json) {
// if available, delete from req before creating task
delete req.body.app_json;
// validate possible app_json via verb-specifications
validate(logger, JSON.parse(app_json));
}
} catch (err) {
logger.debug({ err }, `invalid app_json: ${err.message}`);
const {lookupTeamsByAccount, lookupAccountBySid} = srf.locals.dbHelpers;
const account = await lookupAccountBySid(req.body.account_sid);
const accountInfo = await lookupAccountDetails(req.body.account_sid);
const callSid = uuidv4();
opts.headers = {
...opts.headers,
'X-Jambonz-Routing': target.type,
'X-Jambonz-FS-UUID': srf.locals.fsUUID,
'X-Call-Sid': callSid,
'X-Account-Sid': accountSid,
...(restDial.fromHost && {'X-Preferred-From-Host': restDial.fromHost})
};
switch (target.type) {
case 'phone':
case 'teams':
uri = `sip:${target.number}@${sbcAddress}`;
to = target.number;
if ('teams' === target.type) {
const obj = await lookupTeamsByAccount(accountSid);
if (!obj) throw new Error('dial to ms teams not allowed; account must first be configured with teams info');
Object.assign(opts.headers, {
'X-MS-Teams-FQDN': obj.ms_teams_fqdn,
'X-MS-Teams-Tenant-FQDN': target.tenant || obj.tenant_fqdn
});
if (target.vmail === true) uri = `${uri};opaque=app:voicemail`;
}
break;
case 'user':
uri = `sip:${target.name}`;
to = target.name;
if (target.overrideTo) {
Object.assign(opts.headers, {
'X-Override-To': target.overrideTo
});
}
break;
case 'sip':
uri = target.sipUri;
to = uri;
break;
}
logger.debug({body: req.body}, 'got createCall request');
try {
let uri, cs, to;
const restDial = makeTask(logger, { 'rest:dial': req.body });
restDial.appJson = app_json;
const {lookupAccountDetails, lookupCarrierByPhoneNumber, lookupCarrier} = dbUtils(logger, srf);
const {
lookupAppBySid
} = srf.locals.dbHelpers;
const {getSBC, getFreeswitch} = srf.locals;
let sbcAddress = getSBC();
if (!sbcAddress) throw new Error('no available SBCs for outbound call creation');
const target = restDial.to;
const opts = {
callingNumber: restDial.from,
...(restDial.callerName && {callingName: restDial.callerName}),
headers: req.body.headers || {}
};
const {lookupTeamsByAccount, lookupAccountBySid} = srf.locals.dbHelpers;
const account = await lookupAccountBySid(req.body.account_sid);
const accountInfo = await lookupAccountDetails(req.body.account_sid);
const callSid = uuidv4();
const application = req.body.application_sid ? await lookupAppBySid(req.body.application_sid) : null;
const record_all_calls = account.record_all_calls || (application && application.record_all_calls);
const recordOutputFormat = account.record_format || 'mp3';
const rootSpan = new RootSpan('rest-call', {
callSid,
accountSid,
...(req.body?.application_sid && {'X-Application-Sid': req.body.application_sid})
});
opts.headers = {
...opts.headers,
'X-Jambonz-Routing': target.type,
'X-Jambonz-FS-UUID': srf.locals.fsUUID,
'X-Call-Sid': callSid,
'X-Account-Sid': accountSid,
'X-Trace-ID': rootSpan.traceId,
...(req.body?.application_sid && {'X-Application-Sid': req.body.application_sid}),
...(restDial.fromHost && {'X-Preferred-From-Host': restDial.fromHost}),
...(record_all_calls && {'X-Record-All-Calls': recordOutputFormat}),
...(target.proxy && {'X-SIP-Proxy': target.proxy}),
...target.headers
};
switch (target.type) {
case 'phone':
case 'teams':
uri = `sip:${target.number}@${sbcAddress}`;
to = target.number;
if ('teams' === target.type) {
const obj = await lookupTeamsByAccount(accountSid);
if (!obj) throw new Error('dial to ms teams not allowed; account must first be configured with teams info');
Object.assign(opts.headers, {
'X-MS-Teams-FQDN': obj.ms_teams_fqdn,
'X-MS-Teams-Tenant-FQDN': target.tenant || obj.tenant_fqdn
});
if (target.vmail === true) uri = `${uri};opaque=app:voicemail`;
}
break;
case 'user':
uri = `sip:${target.name}`;
to = target.name;
if (target.overrideTo) {
Object.assign(opts.headers, {
'X-Override-To': target.overrideTo
});
}
break;
case 'sip':
uri = target.sipUri;
to = uri;
break;
if (target.type === 'phone' && target.trunk) {
const {lookupCarrier} = dbUtils(this.logger, srf);
const voip_carrier_sid = await lookupCarrier(req.body.account_sid, target.trunk);
logger.info(
`createCall: selected ${voip_carrier_sid} for requested carrier: ${target.trunk || 'unspecified'})`);
if (voip_carrier_sid) {
opts.headers['X-Requested-Carrier-Sid'] = voip_carrier_sid;
}
}
if (target.type === 'phone' && target.trunk) {
const voip_carrier_sid = await lookupCarrier(req.body.account_sid, target.trunk);
logger.info(
`createCall: selected ${voip_carrier_sid} for requested carrier: ${target.trunk || 'unspecified'})`);
if (voip_carrier_sid) {
opts.headers['X-Requested-Carrier-Sid'] = voip_carrier_sid;
}
/* create endpoint for outdial */
const ms = getFreeswitch();
if (!ms) throw new Error('no available Freeswitch for outbound call creation');
const ep = await ms.createEndpoint();
logger.debug(`createCall: successfully allocated endpoint, sending INVITE to ${sbcAddress}`);
/* launch outdial */
let sdp, sipLogger;
const connectStream = async(remoteSdp) => {
if (remoteSdp !== sdp) {
ep.modify(sdp = remoteSdp);
return true;
}
// find handling sbc sip for called user
if (JAMBONES_DIAL_SBC_FOR_REGISTERED_USER && target.type === 'user') {
const { registrar } = srf.locals.dbHelpers;
const reg = await registrar.query(target.name);
if (reg) {
sbcAddress = selectHostPort(logger, reg.sbcAddress, 'tcp')[1];
}
//sbc outbound return 404 Notfound to handle case called user is not reigstered.
}
/**
* trunk isn't specified,
* check if from-number matches any existing numbers on Jambonz
* */
if (target.type === 'phone' && !target.trunk) {
const str = restDial.from || '';
const callingNumber = str.startsWith('+') ? str.substring(1) : str;
const voip_carrier_sid = await lookupCarrierByPhoneNumber(req.body.account_sid, callingNumber);
logger.info(
`createCall: selected ${voip_carrier_sid} for requested phone number: ${callingNumber || 'unspecified'})`);
if (voip_carrier_sid) {
opts.headers['X-Requested-Carrier-Sid'] = voip_carrier_sid;
}
}
/* create endpoint for outdial */
const ms = getFreeswitch();
if (!ms) throw new Error('no available Freeswitch for outbound call creation');
const ep = await ms.createEndpoint();
logger.debug(`createCall: successfully allocated endpoint, sending INVITE to ${sbcAddress}`);
/* launch outdial */
let sdp, sipLogger;
let dualEp;
let localSdp = ep.local.sdp;
if (req.body.dual_streams) {
dualEp = await ms.createEndpoint();
localSdp = mergeSdpMedia(localSdp, dualEp.local.sdp);
}
const connectStream = async(remoteSdp) => {
if (remoteSdp !== sdp) {
sdp = remoteSdp;
if (req.body.dual_streams) {
const [sdpLegA, sdpLebB] = extractSdpMedia(remoteSdp);
await ep.modify(sdpLegA);
await dualEp.modify(sdpLebB);
await ep.bridge(dualEp);
} else {
ep.modify(sdp);
}
return true;
}
return false;
};
Object.assign(opts, {
proxy: `sip:${sbcAddress}`,
localSdp
});
if (target.auth) opts.auth = target.auth;
return false;
};
Object.assign(opts, {
proxy: `sip:${sbcAddress}`,
localSdp: ep.local.sdp
});
if (target.auth) opts.auth = target.auth;
/**
/**
* create our application object -
* we merge the inbound call application,
* with the provided app params from the request body
* not from the database as per an inbound call,
* but from the provided params in the request
*/
const app = {
...application,
...req.body
};
const app = req.body;
/**
/**
* attach our requestor and notifier objects
* these will be used for all http requests we make during this call
*/
if ('WS' === app.call_hook?.method || /^wss?:/.test(app.call_hook.url)) {
logger.debug({call_hook: app.call_hook}, 'creating websocket for call hook');
app.requestor = new WsRequestor(logger, account.account_sid, app.call_hook, account.webhook_secret) ;
if (app.call_hook.url === app.call_status_hook.url || !app.call_status_hook?.url) {
logger.debug('reusing websocket for call status hook');
app.notifier = app.requestor;
}
}
else {
logger.debug({call_hook: app.call_hook}, 'creating http client for call hook');
app.requestor = new HttpRequestor(logger, account.account_sid, app.call_hook, account.webhook_secret);
}
if (!app.notifier && app.call_status_hook) {
app.notifier = new HttpRequestor(logger, account.account_sid, app.call_status_hook, account.webhook_secret);
logger.debug({call_status_hook: app.call_status_hook}, 'creating http client for call status hook');
}
else if (!app.notifier) {
logger.debug('creating null call status hook');
app.notifier = {request: () => {}, close: () => {}};
if ('WS' === app.call_hook?.method || /^wss?:/.test(app.call_hook.url)) {
logger.debug({call_hook: app.call_hook}, 'creating websocket for call hook');
app.requestor = new WsRequestor(logger, account.account_sid, app.call_hook, account.webhook_secret) ;
if (app.call_hook.url === app.call_status_hook.url || !app.call_status_hook?.url) {
logger.debug('reusing websocket for call status hook');
app.notifier = app.requestor;
}
}
else {
logger.debug({call_hook: app.call_hook}, 'creating http client for call hook');
app.requestor = new HttpRequestor(logger, account.account_sid, app.call_hook, account.webhook_secret);
}
if (!app.notifier && app.call_status_hook) {
app.notifier = new HttpRequestor(logger, account.account_sid, app.call_status_hook, account.webhook_secret);
logger.debug({call_hook: app.call_hook}, 'creating http client for call status hook');
}
else if (!app.notifier) {
logger.debug('creating null call status hook');
app.notifier = {request: () => {}, close: () => {}};
}
/* now launch the outdial */
try {
const dlg = await srf.createUAC(uri, {...opts, followRedirects: true, keepUriOnRedirect: true}, {
cbRequest: (err, inviteReq) => {
/* now launch the outdial */
try {
const dlg = await srf.createUAC(uri, {...opts, followRedirects: true, keepUriOnRedirect: true}, {
cbRequest: (err, inviteReq) => {
/* in case of 302 redirect, this gets called twice, ignore the second
except to update the req so that it can later be canceled if need be
*/
if (res.headersSent) {
logger.info(`create-call: got redirect, updating request to new call-id ${req.get('Call-ID')}`);
if (cs) cs.req = inviteReq;
return;
}
if (err) {
logger.error(err, 'createCall Error creating call');
res.status(500).send('Call Failure');
return;
}
inviteReq.srf = srf;
inviteReq.locals = {
...(inviteReq || {}),
callSid,
application_sid: app.application_sid
};
/* ok our outbound INVITE is in flight */
const tasks = [restDial];
sipLogger = logger.child({
callSid,
callId: inviteReq.get('Call-ID'),
accountSid,
traceId: rootSpan.traceId
}, {
...(account.enable_debug_log && {level: 'debug'})
});
app.requestor.logger = app.notifier.logger = sipLogger;
const callInfo = new CallInfo({
direction: CallDirection.Outbound,
req: inviteReq,
to,
tag: app.tag,
callSid,
accountSid: req.body.account_sid,
applicationSid: app.application_sid,
traceId: rootSpan.traceId
});
cs = new RestCallSession({
logger: sipLogger,
application: app,
srf,
req: inviteReq,
ep,
ep2: dualEp,
tasks,
callInfo,
accountInfo,
rootSpan
});
cs.exec(req);
res.status(201).json({sid: cs.callSid, callId: inviteReq.get('Call-ID')});
sipLogger.info({sid: cs.callSid, callId: inviteReq.get('Call-ID')},
`outbound REST call attempt to ${JSON.stringify(target)} has been sent`);
},
cbProvisional: (prov) => {
const callStatus = prov.body ? CallStatus.EarlyMedia : CallStatus.Ringing;
// Update call-id for sbc outbound INVITE
cs.callInfo.sbcCallid = prov.get('X-CID');
if ([180, 183].includes(prov.status) && prov.body) connectStream(prov.body);
restDial.emit('callStatus', prov.status, !!prov.body);
cs.emit('callStatusChange', {callStatus, sipStatus: prov.status});
if (res.headersSent) {
logger.info(`create-call: got redirect, updating request to new call-id ${req.get('Call-ID')}`);
if (cs) cs.req = inviteReq;
return;
}
});
connectStream(dlg.remote.sdp);
cs.emit('callStatusChange', {
callStatus: CallStatus.InProgress,
sipStatus: 200,
sipReason: 'OK'
});
restDial.emit('callStatus', 200);
restDial.emit('connect', dlg);
}
catch (err) {
let callStatus = CallStatus.Failed;
if (err instanceof SipError) {
if ([486, 603].includes(err.status)) callStatus = CallStatus.Busy;
else if (487 === err.status) callStatus = CallStatus.NoAnswer;
if (sipLogger) sipLogger.info(`REST outdial failed with ${err.status}`);
else console.log(`REST outdial failed with ${err.status}`);
if (cs) cs.emit('callStatusChange', {
callStatus,
sipStatus: err.status,
sipReason: err.reason
if (err) {
logger.error(err, 'createCall Error creating call');
res.status(500).send('Call Failure');
return;
}
inviteReq.srf = srf;
inviteReq.locals = {
...(inviteReq || {}),
callSid,
application_sid: app.application_sid
};
/* ok our outbound INVITE is in flight */
const tasks = [restDial];
const rootSpan = new RootSpan('rest-call', inviteReq);
sipLogger = logger.child({
callSid,
callId: inviteReq.get('Call-ID'),
accountSid,
traceId: rootSpan.traceId
});
cs.callGone = true;
}
else {
if (cs) cs.emit('callStatusChange', {
callStatus,
sipStatus: 500,
sipReason: 'Internal Server Error'
app.requestor.logger = app.notifier.logger = sipLogger;
const callInfo = new CallInfo({
direction: CallDirection.Outbound,
req: inviteReq,
to,
tag: app.tag,
callSid,
accountSid: req.body.account_sid,
applicationSid: app.application_sid,
traceId: rootSpan.traceId
});
if (sipLogger) sipLogger.error({err}, 'REST outdial failed');
else console.error(err);
cs = new RestCallSession({
logger: sipLogger,
application: app,
srf,
req: inviteReq,
ep,
tasks,
callInfo,
accountInfo,
rootSpan
});
cs.exec(req);
res.status(201).json({sid: cs.callSid, callId: inviteReq.get('Call-ID')});
sipLogger.info({sid: cs.callSid, callId: inviteReq.get('Call-ID')},
`outbound REST call attempt to ${JSON.stringify(target)} has been sent`);
},
cbProvisional: (prov) => {
const callStatus = prov.body ? CallStatus.EarlyMedia : CallStatus.Ringing;
if ([180, 183].includes(prov.status) && prov.body) connectStream(prov.body);
restDial.emit('callStatus', prov.status, !!prov.body);
cs.emit('callStatusChange', {callStatus, sipStatus: prov.status});
}
ep.destroy();
if (dualEp) {
dualEp.destroy();
}
setTimeout(restDial.kill.bind(restDial, cs), 5000);
}
} catch (err) {
sysError(logger, res, err);
});
connectStream(dlg.remote.sdp);
cs.emit('callStatusChange', {
callStatus: CallStatus.InProgress,
sipStatus: 200,
sipReason: 'OK'
});
restDial.emit('callStatus', 200);
restDial.emit('connect', dlg);
}
});
catch (err) {
let callStatus = CallStatus.Failed;
if (err instanceof SipError) {
if ([486, 603].includes(err.status)) callStatus = CallStatus.Busy;
else if (487 === err.status) callStatus = CallStatus.NoAnswer;
if (sipLogger) sipLogger.info(`REST outdial failed with ${err.status}`);
else console.log(`REST outdial failed with ${err.status}`);
if (cs) cs.emit('callStatusChange', {
callStatus,
sipStatus: err.status,
sipReason: err.reason
});
}
else {
if (cs) cs.emit('callStatusChange', {
callStatus,
sipStatus: 500,
sipReason: 'Internal Server Error'
});
if (sipLogger) sipLogger.error({err}, 'REST outdial failed');
else console.error(err);
}
ep.destroy();
setTimeout(restDial.kill.bind(restDial), 5000);
}
} catch (err) {
sysError(logger, res, err);
}
});
module.exports = router;

View File

@@ -9,29 +9,25 @@ const {CallStatus, CallDirection} = require('../../utils/constants');
*/
function retrieveCallSession(callSid, opts) {
if (opts.call_status_hook && !opts.call_hook) {
throw new DbErrorBadRequest(
`call_status_hook can be updated only when call_hook is also being updated for call_sid ${callSid}`);
throw new DbErrorBadRequest('call_status_hook can be updated only when call_hook is also being updated');
}
const cs = sessionTracker.get(callSid);
if (!cs) {
throw new DbErrorUnprocessableRequest(`call session is gone for call_sid ${callSid}`);
throw new DbErrorUnprocessableRequest('call session is gone');
}
if (opts.call_status === CallStatus.Completed && !cs.hasStableDialog) {
throw new DbErrorUnprocessableRequest(
`current call state is incompatible with requested action for call_sid ${callSid}`);
throw new DbErrorUnprocessableRequest('current call state is incompatible with requested action');
}
else if (opts.call_status === CallStatus.NoAnswer) {
if (cs.direction === CallDirection.Outbound) {
if (!cs.isOutboundCallRinging) {
throw new DbErrorUnprocessableRequest(
`current call state is incompatible with requested action for call_sid ${callSid}`);
throw new DbErrorUnprocessableRequest('current call state is incompatible with requested action');
}
}
else {
if (cs.isInboundCallAnswered) {
throw new DbErrorUnprocessableRequest(
`current call state is incompatible with requested action for call_sid ${callSid}`);
throw new DbErrorUnprocessableRequest('current call state is incompatible with requested action');
}
}
}
@@ -45,7 +41,7 @@ function retrieveCallSession(callSid, opts) {
router.post('/:callSid', async(req, res) => {
const logger = req.app.locals.logger;
const callSid = req.params.callSid;
logger.debug({body: req.body}, 'got updateCall request');
logger.debug({body: req.body}, 'got upateCall request');
try {
const cs = retrieveCallSession(callSid, req.body);
if (!cs) {

View File

@@ -1,134 +0,0 @@
const { checkSchema } = require('express-validator');
/**
* @path api-server {{base_url}}/v1/Accounts/:account_sid/Calls
* @see https://api.jambonz.org/#243a2edd-7999-41db-bd0d-08082bbab401
*/
const createCallSchema = checkSchema({
application_sid: {
isString: true,
optional: true,
isLength: { options: { min: 36, max: 36 } },
errorMessage: 'Invalid application_sid',
},
answerOnBridge: {
isBoolean: true,
optional: true,
errorMessage: 'Invalid answerOnBridge',
},
from: {
errorMessage: 'Invalid from',
isString: true,
isLength: {
options: { min: 1, max: 256 },
},
},
fromHost: {
isString: true,
optional: true,
errorMessage: 'Invalid fromHost',
},
to: {
errorMessage: 'Invalid to',
isObject: true,
},
callerName: {
isString: true,
optional: true,
errorMessage: 'Invalid callerName',
},
amd: {
isObject: true,
optional: true,
},
tag: {
isObject: true,
optional: true,
errorMessage: 'Invalid tag',
},
app_json: {
isString: true,
optional: true,
errorMessage: 'Invalid app_json',
},
account_sid: {
isString: true,
optional: true,
errorMessage: 'Invalid account_sid',
isLength: { options: { min: 36, max: 36 } },
},
timeout: {
isInt: true,
optional: true,
errorMessage: 'Invalid timeout',
},
timeLimit: {
isInt: true,
optional: true,
errorMessage: 'Invalid timeLimit',
},
call_hook: {
isObject: true,
optional: true,
errorMessage: 'Invalid call_hook',
},
call_status_hook: {
isObject: true,
optional: true,
errorMessage: 'Invalid call_status_hook',
},
speech_synthesis_vendor: {
isString: true,
optional: true,
errorMessage: 'Invalid speech_synthesis_vendor',
},
speech_synthesis_language: {
isString: true,
optional: true,
errorMessage: 'Invalid speech_synthesis_language',
},
speech_synthesis_voice: {
isString: true,
optional: true,
errorMessage: 'Invalid speech_synthesis_voice',
},
speech_recognizer_vendor: {
isString: true,
optional: true,
errorMessage: 'Invalid speech_recognizer_vendor',
},
speech_recognizer_language: {
isString: true,
optional: true,
errorMessage: 'Invalid speech_recognizer_language',
}
}, ['body']);
const customSanitizeFunction = (value) => {
try {
if (Array.isArray(value)) {
value = value.map((item) => customSanitizeFunction(item));
} else if (typeof value === 'object') {
Object.keys(value).forEach((key) => {
value[key] = customSanitizeFunction(value[key]);
});
} else if (typeof value === 'string') {
/* trims characters at the beginning and at the end of a string */
value = value.trim();
/* Verify strings including 'http' via new URL */
if (value.includes('http')) {
value = new URL(value).toString();
}
}
} catch (error) {
value = `Error: ${error.message}`;
}
return value;
};
module.exports = {
createCallSchema,
customSanitizeFunction
};

View File

@@ -1,5 +1,5 @@
const uuidv4 = require('uuid-random');
const {CallDirection, AllowedSipRecVerbs, WS_CLOSE_CODES} = require('./utils/constants');
const {CallDirection, AllowedSipRecVerbs} = require('./utils/constants');
const {parseSiprecPayload} = require('./utils/siprec-utils');
const CallInfo = require('./session/call-info');
const HttpRequestor = require('./utils/http-requestor');
@@ -10,12 +10,6 @@ const { normalizeJambones } = require('@jambonz/verb-specifications');
const dbUtils = require('./utils/db-utils');
const RootSpan = require('./utils/call-tracer');
const listTaskNames = require('./utils/summarize-tasks');
const {
JAMBONES_MYSQL_REFRESH_TTL,
JAMBONES_DISABLE_DIRECT_P2P_CALL
} = require('./config');
const { createJambonzApp } = require('./dynamic-apps');
const { decrypt } = require('./utils/encrypt-decrypt');
module.exports = function(srf, logger) {
const {
@@ -23,21 +17,17 @@ module.exports = function(srf, logger) {
lookupAppByRegex,
lookupAppBySid,
lookupAppByRealm,
lookupAppByTeamsTenant,
registrar,
lookupClientByAccountAndUsername
lookupAppByTeamsTenant
} = srf.locals.dbHelpers;
const {
writeAlerts,
AlertType
} = srf.locals;
const {lookupAccountDetails, lookupGoogleCustomVoice} = dbUtils(logger, srf);
const {lookupAccountDetails} = dbUtils(logger, srf);
async function initLocals(req, res, next) {
function initLocals(req, res, next) {
const callId = req.get('Call-ID');
const uri = parseUri(req.uri);
logger.info({
uri,
callId,
callingNumber: req.callingNumber,
calledNumber: req.calledNumber
@@ -49,62 +39,16 @@ module.exports = function(srf, logger) {
const callSid = req.has('X-Retain-Call-Sid') ? req.get('X-Retain-Call-Sid') : uuidv4();
const account_sid = req.get('X-Account-Sid');
req.locals = {callSid, account_sid, callId};
let clientDb = null;
if (req.has('X-Authenticated-User')) {
req.locals.originatingUser = req.get('X-Authenticated-User');
let clientSettings;
const arr = /^(.*)@(.*)/.exec(req.locals.originatingUser);
if (arr) {
[clientSettings] = await lookupClientByAccountAndUsername(account_sid, arr[1]);
}
clientDb = await registrar.query(req.locals.originatingUser);
clientDb = {
...clientDb,
...clientSettings,
};
}
// check for call to application
if (uri.user?.startsWith('app-') && req.locals.originatingUser && clientDb?.allow_direct_app_calling) {
const application_sid = uri.user.match(/app-(.*)/)[1];
logger.debug(`got application from Request URI header: ${application_sid}`);
req.locals.application_sid = application_sid;
} else if (req.has('X-Application-Sid')) {
if (req.has('X-Application-Sid')) {
const application_sid = req.get('X-Application-Sid');
logger.debug(`got application from X-Application-Sid header: ${application_sid}`);
req.locals.application_sid = application_sid;
}
// check for call to queue
else if (uri.user?.startsWith('queue-') && req.locals.originatingUser && clientDb?.allow_direct_queue_calling) {
const queue_name = uri.user.match(/queue-(.*)/)[1];
logger.debug(`got Queue from Request URI header: ${queue_name}`);
req.locals.queue_name = queue_name;
}
// check for call to conference
else if (uri.user?.startsWith('conference-') && req.locals.originatingUser && clientDb?.allow_direct_app_calling) {
const conference_id = uri.user.match(/conference-(.*)/)[1];
logger.debug(`got Conference from Request URI header: ${conference_id}`);
req.locals.conference_id = conference_id;
}
// check for call to registered user
else if (!JAMBONES_DISABLE_DIRECT_P2P_CALL && req.locals.originatingUser && clientDb?.allow_direct_user_calling) {
const arr = /^(.*)@(.*)/.exec(req.locals.originatingUser);
if (arr) {
const sipRealm = arr[2];
const called_user = `${req.calledNumber}@${sipRealm}`;
const reg = await registrar.query(called_user);
if (reg) {
logger.debug(`got called Number is a registered user: ${called_user}`);
req.locals.called_user = called_user;
}
}
}
if (req.has('X-Authenticated-User')) req.locals.originatingUser = req.get('X-Authenticated-User');
if (req.has('X-MS-Teams-Tenant-FQDN')) req.locals.msTeamsTenant = req.get('X-MS-Teams-Tenant-FQDN');
if (req.has('X-Cisco-Recording-Participant')) {
const ciscoParticipants = req.get('X-Cisco-Recording-Participant');
const regex = /sip:[a-zA-Z0-9]+@[a-zA-Z0-9.-_]+/g;
const regex = /sip:[\d]+@[\d]+\.[\d]+\.[\d]+\.[\d]+/g;
const sipURIs = ciscoParticipants.match(regex);
logger.info(`X-Cisco-Recording-Participant : ${sipURIs} `);
if (sipURIs && sipURIs.length > 0) {
@@ -173,7 +117,7 @@ module.exports = function(srf, logger) {
};
logger.info({callId, metadata, sdp}, 'successfully parsed SIPREC payload');
} catch (err) {
logger.info({err, callId}, 'Error parsing multipart payload');
logger.info({callId}, 'Error parsing multipart payload');
return res.send(503);
}
}
@@ -188,20 +132,14 @@ module.exports = function(srf, logger) {
const {span} = rootSpan.startChildSpan('lookupAccountDetails');
try {
const accountDetail = await lookupAccountDetails(account_sid);
const account = accountDetail?.account;
req.locals.accountInfo = accountDetail;
req.locals.service_provider_sid = account?.service_provider_sid;
req.locals.accountInfo = await lookupAccountDetails(account_sid);
req.locals.service_provider_sid = req.locals.accountInfo?.account?.service_provider_sid;
span.end();
if (!account?.is_active) {
if (!req.locals.accountInfo.account.is_active) {
logger.info(`Account is inactive or suspended ${account_sid}`);
// TODO: alert
return res.send(503, {headers: {'X-Reason': 'Account exists but is inactive'}});
}
// Change the default log level to debug
if (account?.enable_debug_log) {
req.locals.logger.level = 'debug';
}
logger.debug({accountInfo: req.locals?.accountInfo?.account}, `retrieved account info for ${account_sid}`);
next();
} catch (err) {
@@ -243,27 +181,15 @@ module.exports = function(srf, logger) {
const {span} = rootSpan.startChildSpan('lookupApplication');
try {
let app;
if (req.locals.queue_name) {
logger.debug(`calling to queue ${req.locals.queue_name}, generating queue app`);
app = createJambonzApp('queue', {account_sid, name: req.locals.queue_name});
} else if (req.locals.called_user) {
logger.debug(`calling to registered user ${req.locals.called_user}, generating dial app`);
app = createJambonzApp('user',
{account_sid, name: req.locals.called_user, caller_id: req.locals.callingNumber});
} else if (req.locals.conference_id) {
logger.debug(`calling to conference ${req.locals.conference_id}, generating conference app`);
app = createJambonzApp('conference', {account_sid, name: req.locals.conference_id});
} else if (req.locals.application_sid) {
app = await lookupAppBySid(req.locals.application_sid);
} else if (req.locals.originatingUser) {
if (req.locals.application_sid) app = await lookupAppBySid(req.locals.application_sid);
else if (req.locals.originatingUser) {
const arr = /^(.*)@(.*)/.exec(req.locals.originatingUser);
if (arr) {
const sipRealm = arr[2];
logger.debug(`looking for device calling app for realm ${sipRealm}`);
app = await lookupAppByRealm(sipRealm);
if (app) {
logger.debug({app}, `retrieved device calling app for realm ${sipRealm}`);
}
if (app) logger.debug({app}, `retrieved device calling app for realm ${sipRealm}`);
}
}
else if (req.locals.msTeamsTenant) {
@@ -316,43 +242,26 @@ module.exports = function(srf, logger) {
*/
/* allow for caching data - when caching treat retrieved data as immutable */
const app2 = JAMBONES_MYSQL_REFRESH_TTL ? JSON.parse(JSON.stringify(app)) : app;
const app2 = process.env.JAMBONES_MYSQL_REFRESH_TTL ? JSON.parse(JSON.stringify(app)) : app;
if ('WS' === app.call_hook?.method ||
app.call_hook?.url.startsWith('ws://') || app.call_hook?.url.startsWith('wss://')) {
const requestor = new WsRequestor(logger, account_sid, app.call_hook, accountInfo.account.webhook_secret) ;
app2.requestor = requestor;
app2.notifier = requestor;
app2.requestor = new WsRequestor(logger, account_sid, app.call_hook, accountInfo.account.webhook_secret) ;
app2.notifier = app.requestor;
app2.call_hook.method = 'WS';
}
else {
app2.requestor = new HttpRequestor(logger, account_sid, app.call_hook, accountInfo.account.webhook_secret);
if (app.call_status_hook) app2.notifier = new HttpRequestor(logger, account_sid, app.call_status_hook,
accountInfo.account.webhook_secret);
else app2.notifier = {request: () => {}, close: () => {}};
}
// Resolve application.speech_synthesis_voice if it's custom voice
if (app2.speech_synthesis_vendor === 'google' && app2.speech_synthesis_voice.startsWith('custom_')) {
const arr = /custom_(.*)/.exec(app2.speech_synthesis_voice);
if (arr) {
const google_custom_voice_sid = arr[1];
const [custom_voice] = await lookupGoogleCustomVoice(google_custom_voice_sid);
//google voice cloning key has size 200kb, jambonz should not resolve the voice here that the app's calling
//webhook will receive big payload, tts-task should resolve the voice later.
if (!custom_voice.use_voice_cloning_key) {
app2.speech_synthesis_voice = {
reportedUsage: custom_voice.reported_usage,
model: custom_voice.model
};
}
}
else app2.notifier = {request: () => {}};
}
req.locals.application = app2;
// eslint-disable-next-line no-unused-vars
const {call_hook, call_status_hook, ...appInfo} = app; // mask sensitive data like user/pass on webhook
// eslint-disable-next-line no-unused-vars
const {requestor, notifier, env_vars, ...loggable} = appInfo;
const {requestor, notifier, ...loggable} = appInfo;
logger.info({app: loggable}, `retrieved application for incoming call to ${req.locals.calledNumber}`);
req.locals.callInfo = new CallInfo({
req,
@@ -360,17 +269,6 @@ module.exports = function(srf, logger) {
direction: CallDirection.Inbound,
traceId: rootSpan.traceId
});
// if transferred call contains callInfo, let update original data to newly created callInfo in this instance.
if (app.transferredCall && app.callInfo) {
const {direction, callerName, from, to, originatingSipIp, originatingSipTrunkName} = app.callInfo;
req.locals.callInfo.direction = direction;
req.locals.callInfo.callerName = callerName;
req.locals.callInfo.from = from;
req.locals.callInfo.to = to;
req.locals.callInfo.originatingSipIp = originatingSipIp;
req.locals.callInfo.originatingSipTrunkName = originatingSipTrunkName;
delete app.callInfo;
}
next();
} catch (err) {
span.end();
@@ -387,7 +285,7 @@ module.exports = function(srf, logger) {
const {rootSpan, siprec, application:app} = req.locals;
let span;
try {
if (app.tasks && app.tasks?.length > 0 && !JAMBONES_MYSQL_REFRESH_TTL) {
if (app.tasks && !process.env.JAMBONES_MYSQL_REFRESH_TTL) {
app.tasks = normalizeJambones(logger, app.tasks).map((tdata) => makeTask(logger, tdata));
if (0 === app.tasks.length) throw new Error('no application provided');
return next();
@@ -397,42 +295,22 @@ module.exports = function(srf, logger) {
if (app.app_json) {
json = JSON.parse(app.app_json);
} else {
const defaults = {
synthesizer: {
vendor: app.speech_synthesis_vendor,
...(app.speech_synthesis_label && {label: app.speech_synthesis_label}),
language: app.speech_synthesis_language,
voice: app.speech_synthesis_voice,
...(app.fallback_speech_synthesis_vendor && {fallback_vendor: app.fallback_speech_synthesis_vendor}),
...(app.fallback_speech_synthesis_label && {fallback_label: app.fallback_speech_synthesis_label}),
...(app.fallback_speech_synthesis_language && {fallback_language: app.fallback_speech_synthesis_language}),
...(app.fallback_speech_synthesis_voice && {fallback_voice: app.fallback_speech_synthesis_voice})
},
recognizer: {
vendor: app.speech_recognizer_vendor,
...(app.speech_recognizer_label && {label: app.speech_recognizer_label}),
language: app.speech_recognizer_language,
...(app.fallback_speech_recognizer_vendor && {fallback_vendor: app.fallback_speech_recognizer_vendor}),
...(app.fallback_speech_recognizer_label && {fallback_label: app.fallback_speech_recognizer_label}),
...(app.fallback_speech_recognizer_language && {fallback_language: app.fallback_speech_recognizer_language})
}
};
let env_vars;
try {
if (app.env_vars) {
const d_env_vars = JSON.parse(decrypt(app.env_vars));
logger.info(`Setting env_vars: ${Object.keys(d_env_vars)}`); // Only log the keys not the values
env_vars = d_env_vars;
}
} catch (error) {
logger.info('Unable to set env_vars', error);
}
const params = Object.assign(['POST', 'WS'].includes(app.call_hook.method) ? { sip: req.msg } : {},
req.locals.callInfo,
{ service_provider_sid: req.locals.service_provider_sid },
{ defaults },
{ env_vars }
);
{
defaults: {
synthesizer: {
vendor: app.speech_synthesis_vendor,
language: app.speech_synthesis_language,
voice: app.speech_synthesis_voice
},
recognizer: {
vendor: app.speech_recognizer_vendor,
language: app.speech_recognizer_language
}
}
});
logger.debug({ params }, 'sending initial webhook');
const obj = rootSpan.startChildSpan('performAppWebhook');
span = obj.span;
@@ -440,7 +318,6 @@ module.exports = function(srf, logger) {
const httpHeaders = b3 && { b3 };
json = await app.requestor.request('session:new', app.call_hook, params, httpHeaders);
}
app.tasks = normalizeJambones(logger, json).map((tdata) => makeTask(logger, tdata));
span?.setAttributes({
'http.statusCode': 200,
@@ -472,7 +349,7 @@ module.exports = function(srf, logger) {
}).catch((err) => this.logger.info({err}, 'Error generating alert for parsing application'));
logger.info({err}, `Error retrieving or parsing application: ${err?.message}`);
res.send(480, {headers: {'X-Reason': err?.message || 'unknown'}});
app.requestor.close(WS_CLOSE_CODES.GoingAway);
app.requestor.close();
}
}

View File

@@ -1,6 +1,4 @@
const CallSession = require('./call-session');
const {CallStatus} = require('../utils/constants');
const moment = require('moment');
/**
* @classdesc Subclass of CallSession. Represents a CallSession
@@ -21,14 +19,12 @@ class AdultingCallSession extends CallSession {
rootSpan
});
this.sd = singleDialer;
this.req = callInfo.req;
this.sd.dlg.on('destroy', () => {
this.logger.info('AdultingCallSession: called party hung up');
this._callReleased();
});
this.sd.emit('adulting');
this.on('callStatusChange', this._notifyCallStatusChange.bind(this));
}
get dlg() {
@@ -45,36 +41,14 @@ class AdultingCallSession extends CallSession {
return this.sd.ep;
}
// When adulting session kicked from conference, replaceEndpoint is a must
set ep(newEp) {
this.sd.ep = newEp;
}
/* see note above */
set ep(newEp) {}
get callSid() {
return this.callInfo.callSid;
}
_callerHungup() {
this._hangup('caller');
}
_jambonzHangup() {
this._hangup();
}
_hangup(terminatedBy = 'jambonz') {
if (this.dlg.connectTime) {
const duration = moment().diff(this.dlg.connectTime, 'seconds');
this.rootSpan.setAttributes({'call.termination': `hangup by ${terminatedBy}`});
this.callInfo.callTerminationBy = terminatedBy;
this.emit('callStatusChange', {
callStatus: CallStatus.Completed,
duration
});
}
this.logger.info(`InboundCallSession: ${terminatedBy} hung up`);
this._callReleased();
this.req.removeAllListeners('cancel');
}
}

View File

@@ -1,7 +1,6 @@
const {CallDirection, CallStatus} = require('../utils/constants');
const parseUri = require('drachtio-srf').parseUri;
const uuidv4 = require('uuid-random');
const {JAMBONES_API_BASE_URL} = require('../config');
/**
* @classdesc Represents the common information for all calls
* that is provided in call status webhooks
@@ -32,7 +31,6 @@ class CallInfo {
this.sipStatus = 100;
this.sipReason = 'Trying';
this.callStatus = CallStatus.Trying;
this.sbcCallid = req.get('X-CID');
this.originatingSipIp = req.get('X-Forwarded-For');
this.originatingSipTrunkName = req.get('X-Originating-Carrier');
const {siprec} = req.locals;
@@ -42,13 +40,13 @@ class CallInfo {
this.participants = [
{
participant: 'caller',
uriUser: caller?.user,
uriHost: caller?.host
uriUser: caller.user,
uriHost: caller.host
},
{
participant: 'callee',
uriUser: callee?.user,
uriHost: callee?.host
uriUser: callee.user,
uriHost: callee.host
}
];
}
@@ -130,7 +128,6 @@ class CallInfo {
from: this.from,
to: this.to,
callId: this.callId,
sbcCallid: this.sbcCallid,
sipStatus: this.sipStatus,
sipReason: this.sipReason,
callStatus: this.callStatus,
@@ -149,8 +146,8 @@ class CallInfo {
Object.assign(obj, {customerData: this._customerData});
}
if (JAMBONES_API_BASE_URL) {
Object.assign(obj, {apiBaseUrl: JAMBONES_API_BASE_URL});
if (process.env.JAMBONES_API_BASE_URL) {
Object.assign(obj, {apiBaseUrl: process.env.JAMBONES_API_BASE_URL});
}
if (this.publicIp) {
Object.assign(obj, {fsPublicIp: this.publicIp});

File diff suppressed because it is too large Load Diff

View File

@@ -8,7 +8,7 @@ const CallSession = require('./call-session');
*/
class ConfirmCallSession extends CallSession {
constructor({logger, application, dlg, ep, tasks, callInfo, accountInfo, memberId, confName, rootSpan, req}) {
constructor({logger, application, dlg, ep, tasks, callInfo, accountInfo, memberId, confName, rootSpan}) {
super({
logger,
application,
@@ -23,7 +23,6 @@ class ConfirmCallSession extends CallSession {
});
this.dlg = dlg;
this.ep = ep;
this.req = req;
}
/**
@@ -35,9 +34,6 @@ class ConfirmCallSession extends CallSession {
_callerHungup() {
}
_jambonzHangup() {
}
}

View File

@@ -67,33 +67,15 @@ class InboundCallSession extends CallSession {
* This is invoked when the caller hangs up, in order to calculate the call duration.
*/
_callerHungup() {
this._hangup('caller');
}
_jambonzHangup(reason) {
this.dlg?.destroy({
headers: {
...(reason && {'X-Reason': reason})
}
});
// kill current task or wakeup the call session.
this._callReleased();
}
_hangup(terminatedBy = 'jambonz') {
if (this.dlg === null) {
this.logger.info('InboundCallSession:_hangup - race condition, dlg cleared by app hangup');
return;
}
this.logger.info(`InboundCallSession: ${terminatedBy} hung up`);
assert(this.dlg.connectTime);
const duration = moment().diff(this.dlg.connectTime, 'seconds');
this.rootSpan.setAttributes({'call.termination': `hangup by ${terminatedBy}`});
this.callInfo.callTerminationBy = terminatedBy;
this.rootSpan.setAttributes({'call.termination': 'hangup by caller'});
this.callInfo.callTerminationBy = 'caller';
this.emit('callStatusChange', {
callStatus: CallStatus.Completed,
duration
});
this.logger.info('InboundCallSession: caller hung up');
this._callReleased();
this.req.removeAllListeners('cancel');
}

View File

@@ -1,13 +1,14 @@
const CallSession = require('./call-session');
const {CallStatus} = require('../utils/constants');
const moment = require('moment');
/**
* @classdesc Subclass of CallSession. This represents a CallSession that is
* created for an outbound call that is initiated via the REST API.
* @extends CallSession
*/
class RestCallSession extends CallSession {
constructor({logger, application, srf, req, ep, ep2, tasks, callInfo, accountInfo, rootSpan}) {
constructor({logger, application, srf, req, ep, tasks, callInfo, accountInfo, rootSpan}) {
super({
logger,
application,
@@ -20,11 +21,6 @@ class RestCallSession extends CallSession {
});
this.req = req;
this.ep = ep;
this.ep2 = ep2;
// keep restDialTask reference for closing AMD
if (tasks.length) {
this.restDialTask = tasks[0];
}
this.on('callStatusChange', this._notifyCallStatusChange.bind(this));
this._notifyCallStatusChange({
@@ -41,29 +37,17 @@ class RestCallSession extends CallSession {
setDialog(dlg) {
this.dlg = dlg;
dlg.on('destroy', this._callerHungup.bind(this));
dlg.on('refer', this._onRefer.bind(this));
dlg.on('modify', this._onReinvite.bind(this));
this.wrapDialog(dlg);
}
/**
* This is invoked when the called party hangs up, in order to calculate the call duration.
*/
_callerHungup() {
this._hangup('caller');
}
_jambonzHangup() {
this._hangup();
}
_hangup(terminatedBy = 'jambonz') {
if (this.restDialTask) {
this.restDialTask.turnOffAmd();
}
this.callInfo.callTerminationBy = terminatedBy;
this.callInfo.callTerminationBy = 'caller';
const duration = moment().diff(this.dlg.connectTime, 'seconds');
this.emit('callStatusChange', {callStatus: CallStatus.Completed, duration});
this.logger.info(`RestCallSession: called party hung up by ${terminatedBy}`);
this.logger.debug('RestCallSession: called party hung up');
this._callReleased();
}

View File

@@ -1,22 +0,0 @@
const Task = require('./task');
const {TaskName, TaskPreconditions} = require('../utils/constants');
/**
* Answer the call.
* Note: This is rarely used, as the call is typically answered automatically when required by the app,
* but it can be useful to force an answer before a pause in some cases
*/
class TaskAnswer extends Task {
constructor(logger, opts) {
super(logger, opts);
this.preconditions = TaskPreconditions.Endpoint;
}
get name() { return TaskName.Answer; }
async exec(cs) {
super.exec(cs);
}
}
module.exports = TaskAnswer;

View File

@@ -6,7 +6,6 @@ const { normalizeJambones } = require('@jambonz/verb-specifications');
const makeTask = require('./make_task');
const bent = require('bent');
const assert = require('assert');
const HttpRequestor = require('../utils/http-requestor');
const WAIT = 'wait';
const JOIN = 'join';
const START = 'start';
@@ -49,7 +48,7 @@ class Conference extends Task {
this.confName = this.data.name;
[
'beep', 'startConferenceOnEnter', 'endConferenceOnExit', 'joinMuted',
'maxParticipants', 'waitHook', 'statusHook', 'endHook', 'enterHook', 'endConferenceDuration'
'maxParticipants', 'waitHook', 'statusHook', 'endHook', 'enterHook'
].forEach((attr) => this[attr] = this.data[attr]);
this.record = this.data.record || {};
this.statusEvents = [];
@@ -61,8 +60,6 @@ class Conference extends Task {
this.emitter = new Emitter();
this.results = {};
this.coaching = [];
this.speakOnlyTo = this.data.speakOnlyTo;
// transferred from another server in order to bridge to a local caller?
if (this.data._ && this.data._.connectTime) {
@@ -83,11 +80,7 @@ class Conference extends Task {
// reset answer time if we were transferred from another feature server
if (this.connectTime) dlg.connectTime = this.connectTime;
if (cs.sipRequestWithinDialogHook) {
/* remove any existing listener to escape from duplicating events */
this._removeSipIndialogRequestListener(this.dlg);
this._initSipIndialogRequestListener(cs, dlg);
}
this.ep.on('destroy', this._kicked.bind(this, cs, dlg));
try {
@@ -107,7 +100,6 @@ class Conference extends Task {
this.logger.debug(`Conference:exec - conference ${this.confName} is over`);
if (this.callMoved !== false) await this.performAction(this.results);
this._removeSipIndialogRequestListener(dlg);
} catch (err) {
this.logger.info(err, `TaskConference:exec - error in conference ${this.confName}`);
}
@@ -122,14 +114,7 @@ class Conference extends Task {
}
this.emitter.emit('kill');
await this._doFinalMemberCheck(cs);
if (this.ep && this.ep.connected) {
// drachtio-fsmrf override esl::event::CUSTOM to conference join listerner, After finish the conference
// the application need to reset the esl::event::CUSTOM for another use on the same endpoint
this.ep.resetEslCustomEvent();
this.ep.api(`conference ${this.confName} kick ${this.memberId}`)
.catch((err) => this.logger.info({err}, 'Error kicking participant'));
}
cs.clearConferenceDetails();
if (this.ep && this.ep.connected) this.ep.conn.removeAllListeners('esl::event::CUSTOM::*') ;
this.notifyTaskDone();
}
@@ -142,10 +127,15 @@ class Conference extends Task {
* @param {SipDialog} dlg
*/
async _init(cs, dlg) {
const friendlyName = this.confName;
const {createHash, retrieveHash} = cs.srf.locals.dbHelpers;
this.friendlyName = this.confName;
this.confName = `conf:${cs.accountSid}:${this.confName}`;
this.statusParams = Object.assign({
conferenceSid: this.confName,
friendlyName
}, cs.callInfo);
// check if conference is in progress
const obj = await retrieveHash(this.confName);
if (obj) {
@@ -349,33 +339,16 @@ class Conference extends Task {
}
const opts = {};
if (this.endConferenceOnExit || this.startConferenceOnEnter || this.joinMuted) {
Object.assign(opts, {flags: {
...(this.endConferenceOnExit && {endconf: true}),
...(this.startConferenceOnEnter && {moderator: true}),
//https://developer.signalwire.com/freeswitch/FreeSWITCH-Explained/Modules/mod_conference_3965534/
// mute | Enter conference muted
...((this.joinMuted || this.speakOnlyTo) && {mute: true}),
}});
/**
* Note on the above: if we are joining in "coaching" mode (ie only going to heard by a subset of participants)
* then we join muted temporarily, and then unmute ourselves once we have identified the subset of participants
* to whom we will be speaking.
*/
}
if (this.endConferenceOnExit) Object.assign(opts, {flags: {endconf: true}});
if (this.startConferenceOnEnter) Object.assign(opts, {flags: {moderator: true}});
if (this.joinMuted) Object.assign(opts, {flags: {mute: true}});
try {
const {memberId, confUuid} = await this.ep.join(this.confName, opts);
this.logger.debug({memberId, confUuid}, `Conference:_joinConference: successfully joined ${this.confName}`);
this.memberId = parseInt(memberId, 10);
this.memberId = memberId;
this.confUuid = confUuid;
// set a tag for this member, if provided
if (this.data.memberTag) {
this.setMemberTag(this.data.memberTag);
}
cs.setConferenceDetails(memberId, this.confName, confUuid);
const response = await this.ep.api('conference', [this.confName, 'get', 'count']);
if (response.body && /\d+/.test(response.body)) this.participantCount = parseInt(response.body);
@@ -402,9 +375,6 @@ class Conference extends Task {
.catch((err) => {});
}
if (this.speakOnlyTo) {
this.setCoachMode(this.speakOnlyTo);
}
} catch (err) {
this.logger.error(err, `Failed to join conference ${this.confName}`);
throw err;
@@ -414,25 +384,6 @@ class Conference extends Task {
this.ep.api('conference', `${this.confName} set max_members ${this.maxParticipants}`)
.catch((err) => this.logger.error(err, `Error setting max participants to ${this.maxParticipants}`));
}
if (typeof this.endConferenceDuration === 'number' && this.endConferenceDuration >= 0) {
this.ep.api('conference', `${this.confName} set endconference_grace_time ${this.endConferenceDuration}`)
.catch((err) => this.logger.error(err, `Error setting end conference time to ${this.endConferenceDuration}`));
}
}
_initSipIndialogRequestListener(cs, dlg) {
dlg.on('info', this._onRequestWithinDialog.bind(this, cs));
dlg.on('message', this._onRequestWithinDialog.bind(this, cs));
}
_removeSipIndialogRequestListener(dlg) {
dlg && dlg.removeAllListeners('message');
dlg && dlg.removeAllListeners('info');
}
_onRequestWithinDialog(cs, req, res) {
cs._onRequestWithinDialog(req, res);
}
/**
@@ -463,15 +414,7 @@ class Conference extends Task {
}
}
doConferenceMute(cs, opts) {
assert (cs.isInConference);
const mute = opts.conf_mute_status === 'mute';
this.ep.api(`conference ${this.confName} ${mute ? 'mute' : 'unmute'} ${this.memberId}`)
.catch((err) => this.logger.info({err}, 'Error muting or unmuting participant'));
}
doConferenceHold(cs, opts) {
async doConferenceHold(cs, opts) {
assert (cs.isInConference);
const {conf_hold_status, wait_hook} = opts;
@@ -508,46 +451,6 @@ class Conference extends Task {
}
}
async doConferenceParticipantAction(cs, opts) {
const {action, tag, wait_hook } = opts;
switch (action) {
case 'tag':
await this.setMemberTag(tag);
break;
case 'untag':
await this.clearMemberTag();
break;
case 'coach':
await this.setCoachMode(tag);
break;
case 'uncoach':
await this.clearCoachMode();
break;
case 'hold':
this.doConferenceHold(cs, {
conf_hold_status: 'hold',
...(wait_hook && {wait_hook})
});
break;
case 'unhold':
this.doConferenceHold(cs, {conf_hold_status: 'unhold'});
break;
case 'mute':
this.doConferenceMute(cs, {conf_mute_status: 'mute'});
break;
case 'unmute':
this.doConferenceMute(cs, {conf_mute_status: 'unmute'});
break;
case 'kick':
this.kickMember(cs);
break;
default:
this.logger.info(`Conference:doConferenceParticipantAction - unhandled action ${action}`);
break;
}
}
async _doWaitHookWhileOnHold(cs, dlg, wait_hook) {
do {
try {
@@ -565,13 +468,6 @@ class Conference extends Task {
} while (!this.killed && this.conf_hold_status === 'hold');
}
/**
* mute or unmute side of the call
*/
mute(callSid, doMute) {
this.doConferenceMute(this.callSession, {conf_mute_status: doMute});
}
/**
* Add ourselves to the waitlist of sessions to be notified once
* the conference starts
@@ -601,7 +497,7 @@ class Conference extends Task {
_normalizeHook(cs, hook) {
if (typeof hook === 'object') return hook;
const url = hook.startsWith('/') ?
`${cs.application.requestor instanceof HttpRequestor ? cs.application.requestor.baseUrl : ''}${hook}` :
`${cs.application.requestor.baseUrl}${hook}` :
hook;
return { url } ;
@@ -620,7 +516,7 @@ class Conference extends Task {
const response = await this.ep.api('conference', [this.confName, 'get', 'count']);
if (response.body && confNoMatch(response.body)) this.participantCount = 0;
else if (response.body && /^\d+$/.test(response.body)) this.participantCount = parseInt(response.body) - 1;
this.logger.debug(`Conference:_doFinalMemberCheck conference count ${this.participantCount}`);
this.logger.debug({response}, `Conference:_doFinalMemberCheck conference count ${this.participantCount}`);
} catch (err) {
this.logger.info({err}, 'Conference:_doFinalMemberCheck error retrieving count (we were probably kicked');
}
@@ -630,7 +526,7 @@ class Conference extends Task {
* when we hang up as the last member, the current member count = 1
* when we are kicked out of the call when the moderator leaves, the member count = 0
*/
if (this.participantCount === 0 || this.endConferenceOnExit) {
if (this.participantCount === 0) {
const {deleteKey} = cs.srf.locals.dbHelpers;
try {
this._notifyConferenceEvent(cs, 'end');
@@ -638,8 +534,7 @@ class Conference extends Task {
this.logger.info(`conf ${this.confName} deprovisioned: ${removed ? 'success' : 'failure'}`);
}
catch (err) {
this.logger.error(err, `Error deprovisioning conference ${this.confName},
might be the conference already cleaned by another moderator`);
this.logger.error(err, `Error deprovisioning conference ${this.confName}`);
}
}
}
@@ -672,8 +567,7 @@ class Conference extends Task {
memberId: this.memberId,
confName: this.confName,
tasks,
rootSpan: cs.rootSpan,
req: cs.req
rootSpan: cs.rootSpan
});
await this._playSession.exec();
this._playSession = null;
@@ -717,24 +611,8 @@ class Conference extends Task {
if (!params.time) params.time = (new Date()).toISOString();
if (!params.members && typeof this.participantCount === 'number') params.members = this.participantCount;
cs.application.requestor
.request(
'verb:hook',
this.statusHook,
Object.assign(
params,
Object.assign(
{
conferenceSid: this.confName,
friendlyName: this.friendlyName,
},
cs.callInfo.toJSON()
),
httpHeaders
)
)
.catch((err) =>
this.logger.info(err, 'Conference:notifyConferenceEvent - error')
);
.request('verb:hook', this.statusHook, Object.assign(params, this.statusParams, httpHeaders))
.catch((err) => this.logger.info(err, 'Conference:notifyConferenceEvent - error'));
}
}
@@ -750,19 +628,11 @@ class Conference extends Task {
}
// conference event handlers
_onAddMember(logger, cs, evt) {
const memberId = parseInt(evt.getHeader('Member-ID')) ;
if (this.speakOnlyTo) {
logger.debug(`Conference:_onAddMember - member ${memberId} added to ${this.confName}, updating coaching mode`);
this.setCoachMode(this.speakOnlyTo).catch(() => {});
}
else logger.debug(`Conference:_onAddMember - member ${memberId} added to conference ${this.confName}`);
}
_onDelMember(logger, cs, evt) {
const memberId = parseInt(evt.getHeader('Member-ID')) ;
this.participantCount = parseInt(evt.getHeader('Conference-Size'));
if (memberId === this.memberId) {
logger.info(`Conference:_onDelMember - I was dropped from conference ${this.confName}, task is complete`);
this.logger.info(`Conference:_onDelMember - I was dropped from conference ${this.confName}, task is complete`);
this.replaceEndpointAndEnd(cs);
}
}
@@ -791,99 +661,6 @@ class Conference extends Task {
}
}
_onTag(logger, cs, evt) {
const memberId = parseInt(evt.getHeader('Member-ID')) ;
const tag = evt.getHeader('Tag') || '';
if (memberId !== this.memberId && this.speakOnlyTo) {
logger.info(`Conference:_onTag - member ${memberId} set tag to '${tag }'; updating coach mode accordingly`);
this.setCoachMode(this.speakOnlyTo).catch(() => {});
}
}
/**
* Set the conference to "coaching" mode, where the audio of the participant is only heard
* by a subset of the participants in the conference.
* We do this by first getting all of the members who do *not* have this tag, and then
* we configure this members audio to not be sent to them.
* @param {string} speakOnlyTo - tag of the members who should receive our audio
*
* N.B.: this feature requires jambonz patches to freeswitch mod_conference
*/
async setCoachMode(speakOnlyTo) {
this.speakOnlyTo = speakOnlyTo;
if (!this.memberId) {
this.logger.info('Conference:_setCoachMode: no member id yet');
return;
}
try {
const members = (await this.ep.getNonMatchingConfParticipants(this.confName, speakOnlyTo))
.filter((m) => m !== this.memberId);
if (members.length === 0) {
this.logger.info({members}, 'Conference:_setCoachMode: all participants have the tag, so all will hear me');
if (this.coaching.length) {
await this.ep.api('conference', [this.confName, 'relate', this.memberId, this.coaching.join(','), 'clear']);
this.coaching = [];
}
}
else {
const memberList = members.join(',');
this.logger.info(`Conference:_setCoachMode: my audio will NOT be sent to ${memberList}`);
await this.ep.api('conference', [this.confName, 'relate', this.memberId, memberList, 'nospeak']);
this.coaching = members;
}
} catch (err) {
this.logger.error({err, speakOnlyTo}, '_setCoachMode: Error');
}
}
async clearCoachMode() {
if (!this.memberId) return;
try {
if (this.coaching.length === 0) {
this.logger.info('Conference:_clearCoachMode: no coaching mode to clear');
}
else {
const memberList = this.coaching.join(',');
this.logger.info(`Conference:_clearCoachMode: now sending my audio to all, including ${memberList}`);
await this.ep.api('conference', [this.confName, 'relate', this.memberId, memberList, 'clear']);
}
this.speakOnlyTo = null;
this.coaching = [];
} catch (err) {
this.logger.error({err}, '_clearCoachMode: Error');
}
}
async setMemberTag(tag) {
try {
await this.ep.api('conference', [this.confName, 'tag', this.memberId, tag]);
this.logger.info(`Conference:setMemberTag: set tag for ${this.memberId} to ${tag}`);
this.memberTag = tag;
} catch (err) {
this.logger.error({err}, `Error setting tag for ${this.memberId} to ${tag}`);
}
}
async clearMemberTag() {
try {
await this.ep.api('conference', [this.confName, 'tag', this.memberId]);
this.logger.info(`Conference:setMemberTag: clearing tag for ${this.memberId}`);
this.memberTag = null;
} catch (err) {
this.logger.error({err}, `Error clearing tag for ${this.memberId}`);
}
}
async kickMember(cs) {
assert(cs.isInConference);
try {
await this.ep.api('conference', [this.confName, 'kick', this.memberId]);
this.logger.info(`Conference:kickMember: kick ${this.memberId} out of conference ${this.confName}`);
} catch (err) {
this.logger.error({err}, `Error kicking member out of conference for ${this.memberId}`);
}
}
}
module.exports = Conference;

View File

@@ -1,23 +1,15 @@
const Task = require('./task');
const {TaskName, TaskPreconditions} = require('../utils/constants');
const parseDecibels = require('../utils/parse-decibels');
class TaskConfig extends Task {
constructor(logger, opts) {
super(logger, opts);
[
'synthesizer',
'recognizer',
'bargeIn',
'record',
'listen',
'transcribe',
'fillerNoise',
'actionHookDelayAction',
'boostAudioSignal',
'vad',
'ttsStream'
'listen'
].forEach((k) => this[k] = this.data[k] || {});
if ('notifyEvents' in this.data) {
@@ -35,40 +27,13 @@ class TaskConfig extends Task {
'finishOnKey', 'input', 'numDigits', 'minDigits', 'maxDigits',
'interDigitTimeout', 'bargein', 'dtmfBargein', 'minBargeinWordCount', 'actionHook'
].forEach((k) => {
const val = this.bargeIn[k];
if (val !== undefined && val !== null) this.gatherOpts[k] = val;
if (this.bargeIn[k]) this.gatherOpts[k] = this.bargeIn[k];
});
}
if (this.transcribe?.enable) {
this.transcribeOpts = {
verb: 'transcribe',
...this.transcribe
};
delete this.transcribeOpts.enable;
}
if (this.ttsStream.enable) {
this.sayOpts = {
verb: 'say',
stream: true
};
}
if (this.data.reset) {
if (typeof this.data.reset === 'string') this.data.reset = [this.data.reset];
}
else this.data.reset = [];
if (this.bargeIn.sticky) this.autoEnable = true;
this.preconditions = (this.bargeIn.enable ||
this.record?.action ||
this.listen?.url ||
this.data.amd ||
'boostAudioSignal' in this.data ||
this.transcribe?.enable) ?
this.preconditions = (this.bargeIn.enable || this.record?.action || this.listen?.url || this.data.amd) ?
TaskPreconditions.Endpoint :
TaskPreconditions.None;
this.onHoldMusic = this.data.onHoldMusic;
}
get name() { return TaskName.Config; }
@@ -77,47 +42,27 @@ class TaskConfig extends Task {
get hasRecognizer() { return Object.keys(this.recognizer).length; }
get hasRecording() { return Object.keys(this.record).length; }
get hasListen() { return Object.keys(this.listen).length; }
get hasTranscribe() { return Object.keys(this.transcribe).length; }
get hasDub() { return Object.keys(this.dub).length; }
get hasVad() { return Object.keys(this.vad).length; }
get hasFillerNoise() { return Object.keys(this.fillerNoise).length; }
get hasReferHook() { return Object.keys(this.data).includes('referHook'); }
get hasTtsStream() { return Object.keys(this.ttsStream).length; }
get summary() {
const phrase = [];
/* reset recognizer and/or synthesizer to default values? */
if (this.data.reset.length) phrase.push(`reset ${this.data.reset.join(',')}`);
if (this.bargeIn.enable) phrase.push('enable barge-in');
if (this.hasSynthesizer) {
const {vendor:v, language:l, voice, label} = this.synthesizer;
const s = `{${v},${l},${voice},${label || 'None'}}`;
const {vendor:v, language:l, voice} = this.synthesizer;
const s = `{${v},${l},${voice}}`;
phrase.push(`set synthesizer${s}`);
}
if (this.hasRecognizer) {
const {vendor:v, language:l, label} = this.recognizer;
const s = `{${v},${l},${label || 'None'}}`;
const {vendor:v, language:l} = this.recognizer;
const s = `{${v},${l}}`;
phrase.push(`set recognizer${s}`);
}
if (this.hasRecording) phrase.push(this.record.action);
if (this.hasListen) {
phrase.push(this.listen.enable ? `listen ${this.listen.url}` : 'stop listen');
}
if (this.hasTranscribe) {
phrase.push(this.transcribe.enable ? `transcribe ${this.transcribe.transcriptionHook}` : 'stop transcribe');
}
if (this.hasFillerNoise) phrase.push(`fillerNoise ${this.fillerNoise.enable ? 'on' : 'off'}`);
if (this.data.amd) phrase.push('enable amd');
if (this.notifyEvents) phrase.push(`event notification ${this.notifyEvents ? 'on' : 'off'}`);
if (this.onHoldMusic) phrase.push(`onHoldMusic: ${this.onHoldMusic}`);
if ('boostAudioSignal' in this.data) phrase.push(`setGain ${this.data.boostAudioSignal}`);
if (this.hasReferHook) phrase.push('set referHook');
if (this.hasTtsStream) {
phrase.push(`${this.ttsStream.enable ? 'enable' : 'disable'} ttsStream`);
}
return `${this.name}{${phrase.join(',')}}`;
return `${this.name}{${phrase.join(',')}`;
}
async exec(cs, {ep} = {}) {
@@ -128,10 +73,6 @@ class TaskConfig extends Task {
cs.notifyEvents = !!this.data.notifyEvents;
}
if (this.onHoldMusic) {
cs.onHoldMusic = this.onHoldMusic;
}
if (this.data.amd) {
this.startAmd = cs.startAmd;
this.stopAmd = cs.stopAmd;
@@ -145,65 +86,25 @@ class TaskConfig extends Task {
}
}
this.data.reset.forEach((k) => {
if (k === 'synthesizer') cs.resetSynthesizer();
else if (k === 'recognizer') cs.resetRecognizer();
});
if (this.hasSynthesizer) {
cs.synthesizer = this.synthesizer;
cs.speechSynthesisVendor = this.synthesizer.vendor !== 'default'
? this.synthesizer.vendor
: cs.speechSynthesisVendor;
cs.speechSynthesisLabel = this.synthesizer.label === 'default'
? cs.speechSynthesisLabel : this.synthesizer.label;
cs.speechSynthesisLanguage = this.synthesizer.language !== 'default'
? this.synthesizer.language
: cs.speechSynthesisLanguage;
cs.speechSynthesisVoice = this.synthesizer.voice !== 'default'
? this.synthesizer.voice
: cs.speechSynthesisVoice;
// fallback vendor
cs.fallbackSpeechSynthesisVendor = this.synthesizer.fallbackVendor !== 'default'
? this.synthesizer.fallbackVendor
: cs.fallbackSpeechSynthesisVendor;
cs.fallbackSpeechSynthesisLabel = this.synthesizer.fallbackLabel === 'default'
? cs.fallbackSpeechSynthesisLabel : this.synthesizer.fallbackLabel;
cs.fallbackSpeechSynthesisLanguage = this.synthesizer.fallbackLanguage !== 'default'
? this.synthesizer.fallbackLanguage
: cs.fallbackSpeechSynthesisLanguage;
cs.fallbackSpeechSynthesisVoice = this.synthesizer.fallbackVoice !== 'default'
? this.synthesizer.fallbackVoice
: cs.fallbackSpeechSynthesisVoice;
// new vendor is set, reset fallback vendor
cs.hasFallbackTts = false;
this.logger.info({synthesizer: this.synthesizer}, 'Config: updated synthesizer');
}
if (this.hasRecognizer) {
cs.recognizer = this.recognizer;
cs.speechRecognizerVendor = this.recognizer.vendor !== 'default'
? this.recognizer.vendor
: cs.speechRecognizerVendor;
cs.speechRecognizerLabel = this.recognizer.label === 'default'
? cs.speechRecognizerLabel : this.recognizer.label;
cs.speechRecognizerLanguage = this.recognizer.language !== undefined && this.recognizer.language !== 'default'
cs.speechRecognizerLanguage = this.recognizer.language !== 'default'
? this.recognizer.language
: cs.speechRecognizerLanguage;
//fallback
cs.fallbackSpeechRecognizerVendor = this.recognizer.fallbackVendor !== undefined &&
this.recognizer.fallbackVendor !== 'default'
? this.recognizer.fallbackVendor
: cs.fallbackSpeechRecognizerVendor;
cs.fallbackSpeechRecognizerLabel = this.recognizer.fallbackLabel === 'default' ?
cs.fallbackSpeechRecognizerLabel :
this.recognizer.fallbackLabel;
cs.fallbackSpeechRecognizerLanguage = this.recognizer.fallbackLanguage !== undefined &&
this.recognizer.fallbackLanguage !== 'default'
? this.recognizer.fallbackLanguage
: cs.fallbackSpeechRecognizerLanguage;
cs.isContinuousAsr = typeof this.recognizer.asrTimeout === 'number' ? true : false;
if (cs.isContinuousAsr) {
cs.asrTimeout = this.recognizer.asrTimeout;
@@ -223,8 +124,6 @@ class TaskConfig extends Task {
if ('punctuation' in this.recognizer) {
cs.globalSttPunctuation = this.recognizer.punctuation;
}
// new vendor is set, reset fallback vendor
cs.hasFallbackAsr = false;
this.logger.info({
recognizer: this.recognizer,
isContinuousAsr: cs.isContinuousAsr
@@ -257,83 +156,12 @@ class TaskConfig extends Task {
const {enable, ...opts} = this.listen;
if (enable) {
this.logger.debug({opts}, 'Config: enabling listen');
cs.startBackgroundTask('listen', {verb: 'listen', ...opts});
cs.startBackgroundListen({verb: 'listen', ...opts});
} else {
this.logger.info('Config: disabling listen');
cs.stopBackgroundTask('listen');
cs.stopBackgroundListen();
}
}
if (this.hasTranscribe) {
if (this.transcribe.enable) {
if (!this.transcribeOpts.recognizer) {
this.transcribeOpts.recognizer = this.hasRecognizer ?
this.recognizer :
{
vendor: cs.speechRecognizerVendor,
language: cs.speechRecognizerLanguage
};
}
this.logger.debug(this.transcribeOpts, 'Config: enabling transcribe');
cs.startBackgroundTask('transcribe', this.transcribeOpts);
} else {
this.logger.info('Config: disabling transcribe');
cs.stopBackgroundTask('transcribe');
}
}
if (Object.keys(this.actionHookDelayAction).length !== 0) {
cs.actionHookDelayProperties = this.actionHookDelayAction;
}
if (this.data.sipRequestWithinDialogHook) {
cs.sipRequestWithinDialogHook = this.data.sipRequestWithinDialogHook;
}
if ('boostAudioSignal' in this.data) {
const db = parseDecibels(this.data.boostAudioSignal);
this.logger.info(`Config: boosting audio signal by ${db} dB`);
const args = [ep.uuid, 'setGain', db];
ep.api('uuid_dub', args).catch((err) => {
this.logger.error(err, 'Error boosting audio signal');
});
}
if (this.hasFillerNoise) {
const {enable, ...opts} = this.fillerNoise;
this.logger.info({fillerNoise: this.fillerNoise}, 'Config: fillerNoise');
if (!enable) cs.disableFillerNoise();
else {
cs.enableFillerNoise(opts);
}
}
if (this.hasVad) {
cs.vad = {
enable: this.vad.enable || false,
voiceMs: this.vad.voiceMs || 250,
silenceMs: this.vad.silenceMs || 150,
strategy: this.vad.strategy || 'one-shot',
mode: (this.vad.mode !== undefined && this.vad.mode !== null) ? this.vad.mode : 2
};
}
if (this.hasReferHook) {
cs.referHook = this.data.referHook;
}
if (this.ttsStream.enable && this.sayOpts) {
this.sayOpts.synthesizer = this.hasSynthesizer ? this.synthesizer : {
vendor: cs.speechSynthesisVendor,
language: cs.speechSynthesisLanguage,
voice: cs.speechSynthesisVoice,
...(cs.speechSynthesisLabel && {
label: cs.speechSynthesisLabel
})
};
this.logger.info({opts: this.gatherOpts}, 'Config: enabling ttsStream');
cs.enableBackgroundTtsStream(this.sayOpts);
} else if (!this.ttsStream.enable) {
this.logger.info('Config: disabling ttsStream');
cs.disableTtsStream();
}
}
async kill(cs) {

View File

@@ -3,7 +3,8 @@ const {TaskName, TaskPreconditions, DequeueResults, BONG_TONE} = require('../uti
const Emitter = require('events');
const bent = require('bent');
const assert = require('assert');
const { sleepFor } = require('../utils/helpers');
const sleepFor = (ms) => new Promise((resolve) => setTimeout(() => resolve(), ms));
const getUrl = (cs) => `${cs.srf.locals.serviceUrl}/v1/dequeue/${cs.callSid}`;
@@ -15,7 +16,6 @@ class TaskDequeue extends Task {
this.queueName = this.data.name;
this.timeout = this.data.timeout || 0;
this.beep = this.data.beep === true;
this.callSid = this.data.callSid;
this.emitter = new Emitter();
this.state = DequeueResults.Timeout;
@@ -53,7 +53,7 @@ class TaskDequeue extends Task {
}
_getMemberFromQueue(cs) {
const {retrieveFromSortedSet, retrieveByPatternSortedSet} = cs.srf.locals.dbHelpers;
const {popFront} = cs.srf.locals.dbHelpers;
return new Promise(async(resolve) => {
let timer;
@@ -70,13 +70,7 @@ class TaskDequeue extends Task {
do {
try {
let url;
if (this.callSid) {
const r = await retrieveByPatternSortedSet(this.queueName, `*${this.callSid}`);
url = r[0];
} else {
url = await retrieveFromSortedSet(this.queueName);
}
const url = await popFront(this.queueName);
if (url) {
found = true;
clearTimeout(timer);
@@ -84,7 +78,7 @@ class TaskDequeue extends Task {
resolve(url);
}
} catch (err) {
this.logger.debug({err}, 'TaskDequeue:_getMemberFromQueue error Sorted Set');
this.logger.debug({err}, 'TaskDequeue:_getMemberFromQueue error popFront');
}
await sleepFor(5000);
} while (!this.killed && !timedout && !found);

View File

@@ -6,25 +6,15 @@ const {
TaskName,
TaskPreconditions,
MAX_SIMRINGS,
MediaPath,
KillReason
} = require('../utils/constants');
const assert = require('assert');
const placeCall = require('../utils/place-outdial');
const sessionTracker = require('../session/session-tracker');
const DtmfCollector = require('../utils/dtmf-collector');
const ConfirmCallSession = require('../session/confirm-call-session');
const dbUtils = require('../utils/db-utils');
const parseDecibels = require('../utils/parse-decibels');
const debug = require('debug')('jambonz:feature-server');
const {parseUri} = require('drachtio-srf');
const {ANCHOR_MEDIA_ALWAYS,
JAMBONZ_DISABLE_DIAL_PAI_HEADER,
JAMBONES_DIAL_SBC_FOR_REGISTERED_USER} = require('../config');
const { isOnhold, isOpusFirst } = require('../utils/sdp-utils');
const { normalizeJambones } = require('@jambonz/verb-specifications');
const { selectHostPort } = require('../utils/network');
const { sleepFor } = require('../utils/helpers');
function parseDtmfOptions(logger, dtmfCapture) {
let parentDtmfCollector, childDtmfCollector;
@@ -94,7 +84,6 @@ class TaskDial extends Task {
this.earlyMedia = this.data.answerOnBridge === true;
this.callerId = this.data.callerId;
this.callerName = this.data.callerName;
this.dialMusic = this.data.dialMusic;
this.headers = this.data.headers || {};
this.method = this.data.method || 'POST';
@@ -106,9 +95,6 @@ class TaskDial extends Task {
this.referHook = this.data.referHook;
this.dtmfHook = this.data.dtmfHook;
this.proxy = this.data.proxy;
this.tag = this.data.tag;
this.boostAudioSignal = this.data.boostAudioSignal;
this._mediaPath = MediaPath.FullMedia;
if (this.dtmfHook) {
const {parentDtmfCollector, childDtmfCollector} = parseDtmfOptions(logger, this.data.dtmfCapture || {});
@@ -120,16 +106,12 @@ class TaskDial extends Task {
}
}
const listenData = this.data.listen || this.data.stream;
if (listenData) {
this.listenTask = makeTask(logger, {'listen': listenData }, this);
if (this.data.listen) {
this.listenTask = makeTask(logger, {'listen': this.data.listen}, this);
}
if (this.data.transcribe) {
this.transcribeTask = makeTask(logger, {'transcribe' : this.data.transcribe}, this);
}
if (this.data.dub && Array.isArray(this.data.dub) && this.data.dub.length > 0) {
this.dubTasks = this.data.dub.map((d) => makeTask(logger, {'dub': d}, this));
}
this.results = {};
this.bridged = false;
@@ -151,25 +133,11 @@ class TaskDial extends Task {
get name() { return TaskName.Dial; }
get isOnHoldEnabled() {
return !!this.data.onHoldHook;
}
get canReleaseMedia() {
const keepAnchor = this.data.anchorMedia ||
this.cs.isBackGroundListen ||
this.cs.onHoldMusic ||
ANCHOR_MEDIA_ALWAYS ||
this.listenTask ||
this.dubTasks ||
this.transcribeTask ||
this.startAmd;
return !keepAnchor;
}
get shouldExitMediaPathEntirely() {
return this.data.exitMediaPath;
return !process.env.ANCHOR_MEDIA_ALWAYS &&
!this.listenTask &&
!this.transcribeTask &&
!this.startAmd;
}
get summary() {
@@ -192,27 +160,7 @@ class TaskDial extends Task {
async exec(cs) {
await super.exec(cs);
if (this.data.anchorMedia && this.data.exitMediaPath) {
this.logger.info('Dial:exec - incompatible anchorMedia and exitMediaPath are both set, will obey anchorMedia');
delete this.data.exitMediaPath;
}
if (!this.canReleaseMedia && this.data.exitMediaPath) {
this.logger.info(
'Dial:exec - exitMediaPath is set so features such as transcribe and record will not work on this call');
}
try {
if (this.listenTask) {
const {span, ctx} = this.startChildSpan(`nested:${this.listenTask.summary}`);
this.listenTask.span = span;
this.listenTask.ctx = ctx;
}
if (this.transcribeTask) {
const {span, ctx} = this.startChildSpan(`nested:${this.transcribeTask.summary}`);
this.transcribeTask.span = span;
this.transcribeTask.ctx = ctx;
}
if (this.data.amd) {
this.startAmd = cs.startAmd;
this.stopAmd = cs.stopAmd;
@@ -224,16 +172,7 @@ class TaskDial extends Task {
else {
this.epOther = cs.ep;
if (this.dialMusic && this.epOther && this.epOther.connected) {
(async() => {
do {
try {
await this.epOther.play(this.dialMusic);
} catch (err) {
this.logger.error(err, `TaskDial:exec error playing dialMusic ${this.dialMusic}`);
await sleepFor(1000);
}
} while (!this.killed && !this.bridged && this._mediaPath === MediaPath.FullMedia);
})();
this.epOther.play(this.dialMusic).catch((err) => {});
}
}
if (!this.killed) await this._attemptCalls(cs);
@@ -242,7 +181,6 @@ class TaskDial extends Task {
await this.performAction(this.results, this.killReason !== KillReason.Replaced);
this._removeDtmfDetection(cs.dlg);
this._removeDtmfDetection(this.dlg);
this._removeSipIndialogRequestListener(this.dlg);
} catch (err) {
this.logger.error({err}, 'TaskDial:exec terminating with error');
this.kill(cs);
@@ -271,23 +209,19 @@ class TaskDial extends Task {
}
this._removeDtmfDetection(cs.dlg);
this._removeDtmfDetection(this.dlg);
await this._killOutdials();
this._killOutdials();
if (this.sd) {
const byeReasonHeader = this.killReason === KillReason.MediaTimeout ? 'Media Timeout' : undefined;
this.sd.kill(byeReasonHeader);
this.sd.ep?.removeListener('destroy', this._handleMediaTimeout.bind(this));
this.sd.kill();
this.sd.removeAllListeners();
this.sd = null;
}
if (this.callSid) sessionTracker.remove(this.callSid);
if (this.listenTask) {
await this.listenTask.kill(cs);
this.listenTask.span.end();
this.listenTask = null;
}
if (this.transcribeTask) {
await this.transcribeTask.kill(cs);
this.transcribeTask.span.end();
this.transcribeTask = null;
}
this.notifyTaskDone();
@@ -321,7 +255,7 @@ class TaskDial extends Task {
if (!cs.callGone && this.epOther) {
/* if we can release the media back to the SBC, do so now */
if (this.canReleaseMedia) this._releaseMedia(cs, this.sd, this.shouldExitMediaPathEntirely);
if (this.canReleaseMedia) this._releaseMedia(cs, this.sd);
else this.epOther.bridge(this.ep);
}
} catch (err) {
@@ -361,58 +295,22 @@ class TaskDial extends Task {
const to = parseUri(req.getParsedHeader('Refer-To').uri);
const by = parseUri(req.getParsedHeader('Referred-By').uri);
const referredBy = req.get('Referred-By');
const userAgent = req.get('User-Agent');
const customHeaders = Object.keys(req.headers)
.filter((h) => h.toLowerCase().startsWith('x-'))
.reduce((acc, h) => {
acc[h] = req.get(h);
return acc;
}, {});
this.logger.info({to}, 'refer to parsed');
const json = await cs.requestor.request('verb:hook', this.referHook, {
...(callInfo.toJSON()),
await cs.requestor.request('verb:hook', this.referHook, {
...callInfo,
refer_details: {
sip_refer_to: req.get('Refer-To'),
refer_to_user: to.scheme === 'tel' ? to.number : to.user,
...(referredBy && {sip_referred_by: referredBy}),
...(userAgent && {sip_user_agent: userAgent}),
...(by && {referred_by_user: by.scheme === 'tel' ? by.number : by.user}),
sip_referred_by: req.get('Referred-By'),
sip_user_agent: req.get('User-Agent'),
refer_to_user: to.user,
referred_by_user: by.user,
referring_call_sid,
referred_call_sid,
...customHeaders
referred_call_sid
}
}, httpHeaders);
if (json && Array.isArray(json)) {
try {
const logger = isChild ? this.logger : this.sd.logger;
const tasks = normalizeJambones(logger, json).map((tdata) => makeTask(this.logger, tdata));
if (tasks && tasks.length > 0) {
const legs = isChild ? ['child', 'parent'] : ['parent', 'child'];
logger.info(`Dial:handleRefer received REFER on ${legs[0]} leg, setting new app on ${legs[1]} leg`);
if (isChild) this.redirect(cs, tasks);
else {
logger.info({tasks: json}, 'Dial:handleRefer - new application for for child leg');
const adultingSession = await this.sd.doAdulting({
logger,
application: cs.application,
tasks
});
/* need to update the callSid of the child with its own (new) AdultingCallSession */
sessionTracker.add(adultingSession.callSid, adultingSession);
}
}
} catch (err) {
this.logger.info(err, 'Dial:handleRefer - error setting new application after receiving REFER');
}
}
//caller and callee legs are briged together, accept refer with 202 will release callee leg endpoint
//that makes freeswitch release endpoint for caller leg.
if (this.ep) this.ep.unbridge();
res.send(202);
this.logger.info('DialTask:handleRefer - sent 202 Accepted');
} catch (err) {
this.logger.info({err}, 'DialTask:handleRefer - error processing incoming REFER');
res.send(err.statusCode || 501);
}
}
@@ -430,16 +328,11 @@ class TaskDial extends Task {
sd.removeAllListeners('callCreateFail');
}
async _killOutdials() {
_killOutdials() {
for (const [callSid, sd] of Array.from(this.dials)) {
this.logger.debug(`Dial:_killOutdials killing callSid ${callSid}`);
try {
await sd.kill();
} catch (err) {
this.logger.info(err, `Dial:_killOutdials Error killing ${callSid}`);
}
sd.kill().catch((err) => this.logger.info(err, `Dial:_killOutdials Error killing ${callSid}`));
this._removeHandlers(sd);
this.logger.debug(`Dial:_killOutdials killed callSid ${callSid}`);
}
this.dials.clear();
}
@@ -452,14 +345,8 @@ class TaskDial extends Task {
}
_onInfo(cs, dlg, req, res) {
// SIP Indialog will be handled by another handler
if (cs.sipRequestWithinDialogHook) {
return;
}
res.send(200);
if (req.get('Content-Type') !== 'application/dtmf-relay') {
return;
}
if (req.get('Content-Type') !== 'application/dtmf-relay') return;
const dtmfDetector = dlg === cs.dlg ? this.parentDtmfCollector : this.childDtmfCollector;
if (!dtmfDetector) return;
@@ -488,20 +375,6 @@ class TaskDial extends Task {
}
}
_initSipIndialogRequestListener(cs, dlg) {
dlg.on('info', this._onRequestWithinDialog.bind(this, cs));
dlg.on('message', this._onRequestWithinDialog.bind(this, cs));
}
_removeSipIndialogRequestListener(dlg) {
dlg && dlg.removeAllListeners('message');
dlg && dlg.removeAllListeners('info');
}
_onRequestWithinDialog(cs, req, res) {
cs._onRequestWithinDialog(req, res);
}
async _initializeInbound(cs) {
const {ep} = await cs._evalEndpointPrecondition(this);
this.epOther = ep;
@@ -518,11 +391,11 @@ class TaskDial extends Task {
}
async _attemptCalls(cs) {
const {req, callInfo, direction, srf} = cs;
const {req, srf} = cs;
const {getSBC} = srf.locals;
const {lookupTeamsByAccount, lookupAccountBySid} = srf.locals.dbHelpers;
const {lookupCarrier, lookupCarrierByPhoneNumber, lookupVoipCarrierBySid} = dbUtils(this.logger, cs.srf);
let sbcAddress = this.proxy || getSBC();
const {lookupCarrier} = dbUtils(this.logger, cs.srf);
const sbcAddress = this.proxy || getSBC();
const teamsInfo = {};
let fqdn;
@@ -530,28 +403,15 @@ class TaskDial extends Task {
this.headers = {
'X-Account-Sid': cs.accountSid,
...(req && req.has('X-CID') && {'X-CID': req.get('X-CID')}),
...(direction === 'outbound' && callInfo.sbcCallid && {'X-CID': callInfo.sbcCallid}),
...(!JAMBONZ_DISABLE_DIAL_PAI_HEADER && req && {
...(req.has('P-Asserted-Identity') && {'P-Asserted-Identity': req.get('P-Asserted-Identity')}),
...(req.has('Privacy') && {'Privacy': req.get('Privacy')}),
}),
...(req && req.has('X-Voip-Carrier-Sid') && {'X-Voip-Carrier-Sid': req.get('X-Voip-Carrier-Sid')}),
...(req && req.has('P-Asserted-Identity') && {'P-Asserted-Identity': req.get('P-Asserted-Identity')}),
// Put headers at the end to make sure opt.headers override all default behavior.
...this.headers
};
// default to inband dtmf if not specified
this.inbandDtmfEnabled = cs.inbandDtmfEnabled;
// get calling user from From header
const parsedFrom = req.getParsedHeader('from');
const fromUri = parseUri(parsedFrom.uri);
const opts = {
headers: this.headers,
proxy: `sip:${sbcAddress}`,
callingNumber: this.callerId || fromUri.user,
...(this.callerName && {callingName: this.callerName}),
opusFirst: isOpusFirst(this.cs.ep.remote.sdp),
isVideoCall: this.cs.ep.remote.sdp.includes('m=video')
callingNumber: this.callerId || req.callingNumber
};
const t = this.target.find((t) => t.type === 'teams');
@@ -562,14 +422,10 @@ class TaskDial extends Task {
}
const ms = await cs.getMS();
this.timerRing = setTimeout(async() => {
this.timerRing = setTimeout(() => {
this.logger.info(`Dial:_attemptCall: ring no answer timer ${this.timeout}s exceeded`);
this.timerRing = null;
try {
await this._killOutdials();
} catch (err) {
this.logger.info(err, 'Dial:_attemptCall - error killing outdials');
}
this._killOutdials();
this.result = {
dialCallStatus: CallStatus.NoAnswer,
dialSipStatus: 487
@@ -595,45 +451,14 @@ class TaskDial extends Task {
this.logger.error({err}, 'Error looking up account by sid');
}
}
// find handling sbc sip for called user
if (JAMBONES_DIAL_SBC_FOR_REGISTERED_USER && t.type === 'user') {
const { registrar } = srf.locals.dbHelpers;
const reg = await registrar.query(t.name);
if (reg) {
sbcAddress = selectHostPort(this.logger, reg.sbcAddress, 'tcp')[1];
}
//sbc outbound return 404 Notfound to handle case called user is not reigstered.
}
if (t.type === 'phone' && t.trunk) {
const voip_carrier_sid = await lookupCarrier(cs.accountSid, t.trunk);
this.logger.info(`Dial:_attemptCalls: selected ${voip_carrier_sid} for requested carrier: ${t.trunk}`);
this.logger.info(`Dial:_attemptCalls: selected ${voip_carrier_sid} for requested carrier: ${t.trunk})`);
if (voip_carrier_sid) {
opts.headers['X-Requested-Carrier-Sid'] = voip_carrier_sid;
}
}
/**
* trunk isn't specified,
* check if number matches any existing numbers
* */
if (t.type === 'phone' && !t.trunk) {
const str = this.callerId || req.callingNumber || '';
const callingNumber = str.startsWith('+') ? str.substring(1) : str;
const voip_carrier_sid = await lookupCarrierByPhoneNumber(cs.accountSid, callingNumber);
const req_voip_carrier_sid = req.has('X-Voip-Carrier-Sid') ? req.get('X-Voip-Carrier-Sid') : null;
if (voip_carrier_sid) {
this.logger.info(
`Dial:_attemptCalls: selected voip_carrier_sid ${voip_carrier_sid} for callingNumber: ${callingNumber}`);
opts.headers['X-Requested-Carrier-Sid'] = voip_carrier_sid;
// Checking if outbound carrier is different from inbound carrier and has dtmf type tones
if (voip_carrier_sid !== req_voip_carrier_sid) {
const [voipCarrier] = await lookupVoipCarrierBySid(voip_carrier_sid);
this.inbandDtmfEnabled = voipCarrier?.dtmf_type === 'tones';
}
}
}
if (this.killed) return;
const sd = placeCall({
@@ -647,9 +472,7 @@ class TaskDial extends Task {
callInfo: cs.callInfo,
accountInfo: cs.accountInfo,
rootSpan: cs.rootSpan,
startSpan: this.startSpan.bind(this),
dialTask: this,
onHoldMusic: this.cs.onHoldMusic
startSpan: this.startSpan.bind(this)
});
this.dials.set(sd.callSid, sd);
@@ -665,13 +488,11 @@ class TaskDial extends Task {
}
})
.on('callStatusChange', (obj) => {
if (this.results.dialCallStatus !== CallStatus.Completed &&
this.results.dialCallStatus !== CallStatus.NoAnswer) {
if (this.results.dialCallStatus !== CallStatus.Completed) {
Object.assign(this.results, {
dialCallStatus: obj.callStatus,
dialSipStatus: obj.sipStatus,
dialCallSid: sd.callSid,
dialSbcCallid: sd.callInfo.sbcCallid
});
}
switch (obj.callStatus) {
@@ -707,8 +528,6 @@ class TaskDial extends Task {
await this._connectSingleDial(cs, sd);
} catch (err) {
this.logger.info({err}, 'Dial:_attemptCalls - Error calling _connectSingleDial ');
sd.removeAllListeners();
this.kill(cs);
}
})
.on('decline', () => {
@@ -722,7 +541,11 @@ class TaskDial extends Task {
}
})
.on('reinvite', (req, res) => {
this._onReinvite(req, res);
try {
cs.handleReinviteAfterMediaReleased(req, res);
} catch (err) {
this.logger.error(err, 'Error in dial einvite from B leg');
}
})
.on('refer', (callInfo, req, res) => {
@@ -758,56 +581,6 @@ class TaskDial extends Task {
this._killOutdials(); // NB: order is important
}
async _onReinvite(req, res) {
try {
let isHandled = false;
if (this.isOnHoldEnabled) {
if (isOnhold(req.body)) {
this.logger.debug('Dial: _onReinvite receive hold Request');
if (!this.epOther && !this.ep) {
this.logger.debug(`Dial: _onReinvite receive hold Request,
media already released, reconnect media server`);
// update caller leg for new SDP from callee.
await this.cs.handleReinviteAfterMediaReleased(req, res);
// Freeswitch media is released, reconnect
await this.reAnchorMedia(this.cs, this.sd);
this.isOutgoingLegHold = true;
} else {
this.logger.debug('Dial: _onReinvite receive hold Request, update SDP');
const newSdp = await this.ep.modify(req.body);
res.send(200, {body: newSdp});
}
isHandled = true;
// Media already connected, ask for onHoldHook
this._onHoldHook(req);
} else if (!isOnhold(req.body)) {
this.logger.debug('Dial: _onReinvite receive unhold Request');
if (this.epOther && this.ep && this.isOutgoingLegHold && this.canReleaseMedia) {
this.logger.debug('Dial: _onReinvite receive unhold Request, release media');
// Offhold, time to release media
const newSdp = await this.ep.modify(req.body);
await res.send(200, {body: newSdp});
await this._releaseMedia(this.cs, this.sd, this.shouldExitMediaPathEntirely);
this.isOutgoingLegHold = false;
} else {
this.logger.debug('Dial: _onReinvite receive unhold Request, update media server');
const newSdp = await this.ep.modify(req.body);
res.send(200, {body: newSdp});
}
if (this._onHoldSession) {
this._onHoldSession.kill();
}
isHandled = true;
}
}
if (!isHandled) {
this.cs.handleReinviteAfterMediaReleased(req, res);
}
} catch (err) {
this.logger.error(err, 'Error in dial einvite from B leg');
}
}
_onMaxCallDuration(cs) {
this.logger.info(`Dial:_onMaxCallDuration tearing down call as it has reached ${this.timeLimit}s`);
this.ep && this.ep.unbridge();
@@ -858,27 +631,11 @@ class TaskDial extends Task {
dialCallSid: sd.callSid,
});
if (this.dubTasks) {
for (const dub of this.dubTasks) {
try {
await dub.exec(cs, {ep: sd.ep});
}
catch (err) {
this.logger.error({err}, 'Dial:_selectSingleDial - error executing dubTask');
}
}
}
if (this.parentDtmfCollector) this._installDtmfDetection(cs, cs.dlg);
if (this.childDtmfCollector) this._installDtmfDetection(cs, this.dlg);
if (cs.sipRequestWithinDialogHook) {
/* remove any existing listener to escape from duplicating events */
this._removeSipIndialogRequestListener(this.dlg);
this._initSipIndialogRequestListener(cs, this.dlg);
}
if (this.transcribeTask) this.transcribeTask.exec(cs, {ep: this.epOther, ep2:this.ep});
if (this.listenTask) this.listenTask.exec(cs, {ep: this.listenTask.channel === 2 ? this.ep : this.epOther});
if (this.transcribeTask) this.transcribeTask.exec(cs, {ep2: this.epOther, ep:this.ep});
if (this.listenTask) this.listenTask.exec(cs, {ep: this.epOther});
if (this.startAmd) {
try {
this.startAmd(cs, this.ep, this, this.data.amd);
@@ -887,30 +644,8 @@ class TaskDial extends Task {
}
}
/* boost audio signal if requested */
if (this.boostAudioSignal) {
try {
const db = parseDecibels(this.boostAudioSignal);
this.logger.info(`Dial: boosting audio signal by ${db} dB`);
const args = [this.ep.uuid, 'setGain', db];
await this.ep.api('uuid_dub', args);
} catch (err) {
this.logger.info({err}, 'Dial:_selectSingleDial - Error boosting audio signal');
}
}
/* if we can release the media back to the SBC, do so now */
if (this.canReleaseMedia || this.shouldExitMediaPathEntirely) {
setTimeout(this._releaseMedia.bind(this, cs, sd, this.shouldExitMediaPathEntirely), 200);
}
this.sd.ep.once('destroy', this._handleMediaTimeout.bind(this));
}
_handleMediaTimeout(evt) {
if (evt?.reason === 'MEDIA_TIMEOUT' && this.sd && this.bridged) {
this.kill(this.cs, KillReason.MediaTimeout);
}
if (this.canReleaseMedia) setTimeout(this._releaseMedia.bind(this, cs, sd), 200);
}
_bridgeEarlyMedia(sd) {
@@ -922,57 +657,20 @@ class TaskDial extends Task {
}
}
/* public api */
async updateMediaPath(desiredPath) {
this.logger.info(`Dial:updateMediaPath - ${this._mediaPath} => ${desiredPath}`);
switch (desiredPath) {
case MediaPath.NoMedia:
assert(this._mediaPath !== MediaPath.NoMedia, 'updateMediaPath: already no-media');
await this._releaseMedia(this.cs, this.sd, true);
break;
case MediaPath.PartialMedia:
assert(this._mediaPath !== MediaPath.PartialMedia, 'updateMediaPath: already partial-media');
if (this._mediaPath === MediaPath.FullMedia) {
await this._releaseMedia(this.cs, this.sd, false);
}
else {
// to go from no-media to partial-media we need to go through full-media first
await this.reAnchorMedia(this.cs, this.sd);
await this._releaseMedia(this.cs, this.sd, false);
}
assert(!this.epOther, 'updateMediaPath: epOther should be null');
assert(!this.ep, 'updateMediaPath: ep should be null');
break;
case MediaPath.FullMedia:
assert(this._mediaPath !== MediaPath.FullMedia, 'updateMediaPath: already full-media');
await this.reAnchorMedia(this.cs, this.sd);
break;
default:
assert(false, `updateMediaPath: invalid path request ${desiredPath}`);
}
}
/**
* Release the media from freeswitch
* @param {*} cs
* @param {*} sd
*/
async _releaseMedia(cs, sd, releaseEntirely = false) {
async _releaseMedia(cs, sd) {
assert(cs.ep && sd.ep);
try {
// Wait until we got new SDP from B leg to ofter to A Leg
const aLegSdp = cs.ep.remote.sdp;
await sd.releaseMediaToSBC(aLegSdp, cs.ep.local.sdp, releaseEntirely);
const bLegSdp = sd.dlg.remote.sdp;
await cs.releaseMediaToSBC(bLegSdp, releaseEntirely);
await Promise.all[sd.releaseMediaToSBC(aLegSdp, cs.ep.local.sdp), cs.releaseMediaToSBC(bLegSdp)];
this.epOther = null;
this._mediaPath = releaseEntirely ? MediaPath.NoMedia : MediaPath.PartialMedia;
this.logger.info(
`Dial:_releaseMedia - successfully released media from freewitch, media path is now ${this._mediaPath}`);
this.logger.info('Dial:_releaseMedia - successfully released media from freewitch');
} catch (err) {
this.logger.info({err}, 'Dial:_releaseMedia error');
}
@@ -982,52 +680,14 @@ class TaskDial extends Task {
if (cs.ep && sd.ep) return;
this.logger.info('Dial:reAnchorMedia - re-anchoring media to freewitch');
await Promise.all([sd.reAnchorMedia(this._mediaPath), cs.reAnchorMedia(this._mediaPath)]);
await Promise.all([sd.reAnchorMedia(), cs.reAnchorMedia()]);
this.epOther = cs.ep;
this.epOther.bridge(this.ep);
this._mediaPath = MediaPath.FullMedia;
this.logger.info(
`Dial:_releaseMedia - successfully re-anchored media to freewitch, media path is now ${this._mediaPath}`);
}
// Handle RE-INVITE hold from caller leg.
async handleReinviteAfterMediaReleased(req, res) {
let isHandled = false;
if (this.isOnHoldEnabled) {
if (isOnhold(req.body)) {
if (!this.epOther && !this.ep) {
// update callee leg for new SDP from caller.
const sdp = await this.dlg.modify(req.body);
res.send(200, {body: sdp});
// Onhold but media is already released, reconnect
await this.reAnchorMedia(this.cs, this.sd);
isHandled = true;
this.isIncomingLegHold = true;
}
this._onHoldHook(req);
} else if (!isOnhold(req.body)) {
if (this.epOther && this.ep && this.isIncomingLegHold &&
(this.canReleaseMedia || this.shouldExitMediaPathEntirely)) {
// Offhold, time to release media
const newSdp = await this.epOther.modify(req.body);
await res.send(200, {body: newSdp});
await this._releaseMedia(this.cs, this.sd, this.shouldExitMediaPathEntirely);
isHandled = true;
}
this.isIncomingLegHold = false;
if (this._onHoldSession) {
this._onHoldSession.kill();
}
}
}
if (!isHandled) {
const sdp = await this.dlg.modify(req.body);
this.logger.info({sdp}, 'Dial:handleReinviteAfterMediaReleased - sent reinvite to B leg');
res.send(200, {body: sdp});
}
const sdp = await this.dlg.modify(req.body);
this.logger.info({sdp}, 'Dial:handleReinviteAfterMediaReleased - sent reinvite to B leg');
res.send(200, {body: sdp});
}
_onAmdEvent(cs, evt) {
@@ -1038,55 +698,6 @@ class TaskDial extends Task {
this.logger.error({err}, 'Dial:_onAmdEvent - error calling actionHook');
});
}
async _onHoldHook(req, allowed = [TaskName.Play, TaskName.Say, TaskName.Pause]) {
if (this.data.onHoldHook) {
// send silence for keep Voice quality
await this.epOther.play('silence_stream://500');
let allowedTasks;
do {
try {
const b3 = this.getTracingPropagation();
const httpHeaders = b3 && {b3};
const json = await this.cs.application.requestor.
request('verb:hook', this.data.onHoldHook, {
...this.cs.callInfo.toJSON(),
hold_detail: {
from: req.get('From'),
to: req.get('To')
}
}, httpHeaders);
const tasks = normalizeJambones(this.logger, json).map((tdata) => makeTask(this.logger, tdata));
allowedTasks = tasks.filter((t) => allowed.includes(t.name));
if (tasks.length !== allowedTasks.length) {
this.logger.debug({tasks, allowedTasks}, 'unsupported task');
throw new Error(`unsupported verb in enqueue waitHook: only ${JSON.stringify(allowed)}`);
}
this.logger.debug(`DialTask:_onHoldHook: executing ${tasks.length} tasks`);
if (tasks.length) {
this._onHoldSession = new ConfirmCallSession({
logger: this.logger,
application: this.cs.application,
dlg: this.isIncomingLegHold ? this.dlg : this.cs.dlg,
ep: this.isIncomingLegHold ? this.ep : this.cs.ep,
callInfo: this.cs.callInfo,
accountInfo: this.cs.accountInfo,
tasks,
rootSpan: this.cs.rootSpan,
req: this.cs.req
});
await this._onHoldSession.exec();
this._onHoldSession = null;
}
} catch (error) {
this.logger.info(error, 'DialTask:_onHoldHook: failed retrieving waitHook');
this._onHoldSession = null;
break;
}
} while (allowedTasks && allowedTasks.length > 0 && !this.killed && this.isOnHold);
this.logger.info('Finish onHoldHook');
}
}
}
module.exports = TaskDial;

View File

@@ -58,13 +58,6 @@ class Dialogflow extends Task {
this.vendor = this.data.tts.vendor || 'default';
this.language = this.data.tts.language || 'default';
this.voice = this.data.tts.voice || 'default';
this.speechSynthesisLabel = this.data.tts.label;
// fallback tts
this.fallbackVendor = this.data.tts.fallbackVendor || 'default';
this.fallbackLanguage = this.data.tts.fallbackLanguage || 'default';
this.fallbackVoice = this.data.tts.fallbackLanguage || 'default';
this.fallbackLabel = this.data.tts.fallbackLabel;
}
this.bargein = this.data.bargein;
}
@@ -125,15 +118,8 @@ class Dialogflow extends Task {
this.vendor = cs.speechSynthesisVendor;
this.language = cs.speechSynthesisLanguage;
this.voice = cs.speechSynthesisVoice;
this.speechSynthesisLabel = cs.speechSynthesisLabel;
}
if (this.fallbackVendor === 'default') {
this.fallbackVendor = cs.fallbackSpeechSynthesisVendor;
this.fallbackLanguage = cs.fallbackSpeechSynthesisLanguage;
this.fallbackVoice = cs.fallbackSpeechSynthesisVoice;
this.fallbackLabel = cs.fallbackSpeechSynthesisLabel;
}
this.ttsCredentials = cs.getSpeechCredentials(this.vendor, 'tts', this.speechSynthesisLabel);
this.ttsCredentials = cs.getSpeechCredentials(this.vendor, 'tts');
this.ep.addCustomEventListener('dialogflow::intent', this._onIntent.bind(this, ep, cs));
this.ep.addCustomEventListener('dialogflow::transcription', this._onTranscription.bind(this, ep, cs));
@@ -235,8 +221,18 @@ class Dialogflow extends Task {
}
try {
const {filePath} = await this._fallbackSynthAudio(cs, intent, stats, synthAudio);
const obj = {
text: intent.fulfillmentText,
vendor: this.vendor,
language: this.language,
voice: this.voice,
salt: cs.callSid,
credentials: this.ttsCredentials
};
this.logger.debug({obj}, 'Dialogflow:_onIntent - playing message via tts');
const {filePath, servedFromCache} = await synthAudio(stats, obj);
if (filePath) cs.trackTmpFile(filePath);
if (!this.ttsCredentials && !servedFromCache) cs.billForTts(intent.fulfillmentText.length);
if (this.playInProgress) {
await ep.api('uuid_break', ep.uuid).catch((err) => this.logger.info(err, 'Error killing audio'));
@@ -280,46 +276,6 @@ class Dialogflow extends Task {
}
}
async _fallbackSynthAudio(cs, intent, stats, synthAudio) {
try {
const obj = {
account_sid: cs.accountSid,
text: intent.fulfillmentText,
vendor: this.vendor,
language: this.language,
voice: this.voice,
salt: cs.callSid,
credentials: this.ttsCredentials
};
this.logger.debug({obj}, 'Dialogflow:_onIntent - playing message via tts');
return await synthAudio(stats, obj);
} catch (error) {
this.logger.info({error}, 'Failed to synthesize audio from primary vendor');
try {
if (this.fallbackVendor) {
const credentials = cs.getSpeechCredentials(this.fallbackVendor, 'tts', this.fallbackLabel);
const obj = {
account_sid: cs.accountSid,
text: intent.fulfillmentText,
vendor: this.fallbackVendor,
language: this.fallbackLanguage,
voice: this.fallbackVoice,
salt: cs.callSid,
credentials
};
this.logger.debug({obj}, 'Dialogflow:_onIntent - playing message via fallback tts');
return await synthAudio(stats, obj);
}
} catch (err) {
this.logger.info({err}, 'Failed to synthesize audio from falllback vendor');
throw err;
}
throw error;
}
}
/**
* A transcription - either interim or final - has been returned.
* If we are doing barge-in based on hotword detection, check for the hotword or phrase.

View File

@@ -1,144 +0,0 @@
const {TaskName} = require('../utils/constants');
const TtsTask = require('./tts-task');
const assert = require('assert');
const parseDecibels = require('../utils/parse-decibels');
/**
* Dub task: add or remove additional audio tracks into the call
*/
class TaskDub extends TtsTask {
constructor(logger, opts, parentTask) {
super(logger, opts, parentTask);
this.logger.debug({opts: this.data}, 'TaskDub constructor');
['action', 'track', 'play', 'say', 'loop'].forEach((prop) => {
this[prop] = this.data[prop];
});
this.gain = parseDecibels(this.data.gain);
assert.ok(this.action, 'TaskDub: action is required');
assert.ok(this.track, 'TaskDub: track is required');
}
get name() { return TaskName.Dub; }
async exec(cs, {ep}) {
super.exec(cs);
try {
switch (this.action) {
case 'addTrack':
await this._addTrack(cs, ep);
break;
case 'removeTrack':
await this._removeTrack(cs, ep);
break;
case 'silenceTrack':
await this._silenceTrack(cs, ep);
break;
case 'playOnTrack':
await this._playOnTrack(cs, ep);
break;
case 'sayOnTrack':
await this._sayOnTrack(cs, ep);
break;
default:
throw new Error(`TaskDub: unsupported action ${this.action}`);
}
} catch (err) {
this.logger.error(err, 'Error executing dub task');
}
}
async _addTrack(cs, ep) {
this.logger.info(`adding track: ${this.track}`);
await ep.dub({
action: 'addTrack',
track: this.track
});
if (this.play) await this._playOnTrack(cs, ep);
else if (this.say) await this._sayOnTrack(cs, ep);
}
async _removeTrack(_cs, ep) {
this.logger.info(`removing track: ${this.track}`);
await ep.dub({
action: 'removeTrack',
track: this.track
});
}
async _silenceTrack(_cs, ep) {
this.logger.info(`silencing track: ${this.track}`);
await ep.dub({
action: 'silenceTrack',
track: this.track
});
}
async _playOnTrack(_cs, ep) {
this.logger.info(`playing on track: ${this.track}`);
await ep.dub({
action: 'playOnTrack',
track: this.track,
play: this.play,
loop: this.loop ? 'loop' : 'once',
gain: this.gain
});
}
async _sayOnTrack(cs, ep) {
const text = this.say.text || this.say;
this.synthesizer = this.say.synthesizer || {};
if (Object.keys(this.synthesizer).length) {
this.logger.info({synthesizer: this.synthesizer},
`saying on track ${this.track}: ${text} with synthesizer options`);
}
else {
this.logger.info(`saying on track ${this.track}: ${text}`);
}
this.synthesizer = this.synthesizer || {};
this.text = [text];
const vendor = this.synthesizer.vendor && this.synthesizer.vendor !== 'default' ?
this.synthesizer.vendor :
cs.speechSynthesisVendor;
const language = this.synthesizer.language && this.synthesizer.language !== 'default' ?
this.synthesizer.language :
cs.speechSynthesisLanguage ;
const voice = this.synthesizer.voice && this.synthesizer.voice !== 'default' ?
this.synthesizer.voice :
cs.speechSynthesisVoice;
const label = this.synthesizer.label && this.synthesizer.label !== 'default' ?
this.synthesizer.label :
cs.speechSynthesisLabel;
const disableTtsStreaming = false;
const filepath = await this._synthesizeWithSpecificVendor(cs, ep, {
vendor, language, voice, label, disableTtsStreaming
});
assert.ok(filepath.length === 1, 'TaskDub: no filepath returned from synthesizer');
const path = filepath[0];
if (!path.startsWith('say:{')) {
/* we have a local file of mp3 or r8 of synthesized speech audio to play */
this.logger.info(`playing synthesized speech from file on track ${this.track}: ${path}`);
this.play = path;
await this._playOnTrack(cs, ep);
}
else {
this.logger.info(`doing actual text to speech file on track ${this.track}: ${path}`);
await ep.dub({
action: 'sayOnTrack',
track: this.track,
say: path,
gain: this.gain
});
}
}
}
module.exports = TaskDub;

View File

@@ -18,7 +18,6 @@ class TaskEnqueue extends Task {
this.preconditions = TaskPreconditions.Endpoint;
this.queueName = this.data.name;
this.priority = this.data.priority;
this.waitHook = this.data.waitHook;
this.emitter = new Emitter();
@@ -71,22 +70,12 @@ class TaskEnqueue extends Task {
}
async _addToQueue(cs, dlg) {
const {addToSortedSet, sortedSetLength} = cs.srf.locals.dbHelpers;
const {pushBack} = cs.srf.locals.dbHelpers;
const url = getUrl(cs);
this.waitStartTime = Date.now();
this.logger.debug({queue: this.queueName, url}, 'pushing url onto queue');
if (this.priority < 0) {
this.logger.warn(`priority ${this.priority} is invalid, need to be non-negative integer,
999 will be used for priority`);
}
let members = await addToSortedSet(this.queueName, url, this.priority);
if (members === 1) {
this.logger.info('TaskEnqueue:_addToQueue: added to queue');
} else {
this.logger.info('TaskEnqueue:_addToQueue: failed to add to queue');
}
members = await sortedSetLength(this.queueName);
const members = await pushBack(this.queueName, url);
this.logger.info(`TaskEnqueue:_addToQueue: added to queue, length now ${members}`);
this.notifyUrl = url;
/* invoke account-level webhook for queue event notifications */
@@ -101,9 +90,9 @@ class TaskEnqueue extends Task {
}
async _removeFromQueue(cs) {
const {retrieveByPatternSortedSet, sortedSetLength} = cs.srf.locals.dbHelpers;
await retrieveByPatternSortedSet(this.queueName, `*${getUrl(cs)}`);
return await sortedSetLength(this.queueName);
const {removeFromList, lengthOfList} = cs.srf.locals.dbHelpers;
await removeFromList(this.queueName, getUrl(cs));
return await lengthOfList(this.queueName);
}
async performAction() {
@@ -290,13 +279,13 @@ class TaskEnqueue extends Task {
this.emitter.emit('dequeue', opts);
try {
const {sortedSetLength} = cs.srf.locals.dbHelpers;
const members = await sortedSetLength(this.queueName);
const {lengthOfList} = cs.srf.locals.dbHelpers;
const members = await lengthOfList(this.queueName);
this.dequeued = true;
cs.performQueueWebhook({
event: 'leave',
queue: this.data.name,
length: Math.max(members, 0),
length: Math.max(members - 1, 0),
leaveReason: 'dequeued',
leaveTime: Date.now(),
dequeuer: opts.dequeuer
@@ -311,9 +300,8 @@ class TaskEnqueue extends Task {
}
}
async _playHook(cs, dlg, hook,
allowed = [TaskName.Play, TaskName.Say, TaskName.Pause, TaskName.Leave, TaskName.Tag]) {
const {sortedSetLength, sortedSetPositionByPattern} = cs.srf.locals.dbHelpers;
async _playHook(cs, dlg, hook, allowed = [TaskName.Play, TaskName.Say, TaskName.Pause, TaskName.Leave]) {
const {lengthOfList, getListPosition} = cs.srf.locals.dbHelpers;
const b3 = this.getTracingPropagation();
const httpHeaders = b3 && {b3};
@@ -325,20 +313,13 @@ class TaskEnqueue extends Task {
queueTime: getElapsedTime(this.waitStartTime)
};
try {
const queueSize = await sortedSetLength(this.queueName);
const queuePosition = await sortedSetPositionByPattern(this.queueName, `*${this.notifyUrl}`);
Object.assign(params, {
queueSize,
queuePosition: queuePosition.length ? queuePosition[0] : 0,
callSid: this.cs.callSid,
callId: this.cs.callId,
customerData: this.cs.callInfo.customerData
});
const queueSize = await lengthOfList(this.queueName);
const queuePosition = await getListPosition(this.queueName, this.notifyUrl);
Object.assign(params, {queueSize, queuePosition});
} catch (err) {
this.logger.error({err}, `TaskEnqueue:_playHook error retrieving list info for queue ${this.queueName}`);
}
const json = await cs.application.requestor.request('verb:hook', hook, params, httpHeaders);
this.logger.debug({json}, 'TaskEnqueue:_playHook: received response from waitHook');
const tasks = normalizeJambones(this.logger, json).map((tdata) => makeTask(this.logger, tdata));
const allowedTasks = tasks.filter((t) => allowed.includes(t.name));
@@ -369,8 +350,7 @@ class TaskEnqueue extends Task {
callInfo: cs.callInfo,
accountInfo: cs.accountInfo,
tasks: tasksToRun,
rootSpan: cs.rootSpan,
req: cs.req
rootSpan: cs.rootSpan
});
await this._playSession.exec();
this._playSession = null;

File diff suppressed because it is too large Load Diff

View File

@@ -25,13 +25,6 @@ class Lex extends Task {
this.vendor = this.data.tts.vendor || 'default';
this.language = this.data.tts.language || 'default';
this.voice = this.data.tts.voice || 'default';
this.speechCredentialLabel = this.data.tts.label || 'default';
// fallback tts
this.fallbackVendor = this.data.tts.fallbackVendor || 'default';
this.fallbackLanguage = this.data.tts.fallbackLanguage || 'default';
this.fallbackVoice = this.data.tts.fallbackLanguage || 'default';
this.fallbackLabel = this.data.tts.fallbackLabel || 'default';
}
this.botName = `${this.bot}:${this.alias}:${this.region}`;
@@ -109,16 +102,8 @@ class Lex extends Task {
this.vendor = cs.speechSynthesisVendor;
this.language = cs.speechSynthesisLanguage;
this.voice = cs.speechSynthesisVoice;
this.speechCredentialLabel = cs.speechSynthesisLabel;
}
if (this.fallbackVendor === 'default') {
this.fallbackVendor = cs.fallbackSpeechSynthesisVendor;
this.fallbackLanguage = cs.fallbackSpeechSynthesisLanguage;
this.fallbackVoice = cs.fallbackSpeechSynthesisVoice;
this.fallbackLabel = cs.fallbackSpeechSynthesisLabel;
}
this.ttsCredentials = cs.getSpeechCredentials(this.vendor, 'tts', this.speechCredentialLabel);
this.ttsCredentials = cs.getSpeechCredentials(this.vendor, 'tts');
this.ep.addCustomEventListener('lex::intent', this._onIntent.bind(this, ep, cs));
this.ep.addCustomEventListener('lex::transcription', this._onTranscription.bind(this, ep, cs));
@@ -183,41 +168,6 @@ class Lex extends Task {
}
}
async _fallbackSynthAudio(cs, msg, stats, synthAudio) {
try {
const {filePath} = await synthAudio(stats, {
account_sid: cs.accountSid,
text: msg,
vendor: this.vendor,
language: this.language,
voice: this.voice,
salt: cs.callSid,
credentials: this.ttsCredentials
});
return filePath;
} catch (error) {
this.logger.info({error}, 'failed to synth audio from primary vendor');
if (this.fallbackVendor) {
try {
const credential = cs.getSpeechCredentials(this.fallbackVendor, 'tts', this.fallbackLabel);
const {filePath} = await synthAudio(stats, {
account_sid: cs.accountSid,
text: msg,
vendor: this.fallbackVendor,
language: this.fallbackLanguage,
voice: this.fallbackVoice,
salt: cs.callSid,
credentials: credential
});
return filePath;
} catch (err) {
this.logger.info({err}, 'failed to synth audio from fallback vendor');
}
}
}
}
/**
* @param {*} evt - event data
*/
@@ -237,7 +187,15 @@ class Lex extends Task {
try {
this.logger.debug(`tts with ${this.vendor} ${this.voice}`);
const filePath = await this._fallbackSynthAudio(cs, msg, stats, synthAudio);
// eslint-disable-next-line no-unused-vars
const {filePath, servedFromCache} = await synthAudio(stats, {
text: msg,
vendor: this.vendor,
language: this.language,
voice: this.voice,
salt: cs.callSid,
credentials: this.ttsCredentials
});
if (filePath) cs.trackTmpFile(filePath);
if (this.events.includes('start-play')) {

View File

@@ -3,21 +3,15 @@ const {TaskName, TaskPreconditions, ListenEvents, ListenStatus} = require('../ut
const makeTask = require('./make_task');
const moment = require('moment');
const MAX_PLAY_AUDIO_QUEUE_SIZE = 10;
const DTMF_SPAN_NAME = 'dtmf';
class TaskListen extends Task {
constructor(logger, opts, parentTask) {
super(logger, opts);
/**
* @deprecated
* use bidirectionalAudio.enabled
*/
this.disableBidirectionalAudio = opts.disableBidirectionalAudio;
this.preconditions = TaskPreconditions.Endpoint;
[
'action', 'auth', 'method', 'url', 'finishOnKey', 'maxLength', 'metadata', 'mixType', 'passDtmf', 'playBeep',
'sampleRate', 'timeout', 'transcribe', 'wsAuth', 'disableBidirectionalAudio', 'channel'
'sampleRate', 'timeout', 'transcribe', 'wsAuth', 'disableBidirectionalAudio'
].forEach((k) => this[k] = this.data[k]);
this.mixType = this.mixType || 'mono';
@@ -29,25 +23,12 @@ class TaskListen extends Task {
this.results = {};
this.playAudioQueue = [];
this.isPlayingAudioFromQueue = false;
this.bidirectionalAudio = {
enabled: this.disableBidirectionalAudio === true ? false : true,
...(this.data['bidirectionalAudio']),
};
// From drachtio-version 3.0.40, forkAudioStart will send empty bugname, metadata together with
// bidirectionalAudio params that cause old version of freeswitch missunderstand between bugname and
// bidirectionalAudio params
this._bugname = 'audio_fork';
if (this.transcribe) this.transcribeTask = makeTask(logger, {'transcribe': opts.transcribe}, this);
}
get name() { return TaskName.Listen; }
set bugname(name) { this._bugname = name; }
set ignoreCustomerData(val) { this._ignoreCustomerData = val; }
async exec(cs, {ep}) {
await super.exec(cs);
this.ep = ep;
@@ -84,8 +65,7 @@ class TaskListen extends Task {
if (this.ep && this.ep.connected) {
this.logger.debug('TaskListen:kill closing websocket');
try {
const args = this._bugname ? [this._bugname] : [];
await this.ep.forkAudioStop(...args);
await this.ep.forkAudioStop();
this.logger.debug('TaskListen:kill successfully closed websocket');
} catch (err) {
this.logger.info(err, 'TaskListen:kill');
@@ -105,16 +85,13 @@ class TaskListen extends Task {
async updateListen(status) {
if (!this.killed && this.ep && this.ep.connected) {
const args = this._bugname ? [this._bugname] : [];
this.logger.info(`TaskListen:updateListen status ${status}`);
switch (status) {
case ListenStatus.Pause:
await this.ep.forkAudioPause(...args)
.catch((err) => this.logger.info(err, 'TaskListen: error pausing audio'));
await this.ep.forkAudioPause().catch((err) => this.logger.info(err, 'TaskListen: error pausing audio'));
break;
case ListenStatus.Resume:
await this.ep.forkAudioResume(...args)
.catch((err) => this.logger.info(err, 'TaskListen: error resuming audio'));
await this.ep.forkAudioResume().catch((err) => this.logger.info(err, 'TaskListen: error resuming audio'));
break;
}
}
@@ -127,15 +104,13 @@ class TaskListen extends Task {
async _startListening(cs, ep) {
this._initListeners(ep);
const ci = this.nested ? this.parentTask.sd.callInfo : cs.callInfo.toJSON();
if (this._ignoreCustomerData) {
delete ci.customerData;
}
const metadata = Object.assign(
{sampleRate: this.sampleRate, mixType: this.mixType},
ci,
this.nested ? this.parentTask.sd.callInfo : cs.callInfo.toJSON(),
this.metadata);
if (this.hook.auth) {
this.logger.debug({username: this.hook.auth.username, password: this.hook.auth.password},
'TaskListen:_startListening basic auth');
await this.ep.set({
'MOD_AUDIO_BASIC_AUTH_USERNAME': this.hook.auth.username,
'MOD_AUDIO_BASIC_AUTH_PASSWORD': this.hook.auth.password
@@ -145,9 +120,7 @@ class TaskListen extends Task {
wsUrl: this.hook.url,
mixType: this.mixType,
sampling: this.sampleRate,
...(this._bugname && {bugname: this._bugname}),
metadata,
bidirectionalAudio: this.bidirectionalAudio || {}
metadata
});
this.recordStartTime = moment();
if (this.maxLength) {
@@ -167,7 +140,7 @@ class TaskListen extends Task {
}
/* support bi-directional audio */
if (this.bidirectionalAudio.enabled) {
if (!this.disableBiDirectionalAudio) {
ep.addCustomEventListener(ListenEvents.PlayAudio, this._onPlayAudio.bind(this, ep));
}
ep.addCustomEventListener(ListenEvents.KillAudio, this._onKillAudio.bind(this, ep));
@@ -188,25 +161,12 @@ class TaskListen extends Task {
}
_onDtmf(ep, evt) {
const {dtmf, duration} = evt;
this.logger.debug({evt}, `TaskListen:_onDtmf received dtmf ${dtmf}`);
this.logger.debug({evt}, `TaskListen:_onDtmf received dtmf ${evt.dtmf}`);
if (this.passDtmf && this.ep?.connected) {
const obj = {event: 'dtmf', dtmf, duration};
const args = this._bugname ? [this._bugname, obj] : [obj];
this.ep.forkAudioSendText(...args)
const obj = {event: 'dtmf', dtmf: evt.dtmf, duration: evt.duration};
this.ep.forkAudioSendText(obj)
.catch((err) => this.logger.info({err}, 'TaskListen:_onDtmf error sending dtmf'));
}
/* add a child span for the dtmf event */
const msDuration = Math.floor((duration / 8000) * 1000);
const {span} = this.startChildSpan(`${DTMF_SPAN_NAME}:${dtmf}`);
span.setAttributes({
channel: 1,
dtmf,
duration: `${msDuration}ms`
});
span.end();
if (evt.dtmf === this.finishOnKey) {
this.logger.info(`TaskListen:_onDtmf terminating task due to dtmf ${evt.dtmf}`);
this.results.digits = evt.dtmf;
@@ -221,7 +181,7 @@ class TaskListen extends Task {
}
}
_onConnect(ep) {
this.logger.info('TaskListen:_onConnect');
this.logger.debug('TaskListen:_onConnect');
}
_onConnectFailure(ep, evt) {
this.logger.info(evt, 'TaskListen:_onConnectFailure');
@@ -232,15 +192,7 @@ class TaskListen extends Task {
try {
const results = await ep.play(evt.file);
logger.debug(`Finished playing file, result: ${JSON.stringify(results)}`);
const obj = {
type: 'playDone',
data: {
id: evt.id,
...results
}
};
const args = this._bugname ? [this._bugname, obj] : [obj];
ep.forkAudioSendText(...args);
ep.forkAudioSendText({type: 'playDone', data: Object.assign({id: evt.id}, results)});
} catch (err) {
logger.error({err}, 'Error playing file');
}

View File

@@ -1,144 +0,0 @@
const Task = require('../task');
const {TaskPreconditions} = require('../../utils/constants');
const TaskLlmOpenAI_S2S = require('./llms/openai_s2s');
const TaskLlmVoiceAgent_S2S = require('./llms/voice_agent_s2s');
const TaskLlmUltravox_S2S = require('./llms/ultravox_s2s');
const TaskLlmElevenlabs_S2S = require('./llms/elevenlabs_s2s');
const TaskLlmGoogle_S2S = require('./llms/google_s2s');
const LlmMcpService = require('../../utils/llm-mcp');
class TaskLlm extends Task {
constructor(logger, opts) {
super(logger, opts);
this.preconditions = TaskPreconditions.Endpoint;
['vendor', 'model', 'auth', 'connectOptions'].forEach((prop) => {
this[prop] = this.data[prop];
});
this.eventHandlers = [];
// delegate to the specific llm model
this.llm = this.createSpecificLlm();
// MCP
this.mcpServers = this.data.mcpServers || [];
}
get name() { return this.llm.name ; }
get toolHook() { return this.llm?.toolHook; }
get eventHook() { return this.llm?.eventHook; }
get ep() { return this.cs.ep; }
get mcpService() {
return this.llmMcpService;
}
get isMcpEnabled() {
return this.mcpServers.length > 0;
}
async exec(cs, {ep}) {
await super.exec(cs, {ep});
// create the MCP service if we have MCP servers
if (this.isMcpEnabled) {
this.llmMcpService = new LlmMcpService(this.logger, this.mcpServers);
await this.llmMcpService.init();
}
await this.llm.exec(cs, {ep});
}
async kill(cs) {
super.kill(cs);
await this.llm.kill(cs);
// clean up MCP clients
if (this.isMcpEnabled) {
await this.mcpService.close();
}
}
createSpecificLlm() {
let llm;
switch (this.vendor) {
case 'openai':
case 'microsoft':
llm = new TaskLlmOpenAI_S2S(this.logger, this.data, this);
break;
case 'voiceagent':
case 'deepgram':
llm = new TaskLlmVoiceAgent_S2S(this.logger, this.data, this);
break;
case 'ultravox':
llm = new TaskLlmUltravox_S2S(this.logger, this.data, this);
break;
case 'elevenlabs':
llm = new TaskLlmElevenlabs_S2S(this.logger, this.data, this);
break;
case 'google':
llm = new TaskLlmGoogle_S2S(this.logger, this.data, this);
break;
default:
throw new Error(`Unsupported vendor ${this.vendor} for LLM`);
}
if (!llm) {
throw new Error(`Unsupported vendor:model ${this.vendor}:${this.model}`);
}
return llm;
}
addCustomEventListener(ep, event, handler) {
this.eventHandlers.push({ep, event, handler});
ep.addCustomEventListener(event, handler);
}
removeCustomEventListeners() {
this.eventHandlers.forEach((h) => h.ep.removeCustomEventListener(h.event, h.handler));
}
async sendEventHook(data) {
await this.cs?.requestor.request('llm:event', this.eventHook, data);
}
async sendToolHook(tool_call_id, data) {
const tool_response = await this.cs?.requestor.request('llm:tool-call', this.toolHook, {tool_call_id, ...data});
// if the toolHook was a websocket it will return undefined, otherwise it should return an object
if (typeof tool_response != 'undefined') {
tool_response.type = 'client_tool_result';
tool_response.invocation_id = tool_call_id;
this.processToolOutput(tool_call_id, tool_response);
}
}
async processToolOutput(tool_call_id, data) {
if (!this.ep.connected) {
this.logger.info('TaskLlm:processToolOutput - no connected endpoint');
return;
}
this.llm.processToolOutput(this.ep, tool_call_id, data);
}
async processLlmUpdate(data, callSid) {
if (this.ep.connected) {
if (typeof this.llm.processLlmUpdate === 'function') {
this.llm.processLlmUpdate(this.ep, data, callSid);
}
else {
const {vendor, model} = this.llm;
this.logger.info({data, callSid},
`TaskLlm:_processLlmUpdate: LLM ${vendor}:${model} does not support llm:update`);
}
}
}
}
module.exports = TaskLlm;

View File

@@ -1,327 +0,0 @@
const Task = require('../../task');
const TaskName = 'Llm_Elevenlabs_s2s';
const {LlmEvents_Elevenlabs} = require('../../../utils/constants');
const {request} = require('undici');
const ClientEvent = 'client.event';
const SessionDelete = 'session.delete';
const elevenlabs_server_events = [
'conversation_initiation_metadata',
'user_transcript',
'agent_response',
'client_tool_call'
];
const expandWildcards = (events) => {
const expandedEvents = [];
events.forEach((evt) => {
if (evt.endsWith('.*')) {
const prefix = evt.slice(0, -2); // Remove the wildcard ".*"
const matchingEvents = elevenlabs_server_events.filter((e) => e.startsWith(prefix));
expandedEvents.push(...matchingEvents);
} else {
expandedEvents.push(evt);
}
});
return expandedEvents;
};
class TaskLlmElevenlabs_S2S extends Task {
constructor(logger, opts, parentTask) {
super(logger, opts, parentTask);
this.parent = parentTask;
this.vendor = this.parent.vendor;
this.auth = this.parent.auth;
const {agent_id, api_key} = this.auth || {};
if (!agent_id) throw new Error('auth.agent_id is required for Elevenlabs S2S');
this.agent_id = agent_id;
this.api_key = api_key;
this.actionHook = this.data.actionHook;
this.eventHook = this.data.eventHook;
this.toolHook = this.data.toolHook;
const {
conversation_initiation_client_data,
input_sample_rate = 16000,
output_sample_rate = 16000
} = this.data.llmOptions;
this.conversation_initiation_client_data = conversation_initiation_client_data;
this.input_sample_rate = input_sample_rate;
this.output_sample_rate = output_sample_rate;
this.results = {
completionReason: 'normal conversation end'
};
/**
* only one of these will have items,
* if includeEvents, then these are the events to include
* if excludeEvents, then these are the events to exclude
*/
this.includeEvents = [];
this.excludeEvents = [];
/* default to all events if user did not specify */
this._populateEvents(this.data.events || elevenlabs_server_events);
this.addCustomEventListener = parentTask.addCustomEventListener.bind(parentTask);
this.removeCustomEventListeners = parentTask.removeCustomEventListeners.bind(parentTask);
}
get name() { return TaskName; }
async getSignedUrl() {
if (!this.api_key) {
return {
host: 'api.elevenlabs.io',
path: `/v1/convai/conversation?agent_id=${this.agent_id}`,
};
}
const {statusCode, body} = await request(
`https://api.elevenlabs.io/v1/convai/conversation/get_signed_url?agent_id=${this.agent_id}`, {
method: 'GET',
headers: {
'xi-api-key': this.api_key
},
}
);
const data = await body.json();
if (statusCode !== 200 || !data?.signed_url) {
this.logger.error({statusCode, data}, 'Elevenlabs Error registering call');
throw new Error(`Elevenlabs Error registering call: ${data.message}`);
}
const url = new URL(data.signed_url);
return {
host: url.hostname,
path: url.pathname + url.search,
};
}
async _api(ep, args) {
const res = await ep.api('uuid_elevenlabs_s2s', `^^|${args.join('|')}`);
if (!res.body?.startsWith('+OK')) {
throw new Error({args}, `Error calling uuid_elevenlabs_s2s: ${res.body}`);
}
}
async exec(cs, {ep}) {
await super.exec(cs);
await this._startListening(cs, ep);
await this.awaitTaskDone();
/* note: the parent llm verb started the span, which is why this is necessary */
await this.parent.performAction(this.results);
this._unregisterHandlers();
}
async kill(cs) {
super.kill(cs);
this._api(cs.ep, [cs.ep.uuid, SessionDelete])
.catch((err) => this.logger.info({err}, 'TaskLlmElevenlabs_S2S:kill - error deleting session'));
this.notifyTaskDone();
}
/**
* Send function call output to the Elevenlabs server in the form of conversation.item.create
* per https://elevenlabs.io/docs/conversational-ai/api-reference/conversational-ai/websocket
*/
async processToolOutput(ep, tool_call_id, rawData) {
try {
const {data} = rawData;
this.logger.debug({tool_call_id, data}, 'TaskLlmElevenlabs_S2S:processToolOutput');
if (!data.type || data.type !== 'client_tool_result') {
this.logger.info({data},
'TaskLlmElevenlabs_S2S:processToolOutput - invalid tool output, must be client_tool_result');
}
else {
await this._api(ep, [ep.uuid, ClientEvent, JSON.stringify(data)]);
}
} catch (err) {
this.logger.info({err}, 'TaskLlmElevenlabs_S2S:processToolOutput');
}
}
/**
* Send a session.update to the Elevenlabs server
* Note: creating and deleting conversation items also supported as well as interrupting the assistant
*/
async processLlmUpdate(ep, data, _callSid) {
this.logger.debug({data, _callSid}, 'TaskLlmElevenlabs_S2S:processLlmUpdate, ignored');
}
async _startListening(cs, ep) {
this._registerHandlers(ep);
try {
const {host, path} = await this.getSignedUrl();
const args = this.conversation_initiation_client_data ?
[ep.uuid, 'session.create', this.input_sample_rate, this.output_sample_rate, host, path] :
[ep.uuid, 'session.create', this.input_sample_rate, this.output_sample_rate, host, path, 'no_initial_config'];
await this._api(ep, args);
} catch (err) {
this.logger.error({err}, 'TaskLlmElevenlabs_S2S:_startListening');
this.notifyTaskDone();
}
}
async _sendClientEvent(ep, obj) {
let ok = true;
this.logger.debug({obj}, 'TaskLlmElevenlabs_S2S:_sendClientEvent');
try {
const args = [ep.uuid, ClientEvent, JSON.stringify(obj)];
await this._api(ep, args);
} catch (err) {
ok = false;
this.logger.error({err}, 'TaskLlmElevenlabs_S2S:_sendClientEvent - Error');
}
return ok;
}
async _sendInitialMessage(ep) {
if (this.conversation_initiation_client_data) {
if (!await this._sendClientEvent(ep, {
type: 'conversation_initiation_client_data',
...this.conversation_initiation_client_data
})) {
this.notifyTaskDone();
}
}
}
_registerHandlers(ep) {
this.addCustomEventListener(ep, LlmEvents_Elevenlabs.Connect, this._onConnect.bind(this, ep));
this.addCustomEventListener(ep, LlmEvents_Elevenlabs.ConnectFailure, this._onConnectFailure.bind(this, ep));
this.addCustomEventListener(ep, LlmEvents_Elevenlabs.Disconnect, this._onDisconnect.bind(this, ep));
this.addCustomEventListener(ep, LlmEvents_Elevenlabs.ServerEvent, this._onServerEvent.bind(this, ep));
}
_unregisterHandlers() {
this.removeCustomEventListeners();
}
_onError(ep, evt) {
this.logger.info({evt}, 'TaskLlmElevenlabs_S2S:_onError');
this.notifyTaskDone();
}
_onConnect(ep) {
this.logger.debug('TaskLlmElevenlabs_S2S:_onConnect');
this._sendInitialMessage(ep);
}
_onConnectFailure(_ep, evt) {
this.logger.info(evt, 'TaskLlmElevenlabs_S2S:_onConnectFailure');
this.results = {completionReason: 'connection failure'};
this.notifyTaskDone();
}
_onDisconnect(_ep, evt) {
this.logger.info(evt, 'TaskLlmElevenlabs_S2S:_onConnectFailure');
this.results = {completionReason: 'disconnect from remote end'};
this.notifyTaskDone();
}
async _onServerEvent(ep, evt) {
let endConversation = false;
const type = evt.type;
this.logger.info({evt}, 'TaskLlmElevenlabs_S2S:_onServerEvent');
if (type === 'error') {
endConversation = true;
this.results = {
completionReason: 'server error',
error: evt.error
};
}
/* tool calls */
else if (type === 'client_tool_call') {
this.logger.debug({evt}, 'TaskLlmElevenlabs_S2S:_onServerEvent - function_call');
const {tool_name: name, tool_call_id: call_id, parameters: args} = evt.client_tool_call;
const mcpTools = this.parent.isMcpEnabled ? await this.parent.mcpService.getAvailableMcpTools() : [];
if (mcpTools.some((tool) => tool.name === name)) {
this.logger.debug({name, args}, 'TaskLlmElevenlabs_S2S:_onServerEvent - calling mcp tool');
try {
const res = await this.parent.mcpService.callMcpTool(name, args);
this.logger.debug({res}, 'TaskLlmElevenlabs_S2S:_onServerEvent - function_call - mcp result');
this.processToolOutput(ep, call_id, {
data: {
type: 'client_tool_result',
tool_call_id: call_id,
result: res.content?.length ? res.content[0] : res.content,
is_error: false
}
});
return;
}
catch (err) {
this.logger.info({err, evt}, 'TaskLlmElevenlabs_S2S - error calling mcp tool');
this.results = {
completionReason: 'client error calling mcp function',
error: err
};
endConversation = true;
}
} else if (!this.toolHook) {
this.logger.warn({evt}, 'TaskLlmElevenlabs_S2S:_onServerEvent - no toolHook defined!');
}
else {
try {
await this.parent.sendToolHook(call_id, {name, args});
} catch (err) {
this.logger.info({err, evt}, 'TaskLlmElevenlabs_S2S - error calling function');
this.results = {
completionReason: 'client error calling function',
error: err
};
endConversation = true;
}
}
}
/* check whether we should notify on this event */
if (this.includeEvents.length > 0 ? this.includeEvents.includes(type) : !this.excludeEvents.includes(type)) {
this.parent.sendEventHook(evt)
.catch((err) => this.logger.info({err},
'TaskLlmElevenlabs_S2S:_onServerEvent - error sending event hook'));
}
if (endConversation) {
this.logger.info({results: this.results},
'TaskLlmElevenlabs_S2S:_onServerEvent - ending conversation due to error');
this.notifyTaskDone();
}
}
_populateEvents(events) {
if (events.includes('all')) {
/* work by excluding specific events */
const exclude = events
.filter((evt) => evt.startsWith('-'))
.map((evt) => evt.slice(1));
if (exclude.length === 0) this.includeEvents = elevenlabs_server_events;
else this.excludeEvents = expandWildcards(exclude);
}
else {
/* work by including specific events */
const include = events
.filter((evt) => !evt.startsWith('-'));
this.includeEvents = expandWildcards(include);
}
this.logger.debug({
includeEvents: this.includeEvents,
excludeEvents: this.excludeEvents
}, 'TaskLlmElevenlabs_S2S:_populateEvents');
}
}
module.exports = TaskLlmElevenlabs_S2S;

View File

@@ -1,313 +0,0 @@
const Task = require('../../task');
const TaskName = 'Llm_Google_s2s';
const {LlmEvents_Google} = require('../../../utils/constants');
const ClientEvent = 'client.event';
const SessionDelete = 'session.delete';
const google_server_events = [
'error',
'session.created',
'session.updated',
];
const expandWildcards = (events) => {
const expandedEvents = [];
events.forEach((evt) => {
if (evt.endsWith('.*')) {
const prefix = evt.slice(0, -2); // Remove the wildcard ".*"
const matchingEvents = google_server_events.filter((e) => e.startsWith(prefix));
expandedEvents.push(...matchingEvents);
} else {
expandedEvents.push(evt);
}
});
return expandedEvents;
};
class TaskLlmGoogle_S2S extends Task {
constructor(logger, opts, parentTask) {
super(logger, opts, parentTask);
this.parent = parentTask;
this.vendor = this.parent.vendor;
this.vendor = this.parent.vendor;
this.model = this.parent.model || 'gemini-2o';
this.auth = this.parent.auth;
this.connectionOptions = this.parent.connectOptions;
const {apiKey} = this.auth || {};
if (!apiKey) throw new Error('auth.apiKey is required for Google S2S');
this.apiKey = apiKey;
this.actionHook = this.data.actionHook;
this.eventHook = this.data.eventHook;
this.toolHook = this.data.toolHook;
const {setup} = this.data.llmOptions;
if (typeof setup !== 'object') {
throw new Error('llmOptions with an initial setup is required for Google S2S');
}
this.setup = {
...setup,
// make sure output is always audio
generationConfig: {
...(setup.generationConfig || {}),
responseModalities: 'audio'
}
};
this.results = {
completionReason: 'normal conversation end'
};
/**
* only one of these will have items,
* if includeEvents, then these are the events to include
* if excludeEvents, then these are the events to exclude
*/
this.includeEvents = [];
this.excludeEvents = [];
/* default to all events if user did not specify */
this._populateEvents(this.data.events || google_server_events);
this.addCustomEventListener = parentTask.addCustomEventListener.bind(parentTask);
this.removeCustomEventListeners = parentTask.removeCustomEventListeners.bind(parentTask);
}
get name() { return TaskName; }
async _api(ep, args) {
const res = await ep.api('uuid_google_s2s', `^^|${args.join('|')}`);
if (!res.body?.startsWith('+OK')) {
throw new Error({args}, `Error calling uuid_openai_s2s: ${res.body}`);
}
}
async exec(cs, {ep}) {
await super.exec(cs);
await this._startListening(cs, ep);
await this.awaitTaskDone();
/* note: the parent llm verb started the span, which is why this is necessary */
await this.parent.performAction(this.results);
this._unregisterHandlers();
}
async kill(cs) {
super.kill(cs);
this._api(cs.ep, [cs.ep.uuid, SessionDelete])
.catch((err) => this.logger.info({err}, 'TaskLlmGoogle_S2S:kill - error deleting session'));
this.notifyTaskDone();
}
_populateEvents(events) {
if (events.includes('all')) {
/* work by excluding specific events */
const exclude = events
.filter((evt) => evt.startsWith('-'))
.map((evt) => evt.slice(1));
if (exclude.length === 0) this.includeEvents = google_server_events;
else this.excludeEvents = expandWildcards(exclude);
}
else {
/* work by including specific events */
const include = events
.filter((evt) => !evt.startsWith('-'));
this.includeEvents = expandWildcards(include);
}
this.logger.debug({
includeEvents: this.includeEvents,
excludeEvents: this.excludeEvents
}, 'TaskLlmGoogle_S2S:_populateEvents');
}
async _startListening(cs, ep) {
this._registerHandlers(ep);
try {
const args = [ep.uuid, 'session.create', this.apiKey];
await this._api(ep, args);
} catch (err) {
this.logger.error({err}, 'TaskLlmGoogle_S2S:_startListening');
this.notifyTaskDone();
}
}
async _sendClientEvent(ep, obj) {
let ok = true;
this.logger.debug({obj}, 'TaskLlmGoogle_S2S:_sendClientEvent');
try {
const args = [ep.uuid, ClientEvent, JSON.stringify(obj)];
await this._api(ep, args);
} catch (err) {
ok = false;
this.logger.error({err}, 'TaskLlmGoogle_S2S:_sendClientEvent - Error');
}
return ok;
}
async _sendInitialMessage(ep) {
const setup = this.setup;
const mcpTools = this.parent.isMcpEnabled ? await this.parent.mcpService.getAvailableMcpTools() : [];
if (mcpTools && mcpTools.length > 0) {
const convertedTools = mcpTools.map((tool) => {
return {
functionDeclarations: {
name: tool.name,
description: tool.description,
parameters: tool.inputSchema,
}
};
}
);
// merge with any existing tools
setup.tools = [...convertedTools, ...(this.setup.tools || [])];
}
if (!await this._sendClientEvent(ep, {
setup,
})) {
this.logger.debug(this.setup, 'TaskLlmGoogle_S2S:_sendInitialMessage - sending session.update');
this.notifyTaskDone();
}
}
_registerHandlers(ep) {
this.addCustomEventListener(ep, LlmEvents_Google.Connect, this._onConnect.bind(this, ep));
this.addCustomEventListener(ep, LlmEvents_Google.ConnectFailure, this._onConnectFailure.bind(this, ep));
this.addCustomEventListener(ep, LlmEvents_Google.Disconnect, this._onDisconnect.bind(this, ep));
this.addCustomEventListener(ep, LlmEvents_Google.ServerEvent, this._onServerEvent.bind(this, ep));
}
_unregisterHandlers() {
this.removeCustomEventListeners();
}
_onError(ep, evt) {
this.logger.info({evt}, 'TaskLlmGoogle_S2S:_onError');
this.notifyTaskDone();
}
_onConnect(ep) {
this.logger.debug('TaskLlmGoogle_S2S:_onConnect');
this._sendInitialMessage(ep);
}
_onConnectFailure(_ep, evt) {
this.logger.info(evt, 'TaskLlmGoogle_S2S:_onConnectFailure');
this.results = {completionReason: 'connection failure'};
this.notifyTaskDone();
}
_onDisconnect(_ep, evt) {
this.logger.info(evt, 'TaskLlmGoogle_S2S:_onConnectFailure');
this.results = {completionReason: 'disconnect from remote end'};
this.notifyTaskDone();
}
async _onServerEvent(ep, evt) {
let endConversation = false;
this.logger.debug({evt}, 'TaskLlmGoogle_S2S:_onServerEvent');
const {toolCall /**toolCallCancellation*/} = evt;
if (toolCall) {
this.logger.debug({toolCall}, 'TaskLlmGoogle_S2S:_onServerEvent - toolCall');
if (!this.toolHook) {
this.logger.info({evt}, 'TaskLlmGoogle_S2S:_onServerEvent - no toolHook defined!');
}
else {
const {functionCalls} = toolCall;
const mcpTools = this.parent.isMcpEnabled ? await this.parent.mcpService.getAvailableMcpTools() : [];
const functionResponses = [];
if (mcpTools && mcpTools.length > 0) {
for (const functionCall of functionCalls) {
const {name, args, id} = functionCall;
const tool = mcpTools.find((tool) => tool.name === name);
if (tool) {
const response = await this.parent.mcpService.callMcpTool(name, args);
functionResponses.push({
response: {
output: response,
},
id
});
}
}
}
if (functionResponses && functionResponses.length > 0) {
this.logger.debug({functionResponses}, 'TaskLlmGoogle_S2S:_onServerEvent - function_call - mcp result');
this.processToolOutput(ep, 'tool_call_id', {
toolResponse: {
functionResponses
}
});
} else {
try {
await this.parent.sendToolHook('function_call_id', {type: 'toolCall', functionCalls});
} catch (err) {
this.logger.info({err, evt}, 'TaskLlmGoogle_S2S - error calling function');
this.results = {
completionReason: 'client error calling function',
error: err
};
endConversation = true;
}
}
}
}
this._sendLlmEvent('llm_event', evt);
if (endConversation) {
this.logger.info({results: this.results},
'TaskLlmGoogle_S2S:_onServerEvent - ending conversation due to error');
this.notifyTaskDone();
}
}
_sendLlmEvent(type, evt) {
/* check whether we should notify on this event */
if (this.includeEvents.length > 0 ? this.includeEvents.includes(type) : !this.excludeEvents.includes(type)) {
this.parent.sendEventHook(evt)
.catch((err) => this.logger.info({err}, 'TaskLlmGoogle_S2S:_onServerEvent - error sending event hook'));
}
}
async processLlmUpdate(ep, data, _callSid) {
try {
this.logger.debug({data, _callSid}, 'TaskLlmGoogle_S2S:processLlmUpdate');
await this._api(ep, [ep.uuid, ClientEvent, JSON.stringify(data)]);
} catch (err) {
this.logger.info({err, data}, 'TaskLlmGoogle_S2S:processLlmUpdate - Error processing LLM update');
}
}
async processToolOutput(ep, tool_call_id, data) {
try {
this.logger.debug({tool_call_id, data}, 'TaskLlmGoogle_S2S:processToolOutput');
const {toolResponse} = data;
if (!toolResponse) {
this.logger.info({data},
'TaskLlmGoogle_S2S:processToolOutput - invalid tool output, must be functionResponses');
}
else {
await this._api(ep, [ep.uuid, ClientEvent, JSON.stringify(data)]);
}
} catch (err) {
this.logger.info({err, data}, 'TaskLlmGoogle_S2S:processToolOutput - Error processing tool output');
}
}
}
module.exports = TaskLlmGoogle_S2S;

View File

@@ -1,398 +0,0 @@
const Task = require('../../task');
const TaskName = 'Llm_OpenAI_s2s';
const {LlmEvents_OpenAI} = require('../../../utils/constants');
const ClientEvent = 'client.event';
const SessionDelete = 'session.delete';
const openai_server_events = [
'error',
'session.created',
'session.updated',
'conversation.created',
'input_audio_buffer.committed',
'input_audio_buffer.cleared',
'input_audio_buffer.speech_started',
'input_audio_buffer.speech_stopped',
'conversation.item.created',
'conversation.item.input_audio_transcription.completed',
'conversation.item.input_audio_transcription.failed',
'conversation.item.truncated',
'conversation.item.deleted',
'response.created',
'response.done',
'response.output_item.added',
'response.output_item.done',
'response.content_part.added',
'response.content_part.done',
'response.text.delta',
'response.text.done',
'response.audio_transcript.delta',
'response.audio_transcript.done',
'response.audio.delta',
'response.audio.done',
'response.function_call_arguments.delta',
'response.function_call_arguments.done',
'rate_limits.updated',
'output_audio.playback_started',
'output_audio.playback_stopped',
];
const expandWildcards = (events) => {
const expandedEvents = [];
events.forEach((evt) => {
if (evt.endsWith('.*')) {
const prefix = evt.slice(0, -2); // Remove the wildcard ".*"
const matchingEvents = openai_server_events.filter((e) => e.startsWith(prefix));
expandedEvents.push(...matchingEvents);
} else {
expandedEvents.push(evt);
}
});
return expandedEvents;
};
class TaskLlmOpenAI_S2S extends Task {
constructor(logger, opts, parentTask) {
super(logger, opts, parentTask);
this.parent = parentTask;
this.vendor = this.parent.vendor;
this.model = this.parent.model || 'gpt-4o-realtime-preview-2024-12-17';
this.auth = this.parent.auth;
this.connectionOptions = this.parent.connectOptions;
const {apiKey} = this.auth || {};
if (!apiKey) throw new Error('auth.apiKey is required for OpenAI S2S');
if (['openai', 'microsoft'].indexOf(this.vendor) === -1) {
throw new Error(`Invalid vendor ${this.vendor} for OpenAI S2S`);
}
if ('microsoft' === this.vendor && !this.connectionOptions?.host) {
throw new Error('connectionOptions.host is required for Microsoft OpenAI S2S');
}
this.apiKey = apiKey;
this.authType = 'microsoft' === this.vendor ? 'query' : 'bearer';
this.actionHook = this.data.actionHook;
this.eventHook = this.data.eventHook;
this.toolHook = this.data.toolHook;
const {response_create, session_update} = this.data.llmOptions;
if (typeof response_create !== 'object') {
throw new Error('llmOptions with an initial response.create is required for OpenAI S2S');
}
this.response_create = response_create;
this.session_update = session_update;
this.results = {
completionReason: 'normal conversation end'
};
/**
* only one of these will have items,
* if includeEvents, then these are the events to include
* if excludeEvents, then these are the events to exclude
*/
this.includeEvents = [];
this.excludeEvents = [];
/* default to all events if user did not specify */
this._populateEvents(this.data.events || openai_server_events);
this.addCustomEventListener = parentTask.addCustomEventListener.bind(parentTask);
this.removeCustomEventListeners = parentTask.removeCustomEventListeners.bind(parentTask);
}
get name() { return TaskName; }
get host() {
const {host} = this.connectionOptions || {};
return host || (this.vendor === 'openai' ? 'api.openai.com' : void 0);
}
get path() {
const {path} = this.connectionOptions || {};
if (path) return path;
switch (this.vendor) {
case 'openai':
return `v1/realtime?model=${this.model}`;
case 'microsoft':
return `openai/realtime?api-version=2024-10-01-preview&deployment=${this.model}`;
}
}
async _api(ep, args) {
const res = await ep.api('uuid_openai_s2s', `^^|${args.join('|')}`);
if (!res.body?.startsWith('+OK')) {
throw new Error({args}, `Error calling uuid_openai_s2s: ${res.body}`);
}
}
async exec(cs, {ep}) {
await super.exec(cs);
await this._startListening(cs, ep);
await this.awaitTaskDone();
/* note: the parent llm verb started the span, which is why this is necessary */
await this.parent.performAction(this.results);
this._unregisterHandlers();
}
async kill(cs) {
super.kill(cs);
this._api(cs.ep, [cs.ep.uuid, SessionDelete])
.catch((err) => this.logger.info({err}, 'TaskLlmOpenAI_S2S:kill - error deleting session'));
this.notifyTaskDone();
}
/**
* Send function call output to the OpenAI server in the form of conversation.item.create
* per https://platform.openai.com/docs/guides/realtime/function-calls
*/
async processToolOutput(ep, tool_call_id, data) {
try {
this.logger.debug({tool_call_id, data}, 'TaskLlmOpenAI_S2S:processToolOutput');
if (!data.type || data.type !== 'conversation.item.create') {
this.logger.info({data},
'TaskLlmOpenAI_S2S:processToolOutput - invalid tool output, must be conversation.item.create');
}
else {
await this._api(ep, [ep.uuid, ClientEvent, JSON.stringify(data)]);
// spec also recommends to send immediate response.create
await this._api(ep, [ep.uuid, ClientEvent, JSON.stringify({type: 'response.create'})]);
}
} catch (err) {
this.logger.info({err}, 'TaskLlmOpenAI_S2S:processToolOutput');
}
}
/**
* Send a session.update to the OpenAI server
* Note: creating and deleting conversation items also supported as well as interrupting the assistant
*/
async processLlmUpdate(ep, data, _callSid) {
try {
this.logger.debug({data, _callSid}, 'TaskLlmOpenAI_S2S:processLlmUpdate');
if (!data.type || ![
'session.update',
'conversation.item.create',
'conversation.item.delete',
'response.cancel'
].includes(data.type)) {
this.logger.info({data}, 'TaskLlmOpenAI_S2S:processLlmUpdate - invalid mid-call request');
}
else {
await this._api(ep, [ep.uuid, ClientEvent, JSON.stringify(data)]);
}
} catch (err) {
this.logger.info({err}, 'TaskLlmOpenAI_S2S:processLlmUpdate');
}
}
async _startListening(cs, ep) {
this._registerHandlers(ep);
try {
const args = [ep.uuid, 'session.create', this.host, this.path, this.authType, this.apiKey];
await this._api(ep, args);
} catch (err) {
this.logger.error({err}, 'TaskLlmOpenAI_S2S:_startListening');
this.notifyTaskDone();
}
}
async _sendClientEvent(ep, obj) {
let ok = true;
this.logger.debug({obj}, 'TaskLlmOpenAI_S2S:_sendClientEvent');
try {
const args = [ep.uuid, ClientEvent, JSON.stringify(obj)];
await this._api(ep, args);
} catch (err) {
ok = false;
this.logger.error({err}, 'TaskLlmOpenAI_S2S:_sendClientEvent - Error');
}
return ok;
}
async _sendInitialMessage(ep) {
let obj = {type: 'response.create', response: this.response_create};
if (!await this._sendClientEvent(ep, obj)) {
this.notifyTaskDone();
}
/* send immediate session.update if present */
else if (this.session_update) {
if (this.parent.isMcpEnabled) {
this.logger.debug('TaskLlmOpenAI_S2S:_sendInitialMessage - mcp enabled');
const tools = await this.parent.mcpService.getAvailableMcpTools();
if (tools && tools.length > 0 && this.session_update) {
const convertedTools = tools.map((tool) => ({
name: tool.name,
type: 'function',
description: tool.description,
parameters: tool.inputSchema
}));
this.session_update.tools = [
...convertedTools,
...(this.session_update.tools || [])
];
}
}
obj = {type: 'session.update', session: this.session_update};
this.logger.debug({obj}, 'TaskLlmOpenAI_S2S:_sendInitialMessage - sending session.update');
if (!await this._sendClientEvent(ep, obj)) {
this.notifyTaskDone();
}
}
}
_registerHandlers(ep) {
this.addCustomEventListener(ep, LlmEvents_OpenAI.Connect, this._onConnect.bind(this, ep));
this.addCustomEventListener(ep, LlmEvents_OpenAI.ConnectFailure, this._onConnectFailure.bind(this, ep));
this.addCustomEventListener(ep, LlmEvents_OpenAI.Disconnect, this._onDisconnect.bind(this, ep));
this.addCustomEventListener(ep, LlmEvents_OpenAI.ServerEvent, this._onServerEvent.bind(this, ep));
}
_unregisterHandlers() {
this.removeCustomEventListeners();
}
_onError(ep, evt) {
this.logger.info({evt}, 'TaskLlmOpenAI_S2S:_onError');
this.notifyTaskDone();
}
_onConnect(ep) {
this.logger.debug('TaskLlmOpenAI_S2S:_onConnect');
this._sendInitialMessage(ep);
}
_onConnectFailure(_ep, evt) {
this.logger.info(evt, 'TaskLlmOpenAI_S2S:_onConnectFailure');
this.results = {completionReason: 'connection failure'};
this.notifyTaskDone();
}
_onDisconnect(_ep, evt) {
this.logger.info(evt, 'TaskLlmOpenAI_S2S:_onConnectFailure');
this.results = {completionReason: 'disconnect from remote end'};
this.notifyTaskDone();
}
async _onServerEvent(ep, evt) {
let endConversation = false;
const type = evt.type;
this.logger.info({evt}, 'TaskLlmOpenAI_S2S:_onServerEvent');
/* check for failures, such as rate limit exceeded, that should terminate the conversation */
if (type === 'response.done' && evt.response.status === 'failed') {
endConversation = true;
this.results = {
completionReason: 'server failure',
error: evt.response.status_details?.error
};
}
/* server errors of some sort */
else if (type === 'error') {
endConversation = true;
this.results = {
completionReason: 'server error',
error: evt.error
};
}
/* tool calls */
else if (type === 'response.output_item.done' && evt.item?.type === 'function_call') {
this.logger.debug({evt}, 'TaskLlmOpenAI_S2S:_onServerEvent - function_call');
const {name, call_id} = evt.item;
const args = JSON.parse(evt.item.arguments);
const mcpTools = this.parent.isMcpEnabled ? await this.parent.mcpService.getAvailableMcpTools() : [];
if (mcpTools.some((tool) => tool.name === name)) {
this.logger.debug({call_id, name, args}, 'TaskLlmOpenAI_S2S:_onServerEvent - calling mcp tool');
try {
const res = await this.parent.mcpService.callMcpTool(name, args);
this.logger.debug({res}, 'TaskLlmOpenAI_S2S:_onServerEvent - function_call - mcp result');
this.processToolOutput(ep, call_id, {
type: 'conversation.item.create',
item: {
type: 'function_call_output',
call_id,
output: res.content[0]?.text || 'There is no output from the function call',
}
});
return;
} catch (err) {
this.logger.info({err, evt}, 'TaskLlmOpenAI_S2S - error calling function');
this.results = {
completionReason: 'client error calling mcp function',
error: err
};
endConversation = true;
}
}
else if (!this.toolHook) {
this.logger.warn({evt}, 'TaskLlmOpenAI_S2S:_onServerEvent - no toolHook defined!');
}
else {
try {
await this.parent.sendToolHook(call_id, {name, args});
} catch (err) {
this.logger.info({err, evt}, 'TaskLlmOpenAI - error calling function');
this.results = {
completionReason: 'client error calling function',
error: err
};
endConversation = true;
}
}
}
/* check whether we should notify on this event */
if (this.includeEvents.length > 0 ? this.includeEvents.includes(type) : !this.excludeEvents.includes(type)) {
this.parent.sendEventHook(evt)
.catch((err) => this.logger.info({err}, 'TaskLlmOpenAI_S2S:_onServerEvent - error sending event hook'));
}
if (endConversation) {
this.logger.info({results: this.results}, 'TaskLlmOpenAI_S2S:_onServerEvent - ending conversation due to error');
this.notifyTaskDone();
}
}
_populateEvents(events) {
if (events.includes('all')) {
/* work by excluding specific events */
const exclude = events
.filter((evt) => evt.startsWith('-'))
.map((evt) => evt.slice(1));
if (exclude.length === 0) this.includeEvents = openai_server_events;
else this.excludeEvents = expandWildcards(exclude);
}
else {
/* work by including specific events */
const include = events
.filter((evt) => !evt.startsWith('-'));
this.includeEvents = expandWildcards(include);
}
this.logger.debug({
includeEvents: this.includeEvents,
excludeEvents: this.excludeEvents
}, 'TaskLlmOpenAI_S2S:_populateEvents');
}
}
module.exports = TaskLlmOpenAI_S2S;

View File

@@ -1,344 +0,0 @@
const Task = require('../../task');
const TaskName = 'Llm_Ultravox_s2s';
const {request} = require('undici');
const {LlmEvents_Ultravox} = require('../../../utils/constants');
const ultravox_server_events = [
'createCall',
'pong',
'state',
'transcript',
'conversationText',
'clientToolInvocation',
'playbackClearBuffer',
];
const ClientEvent = 'client.event';
const expandWildcards = (events) => {
// no-op for deepgram
return events;
};
const SessionDelete = 'session.delete';
class TaskLlmUltravox_S2S extends Task {
constructor(logger, opts, parentTask) {
super(logger, opts, parentTask);
this.parent = parentTask;
this.vendor = this.parent.vendor;
this.model = this.parent.model || 'fixie-ai/ultravox';
this.auth = this.parent.auth;
this.connectionOptions = this.parent.connectOptions;
const {apiKey} = this.auth || {};
if (!apiKey) throw new Error('auth.apiKey is required for Vendor: Ultravox');
this.apiKey = apiKey;
this.actionHook = this.data.actionHook;
this.eventHook = this.data.eventHook;
this.toolHook = this.data.toolHook;
this.results = {
completionReason: 'normal conversation end'
};
/**
* only one of these will have items,
* if includeEvents, then these are the events to include
* if excludeEvents, then these are the events to exclude
*/
this.includeEvents = [];
this.excludeEvents = [];
/* default to all events if user did not specify */
this._populateEvents(this.data.events || ultravox_server_events);
this.addCustomEventListener = parentTask.addCustomEventListener.bind(parentTask);
this.removeCustomEventListeners = parentTask.removeCustomEventListeners.bind(parentTask);
}
get name() { return TaskName; }
async _api(ep, args) {
const res = await ep.api('uuid_ultravox_s2s', `^^|${args.join('|')}`);
if (!res.body?.startsWith('+OK')) {
throw new Error(`Error calling uuid_ultravox_s2s: ${JSON.stringify(res.body)}`);
}
}
/**
* Converts a JSON Schema to the dynamic parameters format used in the Ultravox API
* @param {Object} jsonSchema - A JSON Schema object defining parameters
* @param {string} locationDefault - Default location value for parameters (default: 'PARAMETER_LOCATION_BODY')
* @returns {Array} Array of dynamic parameters objects
*/
transformSchemaToParameters(jsonSchema, locationDefault = 'PARAMETER_LOCATION_BODY') {
if (jsonSchema.properties) {
const required = jsonSchema.required || [];
return Object.entries(jsonSchema.properties).map(([name]) => {
return {
name,
location: locationDefault,
required: required.includes(name)
};
});
}
return [];
}
async createCall() {
const mcpTools = this.parent.isMcpEnabled ? await this.parent.mcpService.getAvailableMcpTools() : [];
if (mcpTools && mcpTools.length > 0) {
const convertedTools = mcpTools.map((tool) => {
return {
temporaryTool: {
modelToolName: tool.name,
description: tool.description,
dynamicParameters: this.transformSchemaToParameters(tool.inputSchema),
// use client tool that ultravox call tool via freeswitch module.
client: {}
}
};
}
);
// merge with any existing tools
this.data.llmOptions.selectedTools = [
...convertedTools,
...(this.data.llmOptions.selectedTools || [])
];
}
const payload = {
...this.data.llmOptions,
model: this.model,
medium: {
...(this.data.llmOptions.medium || {}),
serverWebSocket: {
inputSampleRate: 8000,
outputSampleRate: 8000,
}
}
};
const {statusCode, body} = await request('https://api.ultravox.ai/api/calls', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-API-Key': this.apiKey
},
body: JSON.stringify(payload)
});
const data = await body.json();
if (statusCode !== 201 || !data?.joinUrl) {
this.logger.info({statusCode, data}, 'Ultravox Error registering call');
throw new Error(`Ultravox Error registering call:${statusCode} - ${data.detail}`);
}
this.logger.debug({joinUrl: data.joinUrl}, 'Ultravox Call registered');
return data;
}
_unregisterHandlers() {
this.removeCustomEventListeners();
}
_registerHandlers(ep) {
this.addCustomEventListener(ep, LlmEvents_Ultravox.Connect, this._onConnect.bind(this, ep));
this.addCustomEventListener(ep, LlmEvents_Ultravox.ConnectFailure, this._onConnectFailure.bind(this, ep));
this.addCustomEventListener(ep, LlmEvents_Ultravox.Disconnect, this._onDisconnect.bind(this, ep));
this.addCustomEventListener(ep, LlmEvents_Ultravox.ServerEvent, this._onServerEvent.bind(this, ep));
}
async _startListening(cs, ep) {
this._registerHandlers(ep);
try {
const data = await this.createCall();
const {joinUrl} = data;
// split the joinUrl into host and path
const {host, pathname, search} = new URL(joinUrl);
const args = [ep.uuid, 'session.create', host, pathname + search];
await this._api(ep, args);
// Notify the application that the session has been created with detail information
this._sendLlmEvent('createCall', {
type: 'createCall',
...data
});
} catch (err) {
this.logger.info({err}, 'TaskLlmUltraVox_S2S:_startListening - Error sending createCall');
this.results = {completionReason: `connection failure - ${err}`};
this.notifyTaskDone();
}
}
async exec(cs, {ep}) {
await super.exec(cs);
await this._startListening(cs, ep);
await this.awaitTaskDone();
/* note: the parent llm verb started the span, which is why this is necessary */
await this.parent.performAction(this.results);
this._unregisterHandlers();
}
async kill(cs) {
super.kill(cs);
this._api(cs.ep, [cs.ep.uuid, SessionDelete])
.catch((err) => this.logger.info({err}, 'TaskLlmUltravox_S2S:kill - error deleting session'));
this.notifyTaskDone();
}
_onConnect(ep) {
this.logger.info('TaskLlmUltravox_S2S:_onConnect');
}
_onConnectFailure(_ep, evt) {
this.logger.info(evt, 'TaskLlmUltravox_S2S:_onConnectFailure');
this.results = {completionReason: 'connection failure'};
this.notifyTaskDone();
}
_onDisconnect(_ep, evt) {
this.logger.info(evt, 'TaskLlmUltravox_S2S:_onConnectFailure');
this.results = {completionReason: 'disconnect from remote end'};
this.notifyTaskDone();
}
async _onServerEvent(_ep, evt) {
let endConversation = false;
const type = evt.type;
this.logger.debug({evt}, 'TaskLlmUltravox_S2S:_onServerEvent');
/* server errors of some sort */
if (type === 'error') {
endConversation = true;
this.results = {
completionReason: 'server error',
error: evt.error
};
}
/* tool calls */
else if (type === 'client_tool_invocation') {
this.logger.debug({evt}, 'TaskLlmUltravox_S2S:_onServerEvent - function_call');
const {toolName: name, invocationId: call_id, parameters: args} = evt;
const mcpTools = this.parent.isMcpEnabled ? await this.parent.mcpService.getAvailableMcpTools() : [];
if (mcpTools.some((tool) => tool.name === name)) {
this.logger.debug({
name,
input: args
}, 'TaskLlmUltravox_S2S:_onServerEvent - function_call - mcp tool');
try {
const res = await this.parent.mcpService.callMcpTool(name, args);
this.logger.debug({res}, 'TaskLlmUltravox_S2S:_onServerEvent - function_call - mcp result');
this.processToolOutput(_ep, call_id, {
type: 'client_tool_result',
invocation_id: call_id,
result: res.content
});
return;
} catch (err) {
this.logger.info({err, evt}, 'TaskLlmUltravox_S2S - error calling mcp tool');
this.results = {
completionReason: 'client error calling mcp function',
error: err
};
endConversation = true;
}
} else if (!this.toolHook) {
this.logger.info({evt}, 'TaskLlmUltravox_S2S:_onServerEvent - no toolHook defined!');
}
else {
try {
await this.parent.sendToolHook(call_id, {name, args});
} catch (err) {
this.logger.info({err, evt}, 'TaskLlmUltravox_S2S - error calling function');
this.results = {
completionReason: 'client error calling function',
error: err
};
endConversation = true;
}
}
}
this._sendLlmEvent(type, evt);
if (endConversation) {
this.logger.info({results: this.results},
'TaskLlmUltravox_S2S:_onServerEvent - ending conversation due to error');
this.notifyTaskDone();
}
}
_sendLlmEvent(type, evt) {
/* check whether we should notify on this event */
if (this.includeEvents.length > 0 ? this.includeEvents.includes(type) : !this.excludeEvents.includes(type)) {
this.parent.sendEventHook(evt)
.catch((err) => this.logger.info({err}, 'TaskLlmUltravox_S2S:_onServerEvent - error sending event hook'));
}
}
async processLlmUpdate(ep, data, _callSid) {
try {
this.logger.debug({data, _callSid}, 'TaskLlmUltravox_S2S:processLlmUpdate');
if (!data.type || ![
'input_text_message'
].includes(data.type)) {
this.logger.info({data},
'TaskLlmUltravox_S2S:processLlmUpdate - invalid mid-call request, only input_text_message supported');
}
else {
await this._api(ep, [ep.uuid, ClientEvent, JSON.stringify(data)]);
}
} catch (err) {
this.logger.info({err, data}, 'TaskLlmUltravox_S2S:processLlmUpdate - Error processing LLM update');
}
}
async processToolOutput(ep, tool_call_id, data) {
try {
this.logger.debug({tool_call_id, data}, 'TaskLlmUltravox_S2S:processToolOutput');
if (!data.type || data.type !== 'client_tool_result') {
this.logger.info({data},
'TaskLlmUltravox_S2S:processToolOutput - invalid tool output, must be client_tool_result');
}
else {
await this._api(ep, [ep.uuid, ClientEvent, JSON.stringify(data)]);
}
} catch (err) {
this.logger.info({err, data}, 'TaskLlmUltravox_S2S:processToolOutput - Error processing tool output');
}
}
_populateEvents(events) {
if (events.includes('all')) {
/* work by excluding specific events */
const exclude = events
.filter((evt) => evt.startsWith('-'))
.map((evt) => evt.slice(1));
if (exclude.length === 0) this.includeEvents = ultravox_server_events;
else this.excludeEvents = expandWildcards(exclude);
}
else {
/* work by including specific events */
const include = events
.filter((evt) => !evt.startsWith('-'));
this.includeEvents = expandWildcards(include);
}
this.logger.debug({
includeEvents: this.includeEvents,
excludeEvents: this.excludeEvents
}, 'TaskLlmUltravox_S2S:_populateEvents');
}
}
module.exports = TaskLlmUltravox_S2S;

View File

@@ -1,352 +0,0 @@
const Task = require('../../task');
const TaskName = 'Llm_VoiceAgent_s2s';
const {LlmEvents_VoiceAgent} = require('../../../utils/constants');
const ClientEvent = 'client.event';
const SessionDelete = 'session.delete';
const va_server_events = [
'Error',
'Welcome',
'SettingsApplied',
'ConversationText',
'UserStartedSpeaking',
'EndOfThought',
'AgentThinking',
'FunctionCallRequest',
'FunctionCalling',
'AgentStartedSpeaking',
'AgentAudioDone',
];
const expandWildcards = (events) => {
// no-op for deepgram
return events;
};
class TaskLlmVoiceAgent_S2S extends Task {
constructor(logger, opts, parentTask) {
super(logger, opts, parentTask);
this.parent = parentTask;
this.vendor = this.parent.vendor;
this.model = this.parent.model || 'voice-agent';
this.auth = this.parent.auth;
this.connectionOptions = this.parent.connectOptions;
const {apiKey} = this.auth || {};
if (!apiKey) throw new Error('auth.apiKey is required for VoiceAgent S2S');
this.apiKey = apiKey;
this.authType = 'bearer';
this.actionHook = this.data.actionHook;
this.eventHook = this.data.eventHook;
this.toolHook = this.data.toolHook;
const {Settings} = this.data.llmOptions;
if (typeof Settings !== 'object') {
throw new Error('llmOptions with an initial Settings is required for VoiceAgent S2S');
}
// eslint-disable-next-line no-unused-vars
const {audio, ...rest} = Settings;
const cfg = this.Settings = rest;
if (!cfg.agent) throw new Error('llmOptions.Settings.agent is required for VoiceAgent S2S');
if (!cfg.agent.think) {
throw new Error('llmOptions.Settings.agent.think is required for VoiceAgent S2S');
}
if (!cfg.agent.think.provider?.model) {
throw new Error('llmOptions.Settings.agent.think.provider.model is required for VoiceAgent S2S');
}
if (!cfg.agent.think.provider?.type) {
throw new Error('llmOptions.Settings.agent.think.provider.type is required for VoiceAgent S2S');
}
this.results = {
completionReason: 'normal conversation end'
};
/**
* only one of these will have items,
* if includeEvents, then these are the events to include
* if excludeEvents, then these are the events to exclude
*/
this.includeEvents = [];
this.excludeEvents = [];
/* default to all events if user did not specify */
this._populateEvents(this.data.events || va_server_events);
this.addCustomEventListener = parentTask.addCustomEventListener.bind(parentTask);
this.removeCustomEventListeners = parentTask.removeCustomEventListeners.bind(parentTask);
}
get name() { return TaskName; }
get host() {
const {host} = this.connectionOptions || {};
return host || 'agent.deepgram.com';
}
get path() {
const {path} = this.connectionOptions || {};
if (path) return path;
return '/v1/agent/converse';
}
async _api(ep, args) {
const res = await ep.api('uuid_voice_agent_s2s', `^^|${args.join('|')}`);
if (!res.body?.startsWith('+OK')) {
throw new Error(`Error calling uuid_voice_agent_s2s: ${JSON.stringify(res.body)}`);
}
}
async exec(cs, {ep}) {
await super.exec(cs);
await this._startListening(cs, ep);
await this.awaitTaskDone();
/* note: the parent llm verb started the span, which is why this is necessary */
await this.parent.performAction(this.results);
this._unregisterHandlers();
}
async kill(cs) {
super.kill(cs);
this._api(cs.ep, [cs.ep.uuid, SessionDelete])
.catch((err) => this.logger.info({err}, 'TaskLlmVoiceAgent_S2S:kill - error deleting session'));
this.notifyTaskDone();
}
/**
* Send function call response to the VoiceAgent server
*/
async processToolOutput(ep, tool_call_id, data) {
try {
const {data:response} = data;
this.logger.debug({tool_call_id, response}, 'TaskLlmVoiceAgent_S2S:processToolOutput');
if (!response.type || response.type !== 'FunctionCallResponse') {
this.logger.info({response},
'TaskLlmVoiceAgent_S2S:processToolOutput - invalid tool output, must be FunctionCallResponse');
}
else {
await this._api(ep, [ep.uuid, ClientEvent, JSON.stringify(response)]);
}
} catch (err) {
this.logger.info({err}, 'TaskLlmVoiceAgent_S2S:processToolOutput');
}
}
/**
* Send a session.update to the VoiceAgent server
* Note: creating and deleting conversation items also supported as well as interrupting the assistant
*/
async processLlmUpdate(ep, data, _callSid) {
try {
this.logger.debug({data, _callSid}, 'TaskLlmVoiceAgent_S2S:processLlmUpdate');
if (!data.type || ![
'UpdateInstructions',
'UpdateSpeak',
'InjectAgentMessage',
].includes(data.type)) {
this.logger.info({data}, 'TaskLlmVoiceAgent_S2S:processLlmUpdate - invalid mid-call request');
}
else {
await this._api(ep, [ep.uuid, ClientEvent, JSON.stringify(data)]);
}
} catch (err) {
this.logger.info({err}, 'TaskLlmVoiceAgent_S2S:processLlmUpdate');
}
}
async _startListening(cs, ep) {
this._registerHandlers(ep);
try {
const args = [ep.uuid, 'session.create', this.host, this.path, this.authType, this.apiKey];
await this._api(ep, args);
} catch (err) {
this.logger.error({err}, `TaskLlmVoiceAgent_S2S:_startListening: ${JSON.stringify(err)}`);
this.notifyTaskDone();
}
}
async _sendClientEvent(ep, obj) {
let ok = true;
this.logger.debug({obj}, 'TaskLlmVoiceAgent_S2S:_sendClientEvent');
try {
const args = [ep.uuid, ClientEvent, JSON.stringify(obj)];
await this._api(ep, args);
} catch (err) {
ok = false;
this.logger.error({err}, 'TaskLlmVoiceAgent_S2S:_sendClientEvent - Error');
}
return ok;
}
async _sendInitialMessage(ep) {
const mcpTools = this.parent.isMcpEnabled ? await this.parent.mcpService.getAvailableMcpTools() : [];
if (mcpTools && mcpTools.length > 0 && this.Settings.agent?.think) {
const convertedTools = mcpTools.map((tool) => ({
name: tool.name,
description: tool.description,
parameters: tool.inputSchema
}));
this.Settings.agent.think.functions = [
...convertedTools,
...(this.Settings.agent.think?.functions || [])
];
}
if (!await this._sendClientEvent(ep, this.Settings)) {
this.notifyTaskDone();
}
}
_registerHandlers(ep) {
this.addCustomEventListener(ep, LlmEvents_VoiceAgent.Connect, this._onConnect.bind(this, ep));
this.addCustomEventListener(ep, LlmEvents_VoiceAgent.ConnectFailure, this._onConnectFailure.bind(this, ep));
this.addCustomEventListener(ep, LlmEvents_VoiceAgent.Disconnect, this._onDisconnect.bind(this, ep));
this.addCustomEventListener(ep, LlmEvents_VoiceAgent.ServerEvent, this._onServerEvent.bind(this, ep));
}
_unregisterHandlers() {
this.removeCustomEventListeners();
}
_onError(_ep, evt) {
this.logger.info({evt}, 'TaskLlmVoiceAgent_S2S:_onError');
this.notifyTaskDone();
}
_onConnect(ep) {
this.logger.debug('TaskLlmVoiceAgent_S2S:_onConnect');
this._sendInitialMessage(ep);
}
_onConnectFailure(_ep, evt) {
this.logger.info(evt, 'TaskLlmVoiceAgent_S2S:_onConnectFailure');
this.results = {completionReason: 'connection failure'};
this.notifyTaskDone();
}
_onDisconnect(_ep, evt) {
this.logger.info(evt, 'TaskLlmVoiceAgent_S2S:_onConnectFailure');
this.results = {completionReason: 'disconnect from remote end'};
this.notifyTaskDone();
}
async _onServerEvent(_ep, evt) {
let endConversation = false;
const type = evt.type;
this.logger.info({evt}, 'TaskLlmVoiceAgent_S2S:_onServerEvent');
/* check for failures, such as rate limit exceeded, that should terminate the conversation */
if (type === 'response.done' && evt.response.status === 'failed') {
endConversation = true;
this.results = {
completionReason: 'server failure',
error: evt.response.status_details?.error
};
}
/* server errors of some sort */
else if (type === 'error') {
endConversation = true;
this.results = {
completionReason: 'server error',
error: evt.error
};
}
/* tool calls */
else if (type === 'FunctionCallRequest') {
this.logger.debug({evt}, 'TaskLlmVoiceAgent_S2S:_onServerEvent - function_call');
const mcpTools = this.parent.isMcpEnabled ? await this.parent.mcpService.getAvailableMcpTools() : [];
if (!this.toolHook && mcpTools.length === 0) {
this.logger.warn({evt}, 'TaskLlmVoiceAgent_S2S:_onServerEvent - no toolHook defined!');
} else {
const {functions} = evt;
const handledFunctions = [];
try {
if (mcpTools && mcpTools.length > 0) {
for (const func of functions) {
const {name, arguments: args, id} = func;
const tool = mcpTools.find((tool) => tool.name === name);
if (tool) {
handledFunctions.push(name);
const response = await this.parent.mcpService.callMcpTool(name, JSON.parse(args));
this.logger.debug({response}, 'TaskLlmVoiceAgent_S2S:_onServerEvent - function_call - mcp result');
this.processToolOutput(_ep, id, {
data: {
type: 'FunctionCallResponse',
id,
name,
content: response.length > 0 ? response[0].text : 'There is no output from the function call'
}
});
}
}
}
for (const func of functions) {
const {name, arguments: args, id} = func;
if (!handledFunctions.includes(name)) {
await this.parent.sendToolHook(id, {name, args: JSON.parse(args)});
}
}
} catch (err) {
this.logger.info({err, evt}, 'TaskLlmVoiceAgent_S2S:_onServerEvent - error calling function');
this.results = {
completionReason: 'client error calling function',
error: err
};
endConversation = true;
}
}
}
/* check whether we should notify on this event */
if (this.includeEvents.length > 0 ? this.includeEvents.includes(type) : !this.excludeEvents.includes(type)) {
this.parent.sendEventHook(evt)
.catch((err) => this.logger.info({err}, 'TaskLlmVoiceAgent_S2S:_onServerEvent - error sending event hook'));
}
if (endConversation) {
this.logger.info({results: this.results},
'TaskLlmVoiceAgent_S2S:_onServerEvent - ending conversation due to error');
this.notifyTaskDone();
}
}
_populateEvents(events) {
if (events.includes('all')) {
/* work by excluding specific events */
const exclude = events
.filter((evt) => evt.startsWith('-'))
.map((evt) => evt.slice(1));
if (exclude.length === 0) this.includeEvents = va_server_events;
else this.excludeEvents = expandWildcards(exclude);
}
else {
/* work by including specific events */
const include = events
.filter((evt) => !evt.startsWith('-'));
this.includeEvents = expandWildcards(include);
}
this.logger.debug({
includeEvents: this.includeEvents,
excludeEvents: this.excludeEvents
}, 'TaskLlmVoiceAgent_S2S:_populateEvents');
}
}
module.exports = TaskLlmVoiceAgent_S2S;

View File

@@ -14,9 +14,6 @@ function makeTask(logger, obj, parent) {
}
validateVerb(name, data, logger);
switch (name) {
case TaskName.Answer:
const TaskAnswer = require('./answer');
return new TaskAnswer(logger, data, parent);
case TaskName.SipDecline:
const TaskSipDecline = require('./sip_decline');
return new TaskSipDecline(logger, data, parent);
@@ -44,9 +41,6 @@ function makeTask(logger, obj, parent) {
case TaskName.Dtmf:
const TaskDtmf = require('./dtmf');
return new TaskDtmf(logger, data, parent);
case TaskName.Dub:
const TaskDub = require('./dub');
return new TaskDub(logger, data, parent);
case TaskName.Enqueue:
const TaskEnqueue = require('./enqueue');
return new TaskEnqueue(logger, data, parent);
@@ -62,9 +56,6 @@ function makeTask(logger, obj, parent) {
case TaskName.Message:
const TaskMessage = require('./message');
return new TaskMessage(logger, data, parent);
case TaskName.Llm:
const TaskLlm = require('./llm');
return new TaskLlm(logger, data, parent);
case TaskName.Rasa:
const TaskRasa = require('./rasa');
return new TaskRasa(logger, data, parent);
@@ -84,7 +75,6 @@ function makeTask(logger, obj, parent) {
const TaskTranscribe = require('./transcribe');
return new TaskTranscribe(logger, data, parent);
case TaskName.Listen:
case TaskName.Stream:
const TaskListen = require('./listen');
return new TaskListen(logger, data, parent);
case TaskName.Redirect:

View File

@@ -2,7 +2,7 @@ const Task = require('./task');
const {TaskName, TaskPreconditions} = require('../utils/constants');
const bent = require('bent');
const uuidv4 = require('uuid-random');
const {K8S} = require('../config');
class TaskMessage extends Task {
constructor(logger, opts) {
super(logger, opts);
@@ -42,7 +42,7 @@ class TaskMessage extends Task {
}
if (gw) {
this.logger.info({gw, accountSid}, 'Message:exec - using smpp to send message');
url = K8S ? 'http://smpp' : getSmpp();
url = process.env.K8S ? 'http://smpp' : getSmpp();
relativeUrl = '/sms';
payload = {
...payload,

View File

@@ -1,6 +1,6 @@
const Task = require('./task');
const {TaskName, TaskPreconditions} = require('../utils/constants');
const { PlayFileNotFoundError } = require('../utils/error');
class TaskPlay extends Task {
constructor(logger, opts) {
super(logger, opts);
@@ -26,7 +26,6 @@ class TaskPlay extends Task {
let playbackSeconds = 0;
let playbackMilliseconds = 0;
let completed = !(this.timeoutSecs > 0 || this.loop);
cs.playingAudio = true;
if (this.timeoutSecs > 0) {
timeout = setTimeout(async() => {
completed = true;
@@ -40,22 +39,6 @@ class TaskPlay extends Task {
try {
this.notifyStatus({event: 'start-playback'});
while (!this.killed && (this.loop === 'forever' || this.loop--) && this.ep.connected) {
/* Listen for playback-start event and set up a one-time listener for uuid_break
* that will kill the audio playback if the taskIds match. This ensures that
* we only kill the currently playing audio and not audio from other tasks.
* As we are using stickyEventEmitter, even if the event is emitted before the listener is registered,
* the listener will receive the most recent event.
*/
ep.once('playback-start', (evt) => {
this.logger.debug({evt}, 'Play got playback-start');
this.cs.stickyEventEmitter.once('uuid_break', (t) => {
if (t?.taskId === this.taskId) {
this.logger.debug(`Play got kill-playback, executing uuid_break, taskId: ${t?.taskId}`);
this.ep.api('uuid_break', this.ep.uuid).catch((err) => this.logger.info(err, 'Error killing audio'));
this.notifyStatus({event: 'kill-playback'});
}
});
});
if (cs.isInConference) {
const {memberId, confName, confUuid} = cs;
if (Array.isArray(this.url)) {
@@ -83,35 +66,23 @@ class TaskPlay extends Task {
}
}
} catch (err) {
this.logger.info(`TaskPlay:exec - error playing ${this.url}: ${err.message}`);
this.playComplete = true;
if (err.message === 'File Not Found') {
const {writeAlerts, AlertType} = cs.srf.locals;
await this.performAction({status: 'fail', reason: 'playFailed'}, !(this.parentTask || cs.isConfirmCallSession));
this.emit('playDone');
writeAlerts({
account_sid: cs.accountSid,
alert_type: AlertType.PLAY_FILENOTFOUND,
url: this.url,
target_sid: cs.callSid
});
throw new PlayFileNotFoundError(this.url);
}
if (timeout) clearTimeout(timeout);
this.logger.info(err, `TaskPlay:exec - error playing ${this.url}`);
}
this.emit('playDone');
}
async kill(cs) {
super.kill(cs);
if (this.ep?.connected && !this.playComplete) {
if (this.ep.connected && !this.playComplete) {
this.logger.debug('TaskPlay:kill - killing audio');
if (cs.isInConference) {
const {memberId, confName} = cs;
this.killPlayToConfMember(this.ep, memberId, confName);
}
else {
//this.ep.api('uuid_break', this.ep.uuid).catch((err) => this.logger.info(err, 'Error killing audio'));
cs.stickyEventEmitter.emit('uuid_break', this);
this.notifyStatus({event: 'kill-playback'});
this.ep.api('uuid_break', this.ep.uuid).catch((err) => this.logger.info(err, 'Error killing audio'));
}
}
}

View File

@@ -1,8 +1,5 @@
const Task = require('./task');
const {TaskName} = require('../utils/constants');
const WsRequestor = require('../utils/ws-requestor');
const URL = require('url');
const HttpRequestor = require('../utils/http-requestor');
/**
* Redirects to a new application
@@ -16,32 +13,6 @@ class TaskRedirect extends Task {
async exec(cs) {
await super.exec(cs);
if (cs.requestor instanceof WsRequestor && cs.application.requestor._isAbsoluteUrl(this.actionHook)) {
this.logger.info(`Task:performAction redirecting to ${this.actionHook}, requires new ws connection`);
try {
this.cs.requestor.close();
const requestor = new WsRequestor(this.logger, cs.accountSid, {url: this.actionHook}, this.webhook_secret) ;
this.cs.application.requestor = requestor;
} catch (err) {
this.logger.info(err, `Task:performAction error redirecting to ${this.actionHook}`);
}
} else if (cs.application.requestor._isAbsoluteUrl(this.actionHook)) {
const baseUrl = this.cs.application.requestor.baseUrl;
const newUrl = URL.parse(this.actionHook);
const newBaseUrl = newUrl.protocol + '//' + newUrl.host;
if (baseUrl != newBaseUrl) {
try {
this.logger.info(`Task:redirect updating base url to ${newBaseUrl}`);
const newRequestor = new HttpRequestor(this.logger, cs.accountSid, {url: this.actionHook},
cs.accountInfo.account.webhook_secret);
this.cs.requestor.removeAllListeners();
this.cs.application.requestor = newRequestor;
} catch (err) {
this.logger.info(err, `Task:redirect error updating base url to ${this.actionHook}`);
}
}
}
await this.performAction();
}
}

View File

@@ -11,14 +11,10 @@ class TaskRestDial extends Task {
super(logger, opts);
this.from = this.data.from;
this.callerName = this.data.callerName;
this.timeLimit = this.data.timeLimit;
this.fromHost = this.data.fromHost;
this.to = this.data.to;
this.call_hook = this.data.call_hook;
this.timeout = this.data.timeout || 60;
this.sipRequestWithinDialogHook = this.data.sipRequestWithinDialogHook;
this.referHook = this.data.referHook;
this.on('connect', this._onConnect.bind(this));
this.on('callStatus', this._onCallStatus.bind(this));
@@ -26,38 +22,23 @@ class TaskRestDial extends Task {
get name() { return TaskName.RestDial; }
set appJson(app_json) {
this.app_json = app_json;
}
/**
* INVITE has just been sent at this point
*/
async exec(cs) {
await super.exec(cs);
this.cs = cs;
this.canCancel = true;
if (this.data.amd) {
this.startAmd = cs.startAmd;
this.on('amd', this._onAmdEvent.bind(this, cs));
}
this.stopAmd = cs.stopAmd;
this._setCallTimer();
await this.awaitTaskDone();
}
turnOffAmd() {
if (this.callSession.ep && this.callSession.ep.amd) this.stopAmd(this.callSession.ep, this);
}
kill(cs) {
super.kill(cs);
this._clearCallTimer();
if (this.canCancel) {
if (this.canCancel && cs?.req) {
this.canCancel = false;
cs?.req?.cancel();
cs.req.cancel();
}
this.notifyTaskDone();
}
@@ -66,46 +47,25 @@ class TaskRestDial extends Task {
this.canCancel = false;
const cs = this.callSession;
cs.setDialog(dlg);
cs.referHook = this.referHook;
if (this.timeLimit) {
cs.startMaxCallDurationTimer(this.timeLimit);
}
this.logger.debug('TaskRestDial:_onConnect - call connected');
if (this.sipRequestWithinDialogHook) this._initSipRequestWithinDialogHandler(cs, dlg);
try {
const b3 = this.getTracingPropagation();
const httpHeaders = b3 && {b3};
const params = {
...(cs.callInfo.toJSON()),
...cs.callInfo,
defaults: {
synthesizer: {
vendor: cs.speechSynthesisVendor,
language: cs.speechSynthesisLanguage,
voice: cs.speechSynthesisVoice,
label: cs.speechSynthesisLabel,
voice: cs.speechSynthesisVoice
},
recognizer: {
vendor: cs.speechRecognizerVendor,
language: cs.speechRecognizerLanguage,
label: cs.speechRecognizerLabel,
language: cs.speechRecognizerLanguage
}
}
};
if (this.startAmd) {
try {
this.startAmd(this.callSession, this.callSession.ep, this, this.data.amd);
} catch (err) {
this.logger.info({err}, 'Rest:dial:Call established - Error calling startAmd');
}
}
let tasks;
if (this.app_json) {
this.logger.debug('TaskRestDial: using app_json from task data');
tasks = JSON.parse(this.app_json);
} else {
this.logger.debug({call_hook: this.call_hook}, 'TaskRestDial: retrieving application');
tasks = await cs.requestor.request('session:new', this.call_hook, params, httpHeaders);
}
const tasks = await cs.requestor.request('session:new', this.call_hook, params, httpHeaders);
if (tasks && Array.isArray(tasks)) {
this.logger.debug({tasks: tasks}, `TaskRestDial: replacing application with ${tasks.length} tasks`);
cs.replaceApplication(normalizeJambones(this.logger, tasks).map((tdata) => makeTask(this.logger, tdata)));
@@ -137,29 +97,7 @@ class TaskRestDial extends Task {
_onCallTimeout() {
this.logger.debug('TaskRestDial: timeout expired without answer, killing task');
this.timer = null;
if (this.canCancel) {
this.canCancel = false;
this.cs?.req?.cancel();
}
}
_onAmdEvent(cs, evt) {
this.logger.info({evt}, 'Rest:dial:_onAmdEvent');
const {actionHook} = this.data.amd;
this.performHook(cs, actionHook, evt)
.catch((err) => {
this.logger.error({err}, 'Rest:dial:_onAmdEvent - error calling actionHook');
});
}
_initSipRequestWithinDialogHandler(cs, dlg) {
cs.sipRequestWithinDialogHook = this.sipRequestWithinDialogHook;
dlg.on('info', this._onRequestWithinDialog.bind(this, cs));
dlg.on('message', this._onRequestWithinDialog.bind(this, cs));
}
async _onRequestWithinDialog(cs, req, res) {
cs._onRequestWithinDialog(req, res);
this.kill();
}
}

View File

@@ -1,8 +1,6 @@
const assert = require('assert');
const TtsTask = require('./tts-task');
const Task = require('./task');
const {TaskName, TaskPreconditions} = require('../utils/constants');
const pollySSMLSplit = require('polly-ssml-split');
const { SpeechCredentialError } = require('../utils/error');
const breakLengthyTextIfNeeded = (logger, text) => {
const chunkSize = 1000;
@@ -25,387 +23,174 @@ const breakLengthyTextIfNeeded = (logger, text) => {
}
};
const parseTextFromSayString = (text) => {
const closingBraceIndex = text.indexOf('}');
if (closingBraceIndex === -1) return text;
return text.slice(closingBraceIndex + 1);
};
class TaskSay extends TtsTask {
class TaskSay extends Task {
constructor(logger, opts, parentTask) {
super(logger, opts, parentTask);
super(logger, opts);
this.preconditions = TaskPreconditions.Endpoint;
assert.ok((typeof this.data.text === 'string' || Array.isArray(this.data.text)) || this.data.stream === true,
'Say: either text or stream:true is required');
this.text = (Array.isArray(this.data.text) ? this.data.text : [this.data.text])
.map((t) => breakLengthyTextIfNeeded(this.logger, t))
.flat();
if (this.data.stream === true) {
this._isStreamingTts = true;
this.closeOnStreamEmpty = this.data.closeOnStreamEmpty !== false;
}
else {
this._isStreamingTts = false;
this.text = (Array.isArray(this.data.text) ? this.data.text : [this.data.text])
.map((t) => breakLengthyTextIfNeeded(this.logger, t))
.flat();
this.loop = this.data.loop || 1;
this.isHandledByPrimaryProvider = true;
}
this.loop = this.data.loop || 1;
this.earlyMedia = this.data.earlyMedia === true || (parentTask && parentTask.earlyMedia);
this.synthesizer = this.data.synthesizer || {};
this.disableTtsCache = this.data.disableTtsCache;
}
get name() { return TaskName.Say; }
get summary() {
if (this.isStreamingTts) return `${this.name} streaming`;
else {
for (let i = 0; i < this.text.length; i++) {
if (this.text[i].startsWith('silence_stream')) continue;
return `${this.name}{text=${this.text[i].slice(0, 15)}${this.text[i].length > 15 ? '...' : ''}}`;
}
return `${this.name}{${this.text[0]}}`;
for (let i = 0; i < this.text.length; i++) {
if (this.text[i].startsWith('silence_stream')) continue;
return `${this.name}{text=${this.text[i].slice(0, 15)}${this.text[i].length > 15 ? '...' : ''}}`;
}
return `${this.name}{${this.text[0]}}`;
}
get isStreamingTts() { return this._isStreamingTts; }
_validateURL(urlString) {
try {
new URL(urlString);
return true;
} catch (e) {
return false;
}
}
async exec(cs, obj) {
if (this.isStreamingTts && !cs.appIsUsingWebsockets) {
throw new Error('Say: streaming say verb requires applications to use the websocket API');
}
try {
if (this.isStreamingTts) await this.handlingStreaming(cs, obj);
else await this.handling(cs, obj);
this.emit('playDone');
} catch (error) {
if (error instanceof SpeechCredentialError) {
// if say failed due to speech credentials, alarm is writtern and error notification is sent
// finished this say to move to next task.
this.logger.info({error}, 'Say failed due to SpeechCredentialError, finished!');
this.emit('playDone');
return;
}
throw error;
}
}
async handlingStreaming(cs, {ep}) {
const {vendor, language, voice, label} = this.getTtsVendorData(cs);
const credentials = cs.getSpeechCredentials(vendor, 'tts', label);
if (!credentials) {
throw new SpeechCredentialError(
`No text-to-speech service credentials for ${vendor} with labels: ${label} have been configured`);
}
this.ep = ep;
try {
await this.setTtsStreamingChannelVars(vendor, language, voice, credentials, ep);
await cs.startTtsStream();
cs.requestor?.request('tts:streaming-event', '/streaming-event', {event_type: 'stream_open'})
.catch((err) => this.logger.info({err}, 'TaskSay:handlingStreaming - Error sending'));
} catch (err) {
this.logger.info({err}, 'TaskSay:handlingStreaming - Error setting channel vars');
cs.requestor?.request('tts:streaming-event', '/streaming-event', {event_type: 'stream_closed'})
.catch((err) => this.logger.info({err}, 'TaskSay:handlingStreaming - Error sending'));
//TODO: send tts:streaming-event with error?
this.notifyTaskDone();
}
await this.awaitTaskDone();
this.logger.info('TaskSay:handlingStreaming - done');
}
async handling(cs, {ep}) {
const {srf, accountSid:account_sid, callSid:target_sid} = cs;
const {writeAlerts, AlertType} = srf.locals;
const {addFileToCache} = srf.locals.dbHelpers;
const engine = this.synthesizer.engine || cs.synthesizer?.engine || 'neural';
async exec(cs, {ep}) {
await super.exec(cs);
this.ep = ep;
let vendor = this.synthesizer.vendor && this.synthesizer.vendor !== 'default' ?
const {srf} = cs;
const {updateSpeechCredentialLastUsed} = require('../utils/db-utils')(this.logger, srf);
const {writeAlerts, AlertType, stats} = srf.locals;
const {synthAudio} = srf.locals.dbHelpers;
const vendor = this.synthesizer.vendor && this.synthesizer.vendor !== 'default' ?
this.synthesizer.vendor :
cs.speechSynthesisVendor;
let language = this.synthesizer.language && this.synthesizer.language !== 'default' ?
const language = this.synthesizer.language && this.synthesizer.language !== 'default' ?
this.synthesizer.language :
cs.speechSynthesisLanguage ;
let voice = this.synthesizer.voice && this.synthesizer.voice !== 'default' ?
this.synthesizer.voice :
cs.speechSynthesisVoice;
let label = this.taskIncludeSynthesizer ? this.synthesizer.label : cs.speechSynthesisLabel;
const engine = this.synthesizer.engine || 'standard';
const salt = cs.callSid;
const credentials = cs.getSpeechCredentials(vendor, 'tts');
const fallbackVendor = this.synthesizer.fallbackVendor && this.synthesizer.fallbackVendor !== 'default' ?
this.synthesizer.fallbackVendor :
cs.fallbackSpeechSynthesisVendor;
const fallbackLanguage = this.synthesizer.fallbackLanguage && this.synthesizer.fallbackLanguage !== 'default' ?
this.synthesizer.fallbackLanguage :
cs.fallbackSpeechSynthesisLanguage ;
const fallbackVoice = this.synthesizer.fallbackVoice && this.synthesizer.fallbackVoice !== 'default' ?
this.synthesizer.fallbackVoice :
cs.fallbackSpeechSynthesisVoice;
const fallbackLabel = this.taskIncludeSynthesizer ?
this.synthesizer.fallbackLabel : cs.fallbackSpeechSynthesisLabel;
if (cs.hasFallbackTts) {
vendor = fallbackVendor;
language = fallbackLanguage;
voice = fallbackVoice;
label = fallbackLabel;
}
const startFallback = async(error) => {
if (fallbackVendor && this.isHandledByPrimaryProvider && !cs.hasFallbackTts) {
this.notifyError(
{ msg: 'TTS error', details:`TTS vendor ${vendor} error: ${error}`, failover: 'in progress'});
this.isHandledByPrimaryProvider = false;
cs.hasFallbackTts = true;
this.logger.info(`Synthesize error, fallback to ${fallbackVendor}`);
filepath = await this._synthesizeWithSpecificVendor(cs, ep,
{
vendor: fallbackVendor,
language: fallbackLanguage,
voice: fallbackVoice,
label: fallbackLabel
});
} else {
this.notifyError(
{ msg: 'TTS error', details:`TTS vendor ${vendor} error: ${error}`, failover: 'not available'});
throw new SpeechCredentialError(error.message);
/* parse Nuance voices into name and model */
let model;
if (vendor === 'nuance' && voice) {
const arr = /([A-Za-z-]*)\s+-\s+(enhanced|standard)/.exec(voice);
if (arr) {
voice = arr[1];
model = arr[2];
}
};
let filepath;
try {
filepath = await this._synthesizeWithSpecificVendor(cs, ep, {vendor, language, voice, label});
} catch (error) {
await startFallback(error);
}
this.notifyStatus({event: 'start-playback'});
while (!this.killed && (this.loop === 'forever' || this.loop--) && ep?.connected) {
let segment = 0;
while (!this.killed && segment < filepath.length) {
if (cs.isInConference) {
const {memberId, confName, confUuid} = cs;
await this.playToConfMember(ep, memberId, confName, confUuid, filepath[segment]);
this.logger.info({vendor, language, voice, model}, 'TaskSay:exec');
this.ep = ep;
try {
if (!credentials) {
writeAlerts({
account_sid: cs.accountSid,
alert_type: AlertType.TTS_NOT_PROVISIONED,
vendor
}).catch((err) => this.logger.info({err}, 'Error generating alert for no tts'));
this.notifyError({
msg: 'TTS error',
details:`No speech credentials provisioned for selected vendor ${vendor}`
});
throw new Error('no provisioned speech credentials for TTS');
}
// synthesize all of the text elements
let lastUpdated = false;
/* produce an audio segment from the provided text */
const generateAudio = async(text) => {
if (this.killed) return;
if (text.startsWith('silence_stream://')) return text;
/* otel: trace time for tts */
const {span} = this.startChildSpan('tts-generation', {
'tts.vendor': vendor,
'tts.language': language,
'tts.voice': voice
});
try {
const {filePath, servedFromCache, rtt} = await synthAudio(stats, {
text,
vendor,
language,
voice,
engine,
model,
salt,
credentials,
disableTtsCache : this.disableTtsCache
});
this.logger.debug(`file ${filePath}, served from cache ${servedFromCache}`);
if (filePath) cs.trackTmpFile(filePath);
if (!servedFromCache && !lastUpdated) {
lastUpdated = true;
updateSpeechCredentialLastUsed(credentials.speech_credential_sid)
.catch(() => {/*already logged error */});
}
span.setAttributes({'tts.cached': servedFromCache});
span.end();
if (!servedFromCache && rtt) {
this.notifyStatus({
event: 'synthesized-audio',
vendor,
language,
characters: text.length,
elapsedTime: rtt
});
}
return filePath;
} catch (err) {
this.logger.info({err}, 'Error synthesizing tts');
span.end();
writeAlerts({
account_sid: cs.accountSid,
alert_type: AlertType.TTS_FAILURE,
vendor,
detail: err.message
}).catch((err) => this.logger.info({err}, 'Error generating alert for tts failure'));
this.notifyError({msg: 'TTS error', details: err.message || err});
return;
}
else {
const isStreaming = filepath[segment].startsWith('say:{');
if (isStreaming) {
const arr = /^say:\{.*\}\s*(.*)$/.exec(filepath[segment]);
if (arr) this.logger.debug(`Say:exec sending streaming tts request: ${arr[1].substring(0, 64)}..`);
}
else this.logger.debug(`Say:exec sending ${filepath[segment].substring(0, 64)}`);
ep.once('playback-start', (evt) => {
this.logger.debug({evt}, 'Say got playback-start');
if (this.otelSpan) {
this._addStreamingTtsAttributes(this.otelSpan, evt, vendor);
this.otelSpan.end();
this.otelSpan = null;
if (evt.variable_tts_cache_filename) {
cs.trackTmpFile(evt.variable_tts_cache_filename);
}
}
});
ep.once('playback-stop', (evt) => {
this.logger.debug({evt}, 'Say got playback-stop');
this.notifyStatus({event: 'stop-playback'});
this.notifiedPlayBackStop = true;
const tts_error = evt.variable_tts_error;
let response_code = 200;
// Check if any property ends with _response_code
for (const [key, value] of Object.entries(evt)) {
if (key.endsWith('_response_code')) {
response_code = parseInt(value, 10) || 200;
break;
}
}
};
if (tts_error) {
writeAlerts({
account_sid,
alert_type: AlertType.TTS_FAILURE,
vendor,
detail: evt.variable_tts_error,
target_sid
}).catch((err) => this.logger.info({err}, 'Error generating alert for no tts'));
}
if (!tts_error && response_code < 300 && evt.variable_tts_cache_filename && !this.killed) {
const text = parseTextFromSayString(this.text[segment]);
addFileToCache(evt.variable_tts_cache_filename, {
account_sid,
vendor,
language,
voice,
engine,
model: this.model || this.model_id,
text,
instructions: this.instructions
}).catch((err) => this.logger.info({err}, 'Error adding file to cache'));
}
const arr = this.text.map((t) => generateAudio(t));
const filepath = (await Promise.all(arr)).filter((fp) => fp && fp.length);
this.notifyStatus({event: 'start-playback'});
if (this._playResolve) {
(tts_error || response_code >= 300) ? this._playReject(new Error(evt.variable_tts_error)) :
this._playResolve();
}
});
// wait for playback-stop event received to confirm if the playback is successful
this._playPromise = new Promise((resolve, reject) => {
this._playResolve = resolve;
this._playReject = reject;
});
const r = await ep.play(filepath[segment]);
this.logger.debug({r}, 'Say:exec play result');
try {
// wait for playback-stop event received to confirm if the playback is successful
await this._playPromise;
} catch (err) {
try {
await startFallback(err);
continue;
} catch (err) {
this.logger.info({err}, 'Error waiting for playback-stop event');
throw err;
}
} finally {
this._playPromise = null;
this._playResolve = null;
this._playReject = null;
while (!this.killed && (this.loop === 'forever' || this.loop--) && this.ep?.connected) {
let segment = 0;
while (!this.killed && segment < filepath.length) {
if (cs.isInConference) {
const {memberId, confName, confUuid} = cs;
await this.playToConfMember(this.ep, memberId, confName, confUuid, filepath[segment]);
}
if (filepath[segment].startsWith('say:{')) {
const arr = /^say:\{.*\}\s*(.*)$/.exec(filepath[segment]);
if (arr) this.logger.debug(`Say:exec complete playing streaming tts request: ${arr[1].substring(0, 64)}..`);
} else {
// This log will print spech credentials in say command for tts stream mode
else {
this.logger.debug(`Say:exec sending command to play file ${filepath[segment]}`);
await ep.play(filepath[segment]);
this.logger.debug(`Say:exec completed play file ${filepath[segment]}`);
}
segment++;
}
segment++;
}
} catch (err) {
this.logger.info(err, 'TaskSay:exec error');
}
this.emit('playDone');
}
async kill(cs) {
super.kill(cs);
if (this.ep?.connected) {
if (this.ep.connected) {
this.logger.debug('TaskSay:kill - killing audio');
if (cs.isInConference) {
const {memberId, confName} = cs;
this.killPlayToConfMember(this.ep, memberId, confName);
} else if (this.isStreamingTts) {
this.logger.debug('TaskSay:kill - clearing TTS stream for streaming audio');
cs.clearTtsStream();
} else {
if (!this.notifiedPlayBackStop) {
this.notifyStatus({event: 'stop-playback'});
}
}
else {
this.notifyStatus({event: 'kill-playback'});
this.ep.api('uuid_break', this.ep.uuid);
}
this.ep.removeAllListeners('playback-start');
this.ep.removeAllListeners('playback-stop');
// if we are waiting for playback-stop event, resolve the promise
if (this._playResolve) {
this._playResolve();
this._playResolve = null;
}
}
this.notifyTaskDone();
}
_addStreamingTtsAttributes(span, evt, vendor) {
const attrs = {'tts.cached': false};
for (const [key, value] of Object.entries(evt)) {
if (key.startsWith('variable_tts_')) {
let newKey = key.substring('variable_tts_'.length)
.replace('whisper_', 'whisper.')
.replace('nvidia_', 'nvidia.')
.replace('deepgram_', 'deepgram.')
.replace('playht_', 'playht.')
.replace('cartesia_', 'cartesia.')
.replace('rimelabs_', 'rimelabs.')
.replace('verbio_', 'verbio.')
.replace('elevenlabs_', 'elevenlabs.');
if (spanMapping[newKey]) newKey = spanMapping[newKey];
attrs[newKey] = value;
if (key === 'variable_tts_time_to_first_byte_ms' && value) {
this.cs.srf.locals.stats.histogram('tts.response_time', value, [`vendor:${vendor}`]);
}
}
}
delete attrs['cache_filename']; //no value in adding this to the span
span.setAttributes(attrs);
}
notifyTtsStreamIsEmpty() {
if (this.isStreamingTts && this.closeOnStreamEmpty) {
this.logger.info('TaskSay:notifyTtsStreamIsEmpty - stream is empty, killing task');
this.notifyTaskDone();
}
}
}
const spanMapping = {
// IMPORTANT!!! JAMBONZ WEBAPP WILL SHOW TEXT PERFECTLY IF THE SPAN NAME IS SMALLER OR EQUAL 25 CHARACTERS.
// EX: whisper.ratelim_reqs has length 20 <= 25 which is perfect
// Elevenlabs
'elevenlabs.reported_latency_ms': 'elevenlabs.latency_ms',
'elevenlabs.request_id': 'elevenlabs.req_id',
'elevenlabs.history_item_id': 'elevenlabs.item_id',
'elevenlabs.optimize_streaming_latency': 'elevenlabs.optimization',
'elevenlabs.name_lookup_time_ms': 'name_lookup_ms',
'elevenlabs.connect_time_ms': 'connect_ms',
'elevenlabs.final_response_time_ms': 'final_response_ms',
// Whisper
'whisper.reported_latency_ms': 'whisper.latency_ms',
'whisper.request_id': 'whisper.req_id',
'whisper.reported_organization': 'whisper.organization',
'whisper.reported_ratelimit_requests': 'whisper.ratelimit',
'whisper.reported_ratelimit_remaining_requests': 'whisper.ratelimit_remain',
'whisper.reported_ratelimit_reset_requests': 'whisper.ratelimit_reset',
'whisper.name_lookup_time_ms': 'name_lookup_ms',
'whisper.connect_time_ms': 'connect_ms',
'whisper.final_response_time_ms': 'final_response_ms',
// Deepgram
'deepgram.request_id': 'deepgram.req_id',
'deepgram.reported_model_name': 'deepgram.model_name',
'deepgram.reported_model_uuid': 'deepgram.model_uuid',
'deepgram.reported_char_count': 'deepgram.char_count',
'deepgram.name_lookup_time_ms': 'name_lookup_ms',
'deepgram.connect_time_ms': 'connect_ms',
'deepgram.final_response_time_ms': 'final_response_ms',
// Playht
'playht.request_id': 'playht.req_id',
'playht.name_lookup_time_ms': 'name_lookup_ms',
'playht.connect_time_ms': 'connect_ms',
'playht.final_response_time_ms': 'final_response_ms',
// Cartesia
'cartesia.request_id': 'cartesia.req_id',
'cartesia.name_lookup_time_ms': 'name_lookup_ms',
'cartesia.connect_time_ms': 'connect_ms',
'cartesia.final_response_time_ms': 'final_response_ms',
// Rimelabs
'rimelabs.name_lookup_time_ms': 'name_lookup_ms',
'rimelabs.connect_time_ms': 'connect_ms',
'rimelabs.final_response_time_ms': 'final_response_ms',
// verbio
'verbio.name_lookup_time_ms': 'name_lookup_ms',
'verbio.connect_time_ms': 'connect_ms',
'verbio.final_response_time_ms': 'final_response_ms',
};
module.exports = TaskSay;

View File

@@ -18,11 +18,6 @@ class TaskSipDecline extends Task {
super.exec(cs);
res.send(this.data.status, this.data.reason, {
headers: this.headers
}, (err) => {
if (!err) {
// Call was successfully declined
cs._callReleased();
}
});
cs.emit('callStatusChange', {
callStatus: CallStatus.Failed,

View File

@@ -12,7 +12,6 @@ class TaskSipRefer extends Task {
this.referTo = this.data.referTo;
this.referredBy = this.data.referredBy;
this.referredByDisplayName = this.data.referredByDisplayName;
this.headers = this.data.headers || {};
this.eventHook = this.data.eventHook;
}
@@ -95,10 +94,7 @@ class TaskSipRefer extends Task {
}
if (status >= 200) {
this.referSpan.setAttributes({'refer.finalNotify': status});
await this.performAction({refer_status: 202, final_referred_call_status: status})
.catch((err) => {
this.logger.error(err, 'TaskSipRefer:exec - error performing action finalNotify');
});
await this.performAction({refer_status: 202, final_referred_call_status: status});
this.notifyTaskDone();
}
}
@@ -106,7 +102,7 @@ class TaskSipRefer extends Task {
}
_normalizeReferHeaders(cs, dlg) {
let {referTo, referredBy, referredByDisplayName} = this;
let {referTo, referredBy} = this;
/* get IP address of the SBC to use as hostname if needed */
const {host} = parseUri(dlg.remote.uri);
@@ -121,12 +117,9 @@ class TaskSipRefer extends Task {
referredBy = cs.req?.callingNumber || dlg.local.uri;
this.logger.info({referredBy}, 'setting referredby');
}
if (!referredByDisplayName) {
referredByDisplayName = cs.req?.callingName;
}
if (!referredBy.startsWith('<') && !referredBy.startsWith('sip') && !referredBy.startsWith('"')) {
/* they may have only provided a phone number/user */
referredBy = `${referredByDisplayName ? `"${referredByDisplayName}"` : ''}<sip:${referredBy}@${host}>`;
referredBy = `sip:${referredBy}@${host}`;
}
return {referTo, referredBy};
}

View File

@@ -1,434 +0,0 @@
const Task = require('./task');
const assert = require('assert');
const crypto = require('crypto');
const { TaskPreconditions, CobaltTranscriptionEvents } = require('../utils/constants');
const { SpeechCredentialError } = require('../utils/error');
const {JAMBONES_AWS_TRANSCRIBE_USE_GRPC} = require('../config');
/**
* "Please insert turns here: {{turns:4}}"
// -> { processed: 'Please insert turns here: {{turns}}', turns: 4 }
processTurnString("Please insert turns here: {{turns}}"));
// -> { processed: 'Please insert turns here: {{turns}}', turns: null }
*/
const processTurnString = (input) => {
const regex = /\{\{turns(?::(\d+))?\}\}/;
const match = input.match(regex);
if (!match) {
return {
processed: input,
turns: null
};
}
const turns = match[1] ? parseInt(match[1], 10) : null;
const processed = input.replace(regex, '{{turns}}');
return { processed, turns };
};
class SttTask extends Task {
constructor(logger, data, parentTask) {
super(logger, data);
this.parentTask = parentTask;
this.preconditions = TaskPreconditions.Endpoint;
const {
setChannelVarsForStt,
normalizeTranscription,
setSpeechCredentialsAtRuntime,
compileSonioxTranscripts,
consolidateTranscripts,
updateSpeechmaticsPayload
} = require('../utils/transcription-utils')(logger);
this.setChannelVarsForStt = setChannelVarsForStt;
this.normalizeTranscription = normalizeTranscription;
this.compileSonioxTranscripts = compileSonioxTranscripts;
this.consolidateTranscripts = consolidateTranscripts;
this.updateSpeechmaticsPayload = updateSpeechmaticsPayload;
this.eventHandlers = [];
this.isHandledByPrimaryProvider = true;
/**
* Task use taskIncludeRecognizer to identify
* if taskIncludeRecognizer === true, use label from verb.recognizer, even it's empty
* if taskIncludeRecognizer === false, use label from application.recognizer
*/
this.taskIncludeRecognizer = !!this.data.recognizer;
if (this.data.recognizer) {
const recognizer = this.data.recognizer;
this.vendor = recognizer.vendor;
this.language = recognizer.language;
this.label = recognizer.label;
//fallback
this.fallbackVendor = recognizer.fallbackVendor || 'default';
this.fallbackLanguage = recognizer.fallbackLanguage || 'default';
this.fallbackLabel = recognizer.fallbackLabel;
/* let credentials be supplied in the recognizer object at runtime */
this.sttCredentials = setSpeechCredentialsAtRuntime(recognizer);
if (!Array.isArray(this.data.recognizer.altLanguages)) {
this.data.recognizer.altLanguages = [];
}
} else {
this.data.recognizer = {hints: [], altLanguages: []};
}
/* buffer for soniox transcripts */
this._sonioxTranscripts = [];
/*bug name prefix */
this.bugname_prefix = '';
}
async exec(cs, {ep, ep2}) {
super.exec(cs);
this.ep = ep;
this.ep2 = ep2;
// use session preferences if we don't have specific verb-level settings.
if (cs.recognizer) {
for (const k in cs.recognizer) {
const newValue = this.data.recognizer && this.data.recognizer[k] !== undefined ?
this.data.recognizer[k] :
cs.recognizer[k];
if (Array.isArray(newValue)) {
this.data.recognizer[k] = [...(this.data.recognizer[k] || []), ...cs.recognizer[k]];
} else if (typeof newValue === 'object' && newValue !== null) {
this.data.recognizer[k] = { ...(this.data.recognizer[k] || {}), ...cs.recognizer[k] };
} else {
this.data.recognizer[k] = newValue;
}
}
}
if ('default' === this.vendor || !this.vendor) {
this.vendor = cs.speechRecognizerVendor;
if (this.data.recognizer) this.data.recognizer.vendor = this.vendor;
}
if ('default' === this.language || !this.language) {
this.language = cs.speechRecognizerLanguage;
if (this.data.recognizer) this.data.recognizer.language = this.language;
}
if (!this.taskIncludeRecognizer) {
this.label = cs.speechRecognizerLabel;
if (this.data.recognizer) this.data.recognizer.label = this.label;
}
// Fallback options
if ('default' === this.fallbackVendor || !this.fallbackVendor) {
this.fallbackVendor = cs.fallbackSpeechRecognizerVendor;
if (this.data.recognizer) this.data.recognizer.fallbackVendor = this.fallbackVendor;
}
if ('default' === this.fallbackLanguage || !this.fallbackLanguage) {
this.fallbackLanguage = cs.fallbackSpeechRecognizerLanguage;
if (this.data.recognizer) this.data.recognizer.fallbackLanguage = this.fallbackLanguage;
}
if (!this.taskIncludeRecognizer) {
this.fallbackLabel = cs.fallbackSpeechRecognizerLabel;
if (this.data.recognizer) this.data.recognizer.fallbackLabel = this.fallbackLabel;
}
if (cs.hasFallbackAsr) {
if (this.taskIncludeRecognizer) {
// reset fallback ASR from previous run if this verb contains data.recognizer.
cs.hasFallbackAsr = false;
} else {
this.logger.debug('Call session has fallback to 2nd ASR, use 2nd recognizer configuration');
this.vendor = this.fallbackVendor;
this.language = this.fallbackLanguage;
this.label = this.fallbackLabel;
}
}
if (!this.data.recognizer.vendor) {
this.data.recognizer.vendor = this.vendor;
}
if (this.vendor === 'cobalt' && !this.data.recognizer.model) {
// By default, application saves cobalt model in language
this.data.recognizer.model = cs.speechRecognizerLanguage;
}
if (
// not gather task, such as transcribe
(!this.input ||
// gather task with speech
this.input.includes('speech')) &&
!this.sttCredentials) {
try {
this.sttCredentials = await this._initSpeechCredentials(this.cs, this.vendor, this.label);
} catch (error) {
if (this.canFallback) {
this.notifyError(
{
msg: 'ASR error', details:`Invalid vendor ${this.vendor}, Error: ${error}`,
failover: 'in progress'
});
await this._initFallback();
} else {
this.notifyError(
{
msg: 'ASR error', details:`Invalid vendor ${this.vendor}, Error: ${error}`,
failover: 'not available'
});
throw error;
}
}
}
/* when using cobalt model is required */
if (this.vendor === 'cobalt' && !this.data.recognizer.model) {
this.notifyError({ msg: 'ASR error', details:'Cobalt requires a model to be specified'});
throw new Error('Cobalt requires a model to be specified');
}
if (cs.hasAltLanguages) {
this.data.recognizer.altLanguages = this.data.recognizer.altLanguages.concat(cs.altLanguages);
this.logger.debug({altLanguages: this.altLanguages},
'STT:exec - applying altLanguages');
}
if (cs.hasGlobalSttPunctuation && !this.data.recognizer.punctuation) {
this.data.recognizer.punctuation = cs.globalSttPunctuation;
}
}
addCustomEventListener(ep, event, handler) {
this.eventHandlers.push({ep, event, handler});
ep.addCustomEventListener(event, handler);
}
removeCustomEventListeners() {
this.eventHandlers.forEach((h) => h.ep.removeCustomEventListener(h.event, h.handler));
}
async _initSpeechCredentials(cs, vendor, label) {
const {getNuanceAccessToken, getIbmAccessToken, getAwsAuthToken, getVerbioAccessToken} = cs.srf.locals.dbHelpers;
let credentials = cs.getSpeechCredentials(vendor, 'stt', label);
if (!credentials) {
const {writeAlerts, AlertType} = cs.srf.locals;
this.logger.info(`ERROR stt using ${vendor} requested but creds not supplied`);
writeAlerts({
account_sid: cs.accountSid,
alert_type: AlertType.STT_NOT_PROVISIONED,
vendor,
target_sid: cs.callSid
}).catch((err) => this.logger.info({err}, 'Error generating alert for no stt'));
// the ASR might have fallback configuration, should not done task here.
throw new SpeechCredentialError(`No speech-to-text service credentials for ${vendor} have been configured`);
}
if (vendor === 'nuance' && credentials.client_id) {
/* get nuance access token */
const {client_id, secret} = credentials;
const {access_token, servedFromCache} = await getNuanceAccessToken(client_id, secret, 'asr tts');
this.logger.debug({client_id}, `got nuance access token ${servedFromCache ? 'from cache' : ''}`);
credentials = {...credentials, access_token};
}
else if (vendor == 'ibm' && credentials.stt_api_key) {
/* get ibm access token */
const {stt_api_key, stt_region} = credentials;
const {access_token, servedFromCache} = await getIbmAccessToken(stt_api_key);
this.logger.debug({stt_api_key}, `got ibm access token ${servedFromCache ? 'from cache' : ''}`);
credentials = {...credentials, access_token, stt_region};
} else if (['aws', 'polly'].includes(vendor) && credentials.roleArn) {
/* get aws access token */
const {roleArn, region} = credentials;
const {accessKeyId, secretAccessKey, sessionToken, servedFromCache} =
await getAwsAuthToken({
region,
roleArn
});
this.logger.debug({roleArn}, `(roleArn) got aws access token ${servedFromCache ? 'from cache' : ''}`);
// from role ARN, we will get SessionToken, but feature server use it as securityToken.
credentials = {...credentials, accessKeyId, secretAccessKey, securityToken: sessionToken};
}
else if (vendor === 'verbio' && credentials.client_id && credentials.client_secret) {
const {access_token, servedFromCache} = await getVerbioAccessToken(credentials);
this.logger.debug({client_id: credentials.client_id},
`got verbio access token ${servedFromCache ? 'from cache' : ''}`);
credentials.access_token = access_token;
}
else if (vendor == 'aws' && !JAMBONES_AWS_TRANSCRIBE_USE_GRPC) {
/* get AWS access token */
const {speech_credential_sid, accessKeyId, secretAccessKey, securityToken, region } = credentials;
if (!securityToken) {
const { servedFromCache, ...newCredentials} = await getAwsAuthToken({
speech_credential_sid,
accessKeyId,
secretAccessKey,
region});
this.logger.debug({newCredentials}, `got aws security token ${servedFromCache ? 'from cache' : ''}`);
credentials = {...newCredentials, region};
}
}
return credentials;
}
get canFallback() {
return this.fallbackVendor && this.isHandledByPrimaryProvider && !this.cs.hasFallbackAsr;
}
async _initFallback() {
assert(this.fallbackVendor, 'fallback failed without fallbackVendor configuration');
this.logger.info(`Failed to use primary STT provider, fallback to ${this.fallbackVendor}`);
this.isHandledByPrimaryProvider = false;
this.cs.hasFallbackAsr = true;
this.vendor = this.cs.fallbackSpeechRecognizerVendor = this.fallbackVendor;
this.language = this.cs.fallbackSpeechRecognizerLanguage = this.fallbackLanguage;
this.label = this.cs.fallbackSpeechRecognizerLabel = this.fallbackLabel;
this.data.recognizer.vendor = this.vendor;
this.data.recognizer.language = this.language;
this.data.recognizer.label = this.label;
this.sttCredentials = await this._initSpeechCredentials(this.cs, this.vendor, this.label);
// cleanup previous listener from previous vendor
this.removeCustomEventListeners();
}
async compileHintsForCobalt(ep, hostport, model, token, hints) {
const {retrieveKey} = this.cs.srf.locals.dbHelpers;
const hash = crypto.createHash('sha1');
hash.update(`${model}:${hints}`);
const key = `cobalt:${hash.digest('hex')}`;
this.context = await retrieveKey(key);
if (this.context) {
this.logger.debug({model, hints}, 'found cached cobalt context for supplied hints');
return this.context;
}
this.logger.debug({model, hints}, 'compiling cobalt context for supplied hints');
return new Promise((resolve, reject) => {
this.cobaltCompileResolver = resolve;
ep.addCustomEventListener(CobaltTranscriptionEvents.CompileContext, this._onCompileContext.bind(this, ep, key));
ep.api('uuid_cobalt_compile_context', [ep.uuid, hostport, model, token, hints], (err, evt) => {
if (err || 0 !== evt.getBody().indexOf('+OK')) {
ep.removeCustomEventListener(CobaltTranscriptionEvents.CompileContext);
return reject(err);
}
});
});
}
formatOpenAIPrompt(cs, {prompt, hintsTemplate, conversationHistoryTemplate, hints}) {
let conversationHistoryPrompt, hintsPrompt;
/* generate conversation history from template */
if (conversationHistoryTemplate) {
const {processed, turns} = processTurnString(conversationHistoryTemplate);
this.logger.debug({processed, turns}, 'SttTask: processed conversation history template');
conversationHistoryPrompt = cs.getFormattedConversation(turns || 4);
//this.logger.debug({conversationHistoryPrompt}, 'SttTask: conversation history');
if (conversationHistoryPrompt) {
conversationHistoryPrompt = processed.replace('{{turns}}', `\n${conversationHistoryPrompt}\nuser: `);
}
}
/* generate hints from template */
if (hintsTemplate && Array.isArray(hints) && hints.length > 0) {
hintsPrompt = hintsTemplate.replace('{{hints}}', hints);
}
/* combine into final prompt */
let finalPrompt = prompt || '';
if (hintsPrompt) {
finalPrompt = `${finalPrompt}\n${hintsPrompt}`;
}
if (conversationHistoryPrompt) {
finalPrompt = `${finalPrompt}\n${conversationHistoryPrompt}`;
}
this.logger.debug({
finalPrompt,
hints,
hintsPrompt,
conversationHistoryTemplate,
conversationHistoryPrompt
}, 'SttTask: formatted OpenAI prompt');
return finalPrompt?.trimStart();
}
/* some STT engines will keep listening after a final response, so no need to restart */
doesVendorContinueListeningAfterFinalTranscript(vendor) {
return (vendor.startsWith('custom:') || [
'soniox',
'aws',
'microsoft',
'deepgram',
'google',
'speechmatics',
'openai',
].includes(vendor));
}
_onCompileContext(ep, key, evt) {
const {addKey} = this.cs.srf.locals.dbHelpers;
this.logger.debug({evt}, `received cobalt compile context event, will cache under ${key}`);
this.cobaltCompileResolver(evt.compiled_context);
ep.removeCustomEventListener(CobaltTranscriptionEvents.CompileContext);
this.cobaltCompileResolver = null;
//cache the compiled context
addKey(key, evt.compiled_context, 3600 * 12)
.catch((err) => this.logger.info({err}, `Error caching cobalt context for ${key}`));
}
_doContinuousAsrWithDeepgram(asrTimeout) {
/* deepgram has an utterance_end_ms property that simplifies things */
assert(this.vendor === 'deepgram');
if (asrTimeout < 1000) {
this.notifyError({
msg: 'ASR error',
details:`asrTimeout ${asrTimeout} is too short for deepgram; setting it to 1000ms`
});
asrTimeout = 1000;
}
else if (asrTimeout > 5000) {
this.notifyError({
msg: 'ASR error',
details:`asrTimeout ${asrTimeout} is too long for deepgram; setting it to 5000ms`
});
asrTimeout = 5000;
}
this.logger.debug(`_doContinuousAsrWithDeepgram - setting utterance_end_ms to ${asrTimeout}`);
const dgOptions = this.data.recognizer.deepgramOptions = this.data.recognizer.deepgramOptions || {};
dgOptions.utteranceEndMs = dgOptions.utteranceEndMs || asrTimeout;
}
_onVendorConnect(_cs, _ep) {
this.logger.debug(`TaskGather:_on${this.vendor}Connect`);
}
_onVendorError(cs, _ep, evt) {
this.logger.info({evt}, `${this.name}:_on${this.vendor}Error`);
const {writeAlerts, AlertType} = cs.srf.locals;
writeAlerts({
account_sid: cs.accountSid,
alert_type: AlertType.STT_FAILURE,
message: 'STT failure reported by vendor',
detail: evt.error,
vendor: this.vendor,
target_sid: cs.callSid
}).catch((err) => this.logger.info({err}, `Error generating alert for ${this.vendor} connection failure`));
}
_onVendorConnectFailure(cs, _ep, evt) {
const {reason} = evt;
const {writeAlerts, AlertType} = cs.srf.locals;
this.logger.info({evt}, `${this.name}:_on${this.vendor}ConnectFailure`);
writeAlerts({
account_sid: cs.accountSid,
alert_type: AlertType.STT_FAILURE,
message: `Failed connecting to ${this.vendor} speech recognizer: ${reason}`,
vendor: this.vendor,
target_sid: cs.callSid
}).catch((err) => this.logger.info({err}, `Error generating alert for ${this.vendor} connection failure`));
}
}
module.exports = SttTask;

View File

@@ -12,7 +12,7 @@ class TaskTag extends Task {
async exec(cs) {
super.exec(cs);
cs.callInfo.customerData = this.data;
this.logger.debug({customerData: cs.callInfo.customerData}, 'TaskTag:exec set customer data in callInfo');
//this.logger.debug({callInfo: cs.callInfo.toJSON()}, 'TaskTag:exec set customer data in callInfo');
}
}

View File

@@ -19,7 +19,6 @@ class Task extends Emitter {
this.data = data;
this.actionHook = this.data.actionHook;
this.id = data.id;
this.taskId = uuidv4();
this._killInProgress = false;
this._completionPromise = new Promise((resolve) => this._completionResolver = resolve);
@@ -46,10 +45,6 @@ class Task extends Emitter {
return this.name;
}
set disableTracing(val) {
this._disableTracing = val;
}
toJSON() {
return this.data;
}
@@ -160,38 +155,20 @@ class Task extends Emitter {
if (this.actionHook) {
const type = this.name === TaskName.Redirect ? 'session:redirect' : 'verb:hook';
const params = results ? Object.assign(this.cs.callInfo.toJSON(), results) : this.cs.callInfo.toJSON();
const span = this.startSpan(`${type} (${this.actionHook})`);
const span = this.startSpan(type, {'hook.url': this.actionHook});
const b3 = this.getTracingPropagation('b3', span);
const httpHeaders = b3 && {b3};
span.setAttributes({'http.body': JSON.stringify(params)});
try {
if (this.id) params.verb_id = this.id;
const json = await this.cs.requestor.request(type, this.actionHook, params, httpHeaders);
span.setAttributes({'http.statusCode': 200});
const isWsConnection = this.cs.requestor instanceof WsRequestor;
if (!isWsConnection || (expectResponse && json && Array.isArray(json) && json.length)) {
span.end();
} else {
/** we use this span to measure application response latency,
* and with websocket connections we generally get the application's response
* in a subsequent message from the far end, so we terminate the span when the
* first new set of verbs arrive after sending a transcript
* */
this.emit('VerbHookSpanWaitForEnd', {span});
// If actionHook delay action is configured, and ws application have not responded yet any verb for actionHook
// We have to transfer the task to call-session to await on next ws command verbs, and also run action Hook
// delay actions
//if (this.hookDelayActionOpts) {
// this.emit('ActionHookDelayActionOptions', this.hookDelayActionOpts);
//}
}
span.end();
if (expectResponse && json && Array.isArray(json)) {
const makeTask = require('./make_task');
const tasks = normalizeJambones(this.logger, json).map((tdata) => makeTask(this.logger, tdata));
if (tasks && tasks.length > 0) {
this.logger.info({tasks: tasks}, `${this.name} replacing application with ${tasks.length} tasks`);
this.callSession.replaceApplication(tasks);
return true;
}
}
} catch (err) {
@@ -199,7 +176,6 @@ class Task extends Emitter {
span.end();
throw err;
}
return false;
}
}
@@ -279,7 +255,6 @@ class Task extends Emitter {
delete obj.requestor;
delete obj.notifier;
obj.tasks = cs.getRemainingTaskData();
obj.callInfo = cs.callInfo.toJSON();
if (opts && obj.tasks.length > 0) {
const key = Object.keys(obj.tasks[0])[0];
Object.assign(obj.tasks[0][key], {_: opts});

File diff suppressed because it is too large Load Diff

View File

@@ -1,335 +0,0 @@
const Task = require('./task');
const { TaskPreconditions } = require('../utils/constants');
const { SpeechCredentialError } = require('../utils/error');
const dbUtils = require('../utils/db-utils');
class TtsTask extends Task {
constructor(logger, data, parentTask) {
super(logger, data);
this.parentTask = parentTask;
this.preconditions = TaskPreconditions.Endpoint;
this.earlyMedia = this.data.earlyMedia === true || (parentTask && parentTask.earlyMedia);
/**
* Task use taskIncludeSynthesizer to identify
* if taskIncludeSynthesizer === true, use label from verb.synthesizer, even it's empty
* if taskIncludeSynthesizer === false, use label from application.synthesizer
*/
this.taskIncludeSynthesizer = !!this.data.synthesizer;
this.synthesizer = this.data.synthesizer || {};
this.disableTtsCache = this.data.disableTtsCache;
this.options = this.synthesizer.options || {};
this.instructions = this.data.instructions;
}
async exec(cs) {
super.exec(cs);
if (cs.synthesizer) {
this.options = {...cs.synthesizer.options, ...this.options};
this.data.synthesizer = this.data.synthesizer || {};
for (const k in cs.synthesizer) {
const newValue = this.data.synthesizer && this.data.synthesizer[k] !== undefined ?
this.data.synthesizer[k] :
cs.synthesizer[k];
if (Array.isArray(newValue)) {
this.data.synthesizer[k] = [...(this.data.synthesizer[k] || []), ...cs.synthesizer[k]];
} else if (typeof newValue === 'object' && newValue !== null) {
this.data.synthesizer[k] = { ...(this.data.synthesizer[k] || {}), ...cs.synthesizer[k] };
} else {
this.data.synthesizer[k] = newValue;
}
}
}
const fullText = Array.isArray(this.text) ? this.text.join(' ') : this.text;
if (fullText.length > 0) {
cs.emit('botSaid', fullText);
}
}
getTtsVendorData(cs) {
const vendor = this.synthesizer.vendor && this.synthesizer.vendor !== 'default' ?
this.synthesizer.vendor :
cs.speechSynthesisVendor;
const language = this.synthesizer.language && this.synthesizer.language !== 'default' ?
this.synthesizer.language :
cs.speechSynthesisLanguage ;
const voice = this.synthesizer.voice && this.synthesizer.voice !== 'default' ?
this.synthesizer.voice :
cs.speechSynthesisVoice;
const label = this.taskIncludeSynthesizer ? this.synthesizer.label : cs.speechSynthesisLabel;
return {vendor, language, voice, label};
}
async setTtsStreamingChannelVars(vendor, language, voice, credentials, ep) {
const {api_key, model_id, custom_tts_streaming_url, auth_token} = credentials;
let obj;
this.logger.debug({credentials},
`setTtsStreamingChannelVars: vendor: ${vendor}, language: ${language}, voice: ${voice}`);
switch (vendor) {
case 'deepgram':
obj = {
DEEPGRAM_API_KEY: api_key,
DEEPGRAM_TTS_STREAMING_MODEL: voice
};
break;
case 'cartesia':
obj = {
CARTESIA_API_KEY: api_key,
CARTESIA_TTS_STREAMING_MODEL_ID: model_id,
CARTESIA_TTS_STREAMING_VOICE_ID: voice,
CARTESIA_TTS_STREAMING_LANGUAGE: language || 'en',
};
break;
case 'elevenlabs':
const {stability, similarity_boost, use_speaker_boost, style, speed} = this.options.voice_settings || {};
obj = {
ELEVENLABS_API_KEY: api_key,
ELEVENLABS_TTS_STREAMING_MODEL_ID: model_id,
ELEVENLABS_TTS_STREAMING_VOICE_ID: voice,
// 20/12/2024 - only eleven_turbo_v2_5 support multiple language
...(['eleven_turbo_v2_5'].includes(model_id) && {ELEVENLABS_TTS_STREAMING_LANGUAGE: language}),
...(stability && {ELEVENLABS_TTS_STREAMING_VOICE_SETTINGS_STABILITY: stability}),
...(similarity_boost && {ELEVENLABS_TTS_STREAMING_VOICE_SETTINGS_SIMILARITY_BOOST: similarity_boost}),
...(use_speaker_boost && {ELEVENLABS_TTS_STREAMING_VOICE_SETTINGS_USE_SPEAKER_BOOST: use_speaker_boost}),
...(style && {ELEVENLABS_TTS_STREAMING_VOICE_SETTINGS_STYLE: style}),
// speed has value 0.7 to 1.2, 1.0 is default, make sure we send the value event it's 0
...(speed !== null && speed !== undefined && {ELEVENLABS_TTS_STREAMING_VOICE_SETTINGS_SPEED: `${speed}`}),
...(this.options.pronunciation_dictionary_locators &&
Array.isArray(this.options.pronunciation_dictionary_locators) && {
ELEVENLABS_TTS_STREAMING_PRONUNCIATION_DICTIONARY_LOCATORS:
JSON.stringify(this.options.pronunciation_dictionary_locators)
}),
};
break;
case 'rimelabs':
const {
pauseBetweenBrackets, phonemizeBetweenBrackets, inlineSpeedAlpha, speedAlpha, reduceLatency
} = this.options;
obj = {
RIMELABS_API_KEY: api_key,
RIMELABS_TTS_STREAMING_MODEL_ID: model_id,
RIMELABS_TTS_STREAMING_VOICE_ID: voice,
RIMELABS_TTS_STREAMING_LANGUAGE: language || 'en',
...(pauseBetweenBrackets && {RIMELABS_TTS_STREAMING_PAUSE_BETWEEN_BRACKETS: pauseBetweenBrackets}),
...(phonemizeBetweenBrackets &&
{RIMELABS_TTS_STREAMING_PHONEMIZE_BETWEEN_BRACKETS: phonemizeBetweenBrackets}),
...(inlineSpeedAlpha && {RIMELABS_TTS_STREAMING_INLINE_SPEED_ALPHA: inlineSpeedAlpha}),
...(speedAlpha && {RIMELABS_TTS_STREAMING_SPEED_ALPHA: speedAlpha}),
...(reduceLatency && {RIMELABS_TTS_STREAMING_REDUCE_LATENCY: reduceLatency})
};
break;
default:
if (vendor.startsWith('custom:')) {
const use_tls = custom_tts_streaming_url.startsWith('wss://');
obj = {
CUSTOM_TTS_STREAMING_HOST: custom_tts_streaming_url.replace(/^(ws|wss):\/\//, ''),
CUSTOM_TTS_STREAMING_API_KEY: auth_token,
CUSTOM_TTS_STREAMING_VOICE_ID: voice,
CUSTOM_TTS_STREAMING_LANGUAGE: language || 'en',
CUSTOM_TTS_STREAMING_USE_TLS: use_tls
};
} else {
throw new Error(`vendor ${vendor} is not supported for tts streaming yet`);
}
}
this.logger.debug({vendor, credentials, obj}, 'setTtsStreamingChannelVars');
await ep.set(obj);
}
async _synthesizeWithSpecificVendor(cs, ep, {vendor, language, voice, label, preCache = false}) {
const {srf, accountSid:account_sid} = cs;
const {writeAlerts, AlertType, stats} = srf.locals;
const {synthAudio} = srf.locals.dbHelpers;
const engine = this.synthesizer.engine || cs.synthesizer?.engine || 'neural';
const salt = cs.callSid;
let credentials = cs.getSpeechCredentials(vendor, 'tts', label);
if (!credentials) {
throw new SpeechCredentialError(
`No text-to-speech service credentials for ${vendor} with labels: ${label} have been configured`);
}
/* parse Nuance voices into name and model */
if (vendor === 'nuance' && voice) {
const arr = /([A-Za-z-]*)\s+-\s+(enhanced|standard)/.exec(voice);
if (arr) {
voice = arr[1];
this.model = arr[2];
}
} else if (vendor === 'deepgram') {
this.model = voice;
}
/* allow for microsoft custom region voice and api_key to be specified as an override */
if (vendor === 'microsoft' && this.options.deploymentId) {
credentials = credentials || {};
credentials.use_custom_tts = true;
credentials.custom_tts_endpoint = this.options.deploymentId;
credentials.api_key = this.options.apiKey || credentials.apiKey;
credentials.region = this.options.region || credentials.region;
voice = this.options.voice || voice;
} else if (vendor === 'elevenlabs') {
credentials = credentials || {};
credentials.model_id = this.options.model_id || credentials.model_id;
credentials.voice_settings = this.options.voice_settings || {};
credentials.optimize_streaming_latency = this.options.optimize_streaming_latency
|| credentials.optimize_streaming_latency;
voice = this.options.voice_id || voice;
} else if (vendor === 'rimelabs') {
credentials = credentials || {};
credentials.model_id = this.options.model_id || credentials.model_id;
} else if (vendor === 'whisper') {
credentials = credentials || {};
credentials.model_id = this.options.model_id || credentials.model_id;
} else if (vendor === 'verbio') {
credentials = credentials || {};
credentials.engine_version = this.options.engine_version || credentials.engine_version;
} else if (vendor === 'playht') {
credentials = credentials || {};
credentials.voice_engine = this.options.voice_engine || credentials.voice_engine;
} else if (vendor === 'google' && typeof voice === 'string' && voice.startsWith('custom_')) {
const {lookupGoogleCustomVoice} = dbUtils(this.logger, cs.srf);
const arr = /custom_(.*)/.exec(voice);
if (arr) {
const google_custom_voice_sid = arr[1];
const [custom_voice] = await lookupGoogleCustomVoice(google_custom_voice_sid);
if (custom_voice.use_voice_cloning_key) {
voice = {
voice_cloning_key: custom_voice.voice_cloning_key,
};
}
}
} else if (vendor === 'cartesia') {
credentials.model_id = this.options.model_id || credentials.model_id;
}
this.model_id = credentials.model_id;
/**
* note on cache_speech_handles. This was found to be risky.
* It can cause a crash in the following sequence on a single call:
* 1. Stream tts on vendor A with cache_speech_handles=1, then
* 2. Stream tts on vendor B with cache_speech_handles=1
*
* we previously tried to track when vendors were switched and manage the flag accordingly,
* but it difficult to track all the scenarios and the benefit (slightly faster start to tts playout)
* is probably minimal. DH.
*/
ep.set({
tts_engine: vendor.startsWith('custom:') ? 'custom' : vendor,
tts_voice: voice,
//cache_speech_handles: !cs.currentTtsVendor || cs.currentTtsVendor === vendor ? 1 : 0,
cache_speech_handles: 0,
}).catch((err) => this.logger.info({err}, 'Error setting tts_engine on endpoint'));
// set the current vendor on the call session
// If vendor is changed from the previous one, then reset the cache_speech_handles flag
//cs.currentTtsVendor = vendor;
if (!preCache && !this._disableTracing)
this.logger.debug({vendor, language, voice, model: this.model}, 'TaskSay:exec');
try {
if (!credentials) {
writeAlerts({
account_sid,
alert_type: AlertType.TTS_NOT_PROVISIONED,
vendor,
target_sid: cs.callSid
}).catch((err) => this.logger.info({err}, 'Error generating alert for no tts'));
throw new SpeechCredentialError('no provisioned speech credentials for TTS');
}
/* produce an audio segment from the provided text */
const generateAudio = async(text) => {
if (this.killed) return;
if (text.startsWith('silence_stream://')) return text;
/* otel: trace time for tts */
if (!preCache && !this._disableTracing) {
const {span} = this.startChildSpan('tts-generation', {
'tts.vendor': vendor,
'tts.language': language,
'tts.voice': voice,
'tts.label': label || 'None',
});
this.otelSpan = span;
}
try {
const {filePath, servedFromCache, rtt} = await synthAudio(stats, {
account_sid,
text,
instructions: this.instructions,
vendor,
language,
voice,
engine,
model: this.model,
salt,
credentials,
options: this.options,
disableTtsCache : this.disableTtsCache,
renderForCaching: preCache
});
if (!filePath.startsWith('say:')) {
this.logger.debug(`Say: file ${filePath}, served from cache ${servedFromCache}`);
if (filePath) cs.trackTmpFile(filePath);
if (this.otelSpan) {
this.otelSpan.setAttributes({'tts.cached': servedFromCache});
this.otelSpan.end();
this.otelSpan = null;
}
if (!servedFromCache && rtt && !preCache && !this._disableTracing) {
this.notifyStatus({
event: 'synthesized-audio',
vendor,
language,
characters: text.length,
elapsedTime: rtt
});
}
}
else {
this.logger.debug('Say: a streaming tts api will be used');
const modifiedPath = filePath.replace('say:{', `say:{session-uuid=${ep.uuid},`);
return modifiedPath;
}
return filePath;
} catch (err) {
this.logger.info({err}, 'Error synthesizing tts');
if (this.otelSpan) this.otelSpan.end();
writeAlerts({
account_sid: cs.accountSid,
alert_type: AlertType.TTS_FAILURE,
vendor,
detail: err.message,
target_sid: cs.callSid
}).catch((err) => this.logger.info({err}, 'Error generating alert for tts failure'));
throw err;
}
};
const arr = this.text.map((t) => (this._validateURL(t) ? t : generateAudio(t)));
return (await Promise.all(arr)).filter((fp) => fp && fp.length);
} catch (err) {
this.logger.info(err, 'TaskSay:exec error');
throw err;
}
}
_validateURL(urlString) {
try {
new URL(urlString);
return true;
} catch (e) {
return false;
}
}
}
module.exports = TtsTask;

View File

@@ -1,187 +0,0 @@
const makeTask = require('../tasks/make_task');
const Emitter = require('events');
const { normalizeJambones } = require('@jambonz/verb-specifications');
const {TaskName} = require('../utils/constants');
/**
* ActionHookDelayProcessor
* @extends Emitter
*
* @param {Object} logger - logger instance
* @param {Object} opts - options
* @param {Object} cs - call session
* @param {Object} ep - endpoint
*
* @emits {Event} 'giveup' - when associated giveup timer expires
*
* Ref:https://www.jambonz.org/docs/supporting-articles/handling-action-hook-delays/
*/
class ActionHookDelayProcessor extends Emitter {
constructor(logger, opts, cs) {
super();
this.logger = logger;
this.cs = cs;
this._active = false;
const enabled = this.init(opts);
if (enabled && this.noResponseTimeout &&
(!this.actions || !Array.isArray(this.actions) || this.actions.length === 0)) {
throw new Error('ActionHookDelayProcessor: no actions specified');
}
else if (enabled && this.actions &&
this.actions.some((a) => !a.verb || ![TaskName.Say, TaskName.Play].includes(a.verb))) {
throw new Error(`ActionHookDelayProcessor: invalid actions specified: ${JSON.stringify(this.actions)}`);
}
}
get properties() {
return {
actions: this.actions,
retries: this.retries,
noResponseTimeout: this.noResponseTimeout,
noResponseGiveUpTimeout: this.noResponseGiveUpTimeout
};
}
get ep() {
return this.cs.ep;
}
init(opts) {
this.logger.debug({opts}, 'ActionHookDelayProcessor#init');
this.actions = opts.actions;
this.retries = Math.max((opts.retries || 1), opts.actions.length);
this.noResponseTimeout = opts.noResponseTimeout;
this.noResponseGiveUpTimeout = opts.noResponseGiveUpTimeout;
this.giveUpActions = opts.giveUpActions;
// return false if these options actually disable the ahdp
return ('enable' in opts && opts.enable === true) ||
('enabled' in opts && opts.enabled === true) ||
(!('enable' in opts) && !('enabled' in opts));
}
start() {
this.logger.debug('ActionHookDelayProcessor#start');
if (this._active) {
this.logger.debug('ActionHookDelayProcessor#start: already started due to prior gather which is continuing');
return;
}
this._active = true;
this._retryCount = 0;
if (this.noResponseTimeout > 0) {
const timeoutMs = this.noResponseTimeout * 1000;
this._noResponseTimer = setTimeout(this._onNoResponseTimer.bind(this), timeoutMs);
} else {
this.logger.debug(
'ActionHookDelayProcessor#start: noResponseTimeout is 0 or undefined hence not calling _onNoResponseTimer'
);
}
if (this.noResponseGiveUpTimeout > 0) {
const timeoutMs = this.noResponseGiveUpTimeout * 1000;
this._noResponseGiveUpTimer = setTimeout(this._onNoResponseGiveUpTimer.bind(this), timeoutMs);
}
}
async stop() {
this._active = false;
if (this._noResponseTimer) {
clearTimeout(this._noResponseTimer);
this._noResponseTimer = null;
}
if (this._noResponseGiveUpTimer) {
clearTimeout(this._noResponseGiveUpTimer);
this._noResponseGiveUpTimer = null;
}
if (this._taskInProgress) {
this.logger.debug(`ActionHookDelayProcessor#stop: stopping ${this._taskInProgress.name}`);
this._sayResolver = () => {
this.logger.debug('ActionHookDelayProcessor#stop: play/say is done, continue on..');
//this._taskInProgress.kill(this.cs);
this._taskInProgress = null;
};
/* we let Say finish, but interrupt Play */
if (TaskName.Play === this._taskInProgress.name) {
await this._taskInProgress.kill(this.cs);
}
return new Promise((resolve) => this._sayResolver = resolve);
}
this.logger.debug('ActionHookDelayProcessor#stop returning');
}
_onNoResponseTimer() {
this.logger.debug('ActionHookDelayProcessor#_onNoResponseTimer');
this._noResponseTimer = null;
/* get the next play or say action */
const verb = this.actions[this._retryCount % this.actions.length];
const t = normalizeJambones(this.logger, [verb]);
this.logger.debug({verb}, 'ActionHookDelayProcessor#_onNoResponseTimer: starting action');
try {
this._taskInProgress = makeTask(this.logger, t[0]);
this._taskInProgress.disableTracing = true;
this._taskInProgress.exec(this.cs, {ep: this.ep}).catch((err) => {
this.logger.info(`ActionHookDelayProcessor#_onNoResponseTimer: error playing file: ${err.message}`);
this._taskInProgress = null;
this.ep.removeAllListeners('playback-start');
this.ep.removeAllListeners('playback-stop');
});
} catch (err) {
this.logger.info(err, 'ActionHookDelayProcessor#_onNoResponseTimer: error starting action');
this._taskInProgress = null;
return;
}
this.ep.once('playback-start', (evt) => {
this.logger.debug({evt}, 'got playback-start');
if (!this._active) {
this.logger.info({evt}, 'ActionHookDelayProcessor#_onNoResponseTimer: killing audio immediately');
/* note: in race condition we may have just hung up and cs.ep cleared */
this.ep?.api('uuid_break', this.ep?.uuid)
.catch((err) => this.logger.info(err,
'ActionHookDelayProcessor#_onNoResponseTimer Error killing audio'));
}
});
this.ep.once('playback-stop', (evt) => {
this._taskInProgress = null;
if (this._sayResolver) {
/* we were waiting for the play to finish before continuing to next task */
this.logger.debug({evt}, 'ActionHookDelayProcessor#_onNoResponseTimer got playback-stop');
this._sayResolver();
this._sayResolver = null;
}
else {
/* possibly start the no response timer again */
if (this._active && this.retries > 0 && this._retryCount < this.retries && this.noResponseTimeout > 0) {
this.logger.debug({evt}, 'ActionHookDelayProcessor#_onNoResponseTimer: playback-stop on play/say action');
const timeoutMs = this.noResponseTimeout * 1000;
this._noResponseTimer = setTimeout(this._onNoResponseTimer.bind(this), timeoutMs);
}
}
});
this._retryCount++;
}
_onNoResponseGiveUpTimer() {
this._active = false;
if (!this.giveUpActions) {
this.logger.info('ActionHookDelayProcessor#_onNoResponseGiveUpTimer');
this.stop().catch((err) => {});
this.emit('giveup');
} else {
this.logger.info('ActionHookDelayProcessor#_onNoResponseGiveUpTimer - giveUpActions');
this.emit('giveupWithTasks', this.giveUpActions);
}
}
}
module.exports = ActionHookDelayProcessor;

View File

@@ -1,22 +1,14 @@
const Emitter = require('events');
const {readFile} = require('fs');
const {
TaskName,
GoogleTranscriptionEvents,
AwsTranscriptionEvents,
AzureTranscriptionEvents,
NuanceTranscriptionEvents,
NvidiaTranscriptionEvents,
IbmTranscriptionEvents,
SonioxTranscriptionEvents,
CobaltTranscriptionEvents,
DeepgramTranscriptionEvents,
JambonzTranscriptionEvents,
AmdEvents,
AvmdEvents
} = require('./constants');
const bugname = 'amd_bug';
const {VMD_HINTS_FILE} = require('../config');
const {VMD_HINTS_FILE} = process.env;
let voicemailHints = [];
const updateHints = async(file, callback) => {
@@ -45,7 +37,6 @@ if (VMD_HINTS_FILE) {
});
}
class Amd extends Emitter {
constructor(logger, cs, opts) {
super();
@@ -56,21 +47,13 @@ class Amd extends Emitter {
this.language = opts.recognizer?.language || cs.speechRecognizerLanguage;
if ('default' === this.language) this.language = cs.speechRecognizerLanguage;
this.sttCredentials = cs.getSpeechCredentials(this.vendor, 'stt',
opts.recognizer?.label || cs.speechRecognizerLabel);
this.sttCredentials = cs.getSpeechCredentials(this.vendor, 'stt');
if (!this.sttCredentials) throw new Error(`No speech credentials found for vendor ${this.vendor}`);
this.thresholdWordCount = opts.thresholdWordCount || 9;
const {normalizeTranscription} = require('./transcription-utils')(logger);
this.normalizeTranscription = normalizeTranscription;
const {getNuanceAccessToken, getIbmAccessToken} = cs.srf.locals.dbHelpers;
this.getNuanceAccessToken = getNuanceAccessToken;
this.getIbmAccessToken = getIbmAccessToken;
const {setChannelVarsForStt} = require('./transcription-utils')(logger);
this.setChannelVarsForStt = setChannelVarsForStt;
this.digitCount = opts.digitCount || 0;
this.numberRegEx = RegExp(`[0-9]{${this.digitCount}}`);
const {
noSpeechTimeoutMs = 5000,
@@ -156,7 +139,7 @@ class Amd extends Emitter {
const wordCount = t.alternatives[0].transcript.split(' ').length;
const final = t.is_final;
const foundHint = hints.find((h) => t.alternatives[0].transcript.toLowerCase().includes(h.toLowerCase()));
const foundHint = hints.find((h) => t.alternatives[0].transcript.includes(h));
if (foundHint) {
/* we detected a common voice mail greeting */
this.logger.debug(`Amd:evaluateTranscription: found hint ${foundHint}`);
@@ -166,14 +149,6 @@ class Amd extends Emitter {
language: t.language_code
});
}
else if (this.digitCount != 0 && this.numberRegEx.test(t.alternatives[0].transcript)) {
/* a string of numbers is typically a machine */
this.emit(this.decision = AmdEvents.MachineDetected, {
reason: 'digit count',
greeting: t.alternatives[0].transcript,
language: t.language_code
});
}
else if (final && wordCount < this.thresholdWordCount) {
/* a short greeting is typically a human */
this.emit(this.decision = AmdEvents.HumanDetected, {
@@ -209,7 +184,7 @@ module.exports = (logger) => {
const {vendor, language} = ep.amd;
ep.startTranscription({
vendor,
locale: language,
language,
interim: true,
bugname
}).catch((err) => {
@@ -221,8 +196,7 @@ module.exports = (logger) => {
account_sid: cs.accountSid,
alert_type: AlertType.STT_FAILURE,
vendor: vendor,
detail: err.message,
target_sid: cs.callSid
detail: err.message
});
}).catch((err) => logger.info({err}, 'Error generating alert for tts failure'));
@@ -255,99 +229,51 @@ module.exports = (logger) => {
const startAmd = async(cs, ep, task, opts) => {
const amd = ep.amd = new Amd(logger, cs, opts);
const {vendor, language} = amd;
let sttCredentials = amd.sttCredentials;
// hints from configuration might be too long for specific language and vendor that make transcribe freeswitch
// modules cannot connect to the vendor. hints is used in next step to validate if the transcription
// matchs voice mail hints.
const hints = [];
if (vendor === 'nuance' && sttCredentials.client_id) {
/* get nuance access token */
const {getNuanceAccessToken} = amd;
const {client_id, secret} = sttCredentials;
const {access_token, servedFromCache} = await getNuanceAccessToken(client_id, secret, 'asr tts');
logger.debug({client_id}, `Gather:exec - got nuance access token ${servedFromCache ? 'from cache' : ''}`);
sttCredentials = {...sttCredentials, access_token};
}
else if (vendor == 'ibm' && sttCredentials.stt_api_key) {
/* get ibm access token */
const {getIbmAccessToken} = amd;
const {stt_api_key, stt_region} = sttCredentials;
const {access_token, servedFromCache} = await getIbmAccessToken(stt_api_key);
logger.debug({stt_api_key}, `Gather:exec - got ibm access token ${servedFromCache ? 'from cache' : ''}`);
sttCredentials = {...sttCredentials, access_token, stt_region};
}
const {vendor, language, sttCredentials} = amd;
const sttOpts = {};
const hints = voicemailHints[language] || [];
/* set stt options */
logger.info(`starting amd for vendor ${vendor} and language ${language}`);
const sttOpts = amd.setChannelVarsForStt({name: TaskName.Gather}, sttCredentials, language, {
vendor,
hints,
enhancedModel: true,
altLanguages: opts.recognizer?.altLanguages || [],
initialSpeechTimeoutMs: opts.resolveTimeoutMs,
});
if ('google' === vendor) {
sttOpts.GOOGLE_APPLICATION_CREDENTIALS = JSON.stringify(sttCredentials.credentials);
sttOpts.GOOGLE_SPEECH_USE_ENHANCED = true;
sttOpts.GOOGLE_SPEECH_HINTS = hints.join(',');
if (opts.recognizer?.altLanguages) {
sttOpts.GOOGLE_SPEECH_ALTERNATIVE_LANGUAGE_CODES = opts.recognizer.altLanguages.join(',');
}
ep.addCustomEventListener(GoogleTranscriptionEvents.Transcription, onTranscription.bind(null, cs, ep, task));
ep.addCustomEventListener(GoogleTranscriptionEvents.EndOfUtterance, onEndOfUtterance.bind(null, cs, ep, task));
}
else if (['aws', 'polly'].includes(vendor)) {
Object.assign(sttOpts, {
AWS_ACCESS_KEY_ID: sttCredentials.accessKeyId,
AWS_SECRET_ACCESS_KEY: sttCredentials.secretAccessKey,
AWS_REGION: sttCredentials.region
});
ep.addCustomEventListener(AwsTranscriptionEvents.Transcription, onTranscription.bind(null, cs, ep, task));
}
else if ('microsoft' === vendor) {
Object.assign(sttOpts, {
'AZURE_SUBSCRIPTION_KEY': sttCredentials.api_key,
'AZURE_REGION': sttCredentials.region
});
sttOpts.AZURE_SPEECH_HINTS = hints.join(',');
if (opts.recognizer?.altLanguages) {
sttOpts.AZURE_SPEECH_ALTERNATIVE_LANGUAGE_CODES = opts.recognizer.altLanguages.join(',');
}
sttOpts.AZURE_INITIAL_SPEECH_TIMEOUT_MS = opts.resolveTimeoutMs || 20000;
ep.addCustomEventListener(AzureTranscriptionEvents.Transcription, onTranscription.bind(null, cs, ep, task));
ep.addCustomEventListener(AzureTranscriptionEvents.NoSpeechDetected, onNoSpeechDetected.bind(null, cs, ep, task));
}
await ep.set(sttOpts).catch((err) => logger.info(err, 'Error setting channel variables'));
amd.transcriptionHandler = onTranscription.bind(null, cs, ep, task);
amd.EndOfUtteranceHandler = onEndOfUtterance.bind(null, cs, ep, task);
amd.noSpeechHandler = onNoSpeechDetected.bind(null, cs, ep, task);
switch (vendor) {
case 'google':
ep.addCustomEventListener(GoogleTranscriptionEvents.Transcription, amd.transcriptionHandler);
ep.addCustomEventListener(GoogleTranscriptionEvents.EndOfUtterance, amd.EndOfUtteranceHandler);
break;
case 'aws':
case 'polly':
ep.addCustomEventListener(AwsTranscriptionEvents.Transcription, amd.transcriptionHandler);
break;
case 'microsoft':
ep.addCustomEventListener(AzureTranscriptionEvents.Transcription, amd.transcriptionHandler);
ep.addCustomEventListener(AzureTranscriptionEvents.NoSpeechDetected, amd.noSpeechHandler);
break;
case 'nuance':
ep.addCustomEventListener(NuanceTranscriptionEvents.Transcription, amd.transcriptionHandler);
break;
case 'deepgram':
ep.addCustomEventListener(DeepgramTranscriptionEvents.Transcription, amd.transcriptionHandler);
break;
case 'soniox':
amd.bugname = 'soniox_amd_transcribe';
ep.addCustomEventListener(SonioxTranscriptionEvents.Transcription, amd.transcriptionHandler);
break;
case 'ibm':
ep.addCustomEventListener(IbmTranscriptionEvents.Transcription, amd.transcriptionHandler);
break;
case 'nvidia':
ep.addCustomEventListener(NvidiaTranscriptionEvents.Transcription, amd.transcriptionHandler);
break;
case 'cobalt':
ep.addCustomEventListener(CobaltTranscriptionEvents.Transcription, amd.transcriptionHandler);
break;
default:
if (vendor.startsWith('custom:')) {
ep.addCustomEventListener(JambonzTranscriptionEvents.Transcription, amd.transcriptionHandler);
break;
}
else {
throw new Error(`Invalid vendor ${this.vendor}`);
}
}
amd
.on(AmdEvents.NoSpeechDetected, (evt) => {
task.emit('amd', {type: AmdEvents.NoSpeechDetected, ...evt});
try {
stopAmd(ep, task);
ep.connected && ep.stopTranscription({vendor, bugname});
} catch (err) {
logger.info({err}, 'Error stopping transcription');
}
@@ -355,7 +281,7 @@ module.exports = (logger) => {
.on(AmdEvents.HumanDetected, (evt) => {
task.emit('amd', {type: AmdEvents.HumanDetected, ...evt});
try {
stopAmd(ep, task);
ep.connected && ep.stopTranscription({vendor, bugname});
} catch (err) {
logger.info({err}, 'Error stopping transcription');
}
@@ -366,7 +292,7 @@ module.exports = (logger) => {
.on(AmdEvents.DecisionTimeout, (evt) => {
task.emit('amd', {type: AmdEvents.DecisionTimeout, ...evt});
try {
stopAmd(ep, task);
ep.connected && ep.stopTranscription({vendor, bugname});
} catch (err) {
logger.info({err}, 'Error stopping transcription');
}
@@ -374,7 +300,7 @@ module.exports = (logger) => {
.on(AmdEvents.ToneTimeout, (evt) => {
//task.emit('amd', {type: AmdEvents.ToneTimeout, ...evt});
try {
stopAmd(ep, task);
ep.connected && ep.execute('avmd_stop').catch((err) => logger.info(err, 'Error stopping avmd'));
} catch (err) {
logger.info({err}, 'Error stopping avmd');
}
@@ -382,7 +308,7 @@ module.exports = (logger) => {
.on(AmdEvents.MachineStoppedSpeaking, () => {
task.emit('amd', {type: AmdEvents.MachineStoppedSpeaking});
try {
stopAmd(ep, task);
ep.connected && ep.stopTranscription({vendor, bugname});
} catch (err) {
logger.info({err}, 'Error stopping transcription');
}
@@ -401,19 +327,6 @@ module.exports = (logger) => {
if (ep.amd) {
vendor = ep.amd.vendor;
ep.amd.stopAllTimers();
ep.removeListener(GoogleTranscriptionEvents.Transcription, ep.amd.transcriptionHandler);
ep.removeListener(GoogleTranscriptionEvents.EndOfUtterance, ep.amd.EndOfUtteranceHandler);
ep.removeListener(AwsTranscriptionEvents.Transcription, ep.amd.transcriptionHandler);
ep.removeListener(AzureTranscriptionEvents.Transcription, ep.amd.transcriptionHandler);
ep.removeListener(AzureTranscriptionEvents.NoSpeechDetected, ep.amd.noSpeechHandler);
ep.removeListener(NuanceTranscriptionEvents.Transcription, ep.amd.transcriptionHandler);
ep.removeListener(DeepgramTranscriptionEvents.Transcription, ep.amd.transcriptionHandler);
ep.removeListener(SonioxTranscriptionEvents.Transcription, ep.amd.transcriptionHandler);
ep.removeListener(IbmTranscriptionEvents.Transcription, ep.amd.transcriptionHandler);
ep.removeListener(NvidiaTranscriptionEvents.Transcription, ep.amd.transcriptionHandler);
ep.removeListener(JambonzTranscriptionEvents.Transcription, ep.amd.transcriptionHandler);
ep.amd = null;
}

View File

@@ -1,30 +1,20 @@
const Emitter = require('events');
const bent = require('bent');
const assert = require('assert');
const {
AWS_REGION,
AWS_SNS_PORT: PORT,
AWS_SNS_TOPIC_ARN,
AWS_SNS_PORT_MAX,
} = require('../config');
const PORT = process.env.AWS_SNS_PORT || 3010;
const {LifeCycleEvents} = require('./constants');
const express = require('express');
const app = express();
const getString = bent('string');
const {
SNSClient,
SubscribeCommand,
UnsubscribeCommand } = require('@aws-sdk/client-sns');
const snsClient = new SNSClient({ region: AWS_REGION, apiVersion: '2010-03-31' });
const {
AutoScalingClient,
DescribeAutoScalingGroupsCommand,
CompleteLifecycleActionCommand } = require('@aws-sdk/client-auto-scaling');
const autoScalingClient = new AutoScalingClient({ region: AWS_REGION, apiVersion: '2011-01-01' });
const AWS = require('aws-sdk');
const sns = new AWS.SNS({apiVersion: '2010-03-31'});
const autoscaling = new AWS.AutoScaling({apiVersion: '2011-01-01'});
const {Parser} = require('xml2js');
const parser = new Parser();
const {validatePayload} = require('verify-aws-sns-signature');
AWS.config.update({region: process.env.AWS_REGION});
class SnsNotifier extends Emitter {
constructor(logger) {
super();
@@ -41,8 +31,8 @@ class SnsNotifier extends Emitter {
_handleErrors(logger, app, resolve, reject, e) {
if (e.code === 'EADDRINUSE' &&
AWS_SNS_PORT_MAX &&
e.port < AWS_SNS_PORT_MAX) {
process.env.AWS_SNS_PORT_MAX &&
e.port < process.env.AWS_SNS_PORT_MAX) {
logger.info(`SNS lifecycle server failed to bind port on ${e.port}, will try next port`);
const server = this._doListen(logger, app, ++e.port, resolve);
@@ -55,12 +45,12 @@ class SnsNotifier extends Emitter {
async _handlePost(req, res) {
try {
const parsedBody = JSON.parse(req.body);
this.logger.info({headers: req.headers, body: parsedBody}, 'Received HTTP POST from AWS');
this.logger.debug({headers: req.headers, body: parsedBody}, 'Received HTTP POST from AWS');
if (!validatePayload(parsedBody)) {
this.logger.info('incoming AWS SNS HTTP POST failed signature validation');
return res.sendStatus(403);
}
this.logger.info('incoming HTTP POST passed validation');
this.logger.debug('incoming HTTP POST passed validation');
res.sendStatus(200);
switch (parsedBody.Type) {
@@ -74,18 +64,7 @@ class SnsNotifier extends Emitter {
subscriptionRequestId: this.subscriptionRequestId
}, 'response from SNS SubscribeURL');
const data = await this.describeInstance();
const group = data.AutoScalingGroups.find((group) =>
group.Instances && group.Instances.some((instance) => instance.InstanceId === this.instanceId)
);
if (!group) {
this.logger.error('Current instance not found in any Auto Scaling group', data);
} else {
const instance = group.Instances.find((instance) => instance.InstanceId === this.instanceId);
this.lifecycleState = instance.LifecycleState;
}
//this.lifecycleState = data.AutoScalingGroups[0].Instances[0].LifecycleState;
this.lifecycleState = data.AutoScalingInstances[0].LifecycleState;
this.emit('SubscriptionConfirmation', {publicIp: this.publicIp});
break;
@@ -105,7 +84,7 @@ class SnsNotifier extends Emitter {
this.unsubscribe();
}
else {
this.logger.info(`SnsNotifier - instance ${msg.EC2InstanceId} is scaling in (not us)`);
this.logger.debug(`SnsNotifier - instance ${msg.EC2InstanceId} is scaling in (not us)`);
}
}
break;
@@ -122,7 +101,7 @@ class SnsNotifier extends Emitter {
async init() {
try {
this.logger.info('SnsNotifier: retrieving instance data');
this.logger.debug('SnsNotifier: retrieving instance data');
this.instanceId = await getString('http://169.254.169.254/latest/meta-data/instance-id');
this.publicIp = await getString('http://169.254.169.254/latest/meta-data/public-ipv4');
this.logger.info({
@@ -151,56 +130,51 @@ class SnsNotifier extends Emitter {
async subscribe() {
try {
const params = {
const response = await sns.subscribe({
Protocol: 'http',
TopicArn: AWS_SNS_TOPIC_ARN,
TopicArn: process.env.AWS_SNS_TOPIC_ARM,
Endpoint: this.snsEndpoint
};
const response = await snsClient.send(new SubscribeCommand(params));
this.logger.info({response}, `response to SNS subscribe to ${AWS_SNS_TOPIC_ARN}`);
}).promise();
this.logger.info({response}, `response to SNS subscribe to ${process.env.AWS_SNS_TOPIC_ARM}`);
} catch (err) {
this.logger.error({err}, `Error subscribing to SNS topic arn ${AWS_SNS_TOPIC_ARN}`);
this.logger.error({err}, `Error subscribing to SNS topic arn ${process.env.AWS_SNS_TOPIC_ARM}`);
}
}
async unsubscribe() {
if (!this.subscriptionArn) throw new Error('SnsNotifier#unsubscribe called without an active subscription');
try {
const params = {
const response = await sns.unsubscribe({
SubscriptionArn: this.subscriptionArn
};
const response = await snsClient.send(new UnsubscribeCommand(params));
this.logger.info({response}, `response to SNS unsubscribe to ${AWS_SNS_TOPIC_ARN}`);
}).promise();
this.logger.info({response}, `response to SNS unsubscribe to ${process.env.AWS_SNS_TOPIC_ARM}`);
} catch (err) {
this.logger.error({err}, `Error unsubscribing to SNS topic arn ${AWS_SNS_TOPIC_ARN}`);
this.logger.error({err}, `Error unsubscribing to SNS topic arn ${process.env.AWS_SNS_TOPIC_ARM}`);
}
}
completeScaleIn() {
assert(this.scaleInParams);
autoScalingClient.send(new CompleteLifecycleActionCommand(this.scaleInParams))
.then((data) => {
return this.logger.info({data}, 'Successfully completed scale-in action');
})
.catch((err) => {
this.logger.error({err}, 'Error completing scale-in');
});
autoscaling.completeLifecycleAction(this.scaleInParams, (err, response) => {
if (err) return this.logger.error({err}, 'Error completing scale-in');
this.logger.info({response}, 'Successfully completed scale-in action');
});
}
describeInstance() {
return new Promise((resolve, reject) => {
if (!this.instanceId) return reject('instance-id unknown');
autoScalingClient.send(new DescribeAutoScalingGroupsCommand({
autoscaling.describeAutoScalingInstances({
InstanceIds: [this.instanceId]
}))
.then((data) => {
this.logger.info({data}, 'SnsNotifier: describeInstance');
return resolve(data);
})
.catch((err) => {
}, (err, data) => {
if (err) {
this.logger.error({err}, 'Error describing instances');
reject(err);
});
} else {
this.logger.info({data}, 'SnsNotifier: describeInstance');
resolve(data);
}
});
});
}
@@ -214,7 +188,7 @@ module.exports = async function(logger) {
process.on('SIGHUP', async() => {
try {
const data = await notifier.describeInstance();
const state = data.AutoScalingGroups[0].Instances[0].LifecycleState;
const state = data.AutoScalingInstances[0].LifecycleState;
if (state !== notifier.lifecycleState) {
notifier.lifecycleState = state;
switch (state) {

View File

@@ -1,221 +0,0 @@
const { normalizeJambones } = require('@jambonz/verb-specifications');
const makeTask = require('../tasks/make_task');
const { JAMBONZ_RECORD_WS_BASE_URL, JAMBONZ_RECORD_WS_USERNAME, JAMBONZ_RECORD_WS_PASSWORD } = require('../config');
const Emitter = require('events');
class BackgroundTaskManager extends Emitter {
constructor({cs, logger, rootSpan}) {
super();
this.tasks = new Map();
this.cs = cs;
this.logger = logger;
this.rootSpan = rootSpan;
}
isTaskRunning(type) {
return this.tasks.has(type);
}
getTask(type) {
if (this.tasks.has(type)) {
return this.tasks.get(type);
}
}
count() {
return this.tasks.size;
}
async newTask(type, opts, sticky = false) {
this.logger.info({opts}, `initiating Background task ${type}`);
if (this.tasks.has(type)) {
this.logger.info(`Background task ${type} is running, skipped`);
return;
}
let task;
switch (type) {
case 'listen':
task = await this._initListen(opts);
break;
case 'bargeIn':
task = await this._initBargeIn(opts);
break;
case 'record':
task = await this._initRecord();
break;
case 'transcribe':
task = await this._initTranscribe(opts);
break;
case 'ttsStream':
task = await this._initTtsStream(opts);
break;
default:
break;
}
if (task) {
this.tasks.set(type, task);
}
if (task && sticky) task.sticky = true;
return task;
}
stop(type) {
const task = this.getTask(type);
if (task) {
this.logger.info(`stopping background task: ${type}`);
task.removeAllListeners();
task.span.end();
task.kill();
// Remove task from managed List
this.tasks.delete(type);
}
}
stopAll() {
this.logger.debug('BackgroundTaskManager:stopAll');
for (const key of this.tasks.keys()) {
this.stop(key);
}
}
// Initiate Listen
async _initListen(opts, bugname = 'jambonz-background-listen', ignoreCustomerData = false, type = 'listen') {
let task;
try {
const t = normalizeJambones(this.logger, [opts]);
task = makeTask(this.logger, t[0]);
task.bugname = bugname;
task.ignoreCustomerData = ignoreCustomerData;
const resources = await this.cs._evaluatePreconditions(task);
const {span, ctx} = this.rootSpan.startChildSpan(`background-${type}:${task.summary}`);
task.span = span;
task.ctx = ctx;
task.exec(this.cs, resources)
.then(this._taskCompleted.bind(this, type, task))
.catch(this._taskError.bind(this, type, task));
} catch (err) {
this.logger.info({err, opts}, `BackgroundTaskManager:_initListen - Error creating ${bugname} task`);
}
return task;
}
// Initiate Gather
async _initBargeIn(opts) {
let task;
try {
const copy = JSON.parse(JSON.stringify(opts));
const t = normalizeJambones(this.logger, [opts]);
task = makeTask(this.logger, t[0]);
task
.once('dtmf', this._bargeInTaskCompleted.bind(this))
.once('vad', this._bargeInTaskCompleted.bind(this))
.once('transcription', this._bargeInTaskCompleted.bind(this))
.once('timeout', this._bargeInTaskCompleted.bind(this));
const resources = await this.cs._evaluatePreconditions(task);
const {span, ctx} = this.rootSpan.startChildSpan(`background-bargeIn:${task.summary}`);
task.span = span;
task.ctx = ctx;
task.bugname_prefix = 'background_bargeIn_';
task.exec(this.cs, resources)
.then(() => {
this._taskCompleted('bargeIn', task);
if (task.sticky && !this.cs.callGone && !this.cs._stopping) {
this.logger.info('BackgroundTaskManager:_initBargeIn: restarting background bargeIn');
this._bargeInHandled = false;
this.newTask('bargeIn', copy, true);
}
return;
})
.catch(this._taskError.bind(this, 'bargeIn', task));
} catch (err) {
this.logger.info(err, 'BackgroundTaskManager:_initGather - Error creating bargeIn task');
}
return task;
}
// Initiate Record
async _initRecord() {
if (this.cs.accountInfo.account.record_all_calls || this.cs.application.record_all_calls) {
if (!JAMBONZ_RECORD_WS_BASE_URL || !this.cs.accountInfo.account.bucket_credential) {
this.logger.error('_initRecord: invalid cfg - missing JAMBONZ_RECORD_WS_BASE_URL or bucket config');
return undefined;
}
const listenOpts = {
url: `${JAMBONZ_RECORD_WS_BASE_URL}/record/${this.cs.accountInfo.account.bucket_credential.vendor}`,
disableBidirectionalAudio: true,
mixType : 'stereo',
passDtmf: true
};
if (JAMBONZ_RECORD_WS_USERNAME && JAMBONZ_RECORD_WS_PASSWORD) {
listenOpts.wsAuth = {
username: JAMBONZ_RECORD_WS_USERNAME,
password: JAMBONZ_RECORD_WS_PASSWORD
};
}
this.logger.debug({listenOpts}, '_initRecord: enabling listen');
return await this._initListen({verb: 'listen', ...listenOpts}, 'jambonz-session-record', true, 'record');
}
}
// Initiate Transcribe
async _initTranscribe(opts) {
let task;
try {
const t = normalizeJambones(this.logger, [opts]);
task = makeTask(this.logger, t[0]);
const resources = await this.cs._evaluatePreconditions(task);
const {span, ctx} = this.rootSpan.startChildSpan(`background-transcribe:${task.summary}`);
task.span = span;
task.ctx = ctx;
task.bugname_prefix = 'background_transcribe_';
task.exec(this.cs, resources)
.then(this._taskCompleted.bind(this, 'transcribe', task))
.catch(this._taskError.bind(this, 'transcribe', task));
} catch (err) {
this.logger.info(err, 'BackgroundTaskManager:_initTranscribe - Error creating transcribe task');
}
return task;
}
// Initiate Tts Stream
async _initTtsStream(opts) {
let task;
try {
const t = normalizeJambones(this.logger, [opts]);
task = makeTask(this.logger, t[0]);
const resources = await this.cs._evaluatePreconditions(task);
const {span, ctx} = this.rootSpan.startChildSpan(`background-ttsStream:${task.summary}`);
task.span = span;
task.ctx = ctx;
task.exec(this.cs, resources)
.then(this._taskCompleted.bind(this, 'ttsStream', task))
.catch(this._taskError.bind(this, 'ttsStream', task));
} catch (err) {
this.logger.info(err, 'BackgroundTaskManager:_initTtsStream - Error creating ttsStream task');
}
return task;
}
_taskCompleted(type, task) {
this.logger.debug({type, task}, `BackgroundTaskManager:_taskCompleted: task completed, sticky: ${task.sticky}`);
task.removeAllListeners();
task.span.end();
this.tasks.delete(type);
}
_taskError(type, task, error) {
this.logger.info({type, task, error}, 'BackgroundTaskManager:_taskError: task Error');
task.removeAllListeners();
task.span.end();
this.tasks.delete(type);
}
_bargeInTaskCompleted(evt) {
if (this._bargeInHandled) return;
this._bargeInHandled = true;
this.logger.debug({evt},
'BackgroundTaskManager:_bargeInTaskCompleted on event from background bargeIn, emitting bargein-done event');
this.emit('bargeIn-done', evt);
}
}
module.exports = BackgroundTaskManager;

View File

@@ -1,9 +1,7 @@
const assert = require('assert');
const Emitter = require('events');
const crypto = require('crypto');
const parseUrl = require('parse-url');
const timeSeries = require('@jambonz/time-series');
const {NODE_ENV, JAMBONES_TIME_SERIES_HOST} = require('../config');
let alerter ;
class BaseRequestor extends Emitter {
@@ -22,23 +20,15 @@ class BaseRequestor extends Emitter {
const {stats} = require('../../').srf.locals;
this.stats = stats;
const u = this._parsedUrl = parseUrl(this.url);
if (u.port) this._baseUrl = `${u.protocol}://${u.resource}:${u.port}`;
else this._baseUrl = `${u.protocol}://${u.resource}`;
if (!alerter) {
alerter = timeSeries(logger, {
host: JAMBONES_TIME_SERIES_HOST,
host: process.env.JAMBONES_TIME_SERIES_HOST,
commitSize: 50,
commitInterval: 'test' === NODE_ENV ? 7 : 20
commitInterval: 'test' === process.env.NODE_ENV ? 7 : 20
});
}
}
get baseUrl() {
return this._baseUrl;
}
get Alerter() {
return alerter;
}

View File

@@ -2,24 +2,17 @@ const {context, trace} = require('@opentelemetry/api');
const {Dialog} = require('drachtio-srf');
class RootSpan {
constructor(callType, req) {
const {srf} = require('../../');
const tracer = srf.locals.otel.tracer;
let callSid, accountSid, applicationSid, linkedSpanId;
let tracer, callSid, linkedSpanId;
if (req instanceof Dialog) {
const dlg = req;
tracer = dlg.srf.locals.otel.tracer;
callSid = dlg.callSid;
linkedSpanId = dlg.linkedSpanId;
}
else if (req.srf) {
callSid = req.locals.callSid;
accountSid = req.get('X-Account-Sid'),
applicationSid = req.locals.application_sid;
}
else {
callSid = req.callSid;
accountSid = req.accountSid;
applicationSid = req.applicationSid;
tracer = req.srf.locals.otel.tracer;
callSid = req.locals.callSid;
}
this._span = tracer.startSpan(callType || 'incoming-call');
if (req instanceof Dialog) {
@@ -29,20 +22,13 @@ class RootSpan {
callId: dlg.sip.callId
});
}
else if (req.srf) {
this._span.setAttributes({
callSid,
accountSid,
applicationSid,
callId: req.get('Call-ID'),
externalCallId: req.get('X-CID')
});
}
else {
this._span.setAttributes({
callSid,
accountSid,
applicationSid
accountSid: req.get('X-Account-Sid'),
applicationSid: req.locals.application_sid,
callId: req.get('Call-ID'),
externalCallId: req.get('X-CID')
});
}

View File

@@ -1,20 +1,18 @@
{
"TaskName": {
"Answer": "answer",
"Cognigy": "cognigy",
"Conference": "conference",
"Config": "config",
"Dequeue": "dequeue",
"Dial": "dial",
"Dialogflow": "dialogflow",
"Dtmf": "dtmf",
"Dub": "dub",
"Enqueue": "enqueue",
"Gather": "gather",
"Hangup": "hangup",
"Leave": "leave",
"Lex": "lex",
"Listen": "listen",
"Llm": "llm",
"Message": "message",
"Pause": "pause",
"Play": "play",
@@ -28,12 +26,10 @@
"SipRedirect": "sip:redirect",
"Say": "say",
"SayLegacy": "say:legacy",
"Stream": "stream",
"Tag": "tag",
"Transcribe": "transcribe"
},
"AllowedSipRecVerbs": ["answer", "config", "gather", "transcribe", "listen", "tag", "hangup", "sip:decline"],
"AllowedConfirmSessionVerbs": ["config", "gather", "plays", "say", "tag"],
"AllowedSipRecVerbs": ["config", "gather", "transcribe", "listen"],
"CallStatus": {
"Trying": "trying",
"Ringing": "ringing",
@@ -55,11 +51,6 @@
"Silence": "silence",
"Resume": "resume"
},
"TranscribeStatus": {
"Pause": "pause",
"Silence": "silence",
"Resume": "resume"
},
"TaskPreconditions": {
"None": "none",
"Endpoint": "endpoint",
@@ -99,15 +90,6 @@
"Transcription": "soniox_transcribe::transcription",
"Error": "soniox_transcribe::error"
},
"VerbioTranscriptionEvents": {
"Transcription": "verbio_transcribe::transcription",
"Error": "verbio_transcribe::error"
},
"CobaltTranscriptionEvents": {
"Transcription": "cobalt_speech::transcription",
"CompileContext": "cobalt_speech::compile_context_response",
"Error": "cobalt_speech::error"
},
"IbmTranscriptionEvents": {
"Transcription": "ibm_transcribe::transcription",
"ConnectFailure": "ibm_transcribe::connect_failed",
@@ -128,48 +110,12 @@
"NoSpeechDetected": "azure_transcribe::no_speech_detected",
"VadDetected": "azure_transcribe::vad_detected"
},
"SpeechmaticsTranscriptionEvents": {
"Transcription": "speechmatics_transcribe::transcription",
"Translation": "speechmatics_transcribe::translation",
"Info": "speechmatics_transcribe::info",
"RecognitionStarted": "speechmatics_transcribe::recognition_started",
"ConnectFailure": "speechmatics_transcribe::connect_failed",
"Connect": "speechmatics_transcribe::connect",
"Error": "speechmatics_transcribe::error"
},
"OpenAITranscriptionEvents": {
"Transcription": "openai_transcribe::transcription",
"Translation": "openai_transcribe::translation",
"SpeechStarted": "openai_transcribe::speech_started",
"SpeechStopped": "openai_transcribe::speech_stopped",
"PartialTranscript": "openai_transcribe::partial_transcript",
"Info": "openai_transcribe::info",
"RecognitionStarted": "openai_transcribe::recognition_started",
"ConnectFailure": "openai_transcribe::connect_failed",
"Connect": "openai_transcribe::connect",
"Error": "openai_transcribe::error"
},
"JambonzTranscriptionEvents": {
"Transcription": "jambonz_transcribe::transcription",
"ConnectFailure": "jambonz_transcribe::connect_failed",
"Connect": "jambonz_transcribe::connect",
"Error": "jambonz_transcribe::error"
},
"AssemblyAiTranscriptionEvents": {
"Transcription": "assemblyai_transcribe::transcription",
"Error": "assemblyai_transcribe::error",
"ConnectFailure": "assemblyai_transcribe::connect_failed",
"Connect": "assemblyai_transcribe::connect"
},
"VoxistTranscriptionEvents": {
"Transcription": "voxist_transcribe::transcription",
"Error": "voxist_transcribe::error",
"ConnectFailure": "voxist_transcribe::connect_failed",
"Connect": "voxist_transcribe::connect"
},
"VadDetection": {
"Detection": "vad_detect:detection"
},
"ListenEvents": {
"Connect": "mod_audio_fork::connect",
"ConnectFailure": "mod_audio_fork::connect_failed",
@@ -187,41 +133,6 @@
"StandbyEnter": "standby-enter",
"StandbyExit": "standby-exit"
},
"LlmEvents_OpenAI": {
"Error": "error",
"Connect": "openai_s2s::connect",
"ConnectFailure": "openai_s2s::connect_failed",
"Disconnect": "openai_s2s::disconnect",
"ServerEvent": "openai_s2s::server_event"
},
"LlmEvents_Google": {
"Error": "error",
"Connect": "google_s2s::connect",
"ConnectFailure": "google_s2s::connect_failed",
"Disconnect": "google_s2s::disconnect",
"ServerEvent": "google_s2s::server_event"
},
"LlmEvents_Elevenlabs": {
"Error": "error",
"Connect": "elevenlabs_s2s::connect",
"ConnectFailure": "elevenlabs_s2s::connect_failed",
"Disconnect": "elevenlabs_s2s::disconnect",
"ServerEvent": "elevenlabs_s2s::server_event"
},
"LlmEvents_VoiceAgent": {
"Error": "error",
"Connect": "voice_agent_s2s::connect",
"ConnectFailure": "voice_agent_s2s::connect_failed",
"Disconnect": "voice_agent_s2s::disconnect",
"ServerEvent": "voice_agent_s2s::server_event"
},
"LlmEvents_Ultravox": {
"Error": "error",
"Connect": "ultravox_s2s::connect",
"ConnectFailure": "ultravox_s2s::connect_failed",
"Disconnect": "ultravox_s2s::disconnect",
"ServerEvent": "ultravox_s2s::server_event"
},
"QueueResults": {
"Bridged": "bridged",
"Error": "error",
@@ -236,23 +147,17 @@
},
"KillReason": {
"Hangup": "hangup",
"Replaced": "replaced",
"MediaTimeout": "media_timeout"
"Replaced": "replaced"
},
"HookMsgTypes": [
"session:new",
"session:reconnect",
"session:redirect",
"session:adulting",
"call:status",
"queue:status",
"dial:confirm",
"verb:hook",
"verb:status",
"llm:event",
"llm:tool-call",
"tts:tokens-result",
"tts:streaming-event",
"jambonz:error"
],
"RecordState": {
@@ -271,59 +176,7 @@
"ToneTimeout": "amd_tone_timeout",
"Stopped": "amd_stopped"
},
"MediaPath": {
"NoMedia": "no-media",
"PartialMedia": "partial-media",
"FullMedia": "full-media"
},
"DeepgramTtsStreamingEvents": {
"Empty": "deepgram_tts_streaming::empty",
"ConnectFailure": "deepgram_tts_streaming::connect_failed",
"Connect": "deepgram_tts_streaming::connect"
},
"CartesiaTtsStreamingEvents": {
"Empty": "cartesia_tts_streaming::empty",
"ConnectFailure": "cartesia_tts_streaming::connect_failed",
"Connect": "cartesia_tts_streaming::connect"
},
"ElevenlabsTtsStreamingEvents": {
"Empty": "elevenlabs_tts_streaming::empty",
"ConnectFailure": "elevenlabs_tts_streaming::connect_failed",
"Connect": "elevenlabs_tts_streaming::connect"
},
"RimelabsTtsStreamingEvents": {
"Empty": "rimelabs_tts_streaming::empty",
"ConnectFailure": "rimelabs_tts_streaming::connect_failed",
"Connect": "rimelabs_tts_streaming::connect"
},
"CustomTtsStreamingEvents": {
"Empty": "custom_tts_streaming::empty",
"ConnectFailure": "custom_tts_streaming::connect_failed",
"Connect": "custom_tts_streaming::connect"
},
"TtsStreamingEvents": {
"Empty": "tts_streaming::empty",
"Pause": "tts_streaming::pause",
"Resume": "tts_streaming::resume",
"ConnectFailure": "tts_streaming::connect_failed"
},
"TtsStreamingConnectionStatus": {
"NotConnected": "not_connected",
"Connected": "connected",
"Connecting": "connecting",
"Failed": "failed"
},
"MAX_SIMRINGS": 10,
"BONG_TONE": "tone_stream://v=-7;%(100,0,941.0,1477.0);v=-7;>=2;+=.1;%(1400,0,350,440)",
"FS_UUID_SET_NAME": "fsUUIDs",
"SystemState" : {
"Online": "ONLINE",
"Offline": "OFFLINE",
"GracefulShutdownInProgress":"SHUTDOWN_IN_PROGRESS"
},
"FEATURE_SERVER" : "feature-server",
"WS_CLOSE_CODES": {
"NormalClosure": 1000,
"GoingAway": 1001
}
"FS_UUID_SET_NAME": "fsUUIDs"
}

View File

@@ -1,24 +1,19 @@
const {execSync} = require('child_process');
const {
JAMBONES_FREESWITCH,
NODE_ENV,
JAMBONES_FREESWITCH_MAX_CALL_DURATION_MINS,
} = require('../config');
const now = Date.now();
const fsInventory = JAMBONES_FREESWITCH
const fsInventory = process.env.JAMBONES_FREESWITCH
.split(',')
.map((fs) => {
const arr = /^([^:]*):([^:]*):([^:]*)(?::([^:]*))?/.exec(fs);
const opts = {address: arr[1], port: arr[2], secret: arr[3]};
if (arr.length > 4) opts.advertisedAddress = arr[4];
if (NODE_ENV === 'test') opts.listenAddress = '0.0.0.0';
if (process.env.NODE_ENV === 'test') opts.listenAddress = '0.0.0.0';
return opts;
});
const clearChannels = () => {
const {logger} = require('../..');
const pwd = fsInventory[0].secret;
const maxDurationMins = JAMBONES_FREESWITCH_MAX_CALL_DURATION_MINS;
const maxDurationMins = process.env.JAMBONES_FREESWITCH_MAX_CALL_DURATION_MINS || 180;
const calls = execSync(`/usr/local/freeswitch/bin/fs_cli -p ${pwd} -x "show calls"`, {encoding: 'utf8'})
.split('\n')

View File

@@ -3,10 +3,13 @@ const {decrypt} = require('./encrypt-decrypt');
const sqlAccountDetails = `SELECT *
FROM accounts account
WHERE account.account_sid = ?`;
const sqlSpeechCredentialsForAccount = `SELECT *
const sqlSpeechCredentials = `SELECT *
FROM speech_credentials
WHERE account_sid = ? OR (account_sid is NULL AND service_provider_sid =
(SELECT service_provider_sid from accounts where account_sid = ?))`;
WHERE account_sid = ? `;
const sqlSpeechCredentialsForSP = `SELECT *
FROM speech_credentials
WHERE service_provider_sid =
(SELECT service_provider_sid from accounts where account_sid = ?)`;
const sqlQueryAccountCarrierByName = `SELECT voip_carrier_sid
FROM voip_carriers vc
WHERE vc.account_sid = ?
@@ -17,19 +20,6 @@ WHERE vc.account_sid IS NULL
AND vc.service_provider_sid =
(SELECT service_provider_sid from accounts where account_sid = ?)
AND vc.name = ?`;
const sqlQueryAccountPhoneNumber = `SELECT voip_carrier_sid
FROM phone_numbers pn
WHERE pn.account_sid = ?
AND pn.number = ?`;
const sqlQuerySPPhoneNumber = `SELECT voip_carrier_sid
FROM phone_numbers pn
WHERE pn.account_sid IS NULL
AND pn.service_provider_sid =
(SELECT service_provider_sid from accounts where account_sid = ?)
AND pn.number = ?`;
const sqlQueryGoogleCustomVoices = `SELECT *
FROM google_custom_voices
WHERE google_custom_voice_sid = ?`;
const speechMapper = (cred) => {
const {credential, ...obj} = cred;
@@ -41,7 +31,6 @@ const speechMapper = (cred) => {
const o = JSON.parse(decrypt(credential));
obj.access_key_id = o.access_key_id;
obj.secret_access_key = o.secret_access_key;
obj.role_arn = o.role_arn;
obj.aws_region = o.aws_region;
}
else if ('microsoft' === obj.vendor) {
@@ -50,10 +39,8 @@ const speechMapper = (cred) => {
obj.region = o.region;
obj.use_custom_stt = o.use_custom_stt;
obj.custom_stt_endpoint = o.custom_stt_endpoint;
obj.custom_stt_endpoint_url = o.custom_stt_endpoint_url;
obj.use_custom_tts = o.use_custom_tts;
obj.custom_tts_endpoint = o.custom_tts_endpoint;
obj.custom_tts_endpoint_url = o.custom_tts_endpoint_url;
}
else if ('wellsaid' === obj.vendor) {
const o = JSON.parse(decrypt(credential));
@@ -76,83 +63,16 @@ const speechMapper = (cred) => {
else if ('deepgram' === obj.vendor) {
const o = JSON.parse(decrypt(credential));
obj.api_key = o.api_key;
obj.deepgram_stt_uri = o.deepgram_stt_uri;
obj.deepgram_tts_uri = o.deepgram_tts_uri;
obj.deepgram_stt_use_tls = o.deepgram_stt_use_tls;
}
else if ('soniox' === obj.vendor) {
const o = JSON.parse(decrypt(credential));
obj.api_key = o.api_key;
}
else if ('nvidia' === obj.vendor) {
const o = JSON.parse(decrypt(credential));
obj.riva_server_uri = o.riva_server_uri;
}
else if ('cobalt' === obj.vendor) {
const o = JSON.parse(decrypt(credential));
obj.cobalt_server_uri = o.cobalt_server_uri;
}
else if ('elevenlabs' === obj.vendor) {
const o = JSON.parse(decrypt(credential));
obj.api_key = o.api_key;
obj.model_id = o.model_id;
obj.options = o.options;
}
else if ('playht' === obj.vendor) {
const o = JSON.parse(decrypt(credential));
obj.api_key = o.api_key;
obj.user_id = o.user_id;
obj.voice_engine = o.voice_engine;
obj.options = o.options;
}
else if ('cartesia' === obj.vendor) {
const o = JSON.parse(decrypt(credential));
obj.api_key = o.api_key;
obj.model_id = o.model_id;
obj.embedding = o.embedding;
obj.options = o.options;
}
else if ('rimelabs' === obj.vendor) {
const o = JSON.parse(decrypt(credential));
obj.api_key = o.api_key;
obj.model_id = o.model_id;
obj.options = o.options;
}
else if ('assemblyai' === obj.vendor) {
const o = JSON.parse(decrypt(credential));
obj.api_key = o.api_key;
}
else if ('voxist' === obj.vendor) {
const o = JSON.parse(decrypt(credential));
obj.api_key = o.api_key;
}
else if ('whisper' === obj.vendor) {
const o = JSON.parse(decrypt(credential));
obj.api_key = o.api_key;
obj.model_id = o.model_id;
}
else if ('verbio' === obj.vendor) {
const o = JSON.parse(decrypt(credential));
obj.client_id = o.client_id;
obj.client_secret = o.client_secret;
obj.engine_version = o.engine_version;
}
else if ('speechmatics' === obj.vendor) {
const o = JSON.parse(decrypt(credential));
obj.api_key = o.api_key;
obj.speechmatics_stt_uri = o.speechmatics_stt_uri;
}
else if ('openai' === obj.vendor) {
const o = JSON.parse(decrypt(credential));
obj.api_key = o.api_key;
obj.model_id = o.model_id;
}
else if (obj.vendor.startsWith('custom:')) {
const o = JSON.parse(decrypt(credential));
obj.auth_token = o.auth_token;
obj.custom_stt_url = o.custom_stt_url;
obj.custom_tts_url = o.custom_tts_url;
obj.custom_tts_streaming_url = o.custom_tts_streaming_url;
}
} catch (err) {
console.log(err);
@@ -160,28 +80,27 @@ const speechMapper = (cred) => {
return obj;
};
const bucketCredentialDecrypt = (account) => {
const { bucket_credential } = account.account;
if (!bucket_credential || bucket_credential.vendor) return;
account.account.bucket_credential = JSON.parse(decrypt(bucket_credential));
};
module.exports = (logger, srf) => {
const {pool} = srf.locals.dbHelpers;
const pp = pool.promise();
const lookupAccountDetails = async(account_sid) => {
const [r] = await pp.query({sql: sqlAccountDetails, nestTables: true}, [account_sid]);
const [r] = await pp.query({sql: sqlAccountDetails, nestTables: true}, account_sid);
if (0 === r.length) throw new Error(`invalid accountSid: ${account_sid}`);
const [r2] = await pp.query(sqlSpeechCredentialsForAccount, [account_sid, account_sid]);
const [r2] = await pp.query(sqlSpeechCredentials, account_sid);
const speech = r2.map(speechMapper);
const account = r[0];
bucketCredentialDecrypt(account);
/* add service provider creds unless we have that vendor at the account level */
const [r3] = await pp.query(sqlSpeechCredentialsForSP, account_sid);
r3.forEach((s) => {
if (!speech.find((s2) => s2.vendor === s.vendor)) {
speech.push(speechMapper(s));
}
});
return {
...account,
...r[0],
speech
};
};
@@ -209,46 +128,9 @@ module.exports = (logger, srf) => {
}
};
const lookupCarrierByPhoneNumber = async(account_sid, phoneNumber) => {
const pp = pool.promise();
try {
const [r] = await pp.query(sqlQueryAccountPhoneNumber, [account_sid, phoneNumber]);
if (r.length) return r[0].voip_carrier_sid;
const [r2] = await pp.query(sqlQuerySPPhoneNumber, [account_sid, phoneNumber]);
if (r2.length) return r2[0].voip_carrier_sid;
} catch (err) {
logger.error({err}, `lookupPhoneNumber: Error ${account_sid}:${phoneNumber}`);
}
};
const lookupGoogleCustomVoice = async(google_custom_voice_sid) => {
const pp = pool.promise();
try {
const [r] = await pp.query(sqlQueryGoogleCustomVoices, [google_custom_voice_sid]);
return r;
} catch (err) {
logger.error({err}, `lookupGoogleCustomVoices: Error ${google_custom_voice_sid}`);
}
};
const lookupVoipCarrierBySid = async(sid) => {
const pp = pool.promise();
try {
const [r] = await pp.query('SELECT * FROM voip_carriers WHERE voip_carrier_sid = ?', [sid]);
return r;
} catch (err) {
logger.error({err}, `lookupVoipCarrierBySid: Error ${sid}`);
}
};
return {
lookupAccountDetails,
updateSpeechCredentialLastUsed,
lookupCarrier,
lookupCarrierByPhoneNumber,
lookupGoogleCustomVoice,
lookupVoipCarrierBySid
lookupCarrier
};
};

View File

@@ -1,9 +1,8 @@
const crypto = require('crypto');
const {LEGACY_CRYPTO, ENCRYPTION_SECRET, JWT_SECRET} = require('../config');
const algorithm = LEGACY_CRYPTO ? 'aes-256-ctr' : 'aes-256-cbc';
const algorithm = process.env.LEGACY_CRYPTO ? 'aes-256-ctr' : 'aes-256-cbc';
const iv = crypto.randomBytes(16);
const secretKey = crypto.createHash('sha256')
.update(ENCRYPTION_SECRET || JWT_SECRET)
.update(process.env.ENCRYPTION_SECRET || process.env.JWT_SECRET)
.digest('base64')
.substring(0, 32);

View File

@@ -1,33 +0,0 @@
class NonFatalTaskError extends Error {
constructor(msg) {
super(msg);
}
}
class SpeechCredentialError extends NonFatalTaskError {
constructor(msg) {
super(msg);
}
}
class PlayFileNotFoundError extends NonFatalTaskError {
constructor(url) {
super('File not found');
this.url = url;
}
}
class HTTPResponseError extends Error {
constructor(statusCode) {
super('Unexpected HTTP Response');
delete this.stack;
this.statusCode = statusCode;
}
}
module.exports = {
SpeechCredentialError,
NonFatalTaskError,
PlayFileNotFoundError,
HTTPResponseError
};

View File

@@ -1,5 +0,0 @@
const sleepFor = (ms) => new Promise((resolve) => setTimeout(() => resolve(), ms));
module.exports = {
sleepFor
};

View File

@@ -1,21 +1,20 @@
const express = require('express');
const httpRoutes = require('../http-routes');
const {PORT, HTTP_PORT_MAX} = require('../config');
const PORT = process.env.HTTP_PORT || 3000;
const doListen = (logger, app, port, resolve) => {
const server = app.listen(port, () => {
const {srf} = app.locals;
srf.locals.serviceUrl = `http://${srf.locals.ipv4}:${port}`;
logger.info(`listening for HTTP requests on port ${port}, serviceUrl is ${srf.locals.serviceUrl}`);
logger.info(`listening for HTTP requests on port ${PORT}, serviceUrl is ${srf.locals.serviceUrl}`);
resolve({server, app});
});
return server;
};
const handleErrors = (logger, app, resolve, reject, e) => {
if (e.code === 'EADDRINUSE' &&
HTTP_PORT_MAX &&
e.port < HTTP_PORT_MAX) {
process.env.HTTP_PORT_MAX &&
e.port < process.env.HTTP_PORT_MAX) {
logger.info(`HTTP server failed to bind port on ${e.port}, will try next port`);
const server = doListen(logger, app, ++e.port, resolve);

View File

@@ -1,22 +1,11 @@
const {request, getGlobalDispatcher, setGlobalDispatcher, Dispatcher, ProxyAgent, Client, Pool} = require('undici');
const {Client, Pool} = require('undici');
const parseUrl = require('parse-url');
const assert = require('assert');
const BaseRequestor = require('./base-requestor');
const {HookMsgTypes} = require('./constants.json');
const snakeCaseKeys = require('./snakecase-keys');
const pools = new Map();
const {
HTTP_POOL,
HTTP_POOLSIZE,
HTTP_PIPELINING,
HTTP_TIMEOUT,
HTTP_PROXY_IP,
HTTP_PROXY_PORT,
HTTP_PROXY_PROTOCOL,
NODE_ENV,
HTTP_USER_AGENT_HEADER,
} = require('../config');
const {HTTPResponseError} = require('./error');
const HTTP_TIMEOUT = 10000;
const toBase64 = (str) => Buffer.from(str || '', 'utf8').toString('base64');
@@ -27,15 +16,6 @@ function basicAuth(username, password) {
return {Authorization: header};
}
const defaultDispatcher = HTTP_PROXY_IP ?
new ProxyAgent(`${HTTP_PROXY_PROTOCOL}://${HTTP_PROXY_IP}${HTTP_PROXY_PORT ? `:${HTTP_PROXY_PORT}` : ''}`) :
getGlobalDispatcher();
setGlobalDispatcher(new class extends Dispatcher {
dispatch(options, handler) {
return defaultDispatcher.dispatch(options, handler);
}
}());
class HttpRequestor extends BaseRequestor {
constructor(logger, account_sid, hook, secret) {
@@ -48,43 +28,37 @@ class HttpRequestor extends BaseRequestor {
assert(['GET', 'POST'].includes(this.method));
const u = this._parsedUrl = parseUrl(this.url);
if (u.port) this._baseUrl = `${u.protocol}://${u.resource}:${u.port}`;
else this._baseUrl = `${u.protocol}://${u.resource}`;
this._protocol = u.protocol;
this._resource = u.resource;
this._port = u.port;
this._search = u.search;
this._usePools = HTTP_POOL && parseInt(HTTP_POOL);
this._usePools = process.env.HTTP_POOL && parseInt(process.env.HTTP_POOL);
if (this._usePools) {
if (pools.has(this.baseUrl)) {
this.client = pools.get(this.baseUrl);
if (pools.has(this._baseUrl)) {
this.client = pools.get(this._baseUrl);
}
else {
const connections = HTTP_POOLSIZE ? parseInt(HTTP_POOLSIZE) : 10;
const pipelining = HTTP_PIPELINING ? parseInt(HTTP_PIPELINING) : 1;
const pool = this.client = new Pool(this.baseUrl, {
const connections = process.env.HTTP_POOLSIZE ? parseInt(process.env.HTTP_POOLSIZE) : 10;
const pipelining = process.env.HTTP_PIPELINING ? parseInt(process.env.HTTP_PIPELINING) : 1;
const pool = this.client = new Pool(this._baseUrl, {
connections,
pipelining
});
pools.set(this.baseUrl, pool);
this.logger.debug(`HttpRequestor:created pool for ${this.baseUrl}`);
pools.set(this._baseUrl, pool);
this.logger.debug(`HttpRequestor:created pool for ${this._baseUrl}`);
}
}
else {
if (u.port) this.client = new Client(`${u.protocol}://${u.resource}:${u.port}`);
else this.client = new Client(`${u.protocol}://${u.resource}`);
}
}
if (NODE_ENV == 'test' && process.env.JAMBONES_HTTP_PROXY_IP) {
const defDispatcher =
new ProxyAgent(`${process.env.JAMBONES_HTTP_PROXY_PROTOCOL}://${process.env.JAMBONES_HTTP_PROXY_IP}${
process.env.JAMBONES_HTTP_PROXY_PORT ? `:${process.env.JAMBONES_HTTP_PROXY_PORT}` : ''}`);
setGlobalDispatcher(new class extends Dispatcher {
dispatch(options, handler) {
return defDispatcher.dispatch(options, handler);
}
}());
}
get baseUrl() {
return this._baseUrl;
}
close() {
@@ -108,14 +82,10 @@ class HttpRequestor extends BaseRequestor {
assert(HookMsgTypes.includes(type));
const payload = params ? snakeCaseKeys(params, ['customerData', 'sip', 'env_vars', 'args']) : null;
const payload = params ? snakeCaseKeys(params, ['customerData', 'sip']) : null;
const url = hook.url || hook;
const method = hook.method || 'POST';
let buf = '';
httpHeaders = {
...httpHeaders,
...(HTTP_USER_AGENT_HEADER && {'user-agent' : HTTP_USER_AGENT_HEADER})
};
assert.ok(url, 'HttpRequestor:request url was not provided');
assert.ok, (['GET', 'POST'].includes(method), `HttpRequestor:request method must be 'GET' or 'POST' not ${method}`);
@@ -164,18 +134,7 @@ class HttpRequestor extends BaseRequestor {
};
const absUrl = this._isRelativeUrl(url) ? `${this.baseUrl}${url}` : url;
this.logger.debug({url, absUrl, hdrs}, 'send webhook');
const {statusCode, headers, body} = HTTP_PROXY_IP ? await request(
this.baseUrl,
{
path,
query,
method,
headers: hdrs,
...('POST' === method && {body: JSON.stringify(payload)}),
timeout: HTTP_TIMEOUT,
followRedirects: false
}
) : await client.request({
const {statusCode, headers, body} = await client.request({
path,
query,
method,
@@ -185,7 +144,8 @@ class HttpRequestor extends BaseRequestor {
followRedirects: false
});
if (![200, 202, 204].includes(statusCode)) {
const err = new HTTPResponseError(statusCode);
const err = new Error();
err.statusCode = statusCode;
throw err;
}
if (headers['content-type']?.includes('application/json')) {
@@ -219,7 +179,7 @@ class HttpRequestor extends BaseRequestor {
const rtt = this._roundTrip(startAt);
if (buf) this.stats.histogram('app.hook.response_time', rtt, ['hook_type:app']);
if (buf && (Array.isArray(buf) || type == 'llm:tool-call')) {
if (buf && Array.isArray(buf)) {
this.logger.info({response: buf}, `HttpRequestor:request ${method} ${url} succeeded in ${rtt}ms`);
return buf;
}

View File

@@ -1,56 +1,20 @@
const Mrf = require('drachtio-fsmrf');
const os = require('os');
const {
JAMBONES_MYSQL_HOST,
JAMBONES_MYSQL_USER,
JAMBONES_MYSQL_PASSWORD,
JAMBONES_MYSQL_DATABASE,
JAMBONES_MYSQL_CONNECTION_LIMIT,
JAMBONES_MYSQL_PORT,
JAMBONES_FREESWITCH,
SMPP_URL,
JAMBONES_TIME_SERIES_HOST,
JAMBONES_ESL_LISTEN_ADDRESS,
PORT,
HTTP_IP,
NODE_ENV,
} = require('../config');
const Registrar = require('@jambonz/mw-registrar');
const ip = require('ip');
const PORT = process.env.HTTP_PORT || 3000;
const assert = require('assert');
function getLocalIp() {
const interfaces = os.networkInterfaces();
for (const interfaceName in interfaces) {
const interface = interfaces[interfaceName];
for (const iface of interface) {
if (iface.family === 'IPv4' && !iface.internal) {
return iface.address;
}
}
}
return '127.0.0.1'; // Fallback to localhost if no suitable interface found
}
function initMS(logger, wrapper, ms, {
onFreeswitchConnect,
onFreeswitchDisconnect
}) {
function initMS(logger, wrapper, ms) {
Object.assign(wrapper, {ms, active: true, connects: 1});
logger.info(`connected to freeswitch at ${ms.address}`);
onFreeswitchConnect(wrapper);
ms.conn
.on('esl::end', () => {
wrapper.active = false;
wrapper.connects = 0;
logger.info(`lost connection to freeswitch at ${ms.address}`);
onFreeswitchDisconnect(wrapper);
ms.removeAllListeners();
})
.on('esl::ready', () => {
if (wrapper.connects > 0) {
logger.info(`esl::ready connected to freeswitch at ${ms.address}`);
logger.info(`connected to freeswitch at ${ms.address}`);
}
wrapper.connects = 1;
wrapper.active = true;
@@ -64,10 +28,7 @@ function initMS(logger, wrapper, ms, {
});
}
function installSrfLocals(srf, logger, {
onFreeswitchConnect = () => {},
onFreeswitchDisconnect = () => {}
}) {
function installSrfLocals(srf, logger) {
logger.debug('installing srf locals');
assert(!srf.locals.dbHelpers);
const {tracer} = srf.locals.otel;
@@ -81,18 +42,18 @@ function installSrfLocals(srf, logger, {
let idxStart = 0;
(async function() {
const fsInventory = JAMBONES_FREESWITCH
const fsInventory = process.env.JAMBONES_FREESWITCH
.split(',')
.map((fs) => {
const arr = /^([^:]*):([^:]*):([^:]*)(?::([^:]*))?/.exec(fs);
assert.ok(arr, `Invalid syntax JAMBONES_FREESWITCH: ${JAMBONES_FREESWITCH}`);
assert.ok(arr, `Invalid syntax JAMBONES_FREESWITCH: ${process.env.JAMBONES_FREESWITCH}`);
const opts = {address: arr[1], port: arr[2], secret: arr[3]};
if (arr.length > 4) opts.advertisedAddress = arr[4];
/* NB: originally for testing only, but for now all jambonz deployments
have freeswitch installed locally alongside this app
*/
if (NODE_ENV === 'test') opts.listenAddress = '0.0.0.0';
else if (JAMBONES_ESL_LISTEN_ADDRESS) opts.listenAddress = JAMBONES_ESL_LISTEN_ADDRESS;
if (process.env.NODE_ENV === 'test') opts.listenAddress = '0.0.0.0';
else if (process.env.JAMBONES_ESL_LISTEN_ADDRESS) opts.listenAddress = process.env.JAMBONES_ESL_LISTEN_ADDRESS;
return opts;
});
logger.info({fsInventory}, 'freeswitch inventory');
@@ -102,10 +63,7 @@ function installSrfLocals(srf, logger, {
mediaservers.push(val);
try {
const ms = await mrf.connect(fs);
initMS(logger, val, ms, {
onFreeswitchConnect,
onFreeswitchDisconnect
});
initMS(logger, val, ms);
}
catch (err) {
logger.info({err}, `failed connecting to freeswitch at ${fs.address}, will retry shortly: ${err.message}`);
@@ -116,15 +74,9 @@ function installSrfLocals(srf, logger, {
for (const val of mediaservers) {
if (val.connects === 0) {
try {
// make sure all listeners are removed before reconnecting
val.ms?.disconnect();
val.ms = null;
logger.info({mediaserver: val.opts}, 'Retrying initial connection to media server');
const ms = await mrf.connect(val.opts);
initMS(logger, val, ms, {
onFreeswitchConnect,
onFreeswitchDisconnect
});
initMS(logger, val, ms);
} catch (err) {
logger.info({err}, `failed connecting to freeswitch at ${val.opts.address}, will retry shortly`);
}
@@ -171,17 +123,15 @@ function installSrfLocals(srf, logger, {
lookupTeamsByAccount,
lookupAccountBySid,
lookupAccountCapacitiesBySid,
lookupSmppGateways,
lookupClientByAccountAndUsername,
lookupSystemInformation
lookupSmppGateways
} = require('@jambonz/db-helpers')({
host: JAMBONES_MYSQL_HOST,
user: JAMBONES_MYSQL_USER,
port: JAMBONES_MYSQL_PORT || 3306,
password: JAMBONES_MYSQL_PASSWORD,
database: JAMBONES_MYSQL_DATABASE,
connectionLimit: JAMBONES_MYSQL_CONNECTION_LIMIT || 10
}, logger);
host: process.env.JAMBONES_MYSQL_HOST,
user: process.env.JAMBONES_MYSQL_USER,
port: process.env.JAMBONES_MYSQL_PORT || 3306,
password: process.env.JAMBONES_MYSQL_PASSWORD,
database: process.env.JAMBONES_MYSQL_DATABASE,
connectionLimit: process.env.JAMBONES_MYSQL_CONNECTION_LIMIT || 10
}, logger, tracer);
const {
client,
updateCallStatus,
@@ -202,35 +152,30 @@ function installSrfLocals(srf, logger, {
removeFromList,
getListPosition,
lengthOfList,
addToSortedSet,
retrieveFromSortedSet,
retrieveByPatternSortedSet,
sortedSetLength,
sortedSetPositionByPattern,
} = require('@jambonz/realtimedb-helpers')({}, logger, tracer);
const registrar = new Registrar(logger, client);
} = require('@jambonz/realtimedb-helpers')({
host: process.env.JAMBONES_REDIS_HOST,
port: process.env.JAMBONES_REDIS_PORT || 6379
}, logger, tracer);
const {
synthAudio,
addFileToCache,
getNuanceAccessToken,
getIbmAccessToken,
getAwsAuthToken,
getVerbioAccessToken
} = require('@jambonz/speech-utils')({}, logger);
} = require('@jambonz/speech-utils')({
host: process.env.JAMBONES_REDIS_HOST,
port: process.env.JAMBONES_REDIS_PORT || 6379
}, logger, tracer);
const {
writeAlerts,
AlertType,
writeSystemAlerts
AlertType
} = require('@jambonz/time-series')(logger, {
host: JAMBONES_TIME_SERIES_HOST,
host: process.env.JAMBONES_TIME_SERIES_HOST,
commitSize: 50,
commitInterval: 'test' === NODE_ENV ? 7 : 20
commitInterval: 'test' === process.env.NODE_ENV ? 7 : 20
});
let localIp;
try {
// Either use the configured IP address or discover it
localIp = HTTP_IP || getLocalIp();
localIp = ip.address();
} catch (err) {
logger.error({err}, 'installSrfLocals - error detecting local ipv4 address');
}
@@ -238,7 +183,6 @@ function installSrfLocals(srf, logger, {
srf.locals = {...srf.locals,
dbHelpers: {
client,
registrar,
pool,
lookupAppByPhoneNumber,
lookupAppByRegex,
@@ -249,15 +193,11 @@ function installSrfLocals(srf, logger, {
lookupAccountBySid,
lookupAccountCapacitiesBySid,
lookupSmppGateways,
lookupClientByAccountAndUsername,
lookupSystemInformation,
updateCallStatus,
retrieveCall,
listCalls,
deleteCall,
synthAudio,
getAwsAuthToken,
addFileToCache,
createHash,
retrieveHash,
deleteKey,
@@ -273,25 +213,18 @@ function installSrfLocals(srf, logger, {
lengthOfList,
getListPosition,
getNuanceAccessToken,
getIbmAccessToken,
addToSortedSet,
retrieveFromSortedSet,
retrieveByPatternSortedSet,
sortedSetLength,
sortedSetPositionByPattern,
getVerbioAccessToken
getIbmAccessToken
},
parentLogger: logger,
getSBC,
getSmpp: () => {
return SMPP_URL;
return process.env.SMPP_URL;
},
lifecycleEmitter,
getFreeswitch,
stats: stats,
writeAlerts,
AlertType,
writeSystemAlerts
AlertType
};
if (localIp) {

View File

@@ -1,103 +0,0 @@
const { Client } = require('@modelcontextprotocol/sdk/client/index.js');
class LlmMcpService {
constructor(logger, mcpServers) {
this.logger = logger;
this.mcpServers = mcpServers || [];
this.mcpClients = [];
}
// make sure we call init() before using any of the mcp clients
// this is to ensure that we have a valid connection to the MCP server
// and that we have collected the available tools.
async init() {
if (this.mcpClients.length > 0) {
return;
}
const { SSEClientTransport } = await import('@modelcontextprotocol/sdk/client/sse.js');
for (const server of this.mcpServers) {
const { url } = server;
if (url) {
try {
const transport = new SSEClientTransport(new URL(url), {});
const client = new Client({ name: 'Jambonz MCP Client', version: '1.0.0' });
await client.connect(transport);
// collect available tools
const { tools } = await client.listTools();
this.mcpClients.push({
url,
client,
tools
});
} catch (err) {
this.logger.error(`LlmMcpService: Failed to connect to MCP server at ${url}: ${err.message}`);
}
}
}
}
async getAvailableMcpTools() {
// returns a list of available tools from all MCP clients
const tools = [];
for (const mcpClient of this.mcpClients) {
const {tools: availableTools} = mcpClient;
if (availableTools) {
tools.push(...availableTools);
}
}
return tools;
}
async getMcpClientByToolName(name) {
for (const mcpClient of this.mcpClients) {
const { tools } = mcpClient;
if (tools && tools.some((tool) => tool.name === name)) {
return mcpClient.client;
}
}
return null;
}
async getMcpClientByToolId(id) {
for (const mcpClient of this.mcpClients) {
const { tools } = mcpClient;
if (tools && tools.some((tool) => tool.id === id)) {
return mcpClient.client;
}
}
return null;
}
async callMcpTool(name, input) {
const client = await this.getMcpClientByToolName(name);
if (client) {
try {
const result = await client.callTool({
name,
arguments: input,
});
this.logger.debug({result}, 'LlmMcpService - result');
return result;
} catch (err) {
this.logger.error({err}, 'LlmMcpService - error calling tool');
throw err;
}
}
}
async close() {
for (const mcpClient of this.mcpClients) {
const { client } = mcpClient;
if (client) {
await client.close();
this.logger.debug({url: mcpClient.url}, 'LlmMcpService - mcp client closed');
}
}
this.mcpClients = [];
}
}
module.exports = LlmMcpService;

View File

@@ -1,32 +0,0 @@
/**
* Parses a list of hostport entries and selects the first one that matches the specified protocol,
* excluding any entries with the localhost IP address ('127.0.0.1').
*
* Each hostport entry should be in the format: 'protocol/ip:port'
*
* @param {Object} logger - A logging object with a 'debug' method for logging debug messages.
* @param {string} hostport - A comma-separated string containing hostport entries.
* @param {string} protocol - The protocol to match (e.g., 'udp', 'tcp').
* @returns {Array} An array containing:
* 0: protocol
* 1: ip address
* 2: port
*/
const selectHostPort = (logger, hostport, protocol) => {
logger.debug(`selectHostPort: ${hostport}, ${protocol}`);
const sel = hostport
.split(',')
.map((hp) => {
const arr = /(.*)\/(.*):(.*)/.exec(hp);
return [arr[1], arr[2], arr[3]];
})
.filter((hp) => {
return hp[0] === protocol && hp[1] !== '127.0.0.1';
});
return sel[0];
};
module.exports = {
selectHostPort
};

View File

@@ -1,18 +0,0 @@
const parseDecibels = (db) => {
if (!db) return 0;
if (typeof db === 'number') {
return db;
}
else if (typeof db === 'string') {
const match = db.match(/([+-]?\d+(\.\d+)?)\s*db/i);
if (match) {
return Math.trunc(parseFloat(match[1]));
} else {
return 0;
}
} else {
return 0;
}
};
module.exports = parseDecibels;

View File

@@ -1,5 +1,5 @@
const Emitter = require('events');
const {CallStatus, MediaPath} = require('./constants');
const {CallStatus} = require('./constants');
const SipError = require('drachtio-srf').SipError;
const {TaskPreconditions, CallDirection} = require('../utils/constants');
const CallInfo = require('../session/call-info');
@@ -13,20 +13,9 @@ const moment = require('moment');
const stripCodecs = require('./strip-ancillary-codecs');
const RootSpan = require('./call-tracer');
const uuidv4 = require('uuid-random');
const HttpRequestor = require('./http-requestor');
const WsRequestor = require('./ws-requestor');
const {makeOpusFirst, removeVideoSdp} = require('./sdp-utils');
const {
JAMBONES_USE_FREESWITCH_TIMER_FD,
JAMBONES_MEDIA_TIMEOUT_MS,
JAMBONES_MEDIA_HOLD_TIMEOUT_MS,
JAMBONES_TRANSCRIBE_EP_DESTROY_DELAY_MS
} = require('../config');
const { sleepFor } = require('./helpers');
class SingleDialer extends Emitter {
constructor({logger, sbcAddress, target, opts, application, callInfo, accountInfo, rootSpan, startSpan, dialTask,
onHoldMusic}) {
constructor({logger, sbcAddress, target, opts, application, callInfo, accountInfo, rootSpan, startSpan}) {
super();
assert(target.type);
@@ -48,8 +37,6 @@ class SingleDialer extends Emitter {
this.callGone = false;
this.callSid = uuidv4();
this.dialTask = dialTask;
this.onHoldMusic = onHoldMusic;
this.on('callStatusChange', this._notifyCallStatusChange.bind(this));
}
@@ -58,10 +45,6 @@ class SingleDialer extends Emitter {
return this.callInfo.callStatus;
}
get applicationSid() {
return this.application?.application_sid || this.callInfo?.applicationSid;
}
/**
* can be used for all http requests within this session
*/
@@ -88,8 +71,7 @@ class SingleDialer extends Emitter {
...(this.from.host && {'X-Preferred-From-Host': this.from.host}),
'X-Jambonz-Routing': this.target.type,
'X-Call-Sid': this.callSid,
...(this.applicationSid && {'X-Application-Sid': this.applicationSid}),
...(this.target.proxy && {'X-SIP-Proxy': this.target.proxy})
...(this.applicationSid && {'X-Application-Sid': this.applicationSid})
};
if (srf.locals.fsUUID) {
opts.headers = {
@@ -140,7 +122,6 @@ class SingleDialer extends Emitter {
this.serviceUrl = srf.locals.serviceUrl;
this.ep = await ms.createEndpoint();
this._configMsEndpoint();
this.logger.debug(`SingleDialer:exec - created endpoint ${this.ep.uuid}`);
/**
@@ -154,21 +135,15 @@ class SingleDialer extends Emitter {
return;
}
let lastSdp;
const connectStream = async(remoteSdp, isVideoCall) => {
const connectStream = async(remoteSdp) => {
if (remoteSdp === lastSdp) return;
if (process.env.JAMBONES_VIDEO_CALLS_ENABLED_IN_FS && !isVideoCall) {
remoteSdp = removeVideoSdp(remoteSdp);
}
lastSdp = remoteSdp;
return this.ep.modify(remoteSdp);
};
let localSdp = this.ep.local.sdp;
if (process.env.JAMBONES_VIDEO_CALLS_ENABLED_IN_FS && !opts.isVideoCall) {
localSdp = removeVideoSdp(localSdp);
}
Object.assign(opts, {
proxy: `sip:${this.sbcAddress}`,
localSdp: opts.opusFirst ? makeOpusFirst(localSdp) : localSdp
localSdp: this.ep.local.sdp
});
if (this.target.auth) opts.auth = this.target.auth;
inviteSpan = this.startSpan('invite', {
@@ -196,7 +171,6 @@ class SingleDialer extends Emitter {
* (a) create a logger for this call
*/
req.srf = srf;
this.req = req;
this.callInfo = new CallInfo({
direction: CallDirection.Outbound,
parentCallInfo: this.parentCallInfo,
@@ -205,10 +179,6 @@ class SingleDialer extends Emitter {
callSid: this.callSid,
traceId: this.rootSpan.traceId
});
if (this.dialTask && this.dialTask.tag !== null &&
typeof this.dialTask.tag === 'object' && !Array.isArray(this.dialTask.tag)) {
this.callInfo.customerData = this.dialTask.tag;
}
this.logger = srf.locals.parentLogger.child({
callSid: this.callSid,
parentCallSid: this.parentCallInfo.callSid,
@@ -223,20 +193,18 @@ class SingleDialer extends Emitter {
},
cbProvisional: (prov) => {
const status = {sipStatus: prov.status, sipReason: prov.reason};
// Update call-id for sbc outbound INVITE
this.callInfo.sbcCallid = prov.get('X-CID');
if ([180, 183].includes(prov.status) && prov.body) {
if (status.callStatus !== CallStatus.EarlyMedia) {
status.callStatus = CallStatus.EarlyMedia;
this.emit('earlyMedia');
}
connectStream(prov.body, opts.isVideoCall);
connectStream(prov.body);
}
else status.callStatus = CallStatus.Ringing;
this.emit('callStatusChange', status);
}
});
await connectStream(this.dlg.remote.sdp, opts.isVideoCall);
await connectStream(this.dlg.remote.sdp);
this.dlg.callSid = this.callSid;
this.inviteInProgress = null;
this.emit('callStatusChange', {
@@ -275,19 +243,9 @@ class SingleDialer extends Emitter {
.on('modify', async(req, res) => {
try {
if (this.ep) {
if (this.dialTask && this.dialTask.isOnHoldEnabled) {
this.logger.info('dial is onhold, emit event');
this.emit('reinvite', req, res);
} else {
let newSdp = await this.ep.modify(req.body);
// in case of reINVITE if video call is enabled in FS and the call is not a video call,
// remove video media from the SDP
if (process.env.JAMBONES_VIDEO_CALLS_ENABLED_IN_FS && !this.opts?.isVideoCall) {
newSdp = removeVideoSdp(newSdp);
}
res.send(200, {body: newSdp});
this.logger.info({offer: req.body, answer: newSdp}, 'SingleDialer:exec: handling reINVITE');
}
const newSdp = await this.ep.modify(req.body);
res.send(200, {body: newSdp});
this.logger.info({offer: req.body, answer: newSdp}, 'SingleDialer:exec: handling reINVITE');
}
else {
this.logger.info('SingleDialer:exec: handling reINVITE with released media, emit event');
@@ -312,17 +270,17 @@ class SingleDialer extends Emitter {
if (err.status === 487) status.callStatus = CallStatus.NoAnswer;
else if ([486, 600].includes(err.status)) status.callStatus = CallStatus.Busy;
this.logger.info(`SingleDialer:exec outdial failure ${err.status}`);
inviteSpan?.setAttributes({'invite.status_code': err.status});
inviteSpan?.end();
inviteSpan.setAttributes({'invite.status_code': err.status});
inviteSpan.end();
}
else {
this.logger.error(err, 'SingleDialer:exec');
status.sipStatus = 500;
inviteSpan?.setAttributes({
inviteSpan.setAttributes({
'invite.status_code': 500,
'invite.err': err.message
});
inviteSpan?.end();
inviteSpan.end();
}
this.emit('callStatusChange', status);
if (this.ep) this.ep.destroy();
@@ -332,19 +290,14 @@ class SingleDialer extends Emitter {
/**
* kill the call in progress or the stable dialog, whichever we have
*/
async kill(Reason) {
async kill() {
this.killed = true;
if (this.inviteInProgress) await this.inviteInProgress.cancel();
else if (this.dlg && this.dlg.connected) {
const duration = moment().diff(this.dlg.connectTime, 'seconds');
this.logger.debug('SingleDialer:kill hanging up called party');
this.emit('callStatusChange', {callStatus: CallStatus.Completed, duration});
const headers = {
...(Reason && {'X-Reason': Reason})
};
this.dlg.destroy({
headers
});
this.dlg.destroy();
}
if (this.ep) {
this.logger.debug(`SingleDialer:kill - deleting endpoint ${this.ep.uuid}`);
@@ -352,45 +305,6 @@ class SingleDialer extends Emitter {
}
}
_configMsEndpoint() {
const opts = {
...(this.onHoldMusic && {holdMusic: `shout://${this.onHoldMusic.replace(/^https?:\/\//, '')}`}),
...(JAMBONES_USE_FREESWITCH_TIMER_FD && {timer_name: 'timerfd'}),
...(JAMBONES_MEDIA_TIMEOUT_MS && {media_timeout: JAMBONES_MEDIA_TIMEOUT_MS}),
...(JAMBONES_MEDIA_HOLD_TIMEOUT_MS && {media_hold_timeout: JAMBONES_MEDIA_HOLD_TIMEOUT_MS})
};
if (Object.keys(opts).length > 0) {
this.ep.set(opts);
}
if (this.dialTask?.inbandDtmfEnabled && !this.ep.inbandDtmfEnabled) {
// https://developer.signalwire.com/freeswitch/FreeSWITCH-Explained/Modules/mod-dptools/6587132/#0-about
try {
this.ep.execute('start_dtmf');
this.ep.inbandDtmfEnabled = true;
} catch (err) {
this.logger.info(err, 'place-outdial:_configMsEndpoint - error enable inband DTMF');
}
}
const origDestroy = this.ep.destroy.bind(this.ep);
this.ep.destroy = async() => {
try {
if (this.dialTask.transcribeTask && JAMBONES_TRANSCRIBE_EP_DESTROY_DELAY_MS) {
// transcribe task is being used, wait for some time before destroy
// if final transcription is received but endpoint is already closed,
// freeswitch module will not be able to send the transcription
this.logger.info('SingleDialer:_configMsEndpoint -' +
' Dial with transcribe task, wait for some time before destroy');
await sleepFor(JAMBONES_TRANSCRIBE_EP_DESTROY_DELAY_MS);
}
await origDestroy();
} catch (err) {
this.logger.error(err, 'SingleDialer:_configMsEndpoint - error destroying endpoint');
}
};
}
/**
* Run an application on the call after answer, e.g. call screening.
* Once the application completes in some fashion, emit an 'accepted' event
@@ -402,16 +316,10 @@ class SingleDialer extends Emitter {
try {
// retrieve set of tasks
const json = await this.requestor.request('dial:confirm', confirmHook, this.callInfo.toJSON());
if (!json || (Array.isArray(json) && json.length === 0)) {
this.logger.info('SingleDialer:_executeApp: no tasks returned from confirm hook');
this.emit('accept');
return;
}
const tasks = normalizeJambones(this.logger, json).map((tdata) => makeTask(this.logger, tdata));
// verify it contains only allowed verbs
const allowedTasks = tasks.filter((task) => {
return [
TaskPreconditions.None,
TaskPreconditions.StableCall,
TaskPreconditions.Endpoint
].includes(task.preconditions);
@@ -430,8 +338,7 @@ class SingleDialer extends Emitter {
callInfo: this.callInfo,
accountInfo: this.accountInfo,
tasks,
rootSpan: this.rootSpan,
req: this.req
rootSpan: this.rootSpan
});
await cs.exec();
@@ -440,10 +347,7 @@ class SingleDialer extends Emitter {
} catch (err) {
this.logger.debug(err, 'SingleDialer:_executeApp: error');
this.emit('decline');
if (this.dlg.connected) {
this.dlg.destroy();
this.ep.destroy();
}
if (this.dlg.connected) this.dlg.destroy();
}
}
@@ -463,83 +367,40 @@ class SingleDialer extends Emitter {
this.dlg.linkedSpanId = this.rootSpan.traceId;
const rootSpan = new RootSpan('outbound-call', this.dlg);
const newLogger = logger.child({traceId: rootSpan.traceId});
//clone application from parent call with new requestor
//parrent application will be closed in case the parent hangup
const app = {...application};
if ('WS' === app.call_hook?.method ||
app.call_hook?.url.startsWith('ws://') || app.call_hook?.url.startsWith('wss://')) {
if (app.call_hook?.url) app.call_hook.url += '/adulting';
const requestor = new WsRequestor(logger, this.accountInfo.account.account_sid,
app.call_hook, this.accountInfo.account.webhook_secret);
app.requestor = requestor;
app.notifier = requestor;
app.call_hook.method = 'WS';
}
else {
app.requestor = new HttpRequestor(logger, this.accountInfo.account.account_sid,
app.call_hook, this.accountInfo.account.webhook_secret);
if (app.call_status_hook) app.notifier = new HttpRequestor(logger,
this.accountInfo.account.account_sid, app.call_status_hook,
this.accountInfo.account.webhook_secret);
else app.notifier = {request: () => {}, close: () => {}};
}
// Replace old application with new application.
this.application = app;
const cs = new AdultingCallSession({
logger: newLogger,
singleDialer: this,
application: app,
application,
callInfo: this.callInfo,
accountInfo: this.accountInfo,
tasks,
rootSpan
});
app.requestor.request('session:adulting', '/adulting', {
...cs.callInfo.toJSON(),
parentCallInfo: this.parentCallInfo.toJSON()
}).catch((err) => {
newLogger.error({err}, 'doAdulting: error sending adulting request');
});
cs.req = this.req;
// fixed hangup an adulting session does not send status callback Completed
cs.wrapDialog(this.dlg);
cs.exec().catch((err) => newLogger.error({err}, 'doAdulting: error executing session'));
return cs;
}
async releaseMediaToSBC(remoteSdp, localSdp, releaseMediaEntirely) {
async releaseMediaToSBC(remoteSdp, localSdp) {
assert(this.dlg && this.dlg.connected && this.ep && typeof remoteSdp === 'string');
const sdp = stripCodecs(this.logger, remoteSdp, localSdp) || remoteSdp;
await this.dlg.modify(sdp, {
headers: {
'X-Reason': releaseMediaEntirely ? 'release-media-entirely' : 'release-media'
'X-Reason': 'release-media'
}
});
try {
await this.ep.destroy();
} catch (err) {
this.logger.error({err}, 'SingleDialer:releaseMediaToSBC: Error destroying endpoint');
}
this.ep = null;
this.ep.destroy()
.then(() => this.ep = null)
.catch((err) => this.logger.error({err}, 'SingleDialer:releaseMediaToSBC: Error destroying endpoint'));
}
async reAnchorMedia(currentMediaRoute = MediaPath.PartialMedia) {
async reAnchorMedia() {
assert(this.dlg && this.dlg.connected && !this.ep);
this.logger.debug('SingleDialer:reAnchorMedia: re-anchoring media after partial media');
this.ep = await this.ms.createEndpoint({remoteSdp: this.dlg.remote.sdp});
this._configMsEndpoint();
await this.dlg.modify(this.ep.local.sdp, {
headers: {
'X-Reason': 'anchor-media'
}
});
if (currentMediaRoute === MediaPath.NoMedia) {
this.logger.debug('SingleDialer:reAnchorMedia: repoint endpoint after no media');
await this.ep.modify(this.dlg.remote.sdp);
}
}
_notifyCallStatusChange({callStatus, sipStatus, sipReason, duration}) {
@@ -565,13 +426,11 @@ class SingleDialer extends Emitter {
}
function placeOutdial({
logger, srf, ms, sbcAddress, target, opts, application, callInfo, accountInfo, rootSpan, startSpan, dialTask,
onHoldMusic
logger, srf, ms, sbcAddress, target, opts, application, callInfo, accountInfo, rootSpan, startSpan
}) {
const myOpts = deepcopy(opts);
const sd = new SingleDialer({
logger, sbcAddress, target, opts: myOpts, application, callInfo,
accountInfo, rootSpan, startSpan, dialTask, onHoldMusic
logger, sbcAddress, target, myOpts, application, callInfo, accountInfo, rootSpan, startSpan
});
sd.exec(srf, ms, myOpts);
return sd;

View File

@@ -1,9 +1,5 @@
const assert = require('assert');
const timeSeries = require('@jambonz/time-series');
const {
NODE_ENV,
JAMBONES_TIME_SERIES_HOST
} = require('../config');
let alerter ;
function isAbsoluteUrl(u) {
@@ -32,9 +28,9 @@ class Requestor {
if (!alerter) {
alerter = timeSeries(logger, {
host: JAMBONES_TIME_SERIES_HOST,
host: process.env.JAMBONES_TIME_SERIES_HOST,
commitSize: 50,
commitInterval: 'test' === NODE_ENV ? 7 : 20
commitInterval: 'test' === process.env.NODE_ENV ? 7 : 20
});
}
}
@@ -42,9 +38,9 @@ class Requestor {
get Alerter() {
if (!alerter) {
alerter = timeSeries(this.logger, {
host: JAMBONES_TIME_SERIES_HOST,
host: process.env.JAMBONES_TIME_SERIES_HOST,
commitSize: 50,
commitInterval: 'test' === NODE_ENV ? 7 : 20
commitInterval: 'test' === process.env.NODE_ENV ? 7 : 20
});
}
return alerter;

View File

@@ -4,38 +4,28 @@ const {LifeCycleEvents, FS_UUID_SET_NAME} = require('./constants');
const Emitter = require('events');
const debug = require('debug')('jambonz:feature-server');
const noopLogger = {info: () => {}, error: () => {}};
const {
JAMBONES_SBCS,
K8S,
K8S_SBC_SIP_SERVICE_NAME,
AWS_SNS_TOPIC_ARN,
OPTIONS_PING_INTERVAL,
AWS_REGION,
NODE_ENV,
JAMBONES_CLUSTER_ID,
} = require('../config');
module.exports = (logger) => {
logger = logger || noopLogger;
let idxSbc = 0;
let sbcs = [];
if (JAMBONES_SBCS) {
sbcs = JAMBONES_SBCS
if (process.env.JAMBONES_SBCS) {
sbcs = process.env.JAMBONES_SBCS
.split(',')
.map((sbc) => sbc.trim());
assert.ok(sbcs.length, 'JAMBONES_SBCS env var is empty or misconfigured');
logger.info({sbcs}, 'SBC inventory');
}
else if (K8S && K8S_SBC_SIP_SERVICE_NAME) {
sbcs = [`${K8S_SBC_SIP_SERVICE_NAME}:5060`];
else if (process.env.K8S && process.env.K8S_SBC_SIP_SERVICE_NAME) {
sbcs = [`${process.env.K8S_SBC_SIP_SERVICE_NAME}:5060`];
logger.info({sbcs}, 'SBC inventory');
}
// listen for SNS lifecycle changes
let lifecycleEmitter = new Emitter();
let dryUpCalls = false;
if (AWS_SNS_TOPIC_ARN && AWS_REGION) {
if (process.env.AWS_SNS_TOPIC_ARM && process.env.AWS_REGION) {
(async function() {
try {
@@ -46,24 +36,12 @@ module.exports = (logger) => {
const {srf} = require('../..');
srf.locals.publicIp = publicIp;
})
.on(LifeCycleEvents.ScaleIn, async() => {
.on(LifeCycleEvents.ScaleIn, () => {
logger.info('AWS scale-in notification: begin drying up calls');
dryUpCalls = true;
lifecycleEmitter.operationalState = LifeCycleEvents.ScaleIn;
const {srf} = require('../..');
const {writeSystemAlerts} = srf.locals;
if (writeSystemAlerts) {
const {SystemState, FEATURE_SERVER} = require('./constants');
await writeSystemAlerts({
system_component: FEATURE_SERVER,
state : SystemState.GracefulShutdownInProgress,
fields : {
detail: `feature-server with process_id ${process.pid} shutdown in progress`,
host: srf.locals?.ipv4
}
});
}
pingProxies(srf);
// if we have zero calls, we can complete the scale-in right
@@ -97,13 +75,13 @@ module.exports = (logger) => {
}
})();
}
else if (K8S) {
else if (process.env.K8S) {
lifecycleEmitter.scaleIn = () => process.exit(0);
}
async function pingProxies(srf) {
if (NODE_ENV === 'test') return;
if (process.env.NODE_ENV === 'test') return;
for (const sbc of sbcs) {
try {
@@ -113,8 +91,7 @@ module.exports = (logger) => {
method: 'OPTIONS',
headers: {
'X-FS-Status': ms && !dryUpCalls ? 'open' : 'closed',
'X-FS-Calls': srf.locals.sessionTracker.count,
'X-FS-ServiceUrl': srf.locals.serviceUrl
'X-FS-Calls': srf.locals.sessionTracker.count
}
});
req.on('response', (res) => {
@@ -125,7 +102,7 @@ module.exports = (logger) => {
}
}
}
if (K8S) {
if (process.env.K8S) {
setImmediate(() => {
logger.info('disabling OPTIONS pings since we are running as a kubernetes service');
const {srf} = require('../..');
@@ -146,16 +123,16 @@ module.exports = (logger) => {
setInterval(() => {
const {srf} = require('../..');
pingProxies(srf);
}, OPTIONS_PING_INTERVAL);
}, process.env.OPTIONS_PING_INTERVAL || 30000);
// initial ping once we are up
setTimeout(async() => {
// if SBCs are auto-scaling, monitor them as they come and go
const {srf} = require('../..');
if (!JAMBONES_SBCS) {
if (!process.env.JAMBONES_SBCS) {
const {monitorSet} = srf.locals.dbHelpers;
const setName = `${(JAMBONES_CLUSTER_ID || 'default')}:active-sip`;
const setName = `${(process.env.JAMBONES_CLUSTER_ID || 'default')}:active-sip`;
await monitorSet(setName, 10, (members) => {
sbcs = members;
logger.info(`sbc-pinger: SBC roster has changed, list of active SBCs is now ${sbcs}`);

View File

@@ -1,65 +0,0 @@
const sdpTransform = require('sdp-transform');
const isOnhold = (sdp) => {
return sdp && (sdp.includes('a=sendonly') || sdp.includes('a=inactive'));
};
const mergeSdpMedia = (sdp1, sdp2) => {
const parsedSdp1 = sdpTransform.parse(sdp1);
const parsedSdp2 = sdpTransform.parse(sdp2);
parsedSdp1.media.push(...parsedSdp2.media);
return sdpTransform.write(parsedSdp1);
};
const getCodecPlacement = (parsedSdp, codec) => parsedSdp?.media[0]?.rtp?.findIndex((e) => e.codec === codec);
const isOpusFirst = (sdp) => {
return getCodecPlacement(sdpTransform.parse(sdp), 'opus') === 0;
};
const makeOpusFirst = (sdp) => {
const parsedSdp = sdpTransform.parse(sdp);
// Find the index of the OPUS codec
const opusIndex = getCodecPlacement(parsedSdp, 'opus');
// Move OPUS codec to the beginning
if (opusIndex > 0) {
const opusEntry = parsedSdp.media[0].rtp.splice(opusIndex, 1)[0];
parsedSdp.media[0].rtp.unshift(opusEntry);
// Also move the corresponding payload type in the "m" line
const opusPayloadType = parsedSdp.media[0].payloads.split(' ')[opusIndex];
const otherPayloadTypes = parsedSdp.media[0].payloads.split(' ').filter((pt) => pt != opusPayloadType);
parsedSdp.media[0].payloads = [opusPayloadType, ...otherPayloadTypes].join(' ');
}
return sdpTransform.write(parsedSdp);
};
const removeVideoSdp = (sdp) => {
const parsedSdp = sdpTransform.parse(sdp);
// Filter out video media sections, keeping only non-video media
parsedSdp.media = parsedSdp.media.filter((media) => media.type !== 'video');
return sdpTransform.write(parsedSdp);
};
const extractSdpMedia = (sdp) => {
const parsedSdp1 = sdpTransform.parse(sdp);
if (parsedSdp1.media.length > 1) {
parsedSdp1.media = [parsedSdp1.media[0]];
const parsedSdp2 = sdpTransform.parse(sdp);
parsedSdp2.media = [parsedSdp2.media[1]];
return [sdpTransform.write(parsedSdp1), sdpTransform.write(parsedSdp2)];
} else {
return [sdp, sdp];
}
};
module.exports = {
isOnhold,
mergeSdpMedia,
extractSdpMedia,
isOpusFirst,
makeOpusFirst,
removeVideoSdp
};

View File

@@ -97,12 +97,8 @@ const parseSiprecPayload = (req, logger) => {
obj[`${prefix}participantstreamassoc`].forEach((ps) => {
const part = participants[ps.$.participant_id];
if (part) {
if (ps.hasOwnProperty(`${prefix}send`)) {
part.send = ps[`${prefix}send`][0];
}
if (ps.hasOwnProperty(`${prefix}recv`)) {
part.recv = ps[`${prefix}recv`][0];
}
part.send = ps[`${prefix}send`][0];
part.recv = ps[`${prefix}recv`][0];
}
});
}
@@ -113,9 +109,9 @@ const parseSiprecPayload = (req, logger) => {
obj[`${prefix}stream`].forEach((s) => {
const streamId = s.$.stream_id;
let sender;
for (const v of Object.values(participants)) {
for (const [k, v] of Object.entries(participants)) {
if (v.send === streamId) {
sender = v;
sender = k;
break;
}
}
@@ -125,15 +121,9 @@ const parseSiprecPayload = (req, logger) => {
sender.label = s[`${prefix}label`][0];
if (-1 !== ['1', 'a_leg', 'inbound', '10'].indexOf(sender.label)) {
opts.caller.aor = sender.aor;
if (-1 !== ['1', 'a_leg', 'inbound'].indexOf(sender.label)) {
opts.caller.aor = sender.aor ;
if (sender.name) opts.caller.name = sender.name;
// Remap the sdp stream base on sender label
if (!opts.sdp1.includes(`a=label:${sender.label}`)) {
const tmp = opts.sdp1;
opts.sdp1 = opts.sdp2;
opts.sdp2 = tmp;
}
}
else {
opts.callee.aor = sender.aor ;

View File

@@ -1,70 +0,0 @@
const EventEmitter = require('events');
/**
* A specialized EventEmitter that caches the most recent event emissions.
* When new listeners are added, they immediately receive the most recent
* event if it was previously emitted. This is useful for handling state
* changes where late subscribers need to know the current state.
*
* Features:
* - Caches the most recent emission for each event type
* - New listeners immediately receive the cached event if available
* - Supports both regular (on) and one-time (once) listeners
* - Maintains compatibility with Node's EventEmitter interface
*/
class StickyEventEmitter extends EventEmitter {
constructor() {
super();
this._eventCache = new Map();
this._onceListeners = new Map(); // For storing once listeners if needed
}
destroy() {
this._eventCache.clear();
this._onceListeners.clear();
this.removeAllListeners();
}
emit(event, ...args) {
// Store the event and its args
this._eventCache.set(event, args);
// If there are any 'once' listeners waiting, call them
if (this._onceListeners.has(event)) {
const listeners = this._onceListeners.get(event);
for (const listener of listeners) {
listener(...args);
}
if (this.onSuccess) {
this.onSuccess();
}
this._onceListeners.delete(event);
}
return super.emit(event, ...args);
}
on(event, listener) {
if (this._eventCache.has(event)) {
listener(...this._eventCache.get(event));
}
return super.on(event, listener);
}
once(event, listener) {
if (this._eventCache.has(event)) {
listener(...this._eventCache.get(event));
if (this.onSuccess) {
this.onSuccess();
}
} else {
// Store listener in case emit comes before
if (!this._onceListeners.has(event)) {
this._onceListeners.set(event, []);
}
this._onceListeners.get(event).push(listener);
super.once(event, listener); // Also attach to native once
}
return this;
}
}
module.exports = StickyEventEmitter;

View File

@@ -1,4 +1,15 @@
const {TaskName} = require('./constants.json');
const {
TaskName,
AzureTranscriptionEvents,
GoogleTranscriptionEvents,
AwsTranscriptionEvents,
NuanceTranscriptionEvents,
DeepgramTranscriptionEvents,
SonioxTranscriptionEvents,
NvidiaTranscriptionEvents,
JambonzTranscriptionEvents
} = require('./constants');
const stickyVars = {
google: [
'GOOGLE_SPEECH_HINTS',
@@ -22,15 +33,12 @@ const stickyVars = {
'AZURE_SERVICE_ENDPOINT',
'AZURE_INITIAL_SPEECH_TIMEOUT_MS',
'AZURE_USE_OUTPUT_FORMAT_DETAILED',
'AZURE_SPEECH_SEGMENTATION_SILENCE_TIMEOUT_MS'
],
deepgram: [
'DEEPGRAM_SPEECH_KEYWORDS',
'DEEPGRAM_API_KEY',
'DEEPGRAM_SPEECH_TIER',
'DEEPGRAM_SPEECH_MODEL',
'DEEPGRAM_SPEECH_ENABLE_SMART_FORMAT',
'DEEPGRAM_SPEECH_ENABLE_NO_DELAY',
'DEEPGRAM_SPEECH_ENABLE_AUTOMATIC_PUNCTUATION',
'DEEPGRAM_SPEECH_PROFANITY_FILTER',
'DEEPGRAM_SPEECH_REDACT',
@@ -41,23 +49,13 @@ const stickyVars = {
'DEEPGRAM_SPEECH_SEARCH',
'DEEPGRAM_SPEECH_REPLACE',
'DEEPGRAM_SPEECH_ENDPOINTING',
'DEEPGRAM_SPEECH_UTTERANCE_END_MS',
'DEEPGRAM_SPEECH_VAD_TURNOFF',
'DEEPGRAM_SPEECH_TAG',
'DEEPGRAM_SPEECH_MODEL_VERSION',
'DEEPGRAM_SPEECH_FILLER_WORDS',
'DEEPGRAM_SPEECH_KEYTERMS',
'DEEPGRAM_SPEECH_TAG'
],
aws: [
'AWS_VOCABULARY_NAME',
'AWS_VOCABULARY_FILTER_METHOD',
'AWS_VOCABULARY_FILTER_NAME',
'AWS_LANGUAGE_MODEL_NAME',
'AWS_ACCESS_KEY_ID',
'AWS_SECRET_ACCESS_KEY',
'AWS_REGION',
'AWS_SECURITY_TOKEN',
'AWS_PII_ENTITY_TYPES',
'AWS_VOCABULARY_FILTER_NAME'
],
nuance: [
'NUANCE_ACCESS_TOKEN',
@@ -94,159 +92,10 @@ const stickyVars = {
nvidia: [
'NVIDIA_HINTS'
],
cobalt: [
'COBALT_SPEECH_HINTS',
'COBALT_COMPILED_CONTEXT_DATA',
'COBALT_METADATA'
],
soniox: [
'SONIOX_PROFANITY_FILTER',
'SONIOX_MODEL'
],
assemblyai: [
'ASSEMBLYAI_API_KEY',
'ASSEMBLYAI_WORD_BOOST'
],
voxist: [
'VOXIST_API_KEY',
],
speechmatics: [
'SPEECHMATICS_API_KEY',
'SPEECHMATICS_HOST',
'SPEECHMATICS_PATH',
'SPEECHMATICS_SPEECH_HINTS',
'SPEECHMATICS_TRANSLATION_LANGUAGES',
'SPEECHMATICS_TRANSLATION_PARTIALS'
],
openai: [
'OPENAI_API_KEY',
'OPENAI_MODEL',
'OPENAI_INPUT_AUDIO_NOISE_REDUCTION',
'OPENAI_TURN_DETECTION_TYPE',
'OPENAI_TURN_DETECTION_THRESHOLD',
'OPENAI_TURN_DETECTION_PREFIX_PADDING_MS',
'OPENAI_TURN_DETECTION_SILENCE_DURATION_MS',
],
};
/**
* @see https://developers.deepgram.com/docs/models-languages-overview
*/
const optimalDeepramModels = {
zh: ['base', 'base'],
'zh-CN':['base', 'base'],
'zh-TW': ['base', 'base'],
da: ['enhanced', 'enhanced'],
en: ['nova-2-phonecall', 'nova-2'],
'en-US': ['nova-2-phonecall', 'nova-2'],
'en-AU': ['nova-2', 'nova-2'],
'en-GB': ['nova-2', 'nova-2'],
'en-IN': ['nova-2', 'nova-2'],
'en-NZ': ['nova-2', 'nova-2'],
nl: ['nova-2', 'nova-2'],
fr: ['nova-2', 'nova-2'],
'fr-CA': ['nova-2', 'nova-2'],
de: ['nova-2', 'nova-2'],
hi: ['nova-2', 'nova-2'],
'hi-Latn': ['nova-2', 'nova-2'],
id: ['base', 'base'],
it: ['nova-2', 'nova-2'],
ja: ['enhanced', 'enhanced'],
ko: ['nova-2', 'nova-2'],
no: ['nova-2', 'nova-2'],
pl: ['nova-2', 'nova-2'],
pt: ['nova-2', 'nova-2'],
'pt-BR': ['nova-2', 'nova-2'],
'pt-PT': ['nova-2', 'nova-2'],
ru: ['nova-2', 'nova-2'],
es: ['nova-2', 'nova-2'],
'es-419': ['nova-2', 'nova-2'],
'es-LATAM': ['enhanced', 'enhanced'],
sv: ['nova-2', 'nova-2'],
ta: ['enhanced', 'enhanced'],
taq: ['enhanced', 'enhanced'],
tr: ['nova-2', 'nova-2'],
uk: ['nova-2', 'nova-2']
};
const selectDefaultDeepgramModel = (task, language) => {
if (language in optimalDeepramModels) {
const [gather, transcribe] = optimalDeepramModels[language];
return task.name === TaskName.Gather ? gather : transcribe;
}
return 'base';
};
const optimalGoogleModels = {
'v1' : {
'en-IN':['telephony', 'telephony'],
'es-DO':['default', 'default'],
'es-MX':['default', 'default'],
'en-AU':['telephony', 'telephony'],
'en-GB':['telephony', 'telephony'],
'en-NZ':['telephony', 'telephony']
},
'v2' : {
'en-IN':['telephony', 'long']
}
};
const selectDefaultGoogleModel = (task, language, version) => {
const useV2 = version === 'v2';
if (language in optimalGoogleModels[version]) {
const [gather, transcribe] = optimalGoogleModels[version][language];
return task.name === TaskName.Gather ? gather : transcribe;
}
return task.name === TaskName.Gather ?
(useV2 ? 'telephony_short' : 'command_and_search') :
(useV2 ? 'long' : 'latest_long');
};
const consolidateTranscripts = (bufferedTranscripts, channel, language, vendor) => {
if (bufferedTranscripts.length === 1) {
bufferedTranscripts[0].is_final = true;
return bufferedTranscripts[0];
}
let totalConfidence = 0;
const finalTranscript = bufferedTranscripts.reduce((acc, evt) => {
totalConfidence += evt.alternatives[0].confidence;
let newTranscript = evt.alternatives[0].transcript;
// If new transcript consists only of digits, spaces, and a trailing comma or period
if (newTranscript.match(/^[\d\s]+[,.]?$/)) {
newTranscript = newTranscript.replace(/\s/g, ''); // Remove all spaces
if (newTranscript.endsWith(',')) {
newTranscript = newTranscript.slice(0, -1); // Remove the trailing comma
} else if (newTranscript.endsWith('.')) {
newTranscript = newTranscript.slice(0, -1); // Remove the trailing period
}
}
const lastChar = acc.alternatives[0].transcript.slice(-1);
const firstChar = newTranscript.charAt(0);
if (vendor === 'speechmatics' || (lastChar.match(/\d/) && firstChar.match(/\d/))) {
acc.alternatives[0].transcript += newTranscript;
} else {
acc.alternatives[0].transcript += ` ${newTranscript}`;
}
return acc;
}, {
language_code: language,
channel_tag: channel,
is_final: true,
alternatives: [{
transcript: ''
}]
});
finalTranscript.alternatives[0].confidence = bufferedTranscripts.length === 1 ?
bufferedTranscripts[0].alternatives[0].confidence :
totalConfidence / bufferedTranscripts.length;
finalTranscript.alternatives[0].transcript = finalTranscript.alternatives[0].transcript.trim();
finalTranscript.vendor = {
name: vendor,
evt: bufferedTranscripts
};
return finalTranscript;
]
};
const compileSonioxTranscripts = (finalWordChunks, channel, language) => {
@@ -306,28 +155,19 @@ const normalizeSoniox = (evt, channel, language) => {
};
};
const normalizeDeepgram = (evt, channel, language, shortUtterance) => {
const normalizeDeepgram = (evt, channel, language) => {
const copy = JSON.parse(JSON.stringify(evt));
const alternatives = (evt.channel?.alternatives || [])
.map((alt) => ({
confidence: alt.confidence,
transcript: alt.transcript,
}));
/**
* Some models (nova-2-general) return the detected language in the
* alternatives.languages array if the language is set as multi.
* If the language is detected, we use it as the language_code.
*/
const detectedLanguage = evt.channel?.alternatives?.[0]?.languages?.[0];
/**
* note difference between is_final and speech_final in Deepgram:
* https://developers.deepgram.com/docs/understand-endpointing-interim-results
*/
return {
language_code: detectedLanguage || language,
language_code: language,
channel_tag: channel,
is_final: shortUtterance ? evt.is_final : evt.speech_final,
alternatives: alternatives.length ? [alternatives[0]] : [],
is_final: evt.is_final,
alternatives: [alternatives[0]],
vendor: {
name: 'deepgram',
evt: copy
@@ -373,10 +213,8 @@ const normalizeIbm = (evt, channel, language) => {
const normalizeGoogle = (evt, channel, language) => {
const copy = JSON.parse(JSON.stringify(evt));
const language_code = evt.language_code || language;
return {
language_code: language_code,
language_code: language,
channel_tag: channel,
is_final: evt.is_final,
alternatives: [evt.alternatives[0]],
@@ -387,37 +225,12 @@ const normalizeGoogle = (evt, channel, language) => {
};
};
const normalizeCobalt = (evt, channel, language) => {
const copy = JSON.parse(JSON.stringify(evt));
const alternatives = (evt.alternatives || [])
.map((alt) => ({
confidence: alt.confidence,
transcript: alt.transcript_formatted,
}));
const normalizeCustom = (evt, channel, language) => {
return {
language_code: language,
channel_tag: channel,
is_final: evt.is_final,
alternatives,
vendor: {
name: 'cobalt',
evt: copy
}
};
};
const normalizeCustom = (evt, channel, language, vendor) => {
const copy = JSON.parse(JSON.stringify(evt));
return {
language_code: language,
channel_tag: channel,
is_final: evt.is_final,
alternatives: [evt.alternatives[0]],
vendor: {
name: vendor,
evt: copy
}
alternatives: [evt.alternatives[0]]
};
};
@@ -435,34 +248,19 @@ const normalizeNuance = (evt, channel, language) => {
};
};
const normalizeVerbio = (evt, channel, language) => {
const copy = JSON.parse(JSON.stringify(evt));
return {
language_code: language,
channel_tag: channel,
is_final: evt.is_final,
alternatives: evt.alternatives,
vendor: {
name: 'verbio',
evt: copy
}
};
};
const normalizeMicrosoft = (evt, channel, language, punctuation = true) => {
const normalizeMicrosoft = (evt, channel, language) => {
const copy = JSON.parse(JSON.stringify(evt));
const nbest = evt.NBest;
const language_code = evt.PrimaryLanguage?.Language || language;
const alternatives = nbest ? nbest.map((n) => {
return {
confidence: n.Confidence,
// remove all puntuation if needed
transcript: punctuation ? n.Display : n.Display.replace(/\p{P}/gu, '')
transcript: n.Display
};
}) :
[
{
transcript: punctuation ? evt.DisplayText || evt.Text : (evt.DisplayText || evt.Text).replace(/\p{P}/gu, '')
transcript: evt.DisplayText || evt.Text
}
];
@@ -479,145 +277,29 @@ const normalizeMicrosoft = (evt, channel, language, punctuation = true) => {
};
const normalizeAws = (evt, channel, language) => {
const copy = JSON.parse(JSON.stringify(evt));
const isGrpcPayload = Array.isArray(evt);
if (isGrpcPayload) {
/* legacy grpc api */
return {
language_code: language,
channel_tag: channel,
is_final: evt[0].is_final,
alternatives: evt[0].alternatives,
vendor: {
name: 'aws',
evt: copy
}
};
}
else {
/* websocket api */
const alternatives = evt.Transcript?.Results[0]?.Alternatives.map((alt) => {
const items = alt.Items.filter((item) => item.Type === 'pronunciation' && 'Confidence' in item);
const confidence = items.reduce((acc, item) => acc + item.Confidence, 0) / items.length;
return {
transcript: alt.Transcript,
confidence
};
});
return {
language_code: language,
channel_tag: channel,
is_final: evt.Transcript?.Results[0].IsPartial === false,
alternatives,
vendor: {
name: 'aws',
evt: copy
}
};
}
};
const normalizeAssemblyAi = (evt, channel, language) => {
const copy = JSON.parse(JSON.stringify(evt));
return {
language_code: language,
channel_tag: channel,
is_final: evt.message_type === 'FinalTranscript',
alternatives: [
{
confidence: evt.confidence,
transcript: evt.text,
}
],
is_final: evt[0].is_final,
alternatives: evt[0].alternatives,
vendor: {
name: 'assemblyai',
name: 'aws',
evt: copy
}
};
};
const normalizeVoxist = (evt, channel, language) => {
const copy = JSON.parse(JSON.stringify(evt));
return {
language_code: language,
channel_tag: channel,
is_final: evt.type === 'final',
alternatives: [
{
confidence: 1.00,
transcript: evt.text,
}
],
vendor: {
name: 'voxist',
evt: copy
}
};
};
const normalizeSpeechmatics = (evt, channel, language) => {
const copy = JSON.parse(JSON.stringify(evt));
const is_final = evt.message === 'AddTranscript';
const words = evt.results?.filter((r) => r.type === 'word') || [];
const confidence = words.length > 0 ?
words.reduce((acc, word) => acc + word.alternatives[0].confidence, 0) / words.length :
0;
const alternative = {
confidence,
transcript: evt.metadata?.transcript
};
const obj = {
language_code: language,
channel_tag: channel,
is_final,
alternatives: [alternative],
vendor: {
name: 'speechmatics',
evt: copy
}
};
return obj;
};
const calculateConfidence = (logprobsArray) => {
// Sum the individual log probabilities
const totalLogProb = logprobsArray.reduce((sum, tokenInfo) => sum + tokenInfo.logprob, 0);
// Convert the total log probability back to a regular probability
const confidence = Math.exp(totalLogProb);
return confidence;
};
const normalizeOpenAI = (evt, channel, language) => {
const copy = JSON.parse(JSON.stringify(evt));
const obj = {
language_code: language,
channel_tag: channel,
is_final: true,
alternatives: [
{
transcript: evt.transcript,
confidence: evt.logprobs ? calculateConfidence(evt.logprobs) : 1.0,
}
],
vendor: {
name: 'openai',
evt: copy
}
};
return obj;
};
module.exports = (logger) => {
const normalizeTranscription = (evt, vendor, channel, language, shortUtterance, punctuation) => {
const normalizeTranscription = (evt, vendor, channel, language) => {
//logger.debug({ evt, vendor, channel, language }, 'normalizeTranscription');
switch (vendor) {
case 'deepgram':
return normalizeDeepgram(evt, channel, language, shortUtterance);
return normalizeDeepgram(evt, channel, language);
case 'microsoft':
return normalizeMicrosoft(evt, channel, language, punctuation);
return normalizeMicrosoft(evt, channel, language);
case 'google':
return normalizeGoogle(evt, channel, language);
case 'aws':
@@ -630,36 +312,31 @@ module.exports = (logger) => {
return normalizeNvidia(evt, channel, language);
case 'soniox':
return normalizeSoniox(evt, channel, language);
case 'cobalt':
return normalizeCobalt(evt, channel, language);
case 'assemblyai':
return normalizeAssemblyAi(evt, channel, language, shortUtterance);
case 'voxist':
return normalizeVoxist(evt, channel, language);
case 'verbio':
return normalizeVerbio(evt, channel, language);
case 'speechmatics':
return normalizeSpeechmatics(evt, channel, language);
case 'openai':
return normalizeOpenAI(evt, channel, language);
default:
if (vendor.startsWith('custom:')) {
return normalizeCustom(evt, channel, language, vendor);
return normalizeCustom(evt, channel, language);
}
logger.error(`Unknown vendor ${vendor}`);
return evt;
}
};
const setChannelVarsForStt = (task, sttCredentials, language, rOpts = {}) => {
const setChannelVarsForStt = (task, sttCredentials, rOpts = {}) => {
let opts = {};
const {enable, voiceMs = 0, mode = -1} = rOpts.vad || {};
const vad = {enable, voiceMs, mode};
const vendor = rOpts.vendor;
/* voice activity detection works across vendors */
opts = {
...opts,
...(vad.enable && {START_RECOGNIZING_ON_VAD: 1}),
...(vad.enable && vad.voiceMs && {RECOGNIZER_VAD_VOICE_MS: vad.voiceMs}),
...(vad.enable && typeof vad.mode === 'number' && {RECOGNIZER_VAD_MODE: vad.mode}),
};
if ('google' === vendor) {
const useV2 = rOpts.googleOptions?.serviceVersion === 'v2';
const version = useV2 ? 'v2' : 'v1';
let {model} = rOpts;
model = model || selectDefaultGoogleModel(task, language, version);
const model = task.name === TaskName.Gather ? 'command_and_search' : 'latest_long';
opts = {
...opts,
...(sttCredentials && {GOOGLE_APPLICATION_CREDENTIALS: JSON.stringify(sttCredentials.credentials)}),
@@ -679,105 +356,56 @@ module.exports = (logger) => {
...(rOpts.punctuation === false && {GOOGLE_SPEECH_ENABLE_AUTOMATIC_PUNCTUATION: 0}),
...(rOpts.words == false && {GOOGLE_SPEECH_ENABLE_WORD_TIME_OFFSETS: 0}),
...(rOpts.diarization === false && {GOOGLE_SPEECH_SPEAKER_DIARIZATION: 0}),
...(rOpts.hints?.length > 0 && typeof rOpts.hints[0] === 'string' &&
...(rOpts.hints.length > 0 && typeof rOpts.hints[0] === 'string' &&
{GOOGLE_SPEECH_HINTS: rOpts.hints.join(',')}),
...(rOpts.hints?.length > 0 && typeof rOpts.hints[0] === 'object' &&
...(rOpts.hints.length > 0 && typeof rOpts.hints[0] === 'object' &&
{GOOGLE_SPEECH_HINTS: JSON.stringify(rOpts.hints)}),
...(typeof rOpts.hintsBoost === 'number' && {GOOGLE_SPEECH_HINTS_BOOST: rOpts.hintsBoost}),
// When altLanguages is emptylist, we have to send value to freeswitch to clear the previous settings
...(rOpts.altLanguages &&
...(rOpts.altLanguages.length > 0 &&
{GOOGLE_SPEECH_ALTERNATIVE_LANGUAGE_CODES: [...new Set(rOpts.altLanguages)].join(',')}),
...(rOpts.interactionType &&
{GOOGLE_SPEECH_METADATA_INTERACTION_TYPE: rOpts.interactionType}),
...{GOOGLE_SPEECH_MODEL: rOpts.model || model},
...(rOpts.naicsCode > 0 && {GOOGLE_SPEECH_METADATA_INDUSTRY_NAICS_CODE: rOpts.naicsCode}),
GOOGLE_SPEECH_METADATA_RECORDING_DEVICE_TYPE: 'phone_line',
...(useV2 && {
GOOGLE_SPEECH_RECOGNIZER_PARENT: `projects/${sttCredentials.credentials.project_id}/locations/global`,
GOOGLE_SPEECH_CLOUD_SERVICES_VERSION: 'v2',
...(rOpts.googleOptions?.speechStartTimeoutMs && {
GOOGLE_SPEECH_START_TIMEOUT_MS: rOpts.googleOptions.speechStartTimeoutMs
}),
...(rOpts.googleOptions?.speechEndTimeoutMs && {
GOOGLE_SPEECH_END_TIMEOUT_MS: rOpts.googleOptions.speechEndTimeoutMs
}),
...(rOpts.googleOptions?.transcriptNormalization && {
GOOGLE_SPEECH_TRANSCRIPTION_NORMALIZATION: JSON.stringify(rOpts.googleOptions.transcriptNormalization)
}),
...(rOpts.googleOptions?.enableVoiceActivityEvents && {
GOOGLE_SPEECH_ENABLE_VOICE_ACTIVITY_EVENTS: rOpts.googleOptions.enableVoiceActivityEvents
}),
...(rOpts.sgoogleOptions?.recognizerId) && {GOOGLE_SPEECH_RECOGNIZER_ID: rOpts.googleOptions.recognizerId},
...(rOpts.googleOptions?.enableVoiceActivityEvents && {
GOOGLE_SPEECH_ENABLE_VOICE_ACTIVITY_EVENTS: rOpts.googleOptions.enableVoiceActivityEvents
}),
}),
};
}
else if (['aws', 'polly'].includes(vendor)) {
const {awsOptions = {}} = rOpts;
const vocabularyName = awsOptions.vocabularyName || rOpts.vocabularyName;
const vocabularyFilterName = awsOptions.vocabularyFilterName || rOpts.vocabularyFilterName;
const filterMethod = awsOptions.vocabularyFilterMethod || rOpts.filterMethod;
opts = {
...opts,
...(vocabularyName && {AWS_VOCABULARY_NAME: vocabularyName}),
...(vocabularyFilterName && {AWS_VOCABULARY_FILTER_NAME: vocabularyFilterName}),
...(filterMethod && {AWS_VOCABULARY_FILTER_METHOD: filterMethod}),
...(rOpts.vocabularyName && {AWS_VOCABULARY_NAME: rOpts.vocabularyName}),
...(rOpts.vocabularyFilterName && {AWS_VOCABULARY_FILTER_NAME: rOpts.vocabularyFilterName}),
...(rOpts.filterMethod && {AWS_VOCABULARY_FILTER_METHOD: rOpts.filterMethod}),
...(sttCredentials && {
AWS_ACCESS_KEY_ID: sttCredentials.accessKeyId,
AWS_SECRET_ACCESS_KEY: sttCredentials.secretAccessKey,
AWS_REGION: sttCredentials.region,
AWS_SECURITY_TOKEN: sttCredentials.securityToken
AWS_REGION: sttCredentials.region
}),
...(awsOptions.accessKey && {AWS_ACCESS_KEY_ID: awsOptions.accessKey}),
...(awsOptions.secretKey && {AWS_SECRET_ACCESS_KEY: awsOptions.secretKey}),
...(awsOptions.region && {AWS_REGION: awsOptions.region}),
...(awsOptions.securityToken && {AWS_SECURITY_TOKEN: awsOptions.securityToken}),
...(awsOptions.languageModelName && {AWS_LANGUAGE_MODEL_NAME: awsOptions.languageModelName}),
...(awsOptions.piiEntityTypes?.length && {AWS_PII_ENTITY_TYPES: awsOptions.piiEntityTypes.join(',')}),
...(awsOptions.piiIdentifyEntities && {AWS_PII_IDENTIFY_ENTITIES: true}),
...(awsOptions.languageModelName && {AWS_LANGUAGE_MODEL_NAME: awsOptions.languageModelName}),
};
}
else if ('microsoft' === vendor) {
const {azureOptions = {}} = rOpts;
opts = {
...opts,
...(rOpts.hints?.length > 0 && typeof rOpts.hints[0] === 'string' &&
...(rOpts.hints.length > 0 && typeof rOpts.hints[0] === 'string' &&
{AZURE_SPEECH_HINTS: rOpts.hints.map((h) => h.trim()).join(',')}),
...(rOpts.hints?.length > 0 && typeof rOpts.hints[0] === 'object' &&
...(rOpts.hints.length > 0 && typeof rOpts.hints[0] === 'object' &&
{AZURE_SPEECH_HINTS: rOpts.hints.map((h) => h.phrase).join(',')}),
// When altLanguages is emptylist, we have to send value to freeswitch to clear the previous settings
...(rOpts.altLanguages &&
...(rOpts.altLanguages && rOpts.altLanguages.length > 0 &&
{AZURE_SPEECH_ALTERNATIVE_LANGUAGE_CODES: [...new Set(rOpts.altLanguages)].join(',')}),
...(rOpts.requestSnr && {AZURE_REQUEST_SNR: 1}),
...(rOpts.profanityOption && {AZURE_PROFANITY_OPTION: rOpts.profanityOption}),
...(sttCredentials.use_custom_stt && sttCredentials.custom_stt_endpoint_url &&
{AZURE_SERVICE_ENDPOINT: sttCredentials.custom_stt_endpoint_url}),
...(rOpts.azureServiceEndpoint && {AZURE_SERVICE_ENDPOINT: rOpts.azureServiceEndpoint}),
...(rOpts.initialSpeechTimeoutMs > 0 &&
{AZURE_INITIAL_SPEECH_TIMEOUT_MS: rOpts.initialSpeechTimeoutMs}),
...(rOpts.requestSnr && {AZURE_REQUEST_SNR: 1}),
...(rOpts.audioLogging && {AZURE_AUDIO_LOGGING: 1}),
...{AZURE_USE_OUTPUT_FORMAT_DETAILED: 1},
...(azureOptions.speechSegmentationSilenceTimeoutMs &&
{AZURE_SPEECH_SEGMENTATION_SILENCE_TIMEOUT_MS: azureOptions.speechSegmentationSilenceTimeoutMs}),
...(azureOptions.languageIdMode &&
{AZURE_LANGUAGE_ID_MODE: azureOptions.languageIdMode}),
...(azureOptions.postProcessing &&
{AZURE_POST_PROCESSING_OPTION: azureOptions.postProcessing}),
...(sttCredentials && {
...(sttCredentials.api_key && {AZURE_SUBSCRIPTION_KEY: sttCredentials.api_key}),
...(sttCredentials.region && {AZURE_REGION: sttCredentials.region}),
AZURE_SUBSCRIPTION_KEY: sttCredentials.api_key,
AZURE_REGION: sttCredentials.region,
}),
...(sttCredentials.use_custom_stt && sttCredentials.custom_stt_endpoint &&
{AZURE_SERVICE_ENDPOINT_ID: sttCredentials.custom_stt_endpoint}),
//azureSttEndpointId overrides sttCredentials.custom_stt_endpoint
...(rOpts.azureSttEndpointId &&
{AZURE_SERVICE_ENDPOINT_ID: rOpts.azureSttEndpointId}),
...(azureOptions.speechRecognitionMode &&
{AZURE_RECOGNITION_MODE: azureOptions.speechRecognitionMode}),
{AZURE_SERVICE_ENDPOINT_ID: sttCredentials.custom_stt_endpoint})
};
}
else if ('nuance' === vendor) {
@@ -829,34 +457,21 @@ module.exports = (logger) => {
};
}
else if ('deepgram' === vendor) {
let model = rOpts.deepgramOptions?.model || rOpts.model;
const {deepgramOptions = {}} = rOpts;
const deepgramUri = deepgramOptions.deepgramSttUri || sttCredentials.deepgram_stt_uri;
const useTls = deepgramOptions.deepgramSttUseTls || sttCredentials.deepgram_stt_use_tls;
/* default to a sensible model if not supplied */
if (!model) {
model = selectDefaultDeepgramModel(task, language);
}
opts = {
...opts,
DEEPGRAM_SPEECH_MODEL: model,
...(deepgramUri && {DEEPGRAM_URI: deepgramUri}),
...(deepgramUri && useTls && {DEEPGRAM_USE_TLS: 1}),
...(sttCredentials.api_key) &&
{DEEPGRAM_API_KEY: sttCredentials.api_key},
...(deepgramOptions.tier) &&
{DEEPGRAM_SPEECH_TIER: deepgramOptions.tier},
...(deepgramOptions.model) &&
{DEEPGRAM_SPEECH_MODEL: deepgramOptions.model},
...(deepgramOptions.punctuate) &&
{DEEPGRAM_SPEECH_ENABLE_AUTOMATIC_PUNCTUATION: 1},
...(deepgramOptions.smartFormatting) &&
{DEEPGRAM_SPEECH_ENABLE_SMART_FORMAT: 1},
...(deepgramOptions.noDelay) &&
{DEEPGRAM_SPEECH_ENABLE_NO_DELAY: 1},
...(deepgramOptions.profanityFilter) &&
{DEEPGRAM_SPEECH_PROFANITY_FILTER: 1},
...(deepgramOptions.redact) &&
{DEEPGRAM_SPEECH_REDACT: deepgramOptions.redact},
{DEEPGRAM_SPEECH_REDACT: 1},
...(deepgramOptions.diarize) &&
{DEEPGRAM_SPEECH_DIARIZE: 1},
...(deepgramOptions.diarizeVersion) &&
@@ -871,28 +486,18 @@ module.exports = (logger) => {
{DEEPGRAM_SPEECH_SEARCH: deepgramOptions.search.join(',')},
...(deepgramOptions.replace) &&
{DEEPGRAM_SPEECH_REPLACE: deepgramOptions.replace.join(',')},
...(rOpts.hints && rOpts.hints.length > 0 && typeof rOpts.hints[0] === 'string' &&
...(rOpts.hints.length > 0 && typeof rOpts.hints[0] === 'string' &&
{DEEPGRAM_SPEECH_KEYWORDS: rOpts.hints.map((h) => h.trim()).join(',')}),
...(rOpts.hints && rOpts.hints.length > 0 && typeof rOpts.hints[0] === 'object' &&
...(rOpts.hints.length > 0 && typeof rOpts.hints[0] === 'object' &&
{DEEPGRAM_SPEECH_KEYWORDS: rOpts.hints.map((h) => h.phrase).join(',')}),
...(deepgramOptions.keywords) &&
{DEEPGRAM_SPEECH_KEYWORDS: deepgramOptions.keywords.join(',')},
...('endpointing' in deepgramOptions) &&
{DEEPGRAM_SPEECH_ENDPOINTING: deepgramOptions.endpointing === false ? 'false' : deepgramOptions.endpointing,
// default DEEPGRAM_SPEECH_UTTERANCE_END_MS is 1000, will be override by user settings later if there is.
DEEPGRAM_SPEECH_UTTERANCE_END_MS: 1000},
...(deepgramOptions.utteranceEndMs) &&
{DEEPGRAM_SPEECH_UTTERANCE_END_MS: deepgramOptions.utteranceEndMs},
{DEEPGRAM_SPEECH_ENDPOINTING: deepgramOptions.endpointing},
...(deepgramOptions.vadTurnoff) &&
{DEEPGRAM_SPEECH_VAD_TURNOFF: deepgramOptions.vadTurnoff},
...(deepgramOptions.tag) &&
{DEEPGRAM_SPEECH_TAG: deepgramOptions.tag},
...(deepgramOptions.version) &&
{DEEPGRAM_SPEECH_MODEL_VERSION: deepgramOptions.version},
...(deepgramOptions.fillerWords) &&
{DEEPGRAM_SPEECH_FILLER_WORDS: deepgramOptions.fillerWords},
...((Array.isArray(deepgramOptions.keyterms) && deepgramOptions.keyterms.length > 0) &&
{DEEPGRAM_SPEECH_KEYTERMS: deepgramOptions.keyterms.join(',')})
{DEEPGRAM_SPEECH_TAG: deepgramOptions.tag}
};
}
else if ('soniox' === vendor) {
@@ -902,9 +507,9 @@ module.exports = (logger) => {
...opts,
...(sttCredentials.api_key) &&
{SONIOX_API_KEY: sttCredentials.api_key},
...(rOpts.hints?.length > 0 && typeof rOpts.hints[0] === 'string' &&
...(rOpts.hints.length > 0 && typeof rOpts.hints[0] === 'string' &&
{SONIOX_HINTS: rOpts.hints.join(',')}),
...(rOpts.hints?.length > 0 && typeof rOpts.hints[0] === 'object' &&
...(rOpts.hints.length > 0 && typeof rOpts.hints[0] === 'object' &&
{SONIOX_HINTS: JSON.stringify(rOpts.hints)}),
...(typeof rOpts.hintsBoost === 'number' &&
{SONIOX_HINTS_BOOST: rOpts.hintsBoost}),
@@ -944,7 +549,6 @@ module.exports = (logger) => {
}
else if ('nvidia' === vendor) {
const {nvidiaOptions = {}} = rOpts;
const rivaUri = nvidiaOptions.rivaUri || sttCredentials.riva_server_uri;
opts = {
...opts,
...((nvidiaOptions.profanityFilter || rOpts.profanityFilter) && {NVIDIA_PROFANITY_FILTER: 1}),
@@ -956,15 +560,15 @@ module.exports = (logger) => {
...(nvidiaOptions.maxAlternatives && {NVIDIA_MAX_ALTERNATIVES: nvidiaOptions.maxAlternatives}),
...(!nvidiaOptions.maxAlternatives && {NVIDIA_MAX_ALTERNATIVES: 1}),
...(rOpts.model && {NVIDIA_MODEL: rOpts.model}),
...(rivaUri && {NVIDIA_RIVA_URI: rivaUri}),
...(nvidiaOptions.rivaUri && {NVIDIA_RIVA_URI: nvidiaOptions.rivaUri}),
...(nvidiaOptions.verbatimTranscripts && {NVIDIA_VERBATIM_TRANSCRIPTS: 1}),
...(rOpts.diarization && {NVIDIA_SPEAKER_DIARIZATION: 1}),
...(rOpts.diarization && rOpts.diarizationMaxSpeakers > 0 &&
{NVIDIA_DIARIZATION_SPEAKER_COUNT: rOpts.diarizationMaxSpeakers}),
...(rOpts.separateRecognitionPerChannel && {NVIDIA_SEPARATE_RECOGNITION_PER_CHANNEL: 1}),
...(rOpts.hints?.length > 0 && typeof rOpts.hints[0] === 'string' &&
...(rOpts.hints.length > 0 && typeof rOpts.hints[0] === 'string' &&
{NVIDIA_HINTS: rOpts.hints.join(',')}),
...(rOpts.hints?.length > 0 && typeof rOpts.hints[0] === 'object' &&
...(rOpts.hints.length > 0 && typeof rOpts.hints[0] === 'object' &&
{NVIDIA_HINTS: JSON.stringify(rOpts.hints)}),
...(typeof rOpts.hintsBoost === 'number' &&
{NVIDIA_HINTS_BOOST: rOpts.hintsBoost}),
@@ -972,156 +576,23 @@ module.exports = (logger) => {
{NVIDIA_CUSTOM_CONFIGURATION: JSON.stringify(nvidiaOptions.customConfiguration)}),
};
}
else if ('cobalt' === vendor) {
const {cobaltOptions = {}} = rOpts;
const cobaltUri = cobaltOptions.serverUri || sttCredentials.cobalt_server_uri;
opts = {
...opts,
...(rOpts.words && {COBALT_WORD_TIME_OFFSETS: 1}),
...(!rOpts.words && {COBALT_WORD_TIME_OFFSETS: 0}),
...(rOpts.model && {COBALT_MODEL: rOpts.model}),
...(cobaltUri && {COBALT_SERVER_URI: cobaltUri}),
...(rOpts.hints?.length > 0 && typeof rOpts.hints[0] === 'string' &&
{COBALT_SPEECH_HINTS: rOpts.hints.join(',')}),
...(rOpts.hints?.length > 0 && typeof rOpts.hints[0] === 'object' &&
{COBALT_SPEECH_HINTS: JSON.stringify(rOpts.hints)}),
...(rOpts.hints?.length > 0 &&
{COBALT_CONTEXT_TOKEN: cobaltOptions.contextToken || 'unk:default'}),
...(cobaltOptions.metadata && {COBALT_METADATA: cobaltOptions.metadata}),
...(cobaltOptions.enableConfusionNetwork && {COBALT_ENABLE_CONFUSION_NETWORK: 1}),
...(cobaltOptions.compiledContextData && {COBALT_COMPILED_CONTEXT_DATA: cobaltOptions.compiledContextData}),
};
}
else if ('assemblyai' === vendor) {
opts = {
...opts,
...(sttCredentials.api_key) &&
{ASSEMBLYAI_API_KEY: sttCredentials.api_key},
...(rOpts.hints?.length > 0 &&
{ASSEMBLYAI_WORD_BOOST: JSON.stringify(rOpts.hints)})
};
}
else if ('voxist' === vendor) {
opts = {
...opts,
...(sttCredentials.api_key) &&
{VOXIST_API_KEY: sttCredentials.api_key},
};
}
else if ('openai' === vendor) {
const {openaiOptions = {}} = rOpts;
const model = openaiOptions.model || rOpts.model || sttCredentials.model_id || 'whisper-1';
const apiKey = openaiOptions.apiKey || sttCredentials.api_key;
opts = {
OPENAI_MODEL: model,
OPENAI_API_KEY: apiKey,
...opts,
...(openaiOptions.prompt && {OPENAI_PROMPT: openaiOptions.prompt}),
...(openaiOptions.input_audio_noise_reduction &&
{OPENAI_INPUT_AUDIO_NOISE_REDUCTION: openaiOptions.input_audio_noise_reduction}),
};
if (openaiOptions.turn_detection) {
opts = {
...opts,
OPENAI_TURN_DETECTION_TYPE: openaiOptions.turn_detection.type,
...(openaiOptions.turn_detection.threshold && {
OPENAI_TURN_DETECTION_THRESHOLD: openaiOptions.turn_detection.threshold
}),
...(openaiOptions.turn_detection.prefix_padding_ms && {
OPENAI_TURN_DETECTION_PREFIX_PADDING_MS: openaiOptions.turn_detection.prefix_padding_ms
}),
...(openaiOptions.turn_detection.silence_duration_ms && {
OPENAI_TURN_DETECTION_SILENCE_DURATION_MS: openaiOptions.turn_detection.silence_duration_ms
}),
};
}
}
else if ('verbio' === vendor) {
const {verbioOptions = {}} = rOpts;
opts = {
...opts,
...(sttCredentials.access_token && { VERBIO_ACCESS_TOKEN: sttCredentials.access_token}),
...(sttCredentials.engine_version && {VERBIO_ENGINE_VERSION: sttCredentials.engine_version}),
...(language && {VERBIO_LANGUAGE: language}),
...(verbioOptions.enable_formatting && {VERBIO_ENABLE_FORMATTING: verbioOptions.enable_formatting}),
...(verbioOptions.enable_diarization && {VERBIO_ENABLE_DIARIZATION: verbioOptions.enable_diarization}),
...(verbioOptions.topic && {VERBIO_TOPIC: verbioOptions.topic}),
...(verbioOptions.inline_grammar && {VERBIO_INLINE_GRAMMAR: verbioOptions.inline_grammar}),
...(verbioOptions.grammar_uri && {VERBIO_GRAMMAR_URI: verbioOptions.grammar_uri}),
...(verbioOptions.label && {VERBIO_LABEL: verbioOptions.label}),
...(verbioOptions.recognition_timeout && {VERBIO_RECOGNITION_TIMEOUT: verbioOptions.recognition_timeout}),
...(verbioOptions.speech_complete_timeout &&
{VERBIO_SPEECH_COMPLETE_TIMEOUT: verbioOptions.speech_complete_timeout}),
...(verbioOptions.speech_incomplete_timeout &&
{VERBIO_SPEECH_INCOMPLETE_TIMEOUT: verbioOptions.speech_incomplete_timeout}),
};
}
else if ('speechmatics' === vendor) {
const {speechmaticsOptions = {}} = rOpts;
opts = {
...opts,
...(sttCredentials.api_key) && {SPEECHMATICS_API_KEY: sttCredentials.api_key},
...(sttCredentials.speechmatics_stt_uri) && {SPEECHMATICS_HOST: sttCredentials.speechmatics_stt_uri},
...(rOpts.hints?.length > 0 && {SPEECHMATICS_SPEECH_HINTS: rOpts.hints.join(',')}),
...(speechmaticsOptions.translation_config &&
{
SPEECHMATICS_TRANSLATION_LANGUAGES: speechmaticsOptions.translation_config.target_languages.join(','),
SPEECHMATICS_TRANSLATION_PARTIALS: speechmaticsOptions.translation_config.enable_partials ? 1 : 0
}
),
...(speechmaticsOptions.transcription_config?.domain &&
{SPEECHMATICS_DOMAIN: speechmaticsOptions.transcription_config.domain}),
...{SPEECHMATICS_MAX_DELAY: speechmaticsOptions.transcription_config?.max_delay || 0.7},
...{SPEECHMATICS_MAX_DELAY_MODE: speechmaticsOptions.transcription_config?.max_delay_mode || 'flexible'},
...(speechmaticsOptions.transcription_config?.diarization &&
{SPEECHMATICS_DIARIZATION: speechmaticsOptions.transcription_config.diarization}),
...(speechmaticsOptions.transcription_config?.speaker_diarization_config?.speaker_sensitivity &&
{SPEECHMATICS_DIARIZATION_SPEAKER_SENSITIVITY:
speechmaticsOptions.transcription_config.speaker_diarization_config.speaker_sensitivity}),
...(speechmaticsOptions.transcription_config?.speaker_diarization_config?.max_speakers &&
{SPEECHMATICS_DIARIZATION_MAX_SPEAKERS:
speechmaticsOptions.transcription_config.speaker_diarization_config.max_speakers}),
...(speechmaticsOptions.transcription_config?.output_locale &&
{SPEECHMATICS_OUTPUT_LOCALE: speechmaticsOptions.transcription_config.output_locale}),
...(speechmaticsOptions.transcription_config?.punctuation_overrides?.permitted_marks &&
{SPEECHMATICS_PUNCTUATION_ALLOWED:
speechmaticsOptions.transcription_config.punctuation_overrides.permitted_marks.join(',')}),
...(speechmaticsOptions.transcription_config?.punctuation_overrides?.sensitivity &&
{SPEECHMATICS_PUNCTUATION_SENSITIVITY:
speechmaticsOptions.transcription_config?.punctuation_overrides?.sensitivity}),
...(speechmaticsOptions.transcription_config?.operating_point &&
{SPEECHMATICS_OPERATING_POINT: speechmaticsOptions.transcription_config.operating_point}),
...(speechmaticsOptions.transcription_config?.enable_entities &&
{SPEECHMATICS_ENABLE_ENTTIES: speechmaticsOptions.transcription_config.enable_entities}),
...(speechmaticsOptions.transcription_config?.audio_filtering_config?.volume_threshold &&
{SPEECHMATICS_VOLUME_THRESHOLD:
speechmaticsOptions.transcription_config.audio_filtering_config.volume_threshold}),
...(speechmaticsOptions.transcription_config?.transcript_filtering_config?.remove_disfluencies &&
{SPEECHMATICS_REMOVE_DISFLUENCIES:
speechmaticsOptions.transcription_config.transcript_filtering_config.remove_disfluencies})
};
}
else if (vendor.startsWith('custom:')) {
let {options = {}} = rOpts.customOptions || {};
const {sampleRate} = rOpts.customOptions || {};
let {options = {}} = rOpts;
const {auth_token, custom_stt_url} = sttCredentials;
options = {
...options,
...(rOpts.hints?.length > 0 && typeof rOpts.hints[0] === 'string' &&
...(rOpts.hints.length > 0 && typeof rOpts.hints[0] === 'string' &&
{hints: rOpts.hints}),
...(rOpts.hints?.length > 0 && typeof rOpts.hints[0] === 'object' &&
...(rOpts.hints.length > 0 && typeof rOpts.hints[0] === 'object' &&
{hints: JSON.stringify(rOpts.hints)}),
...(typeof rOpts.hintsBoost === 'number' && {hintsBoost: rOpts.hintsBoost}),
...(task.cs?.callSid && {callSid: task.cs.callSid})
...(typeof rOpts.hintsBoost === 'number' && {hintsBoost: rOpts.hintsBoost})
};
opts = {
...opts,
...(auth_token && {JAMBONZ_STT_API_KEY: auth_token}),
JAMBONZ_STT_API_KEY: auth_token,
JAMBONZ_STT_URL: custom_stt_url,
...(Object.keys(options).length > 0 && {JAMBONZ_STT_OPTIONS: JSON.stringify(options)}),
...(sampleRate && {JAMBONZ_STT_SAMPLING: sampleRate})
};
}
@@ -1131,6 +602,41 @@ module.exports = (logger) => {
return opts;
};
const removeSpeechListeners = (ep) => {
ep.removeCustomEventListener(GoogleTranscriptionEvents.Transcription);
ep.removeCustomEventListener(GoogleTranscriptionEvents.EndOfUtterance);
ep.removeCustomEventListener(GoogleTranscriptionEvents.VadDetected);
ep.removeCustomEventListener(AwsTranscriptionEvents.Transcription);
ep.removeCustomEventListener(AwsTranscriptionEvents.VadDetected);
ep.removeCustomEventListener(AzureTranscriptionEvents.Transcription);
ep.removeCustomEventListener(AzureTranscriptionEvents.NoSpeechDetected);
ep.removeCustomEventListener(AzureTranscriptionEvents.VadDetected);
ep.removeCustomEventListener(NuanceTranscriptionEvents.Transcription);
ep.removeCustomEventListener(NuanceTranscriptionEvents.TranscriptionComplete);
ep.removeCustomEventListener(NuanceTranscriptionEvents.StartOfSpeech);
ep.removeCustomEventListener(NuanceTranscriptionEvents.VadDetected);
ep.removeCustomEventListener(DeepgramTranscriptionEvents.Transcription);
ep.removeCustomEventListener(DeepgramTranscriptionEvents.Connect);
ep.removeCustomEventListener(DeepgramTranscriptionEvents.ConnectFailure);
ep.removeCustomEventListener(SonioxTranscriptionEvents.Transcription);
ep.removeCustomEventListener(NvidiaTranscriptionEvents.Transcription);
ep.removeCustomEventListener(NvidiaTranscriptionEvents.TranscriptionComplete);
ep.removeCustomEventListener(NvidiaTranscriptionEvents.StartOfSpeech);
ep.removeCustomEventListener(NvidiaTranscriptionEvents.VadDetected);
ep.removeCustomEventListener(JambonzTranscriptionEvents.Transcription);
ep.removeCustomEventListener(JambonzTranscriptionEvents.Connect);
ep.removeCustomEventListener(JambonzTranscriptionEvents.ConnectFailure);
ep.removeCustomEventListener(JambonzTranscriptionEvents.Error);
};
const setSpeechCredentialsAtRuntime = (recognizer) => {
if (!recognizer) return;
if (recognizer.vendor === 'nuance') {
@@ -1150,10 +656,6 @@ module.exports = (logger) => {
const {apiKey} = recognizer.sonioxOptions || {};
if (apiKey) return {api_key: apiKey};
}
else if (recognizer.vendor === 'cobalt') {
const {serverUri} = recognizer.cobaltOptions || {};
if (serverUri) return {cobalt_server_uri: serverUri};
}
else if (recognizer.vendor === 'ibm') {
const {ttsApiKey, ttsRegion, sttApiKey, sttRegion, instanceId} = recognizer.ibmOptions || {};
if (ttsApiKey || sttApiKey) return {
@@ -1169,8 +671,8 @@ module.exports = (logger) => {
return {
normalizeTranscription,
setChannelVarsForStt,
removeSpeechListeners,
setSpeechCredentialsAtRuntime,
compileSonioxTranscripts,
consolidateTranscripts,
compileSonioxTranscripts
};
};

View File

@@ -1,452 +0,0 @@
const Emitter = require('events');
const assert = require('assert');
const {
TtsStreamingEvents,
TtsStreamingConnectionStatus
} = require('../utils/constants');
const MAX_CHUNK_SIZE = 1800;
const HIGH_WATER_BUFFER_SIZE = 1000;
const LOW_WATER_BUFFER_SIZE = 200;
const TIMEOUT_RETRY_MSECS = 1000; // 1 second
const isWhitespace = (str) => /^\s*$/.test(str);
/**
* Each queue item is an object:
* - { type: 'text', value: '…' } for text tokens.
* - { type: 'flush' } for a flush command.
*/
class TtsStreamingBuffer extends Emitter {
constructor(cs) {
super();
this.cs = cs;
this.logger = cs.logger;
// Use an array to hold our structured items.
this.queue = [];
// Track total number of characters in text items.
this.bufferedLength = 0;
this.eventHandlers = [];
this._isFull = false;
this._connectionStatus = TtsStreamingConnectionStatus.NotConnected;
this.timer = null;
// Record the last time the text buffer was updated.
this.lastUpdateTime = 0;
}
get isEmpty() {
return this.queue.length === 0;
}
get size() {
return this.bufferedLength;
}
get isFull() {
return this._isFull;
}
get ep() {
return this.cs?.ep;
}
async start() {
assert.ok(
this._connectionStatus === TtsStreamingConnectionStatus.NotConnected,
'TtsStreamingBuffer:start already started, or has failed'
);
this.vendor = this.cs.getTsStreamingVendor();
if (!this.vendor) {
this.logger.info('TtsStreamingBuffer:start No TTS streaming vendor configured');
throw new Error('No TTS streaming vendor configured');
}
this.logger.info(`TtsStreamingBuffer:start Connecting to TTS streaming with vendor ${this.vendor}`);
this._connectionStatus = TtsStreamingConnectionStatus.Connecting;
try {
if (this.eventHandlers.length === 0) this._initHandlers(this.ep);
await this._api(this.ep, [this.ep.uuid, 'connect']);
} catch (err) {
this.logger.info({ err }, 'TtsStreamingBuffer:start Error connecting to TTS streaming');
this._connectionStatus = TtsStreamingConnectionStatus.Failed;
}
}
stop() {
clearTimeout(this.timer);
this.removeCustomEventListeners();
if (this.ep) {
this._api(this.ep, [this.ep.uuid, 'close'])
.catch((err) =>
this.logger.info({ err }, 'TtsStreamingBuffer:stop Error closing TTS streaming')
);
}
this.timer = null;
this.queue = [];
this.bufferedLength = 0;
this._connectionStatus = TtsStreamingConnectionStatus.NotConnected;
}
/**
* Buffer new text tokens.
*/
async bufferTokens(tokens) {
if (this._connectionStatus === TtsStreamingConnectionStatus.Failed) {
this.logger.info('TtsStreamingBuffer:bufferTokens TTS streaming connection failed, rejecting request');
return { status: 'failed', reason: `connection to ${this.vendor} failed` };
}
if (0 === this.bufferedLength && isWhitespace(tokens)) {
this.logger.debug({tokens}, 'TtsStreamingBuffer:bufferTokens discarded whitespace tokens');
return { status: 'ok' };
}
const displayedTokens = tokens.length <= 40 ? tokens : tokens.substring(0, 40);
const totalLength = tokens.length;
if (this.bufferedLength + totalLength > HIGH_WATER_BUFFER_SIZE) {
this.logger.info(
`TtsStreamingBuffer throttling: buffer is full, rejecting request to buffer ${totalLength} tokens`
);
if (!this._isFull) {
this._isFull = true;
this.emit(TtsStreamingEvents.Pause);
}
return { status: 'failed', reason: 'full' };
}
this.logger.debug(
`TtsStreamingBuffer:bufferTokens "${displayedTokens}" (length: ${totalLength})`
);
this.queue.push({ type: 'text', value: tokens });
this.bufferedLength += totalLength;
// Update the last update time each time new text is buffered.
this.lastUpdateTime = Date.now();
await this._feedQueue();
return { status: 'ok' };
}
/**
* Insert a flush command. If no text is queued, flush immediately.
* Otherwise, append a flush marker so that all text preceding it will be sent
* (regardless of sentence boundaries) before the flush is issued.
*/
flush() {
if (this._connectionStatus === TtsStreamingConnectionStatus.Connecting) {
this.logger.debug('TtsStreamingBuffer:flush TTS stream is not quite ready - wait for connect');
if (this.queue.length === 0 || this.queue[this.queue.length - 1].type !== 'flush') {
this.queue.push({ type: 'flush' });
}
return;
}
else if (this._connectionStatus === TtsStreamingConnectionStatus.Connected) {
if (this.isEmpty) {
this._doFlush();
}
else {
if (this.queue[this.queue.length - 1].type !== 'flush') {
this.queue.push({ type: 'flush' });
this.logger.debug('TtsStreamingBuffer:flush added flush marker to queue');
}
}
}
else {
this.logger.debug(
`TtsStreamingBuffer:flush TTS stream is not connected, status: ${this._connectionStatus}`
);
}
}
clear() {
this.logger.debug('TtsStreamingBuffer:clear');
if (this._connectionStatus !== TtsStreamingConnectionStatus.Connected) return;
clearTimeout(this.timer);
this._api(this.ep, [this.ep.uuid, 'clear']).catch((err) =>
this.logger.info({ err }, 'TtsStreamingBuffer:clear Error clearing TTS streaming')
);
this.queue = [];
this.bufferedLength = 0;
this.timer = null;
this._isFull = false;
}
/**
* Process the queue in two phases.
*
* Phase 1: Look for flush markers. When a flush marker is found (even if not at the very front),
* send all text tokens that came before it immediately (ignoring sentence boundaries)
* and then send the flush command. Repeat until there are no flush markers left.
*
* Phase 2: With the remaining queue (now containing only text items), accumulate text
* up to MAX_CHUNK_SIZE and use sentence-boundary logic to determine a chunk.
* Then, remove the exact tokens (or portions thereof) that were consumed.
*/
async _feedQueue(handlingTimeout = false) {
this.logger.debug({ queue: this.queue }, 'TtsStreamingBuffer:_feedQueue');
try {
if (!this.cs.isTtsStreamOpen || !this.ep) {
this.logger.debug('TtsStreamingBuffer:_feedQueue TTS stream is not open or no endpoint available');
return;
}
if (
this._connectionStatus === TtsStreamingConnectionStatus.NotConnected ||
this._connectionStatus === TtsStreamingConnectionStatus.Failed
) {
this.logger.debug('TtsStreamingBuffer:_feedQueue TTS stream is not connected');
return;
}
// --- Phase 1: Process flush markers ---
// Process any flush marker that isnt in the very first position.
let flushIndex = this.queue.findIndex((item, idx) => item.type === 'flush' && idx > 0);
while (flushIndex !== -1) {
let flushText = '';
// Accumulate all text tokens preceding the flush marker.
for (let i = 0; i < flushIndex; i++) {
if (this.queue[i].type === 'text') {
flushText += this.queue[i].value;
}
}
// Remove those text items.
for (let i = 0; i < flushIndex; i++) {
const item = this.queue.shift();
if (item.type === 'text') {
this.bufferedLength -= item.value.length;
}
}
// Remove the flush marker (now at the front).
if (this.queue.length > 0 && this.queue[0].type === 'flush') {
this.queue.shift();
}
// Immediately send all accumulated text (ignoring sentence boundaries).
if (flushText.length > 0) {
const modifiedFlushText = flushText.replace(/\n\n/g, '\n \n');
try {
await this._api(this.ep, [this.ep.uuid, 'send', modifiedFlushText]);
} catch (err) {
this.logger.info({ err, flushText }, 'TtsStreamingBuffer:_feedQueue Error sending TTS chunk');
}
}
// Send the flush command.
await this._doFlush();
flushIndex = this.queue.findIndex((item, idx) => item.type === 'flush' && idx > 0);
}
// If a flush marker is at the very front, process it.
while (this.queue.length > 0 && this.queue[0].type === 'flush') {
this.queue.shift();
await this._doFlush();
}
// --- Phase 2: Process remaining text tokens ---
if (this.queue.length === 0) {
this._removeTimer();
return;
}
// Accumulate contiguous text tokens (from the front) up to MAX_CHUNK_SIZE.
let combinedText = '';
for (const item of this.queue) {
if (item.type !== 'text') break;
combinedText += item.value;
if (combinedText.length >= MAX_CHUNK_SIZE) break;
}
if (combinedText.length === 0) {
this._removeTimer();
return;
}
const limit = Math.min(MAX_CHUNK_SIZE, combinedText.length);
let chunkEnd = findSentenceBoundary(combinedText, limit);
if (chunkEnd <= 0) {
if (handlingTimeout) {
chunkEnd = findWordBoundary(combinedText, limit);
if (chunkEnd <= 0) {
this._setTimerIfNeeded();
return;
}
} else {
this._setTimerIfNeeded();
return;
}
}
const chunk = combinedText.slice(0, chunkEnd);
// Now we iterate over the queue items
// and deduct their lengths until we've accounted for chunkEnd characters.
let remaining = chunkEnd;
let tokensProcessed = 0;
for (let i = 0; i < this.queue.length; i++) {
const token = this.queue[i];
if (token.type !== 'text') break;
if (remaining >= token.value.length) {
remaining -= token.value.length;
tokensProcessed = i + 1;
} else {
// Partially consumed token: update its value to remove the consumed part.
token.value = token.value.slice(remaining);
tokensProcessed = i;
remaining = 0;
break;
}
}
// Remove the fully consumed tokens from the front of the queue.
this.queue.splice(0, tokensProcessed);
this.bufferedLength -= chunkEnd;
const modifiedChunk = chunk.replace(/\n\n/g, '\n \n');
this.logger.debug(`TtsStreamingBuffer:_feedQueue sending chunk to tts: ${modifiedChunk}`);
try {
await this._api(this.ep, [this.ep.uuid, 'send', modifiedChunk]);
} catch (err) {
this.logger.info({ err, chunk }, 'TtsStreamingBuffer:_feedQueue Error sending TTS chunk');
}
if (this._isFull && this.bufferedLength <= LOW_WATER_BUFFER_SIZE) {
this.logger.info('TtsStreamingBuffer throttling: buffer is no longer full - resuming');
this._isFull = false;
this.emit(TtsStreamingEvents.Resume);
}
return this._feedQueue();
} catch (err) {
this.logger.info({ err }, 'TtsStreamingBuffer:_feedQueue Error sending TTS chunk');
this.queue = [];
this.bufferedLength = 0;
}
}
async _api(ep, args) {
const apiCmd = `uuid_${this.vendor.startsWith('custom:') ? 'custom' : this.vendor}_tts_streaming`;
const res = await ep.api(apiCmd, `^^|${args.join('|')}`);
if (!res.body?.startsWith('+OK')) {
this.logger.info({ args }, `Error calling ${apiCmd}: ${res.body}`);
throw new Error(`Error calling ${apiCmd}: ${res.body}`);
}
}
_doFlush() {
return this._api(this.ep, [this.ep.uuid, 'flush'])
.then(() => this.logger.debug('TtsStreamingBuffer:_doFlush sent flush command'))
.catch((err) =>
this.logger.info(
{ err },
`TtsStreamingBuffer:_doFlush Error flushing TTS streaming: ${JSON.stringify(err)}`
)
);
}
async _onConnect(vendor) {
this.logger.info(`TtsStreamingBuffer:_onConnect streaming tts connection made to ${vendor} successful`);
this._connectionStatus = TtsStreamingConnectionStatus.Connected;
if (this.queue.length > 0) {
await this._feedQueue();
}
}
_onConnectFailure(vendor) {
this.logger.info(`TtsStreamingBuffer:_onConnectFailure streaming tts connection failed to ${vendor}`);
this._connectionStatus = TtsStreamingConnectionStatus.Failed;
this.queue = [];
this.bufferedLength = 0;
this.emit(TtsStreamingEvents.ConnectFailure, { vendor });
}
_setTimerIfNeeded() {
if (this.bufferedLength > 0 && !this.timer) {
this.logger.debug({queue: this.queue},
`TtsStreamingBuffer:_setTimerIfNeeded setting timer because ${this.bufferedLength} buffered`);
this.timer = setTimeout(this._onTimeout.bind(this), TIMEOUT_RETRY_MSECS);
}
}
_removeTimer() {
if (this.timer) {
this.logger.debug('TtsStreamingBuffer:_removeTimer clearing timer');
clearTimeout(this.timer);
this.timer = null;
}
}
_onTimeout() {
this.logger.debug('TtsStreamingBuffer:_onTimeout Timeout waiting for sentence boundary');
this.timer = null;
// Check if new text has been added since the timer was set.
const now = Date.now();
if (now - this.lastUpdateTime < TIMEOUT_RETRY_MSECS) {
this.logger.debug('TtsStreamingBuffer:_onTimeout New text received recently; postponing flush.');
this._setTimerIfNeeded();
return;
}
this._feedQueue(true);
}
_onTtsEmpty(vendor) {
this.emit(TtsStreamingEvents.Empty, { vendor });
}
addCustomEventListener(ep, event, handler) {
this.eventHandlers.push({ ep, event, handler });
ep.addCustomEventListener(event, handler);
}
removeCustomEventListeners() {
this.eventHandlers.forEach((h) => h.ep.removeCustomEventListener(h.event, h.handler));
}
_initHandlers(ep) {
[
'deepgram',
'cartesia',
'elevenlabs',
'rimelabs',
'custom'
].forEach((vendor) => {
const eventClassName = `${vendor.charAt(0).toUpperCase() + vendor.slice(1)}TtsStreamingEvents`;
const eventClass = require('../utils/constants')[eventClassName];
if (!eventClass) throw new Error(`Event class for vendor ${vendor} not found`);
this.addCustomEventListener(ep, eventClass.Connect, this._onConnect.bind(this, vendor));
this.addCustomEventListener(ep, eventClass.ConnectFailure, this._onConnectFailure.bind(this, vendor));
this.addCustomEventListener(ep, eventClass.Empty, this._onTtsEmpty.bind(this, vendor));
});
}
}
const findSentenceBoundary = (text, limit) => {
// Look for punctuation or double newline that signals sentence end.
const sentenceEndRegex = /[.!?](?=\s|$)|\n\n/g;
let lastSentenceBoundary = -1;
let match;
while ((match = sentenceEndRegex.exec(text)) && match.index < limit) {
const precedingText = text.slice(0, match.index).trim();
if (precedingText.length > 0) {
if (
match[0] === '\n\n' ||
(match.index === 0 || !/\d$/.test(text[match.index - 1]))
) {
lastSentenceBoundary = match.index + (match[0] === '\n\n' ? 2 : 1);
}
}
}
return lastSentenceBoundary;
};
const findWordBoundary = (text, limit) => {
const wordBoundaryRegex = /\s+/g;
let lastWordBoundary = -1;
let match;
while ((match = wordBoundaryRegex.exec(text)) && match.index < limit) {
lastWordBoundary = match.index;
}
return lastWordBoundary;
};
module.exports = TtsStreamingBuffer;

View File

@@ -1,31 +1,11 @@
const assert = require('assert');
const BaseRequestor = require('./base-requestor');
const short = require('short-uuid');
const {HookMsgTypes, WS_CLOSE_CODES} = require('./constants.json');
const {HookMsgTypes} = require('./constants.json');
const Websocket = require('ws');
const snakeCaseKeys = require('./snakecase-keys');
const {
RESPONSE_TIMEOUT_MS,
JAMBONES_WS_PING_INTERVAL_MS,
MAX_RECONNECTS,
JAMBONES_WS_HANDSHAKE_TIMEOUT_MS,
JAMBONES_WS_MAX_PAYLOAD,
HTTP_USER_AGENT_HEADER
} = require('../config');
const MTYPE_WANTS_ACK = [
'call:status',
'verb:status',
'jambonz:error',
'llm:event',
'llm:tool-call',
'tts:streaming-event',
'tts:tokens-result',
];
const MTYPE_NO_DATA = [
'llm:tool-output',
'tts:flush',
'tts:clear'
];
const MAX_RECONNECTS = 5;
const RESPONSE_TIMEOUT_MS = process.env.JAMBONES_WS_API_MSG_RESPONSE_TIMEOUT || 5000;
class WsRequestor extends BaseRequestor {
constructor(logger, account_sid, hook, secret) {
@@ -58,7 +38,6 @@ class WsRequestor extends BaseRequestor {
async request(type, hook, params, httpHeaders = {}) {
assert(HookMsgTypes.includes(type));
const url = hook.url || hook;
const wantsAck = !MTYPE_WANTS_ACK.includes(type);
if (this.maliciousClient) {
this.logger.info({url: this.url}, 'WsRequestor:request - discarding msg to malicious client');
@@ -70,12 +49,6 @@ class WsRequestor extends BaseRequestor {
}
if (type === 'session:new') this.call_sid = params.callSid;
if (type === 'session:reconnect') {
this._reconnectPromise = new Promise((resolve, reject) => {
this._reconnectResolve = resolve;
this._reconnectReject = reject;
});
}
/* if we have an absolute url, and it is http then do a standard webhook */
if (this._isAbsoluteUrl(url) && url.startsWith('http')) {
@@ -91,26 +64,15 @@ class WsRequestor extends BaseRequestor {
}
/* connect if necessary */
const queueMsg = () => {
this.logger.debug(
`WsRequestor:request(${this.id}) - queueing ${type} message since we are connecting`);
if (wantsAck) {
const p = new Promise((resolve, reject) => {
this.queuedMsg.push({type, hook, params, httpHeaders, promise: {resolve, reject}});
});
return p;
}
else {
this.queuedMsg.push({type, hook, params, httpHeaders});
}
return;
};
if (!this.ws) {
if (this.connectInProgress) {
return queueMsg();
this.logger.debug(
`WsRequestor:request(${this.id}) - queueing ${type} message since we are connecting`);
this.queuedMsg.push({type, hook, params, httpHeaders});
return;
}
this.connectInProgress = true;
this.logger.debug(`WsRequestor:request(${this.id}) - connecting since we do not have a connection for ${type}`);
this.logger.debug(`WsRequestor:request(${this.id}) - connecting since we do not have a connection`);
if (this.connections >= MAX_RECONNECTS) {
return Promise.reject(`max attempts connecting to ${this.url}`);
}
@@ -125,15 +87,11 @@ class WsRequestor extends BaseRequestor {
return Promise.reject(err);
}
}
// If jambonz wait for ack from reconnect, queue the msg until reconnect is acked
if (type !== 'session:reconnect' && this._reconnectPromise) {
return queueMsg();
}
assert(this.ws);
/* prepare and send message */
let payload = params ? snakeCaseKeys(params, ['customerData', 'sip', 'env_vars', 'args']) : null;
if (type === 'session:new' || type === 'session:adulting') this._sessionData = payload;
let payload = params ? snakeCaseKeys(params, ['customerData', 'sip']) : null;
if (type === 'session:new') this._sessionData = payload;
if (type === 'session:reconnect') payload = this._sessionData;
assert.ok(url, 'WsRequestor:request url was not provided');
@@ -146,35 +104,16 @@ class WsRequestor extends BaseRequestor {
type,
msgid,
call_sid: this.call_sid,
hook: [
'verb:hook', 'dial:confirm', 'session:redirect', 'llm:event', 'llm:tool-call'
].includes(type) ? url : undefined,
hook: type === 'verb:hook' ? url : undefined,
data: {...payload},
...b3
};
const sendQueuedMsgs = () => {
if (this.queuedMsg.length > 0) {
for (const {type, hook, params, httpHeaders, promise} of this.queuedMsg) {
for (const {type, hook, params, httpHeaders} of this.queuedMsg) {
this.logger.debug(`WsRequestor:request - preparing queued ${type} for sending`);
if (promise) {
this.request(type, hook, params, httpHeaders)
.then((res) => promise.resolve(res))
.catch((err) => promise.reject(err));
}
else setImmediate(this.request.bind(this, type, hook, params, httpHeaders));
}
this.queuedMsg.length = 0;
}
};
const rejectQueuedMsgs = (err) => {
if (this.queuedMsg.length > 0) {
for (const {promise} of this.queuedMsg) {
this.logger.debug(`WsRequestor:request - preparing queued ${type} for rejectQueuedMsgs`);
if (promise) {
promise.reject(err);
}
setImmediate(this.request.bind(this, type, hook, params, httpHeaders));
}
this.queuedMsg.length = 0;
}
@@ -193,8 +132,8 @@ class WsRequestor extends BaseRequestor {
}
/* simple notifications */
if (!wantsAck || reconnectingWithoutAck) {
this.ws?.send(JSON.stringify(obj), () => {
if (['call:status', 'verb:status', 'jambonz:error'].includes(type) || reconnectingWithoutAck) {
this.ws.send(JSON.stringify(obj), () => {
this.logger.debug({obj}, `WsRequestor:request websocket: sent (${url})`);
sendQueuedMsgs();
});
@@ -220,56 +159,27 @@ class WsRequestor extends BaseRequestor {
this.logger.debug({response}, `WsRequestor:request ${url} succeeded in ${rtt}ms`);
this.stats.histogram('app.hook.ws_response_time', rtt, ['hook_type:app']);
resolve(response);
if (this._reconnectResolve) {
this._reconnectResolve();
}
},
failure: (err) => {
if (this._reconnectReject) {
this._reconnectReject(err);
}
clearTimeout(timer);
reject(err);
}
});
/* send the message */
this.ws.send(JSON.stringify(obj), async() => {
this.ws.send(JSON.stringify(obj), () => {
this.logger.debug({obj}, `WsRequestor:request websocket: sent (${url})`);
// If session:reconnect is waiting for ack, hold here until ack to send queuedMsgs
if (this._reconnectPromise) {
try {
await this._reconnectPromise;
} catch (err) {
// bad thing happened to session:recconnect
rejectQueuedMsgs(err);
this.emit('reconnect-error');
return;
} finally {
this._reconnectPromise = null;
this._reconnectResolve = null;
this._reconnectReject = null;
}
}
sendQueuedMsgs();
});
});
}
_stopPingTimer() {
if (this._pingTimer) {
clearInterval(this._pingTimer);
this._pingTimer = null;
}
}
close(code = WS_CLOSE_CODES.NormalClosure) {
close() {
this.closedGracefully = true;
this.logger.debug(`WsRequestor:close closing socket with code ${code}`);
this._stopPingTimer();
this.logger.debug('WsRequestor:close closing socket');
try {
if (this.ws) {
this.ws.close(code);
this.ws.close(1000);
this.ws.removeAllListeners();
this.ws = null;
}
@@ -281,19 +191,15 @@ class WsRequestor extends BaseRequestor {
_connect() {
assert(!this.ws);
this._stopPingTimer();
return new Promise((resolve, reject) => {
const handshakeTimeout = JAMBONES_WS_HANDSHAKE_TIMEOUT_MS ?
parseInt(JAMBONES_WS_HANDSHAKE_TIMEOUT_MS) :
const handshakeTimeout = process.env.JAMBONES_WS_HANDSHAKE_TIMEOUT_MS ?
parseInt(process.env.JAMBONES_WS_HANDSHAKE_TIMEOUT_MS) :
1500;
let opts = {
followRedirects: true,
maxRedirects: 2,
handshakeTimeout,
maxPayload: JAMBONES_WS_MAX_PAYLOAD ? parseInt(JAMBONES_WS_MAX_PAYLOAD) : 24 * 1024,
headers: {
...(HTTP_USER_AGENT_HEADER && {'user-agent' : HTTP_USER_AGENT_HEADER})
}
maxPayload: process.env.JAMBONES_WS_MAX_PAYLOAD ? parseInt(process.env.JAMBONES_WS_MAX_PAYLOAD) : 24 * 1024,
};
if (this.username && this.password) opts = {...opts, auth: `${this.username}:${this.password}`};
@@ -345,15 +251,10 @@ class WsRequestor extends BaseRequestor {
this.connectInProgress = false;
this.connections++;
this.emit('ready', ws);
if (JAMBONES_WS_PING_INTERVAL_MS > 15000) {
this._pingTimer = setInterval(() => this.ws?.ping(), JAMBONES_WS_PING_INTERVAL_MS);
}
}
_onClose(code) {
this.logger.info(`WsRequestor(${this.id}) - closed from far end ${code}`);
this._stopPingTimer();
if (this.connections > 0 && code !== 1000) {
this.logger.info({url: this.url}, 'WsRequestor - socket closed unexpectedly from remote side');
this.emit('socket-closed');
@@ -378,7 +279,6 @@ class WsRequestor extends BaseRequestor {
_onSocketClosed() {
this.ws = null;
this.emit('connection-dropped');
this._stopPingTimer();
if (this.connections > 0 && this.connections < MAX_RECONNECTS && !this.closedGracefully) {
if (!this._initMsgId) this._clearPendingMessages();
this.logger.debug(`WsRequestor:_onSocketClosed waiting ${this.backoffMs} to reconnect`);
@@ -388,9 +288,7 @@ class WsRequestor extends BaseRequestor {
'WsRequestor:_onSocketClosed time to reconnect');
if (!this.ws && !this.connectInProgress) {
this.connectInProgress = true;
return this._connect()
.catch((err) => this.logger.error('WsRequestor:_onSocketClosed There is error while reconnect', err))
.finally(() => this.connectInProgress = false);
this._connect().catch((err) => this.connectInProgress = false);
}
}, this.backoffMs);
this.backoffMs = this.backoffMs < 2000 ? this.backoffMs * 2 : (this.backoffMs + 2000);
@@ -408,10 +306,7 @@ class WsRequestor extends BaseRequestor {
/* messages must be JSON format */
try {
const obj = JSON.parse(content);
this.logger.debug({obj}, 'WsRequestor:_onMessage - received message');
//const {type, msgid, command, call_sid = this.call_sid, queueCommand = false, data} = obj;
const {type, msgid, command, queueCommand = false, tool_call_id, data} = obj;
const call_sid = obj.callSid || this.call_sid;
const {type, msgid, command, call_sid = this.call_sid, queueCommand = false, data} = obj;
//this.logger.debug({obj}, 'WsRequestor:request websocket: received');
assert.ok(type, 'type property not supplied');
@@ -424,8 +319,8 @@ class WsRequestor extends BaseRequestor {
case 'command':
assert.ok(command, 'command property not supplied');
assert.ok(data || MTYPE_NO_DATA.includes(command), 'data property not supplied');
this._recvCommand(msgid, command, call_sid, queueCommand, tool_call_id, data);
assert.ok(data, 'data property not supplied');
this._recvCommand(msgid, command, call_sid, queueCommand, data);
break;
default:
@@ -433,21 +328,6 @@ class WsRequestor extends BaseRequestor {
}
} catch (err) {
this.logger.info({err, content}, 'WsRequestor:_onMessage - invalid incoming message');
const params = {
msg: 'InvalidMessage',
details: err.message,
content: Buffer.from(content).toString('utf-8')
};
const {writeAlerts, AlertType} = this.Alerter;
writeAlerts({
account_sid: this.account_sid,
alert_type: AlertType.INVALID_APP_PAYLOAD,
target_sid: this.call_sid,
message: err.message,
}).catch((err) => this.logger.info({err}, 'Error generating alert for invalid message'));
this.request('jambonz:error', '/error', params)
.catch((err) => this.logger.debug({err}, 'WsRequestor:_onMessage - Error sending'));
}
}
@@ -464,10 +344,10 @@ class WsRequestor extends BaseRequestor {
success && success(data);
}
_recvCommand(msgid, command, call_sid, queueCommand, tool_call_id, data) {
_recvCommand(msgid, command, call_sid, queueCommand, data) {
// TODO: validate command
this.logger.debug({msgid, command, call_sid, queueCommand, data}, 'received command');
this.emit('command', {msgid, command, call_sid, queueCommand, tool_call_id, data});
this.emit('command', {msgid, command, call_sid, queueCommand, data});
}
}

14632
package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,9 +1,9 @@
{
"name": "jambonz-feature-server",
"version": "0.9.3",
"version": "v0.8.2",
"main": "app.js",
"engines": {
"node": ">= 18.x"
"node": ">= 10.16.0"
},
"keywords": [
"sip",
@@ -19,63 +19,59 @@
"bugs": {},
"scripts": {
"start": "node app",
"test": "NODE_ENV=test JAMBONES_HOSTING=1 HTTP_POOL=1 JAMBONES_TTS_TRIM_SILENCE=1 ENCRYPTION_SECRET=foobar DRACHTIO_HOST=127.0.0.1 DRACHTIO_PORT=9060 DRACHTIO_SECRET=cymru JAMBONES_MYSQL_HOST=127.0.0.1 JAMBONES_MYSQL_PORT=3360 JAMBONES_MYSQL_USER=jambones_test JAMBONES_MYSQL_PASSWORD=jambones_test JAMBONES_MYSQL_DATABASE=jambones_test JAMBONES_REDIS_HOST=127.0.0.1 JAMBONES_REDIS_PORT=16379 JAMBONES_LOGLEVEL=error ENABLE_METRICS=0 HTTP_PORT=3000 JAMBONES_SBCS=172.38.0.10 JAMBONES_FREESWITCH=127.0.0.1:8022:JambonzR0ck$:docker-host JAMBONES_TIME_SERIES_HOST=127.0.0.1 JAMBONES_NETWORK_CIDR=172.38.0.0/16 node test/ ",
"test": "NODE_ENV=test JAMBONES_HOSTING=1 HTTP_POOL=1 ENCRYPTION_SECRET=foobar DRACHTIO_HOST=127.0.0.1 DRACHTIO_PORT=9060 DRACHTIO_SECRET=cymru JAMBONES_MYSQL_HOST=127.0.0.1 JAMBONES_MYSQL_PORT=3360 JAMBONES_MYSQL_USER=jambones_test JAMBONES_MYSQL_PASSWORD=jambones_test JAMBONES_MYSQL_DATABASE=jambones_test JAMBONES_REDIS_HOST=127.0.0.1 JAMBONES_REDIS_PORT=16379 JAMBONES_LOGLEVEL=error ENABLE_METRICS=0 HTTP_PORT=3000 JAMBONES_SBCS=172.38.0.10 JAMBONES_FREESWITCH=127.0.0.1:8022:JambonzR0ck$:docker-host JAMBONES_TIME_SERIES_HOST=127.0.0.1 JAMBONES_NETWORK_CIDR=172.38.0.0/16 node test/ ",
"coverage": "./node_modules/.bin/nyc --reporter html --report-dir ./coverage npm run test",
"jslint": "eslint app.js tracer.js lib",
"jslint:fix": "eslint app.js tracer.js lib --fix"
"jslint": "eslint app.js lib"
},
"dependencies": {
"@aws-sdk/client-auto-scaling": "^3.549.0",
"@aws-sdk/client-sns": "^3.549.0",
"@jambonz/db-helpers": "^0.9.12",
"@jambonz/db-helpers": "^0.7.4",
"@jambonz/http-health-check": "^0.0.1",
"@jambonz/mw-registrar": "^0.2.7",
"@jambonz/realtimedb-helpers": "^0.8.13",
"@jambonz/speech-utils": "^0.2.10",
"@jambonz/stats-collector": "^0.1.10",
"@jambonz/time-series": "^0.2.13",
"@jambonz/verb-specifications": "^0.0.104",
"@modelcontextprotocol/sdk": "^1.9.0",
"@opentelemetry/api": "^1.8.0",
"@opentelemetry/exporter-jaeger": "^1.23.0",
"@opentelemetry/exporter-trace-otlp-http": "^0.50.0",
"@opentelemetry/exporter-zipkin": "^1.23.0",
"@opentelemetry/instrumentation": "^0.50.0",
"@opentelemetry/resources": "^1.23.0",
"@opentelemetry/sdk-trace-base": "^1.23.0",
"@opentelemetry/sdk-trace-node": "^1.23.0",
"@opentelemetry/semantic-conventions": "^1.23.0",
"@jambonz/realtimedb-helpers": "^0.7.0",
"@jambonz/speech-utils": "^0.0.12",
"@jambonz/stats-collector": "^0.1.8",
"@jambonz/time-series": "^0.2.5",
"@jambonz/verb-specifications": "^0.0.11",
"@opentelemetry/api": "^1.4.0",
"@opentelemetry/exporter-jaeger": "^1.9.0",
"@opentelemetry/exporter-trace-otlp-http": "^0.35.0",
"@opentelemetry/exporter-zipkin": "^1.9.0",
"@opentelemetry/instrumentation": "^0.35.0",
"@opentelemetry/resources": "^1.9.0",
"@opentelemetry/sdk-trace-base": "^1.9.0",
"@opentelemetry/sdk-trace-node": "^1.9.0",
"@opentelemetry/semantic-conventions": "^1.9.0",
"aws-sdk": "^2.1313.0",
"bent": "^7.3.12",
"debug": "^4.3.4",
"deepcopy": "^2.1.0",
"drachtio-fsmrf": "^4.0.3",
"drachtio-srf": "^5.0.5",
"express": "^4.19.2",
"express-validator": "^7.0.1",
"moment": "^2.30.1",
"parse-url": "^9.2.0",
"pino": "^8.20.0",
"drachtio-fsmrf": "^3.0.20",
"drachtio-srf": "^4.5.23",
"express": "^4.18.2",
"ip": "^1.1.8",
"moment": "^2.29.4",
"parse-url": "^8.1.0",
"pino": "^8.8.0",
"polly-ssml-split": "^0.1.0",
"proxyquire": "^2.1.3",
"sdp-transform": "^2.15.0",
"short-uuid": "^5.1.0",
"sinon": "^17.0.1",
"sdp-transform": "^2.14.1",
"short-uuid": "^4.2.2",
"sinon": "^15.0.1",
"to-snake-case": "^1.0.0",
"undici": "^7.5.0",
"undici": "^5.19.1",
"uuid-random": "^1.3.2",
"verify-aws-sns-signature": "^0.1.0",
"ws": "^8.18.0",
"xml2js": "^0.6.2"
"ws": "^8.9.0",
"xml2js": "^0.5.0"
},
"devDependencies": {
"clear-module": "^4.1.2",
"eslint": "7.32.0",
"eslint-plugin-promise": "^6.1.1",
"eslint": "^7.32.0",
"eslint-plugin-promise": "^4.3.1",
"nyc": "^15.1.0",
"tape": "^5.7.5"
"tape": "^5.6.1"
},
"optionalDependencies": {
"bufferutil": "^4.0.8",
"utf-8-validate": "^6.0.3"
"bufferutil": "^4.0.6",
"utf-8-validate": "^5.0.8"
}
}

View File

@@ -39,7 +39,7 @@ test('\'config: listen\'', async(t) => {
}
];
await provisionCallHook(from, verbs);
provisionCallHook(from, verbs);
// THEN
await sippUac('uac-gather-account-creds-success-send-bye.xml', '172.38.0.10', from);
@@ -86,7 +86,7 @@ test('\'config: listen - stop\'', async(t) => {
}
];
await provisionCallHook(from, verbs);
provisionCallHook(from, verbs);
// THEN
await sippUac('uac-gather-account-creds-success-send-bye.xml', '172.38.0.10', from);

View File

@@ -1,86 +0,0 @@
#
# Recommended minimum configuration:
#
# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 0.0.0.1-0.255.255.255 # RFC 1122 "this" network (LAN)
acl localnet src 10.0.0.0/8 # RFC 1918 local private network (LAN)
acl localnet src 100.64.0.0/10 # RFC 6598 shared address space (CGN)
acl localnet src 169.254.0.0/16 # RFC 3927 link-local (directly plugged) machines
acl localnet src 172.16.0.0/12 # RFC 1918 local private network (LAN)
acl localnet src 172.38.0.0/12 # RFC 1918 local private network (LAN)
acl localnet src 192.168.0.0/16 # RFC 1918 local private network (LAN)
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
#
# Recommended minimum Access Permission configuration:
#
# Deny requests to certain unsafe ports
http_access allow !Safe_ports
# Deny CONNECT to other than secure SSL ports
http_access allow CONNECT !SSL_ports
# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access allow manager
# This default configuration only allows localhost requests because a more
# permissive Squid installation could introduce new attack vectors into the
# network by proxying external TCP connections to unprotected services.
http_access allow localhost
# The two deny rules below are unnecessary in this default configuration
# because they are followed by a "deny all" rule. However, they may become
# critically important when you start allowing external requests below them.
# Protect web applications running on the same server as Squid. They often
# assume that only local users can access them at "localhost" ports.
http_access allow to_localhost
# Protect cloud servers that provide local users with sensitive info about
# their server via certain well-known link-local (a.k.a. APIPA) addresses.
# http_access deny to_linklocal
#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#
# For example, to allow access from your local networks, you may uncomment the
# following rule (and/or add rules that match your definition of "local"):
# http_access allow localnet
# And finally deny all other access to this proxy
http_access allow all
# Squid normally listens to port 3128
http_port 3128
# Uncomment and adjust the following to add a disk cache directory.
#cache_dir ufs /usr/local/var/cache/squid 100 16 256
# Leave coredumps in the first cache dir
coredump_dir /usr/local/var/cache/squid
#
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320

View File

@@ -37,7 +37,7 @@ test('test create-call timeout', async(t) => {
'account_sid':account_sid,
'timeout': 1,
"call_hook": {
"url": "https://public-apps.jambonz.cloud/hello-world",
"url": "https://public-apps.jambonz.us/hello-world",
"method": "POST"
},
"from": "15083718299",
@@ -88,192 +88,17 @@ test('test create-call call-hook basic authentication', async(t) => {
let verbs = [
{
"verb": "pause",
"length": 1
"verb": "say",
"text": "hello"
}
];
await provisionCallHook(from, verbs);
provisionCallHook(from, verbs);
//THEN
await p;
let obj = await getJSON(`http:127.0.0.1:3100/lastRequest/${from}`)
t.ok(obj.headers.Authorization = 'Basic dXNlcm5hbWU6cGFzc3dvcmQ=',
'create-call: call-hook contains basic authentication header');
t.ok(obj.headers['user-agent'] = 'jambonz',
'create-call: call-hook contains user-agent header');
disconnect();
} catch (err) {
console.log(`error received: ${err}`);
disconnect();
t.error(err);
}
});
test('test create-call amd', async(t) => {
clearModule.all();
const {srf, disconnect} = require('../app');
try {
await connect(srf);
// GIVEN
let from = 'create-call-amd';
let account_sid = 'bb845d4b-83a9-4cde-a6e9-50f3743bab3f';
// Give UAS app time to come up
const p = sippUac('uas.xml', '172.38.0.10', from);
await waitFor(1000);
const post = bent('http://127.0.0.1:3000/', 'POST', 'json', 201);
post('v1/createCall', {
'account_sid':account_sid,
"call_hook": {
"url": "http://127.0.0.1:3100/",
"method": "POST",
"username": "username",
"password": "password"
},
"from": from,
"to": {
"type": "phone",
"number": "15583084809"
},
"amd": {
"actionHook": "/actionHook"
},
"speech_recognizer_vendor": "google",
"speech_recognizer_language": "en"
});
let verbs = [
{
"verb": "pause",
"length": 7
}
];
await provisionCallHook(from, verbs);
//THEN
await p;
let obj = await getJSON(`http:127.0.0.1:3100/lastRequest/${from}_actionHook`)
t.ok(obj.body.type = 'amd_no_speech_detected',
'create-call: AMD detected');
disconnect();
} catch (err) {
console.log(`error received: ${err}`);
disconnect();
t.error(err);
}
});
test('test create-call app_json', async(t) => {
clearModule.all();
const {srf, disconnect} = require('../app');
try {
await connect(srf);
// GIVEN
let from = 'create-call-app-json';
let account_sid = 'bb845d4b-83a9-4cde-a6e9-50f3743bab3f';
// Give UAS app time to come up
const p = sippUac('uas.xml', '172.38.0.10', from);
await waitFor(1000);
const app_json = `[
{
"verb": "pause",
"length": 7
}
]`;
const post = bent('http://127.0.0.1:3000/', 'POST', 'json', 201);
post('v1/createCall', {
'account_sid':account_sid,
"call_hook": {
"url": "http://127.0.0.1:3100/",
"method": "POST",
"username": "username",
"password": "password"
},
app_json,
"from": from,
"to": {
"type": "phone",
"number": "15583084809"
},
"amd": {
"actionHook": "/actionHook"
},
"speech_recognizer_vendor": "google",
"speech_recognizer_language": "en"
});
//THEN
await p;
disconnect();
} catch (err) {
console.log(`error received: ${err}`);
disconnect();
t.error(err);
}
});
test('test create-call timeLimit', async(t) => {
clearModule.all();
const {srf, disconnect} = require('../app');
try {
await connect(srf);
// GIVEN
let from = 'create-call-app-json';
let account_sid = 'bb845d4b-83a9-4cde-a6e9-50f3743bab3f';
// Give UAS app time to come up
const p = sippUac('uas.xml', '172.38.0.10', from);
await waitFor(1000);
const startTime = Date.now();
const app_json = `[
{
"verb": "pause",
"length": 7
}
]`;
const post = bent('http://127.0.0.1:3000/', 'POST', 'json', 201);
post('v1/createCall', {
'account_sid':account_sid,
"call_hook": {
"url": "http://127.0.0.1:3100/",
"method": "POST",
"username": "username",
"password": "password"
},
app_json,
"from": from,
"to": {
"type": "phone",
"number": "15583084809"
},
"timeLimit": 1,
"speech_recognizer_vendor": "google",
"speech_recognizer_language": "en"
});
//THEN
await p;
const endTime = Date.now();
t.ok(endTime - startTime < 2000, 'create-call: timeLimit is respected');
disconnect();
} catch (err) {
console.log(`error received: ${err}`);

View File

@@ -2,14 +2,6 @@ const test = require('tape') ;
const exec = require('child_process').exec ;
const fs = require('fs');
const {encrypt} = require('../lib/utils/encrypt-decrypt');
const {
GCP_JSON_KEY,
AWS_ACCESS_KEY_ID,
AWS_SECRET_ACCESS_KEY,
AWS_REGION,
MICROSOFT_REGION,
MICROSOFT_API_KEY,
} = require('../lib/config');
test('creating jambones_test database', (t) => {
exec(`mysql -h 127.0.0.1 -u root --protocol=tcp --port=3360 < ${__dirname}/db/create_test_db.sql`, (err, stdout, stderr) => {
@@ -27,24 +19,24 @@ test('creating schema', (t) => {
t.pass('schema and test data successfully created');
const sql = [];
if (GCP_JSON_KEY) {
const google_credential = encrypt(GCP_JSON_KEY);
if (process.env.GCP_JSON_KEY) {
const google_credential = encrypt(process.env.GCP_JSON_KEY);
t.pass('adding google credentials');
sql.push(`UPDATE speech_credentials SET credential='${google_credential}' WHERE vendor='google';`);
}
if (AWS_ACCESS_KEY_ID && AWS_SECRET_ACCESS_KEY) {
if (process.env.AWS_ACCESS_KEY_ID && process.env.AWS_SECRET_ACCESS_KEY) {
const aws_credential = encrypt(JSON.stringify({
access_key_id: AWS_ACCESS_KEY_ID,
secret_access_key: AWS_SECRET_ACCESS_KEY,
aws_region: AWS_REGION
access_key_id: process.env.AWS_ACCESS_KEY_ID,
secret_access_key: process.env.AWS_SECRET_ACCESS_KEY,
aws_region: process.env.AWS_REGION
}));
t.pass('adding aws credentials');
sql.push(`UPDATE speech_credentials SET credential='${aws_credential}' WHERE vendor='aws';`);
}
if (MICROSOFT_REGION && MICROSOFT_API_KEY) {
if (process.env.MICROSOFT_REGION && process.env.MICROSOFT_API_KEY) {
const microsoft_credential = encrypt(JSON.stringify({
region: MICROSOFT_REGION,
api_key: MICROSOFT_API_KEY
region: process.env.MICROSOFT_REGION,
api_key: process.env.MICROSOFT_API_KEY
}));
t.pass('adding microsoft credentials');
sql.push(`UPDATE speech_credentials SET credential='${microsoft_credential}' WHERE vendor='microsoft';`);

File diff suppressed because it is too large Load Diff

View File

@@ -14,12 +14,8 @@ DROP TABLE IF EXISTS beta_invite_codes;
DROP TABLE IF EXISTS call_routes;
DROP TABLE IF EXISTS clients;
DROP TABLE IF EXISTS dns_records;
DROP TABLE IF EXISTS lcr;
DROP TABLE IF EXISTS lcr_carrier_set_entry;
DROP TABLE IF EXISTS lcr_routes;
@@ -54,12 +50,8 @@ DROP TABLE IF EXISTS signup_history;
DROP TABLE IF EXISTS smpp_addresses;
DROP TABLE IF EXISTS google_custom_voices;
DROP TABLE IF EXISTS speech_credentials;
DROP TABLE IF EXISTS system_information;
DROP TABLE IF EXISTS users;
DROP TABLE IF EXISTS smpp_gateways;
@@ -132,19 +124,6 @@ application_sid CHAR(36) NOT NULL,
PRIMARY KEY (call_route_sid)
) COMMENT='a regex-based pattern match for call routing';
CREATE TABLE clients
(
client_sid CHAR(36) NOT NULL UNIQUE ,
account_sid CHAR(36) NOT NULL,
is_active BOOLEAN NOT NULL DEFAULT 1,
username VARCHAR(64),
password VARCHAR(1024),
allow_direct_app_calling BOOLEAN NOT NULL DEFAULT 1,
allow_direct_queue_calling BOOLEAN NOT NULL DEFAULT 1,
allow_direct_user_calling BOOLEAN NOT NULL DEFAULT 1,
PRIMARY KEY (client_sid)
);
CREATE TABLE dns_records
(
dns_record_sid CHAR(36) NOT NULL UNIQUE ,
@@ -157,23 +136,11 @@ PRIMARY KEY (dns_record_sid)
CREATE TABLE lcr_routes
(
lcr_route_sid CHAR(36),
lcr_sid CHAR(36) NOT NULL,
regex VARCHAR(32) NOT NULL COMMENT 'regex-based pattern match against dialed number, used for LCR routing of PSTN calls',
description VARCHAR(1024),
priority INTEGER NOT NULL COMMENT 'lower priority routes are attempted first',
priority INTEGER NOT NULL UNIQUE COMMENT 'lower priority routes are attempted first',
PRIMARY KEY (lcr_route_sid)
) COMMENT='An ordered list of digit patterns in an LCR table. The patterns are tested in sequence until one matches';
CREATE TABLE lcr
(
lcr_sid CHAR(36) NOT NULL UNIQUE ,
name VARCHAR(64) COMMENT 'User-assigned name for this LCR table',
is_active BOOLEAN NOT NULL DEFAULT 1,
default_carrier_set_entry_sid CHAR(36) COMMENT 'default carrier/route to use when no digit match based results are found.',
service_provider_sid CHAR(36),
account_sid CHAR(36),
PRIMARY KEY (lcr_sid)
) COMMENT='An LCR (least cost routing) table that is used by a service provider or account to make decisions about routing outbound calls when multiple carriers are available.';
) COMMENT='Least cost routing table';
CREATE TABLE password_settings
(
@@ -281,10 +248,7 @@ CREATE TABLE sbc_addresses
sbc_address_sid CHAR(36) NOT NULL UNIQUE ,
ipv4 VARCHAR(255) NOT NULL,
port INTEGER NOT NULL DEFAULT 5060,
tls_port INTEGER,
wss_port INTEGER,
service_provider_sid CHAR(36),
last_updated DATETIME,
PRIMARY KEY (sbc_address_sid)
);
@@ -340,31 +304,9 @@ last_tested DATETIME,
tts_tested_ok BOOLEAN,
stt_tested_ok BOOLEAN,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
label VARCHAR(64),
PRIMARY KEY (speech_credential_sid)
);
CREATE TABLE google_custom_voices
(
google_custom_voice_sid CHAR(36) NOT NULL UNIQUE ,
speech_credential_sid CHAR(36) NOT NULL,
model VARCHAR(512) NOT NULL,
reported_usage ENUM('REPORTED_USAGE_UNSPECIFIED','REALTIME','OFFLINE') DEFAULT 'REALTIME',
name VARCHAR(64) NOT NULL,
voice_cloning_key MEDIUMTEXT,
use_voice_cloning_key BOOLEAN DEFAULT false,
PRIMARY KEY (google_custom_voice_sid)
);
CREATE TABLE system_information
(
domain_name VARCHAR(255),
sip_domain_name VARCHAR(255),
monitoring_domain_name VARCHAR(255),
private_network_cidr VARCHAR(8192),
log_level ENUM('info', 'debug') NOT NULL DEFAULT 'info'
);
CREATE TABLE users
(
user_sid CHAR(36) NOT NULL UNIQUE ,
@@ -415,7 +357,6 @@ smpp_inbound_password VARCHAR(64),
register_from_user VARCHAR(128),
register_from_domain VARCHAR(255),
register_public_ip_in_contact BOOLEAN NOT NULL DEFAULT false,
register_status VARCHAR(4096),
PRIMARY KEY (voip_carrier_sid)
) COMMENT='A Carrier or customer PBX that can send or receive calls';
@@ -444,7 +385,7 @@ PRIMARY KEY (smpp_gateway_sid)
CREATE TABLE phone_numbers
(
phone_number_sid CHAR(36) UNIQUE ,
number VARCHAR(132) NOT NULL,
number VARCHAR(132) NOT NULL UNIQUE ,
voip_carrier_sid CHAR(36),
account_sid CHAR(36),
application_sid CHAR(36),
@@ -457,15 +398,11 @@ CREATE TABLE sip_gateways
sip_gateway_sid CHAR(36),
ipv4 VARCHAR(128) NOT NULL COMMENT 'ip address or DNS name of the gateway. For gateways providing inbound calling service, ip address is required.',
netmask INTEGER NOT NULL DEFAULT 32,
port INTEGER COMMENT 'sip signaling port',
port INTEGER NOT NULL DEFAULT 5060 COMMENT 'sip signaling port',
inbound BOOLEAN NOT NULL COMMENT 'if true, whitelist this IP to allow inbound calls from the gateway',
outbound BOOLEAN NOT NULL COMMENT 'if true, include in least-cost routing when placing calls to the PSTN',
voip_carrier_sid CHAR(36) NOT NULL,
is_active BOOLEAN NOT NULL DEFAULT 1,
send_options_ping BOOLEAN NOT NULL DEFAULT 0,
use_sips_scheme BOOLEAN NOT NULL DEFAULT 0,
pad_crypto BOOLEAN NOT NULL DEFAULT 0,
protocol ENUM('udp','tcp','tls', 'tls/srtp') DEFAULT 'udp' COMMENT 'Outbound call protocol',
PRIMARY KEY (sip_gateway_sid)
) COMMENT='A whitelisted sip gateway used for origination/termination';
@@ -498,24 +435,13 @@ account_sid CHAR(36) COMMENT 'account that this application belongs to (if null,
call_hook_sid CHAR(36) COMMENT 'webhook to call for inbound calls ',
call_status_hook_sid CHAR(36) COMMENT 'webhook to call for call status events',
messaging_hook_sid CHAR(36) COMMENT 'webhook to call for inbound SMS/MMS ',
app_json TEXT,
app_json VARCHAR(16384),
speech_synthesis_vendor VARCHAR(64) NOT NULL DEFAULT 'google',
speech_synthesis_language VARCHAR(12) NOT NULL DEFAULT 'en-US',
speech_synthesis_voice VARCHAR(256),
speech_synthesis_label VARCHAR(64),
speech_synthesis_voice VARCHAR(64),
speech_recognizer_vendor VARCHAR(64) NOT NULL DEFAULT 'google',
speech_recognizer_language VARCHAR(64) NOT NULL DEFAULT 'en-US',
speech_recognizer_label VARCHAR(64),
use_for_fallback_speech BOOLEAN DEFAULT false,
fallback_speech_synthesis_vendor VARCHAR(64),
fallback_speech_synthesis_language VARCHAR(12),
fallback_speech_synthesis_voice VARCHAR(256),
fallback_speech_synthesis_label VARCHAR(64),
fallback_speech_recognizer_vendor VARCHAR(64),
fallback_speech_recognizer_language VARCHAR(64),
fallback_speech_recognizer_label VARCHAR(64),
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
record_all_calls BOOLEAN NOT NULL DEFAULT false,
PRIMARY KEY (application_sid)
) COMMENT='A defined set of behaviors to be applied to phone calls ';
@@ -553,10 +479,6 @@ subspace_client_secret VARCHAR(255),
subspace_sip_teleport_id VARCHAR(255),
subspace_sip_teleport_destinations VARCHAR(255),
siprec_hook_sid CHAR(36),
record_all_calls BOOLEAN NOT NULL DEFAULT false,
record_format VARCHAR(16) NOT NULL DEFAULT 'mp3',
bucket_credential VARCHAR(8192) COMMENT 'credential used to authenticate with storage service',
enable_debug_log BOOLEAN NOT NULL DEFAULT false,
PRIMARY KEY (account_sid)
) COMMENT='An enterprise that uses the platform for comm services';
@@ -577,20 +499,9 @@ ALTER TABLE call_routes ADD FOREIGN KEY account_sid_idxfk_3 (account_sid) REFERE
ALTER TABLE call_routes ADD FOREIGN KEY application_sid_idxfk (application_sid) REFERENCES applications (application_sid);
CREATE INDEX client_sid_idx ON clients (client_sid);
ALTER TABLE clients ADD CONSTRAINT account_sid_idxfk_13 FOREIGN KEY account_sid_idxfk_13 (account_sid) REFERENCES accounts (account_sid);
CREATE INDEX dns_record_sid_idx ON dns_records (dns_record_sid);
ALTER TABLE dns_records ADD FOREIGN KEY account_sid_idxfk_4 (account_sid) REFERENCES accounts (account_sid);
CREATE INDEX lcr_sid_idx ON lcr_routes (lcr_sid);
ALTER TABLE lcr_routes ADD FOREIGN KEY lcr_sid_idxfk (lcr_sid) REFERENCES lcr (lcr_sid);
CREATE INDEX lcr_sid_idx ON lcr (lcr_sid);
ALTER TABLE lcr ADD FOREIGN KEY default_carrier_set_entry_sid_idxfk (default_carrier_set_entry_sid) REFERENCES lcr_carrier_set_entry (lcr_carrier_set_entry_sid);
CREATE INDEX service_provider_sid_idx ON lcr (service_provider_sid);
CREATE INDEX account_sid_idx ON lcr (account_sid);
CREATE INDEX permission_sid_idx ON permissions (permission_sid);
CREATE INDEX predefined_carrier_sid_idx ON predefined_carriers (predefined_carrier_sid);
CREATE INDEX predefined_sip_gateway_sid_idx ON predefined_sip_gateways (predefined_sip_gateway_sid);
@@ -644,6 +555,8 @@ CREATE INDEX smpp_address_sid_idx ON smpp_addresses (smpp_address_sid);
CREATE INDEX service_provider_sid_idx ON smpp_addresses (service_provider_sid);
ALTER TABLE smpp_addresses ADD FOREIGN KEY service_provider_sid_idxfk_4 (service_provider_sid) REFERENCES service_providers (service_provider_sid);
CREATE UNIQUE INDEX speech_credentials_idx_1 ON speech_credentials (vendor,account_sid);
CREATE INDEX speech_credential_sid_idx ON speech_credentials (speech_credential_sid);
CREATE INDEX service_provider_sid_idx ON speech_credentials (service_provider_sid);
ALTER TABLE speech_credentials ADD FOREIGN KEY service_provider_sid_idxfk_5 (service_provider_sid) REFERENCES service_providers (service_provider_sid);
@@ -651,10 +564,6 @@ ALTER TABLE speech_credentials ADD FOREIGN KEY service_provider_sid_idxfk_5 (ser
CREATE INDEX account_sid_idx ON speech_credentials (account_sid);
ALTER TABLE speech_credentials ADD FOREIGN KEY account_sid_idxfk_8 (account_sid) REFERENCES accounts (account_sid);
CREATE INDEX google_custom_voice_sid_idx ON google_custom_voices (google_custom_voice_sid);
CREATE INDEX speech_credential_sid_idx ON google_custom_voices (speech_credential_sid);
ALTER TABLE google_custom_voices ADD FOREIGN KEY speech_credential_sid_idxfk (speech_credential_sid) REFERENCES speech_credentials (speech_credential_sid) ON DELETE CASCADE;
CREATE INDEX user_sid_idx ON users (user_sid);
CREATE INDEX email_idx ON users (email);
CREATE INDEX phone_idx ON users (phone);
@@ -684,8 +593,6 @@ CREATE INDEX smpp_gateway_sid_idx ON smpp_gateways (smpp_gateway_sid);
CREATE INDEX voip_carrier_sid_idx ON smpp_gateways (voip_carrier_sid);
ALTER TABLE smpp_gateways ADD FOREIGN KEY voip_carrier_sid_idxfk (voip_carrier_sid) REFERENCES voip_carriers (voip_carrier_sid);
CREATE UNIQUE INDEX phone_numbers_unique_idx_voip_carrier_number ON phone_numbers (number,voip_carrier_sid);
CREATE INDEX phone_number_sid_idx ON phone_numbers (phone_number_sid);
CREATE INDEX number_idx ON phone_numbers (number);
CREATE INDEX voip_carrier_sid_idx ON phone_numbers (voip_carrier_sid);
@@ -741,4 +648,4 @@ ALTER TABLE accounts ADD FOREIGN KEY device_calling_application_sid_idxfk (devic
ALTER TABLE accounts ADD FOREIGN KEY siprec_hook_sid_idxfk (siprec_hook_sid) REFERENCES applications (application_sid);
SET FOREIGN_KEY_CHECKS=1;
SET FOREIGN_KEY_CHECKS=1;

View File

@@ -3,8 +3,7 @@ const { sippUac } = require('./sipp')('test_fs');
const bent = require('bent');
const getJSON = bent('json')
const clearModule = require('clear-module');
const {provisionCallHook} = require('./utils');
const { sleepFor } = require('../lib/utils/helpers');
const {provisionCallHook} = require('./utils')
process.on('unhandledRejection', (reason, p) => {
console.log('Unhandled Rejection at: Promise', p, 'reason:', reason);
@@ -32,7 +31,6 @@ test('\'dial-phone\'', async(t) => {
{
"verb": "dial",
"callerId": from,
"callerName": "test_callerName",
"actionHook": "/actionHook",
"timeLimit": 5,
"target": [
@@ -44,11 +42,10 @@ test('\'dial-phone\'', async(t) => {
}
];
await provisionCallHook(from, verbs);
provisionCallHook(from, verbs);
// THEN
const p = sippUac('uas-dial.xml', '172.38.0.10', undefined, undefined, 2);
await sleepFor(1000);
let account_sid = '622f62e4-303a-49f2-bbe0-eb1e1714e37a';
let post = bent('http://127.0.0.1:3000/', 'POST', 'json', 201);
@@ -59,7 +56,6 @@ test('\'dial-phone\'', async(t) => {
"method": "POST",
},
"from": from,
"callerName": "Tom",
"to": {
"type": "phone",
"number": "15583084808"
@@ -86,7 +82,7 @@ test('\'dial-sip\'', async(t) => {
try {
await connect(srf);
// wait for fs connected to drachtio server.
await sleepFor(1000);
await new Promise(r => setTimeout(r, 1000));
// GIVEN
const from = "dial_sip";
let verbs = [
@@ -104,7 +100,7 @@ test('\'dial-sip\'', async(t) => {
}
];
await provisionCallHook(from, verbs);
provisionCallHook(from, verbs);
// THEN
const p = sippUac('uas-dial.xml', '172.38.0.10', undefined, undefined, 2);
@@ -171,7 +167,7 @@ test('\'dial-user\'', async(t) => {
}
];
await provisionCallHook(from, verbs);
provisionCallHook(from, verbs);
// THEN
const p = sippUac('uas-dial.xml', '172.38.0.10', undefined, undefined, 2);

View File

@@ -42,7 +42,7 @@ services:
ipv4_address: 172.38.0.7
drachtio:
image: drachtio/drachtio-server:0.8.26
image: drachtio/drachtio-server:latest
restart: always
command: drachtio --contact "sip:*;transport=udp" --mtu 4096 --address 0.0.0.0 --port 9022
ports:
@@ -57,7 +57,7 @@ services:
condition: service_healthy
freeswitch:
image: drachtio/drachtio-freeswitch-mrf:0.9.2-4
image: drachtio/drachtio-freeswitch-mrf:0.4.18
restart: always
command: freeswitch --rtp-range-start 20000 --rtp-range-end 20100
environment:
@@ -92,13 +92,3 @@ services:
networks:
fs:
ipv4_address: 172.38.0.90
squid:
image: ubuntu/squid:edge
ports:
- "3128:3128"
volumes:
- ./configuration/squid.conf:/etc/squid/squid.conf
networks:
fs:
ipv4_address: 172.38.0.91

View File

@@ -4,15 +4,6 @@ const bent = require('bent');
const getJSON = bent('json')
const clearModule = require('clear-module');
const {provisionCallHook} = require('./utils')
const {
GCP_JSON_KEY,
AWS_ACCESS_KEY_ID,
AWS_SECRET_ACCESS_KEY,
SONIOX_API_KEY,
DEEPGRAM_API_KEY,
MICROSOFT_REGION,
MICROSOFT_API_KEY,
} = require('../lib/config');
process.on('unhandledRejection', (reason, p) => {
console.log('Unhandled Rejection at: Promise', p, 'reason:', reason);
@@ -27,7 +18,7 @@ function connect(connectable) {
}
test('\'gather\' test - google', async(t) => {
if (!GCP_JSON_KEY) {
if (!process.env.GCP_JSON_KEY) {
t.pass('skipping google tests');
return t.end();
}
@@ -50,7 +41,7 @@ test('\'gather\' test - google', async(t) => {
}
];
let from = "gather_success";
await provisionCallHook(from, verbs);
provisionCallHook(from, verbs);
// THEN
await sippUac('uac-gather-account-creds-success.xml', '172.38.0.10', from);
let obj = await getJSON(`http://127.0.0.1:3100/lastRequest/${from}_actionHook`);
@@ -67,7 +58,7 @@ test('\'gather\' test - google', async(t) => {
});
test('\'gather\' test - default (google)', async(t) => {
if (!GCP_JSON_KEY) {
if (!process.env.GCP_JSON_KEY) {
t.pass('skipping google tests');
return t.end();
}
@@ -86,7 +77,7 @@ test('\'gather\' test - default (google)', async(t) => {
}
];
let from = "gather_success";
await provisionCallHook(from, verbs);
provisionCallHook(from, verbs);
// THEN
await sippUac('uac-gather-account-creds-success.xml', '172.38.0.10', from);
let obj = await getJSON(`http://127.0.0.1:3100/lastRequest/${from}_actionHook`);
@@ -102,55 +93,8 @@ test('\'gather\' test - default (google)', async(t) => {
}
});
test('\'config\' test - reset to app defaults', async(t) => {
if (!GCP_JSON_KEY) {
t.pass('skipping config tests');
return t.end();
}
clearModule.all();
const {srf, disconnect} = require('../app');
try {
await connect(srf);
// GIVEN
let verbs = [
{
"verb": "config",
"recognizer": {
"vendor": "google",
"language": "fr-FR"
},
},
{
"verb": "config",
"reset": ['recognizer'],
},
{
"verb": "gather",
"input": ["speech"],
"timeout": 10,
"actionHook": "/actionHook"
}
];
let from = "gather_success";
await provisionCallHook(from, verbs);
// THEN
await sippUac('uac-gather-account-creds-success.xml', '172.38.0.10', from);
let obj = await getJSON(`http://127.0.0.1:3100/lastRequest/${from}_actionHook`);
//console.log(JSON.stringify(obj));
t.ok(obj.body.speech.alternatives[0].transcript.toLowerCase() === 'i\'d like to speak to customer support',
'config: resets recognizer to app defaults');
disconnect();
} catch (err) {
console.log(`error received: ${err}`);
disconnect();
t.error(err);
}
});
test('\'gather\' test - microsoft', async(t) => {
if (!MICROSOFT_REGION || !MICROSOFT_API_KEY) {
if (!process.env.MICROSOFT_REGION || !process.env.MICROSOFT_API_KEY) {
t.pass('skipping microsoft tests');
return t.end();
}
@@ -173,7 +117,7 @@ test('\'gather\' test - microsoft', async(t) => {
}
];
let from = "gather_success";
await provisionCallHook(from, verbs);
provisionCallHook(from, verbs);
// THEN
await sippUac('uac-gather-account-creds-success.xml', '172.38.0.10', from);
let obj = await getJSON(`http://127.0.0.1:3100/lastRequest/${from}_actionHook`);
@@ -190,7 +134,7 @@ test('\'gather\' test - microsoft', async(t) => {
});
test('\'gather\' test - aws', async(t) => {
if (!AWS_ACCESS_KEY_ID || !AWS_SECRET_ACCESS_KEY) {
if (!process.env.AWS_ACCESS_KEY_ID || !process.env.AWS_SECRET_ACCESS_KEY) {
t.pass('skipping aws tests');
return t.end();
}
@@ -213,7 +157,7 @@ test('\'gather\' test - aws', async(t) => {
}
];
let from = "gather_success";
await provisionCallHook(from, verbs);
provisionCallHook(from, verbs);
// THEN
await sippUac('uac-gather-account-creds-success.xml', '172.38.0.10', from);
let obj = await getJSON(`http://127.0.0.1:3100/lastRequest/${from}_actionHook`);
@@ -230,7 +174,7 @@ test('\'gather\' test - aws', async(t) => {
});
test('\'gather\' test - deepgram', async(t) => {
if (!DEEPGRAM_API_KEY ) {
if (!process.env.DEEPGRAM_API_KEY ) {
t.pass('skipping deepgram tests');
return t.end();
}
@@ -248,7 +192,7 @@ test('\'gather\' test - deepgram', async(t) => {
"vendor": "deepgram",
"hints": ["customer support", "sales", "human resources", "HR"],
"deepgramOptions": {
"apiKey": DEEPGRAM_API_KEY
"apiKey": process.env.DEEPGRAM_API_KEY
}
},
"timeout": 10,
@@ -256,12 +200,12 @@ test('\'gather\' test - deepgram', async(t) => {
}
];
let from = "gather_success";
await provisionCallHook(from, verbs);
provisionCallHook(from, verbs);
// THEN
await sippUac('uac-gather-account-creds-success.xml', '172.38.0.10', from);
let obj = await getJSON(`http://127.0.0.1:3100/lastRequest/${from}_actionHook`);
//console.log(JSON.stringify(obj));
t.ok(obj.body.speech.alternatives[0].transcript.toLowerCase().includes('like to speak to customer support'),
t.ok(obj.body.speech.alternatives[0].transcript.toLowerCase().startsWith('i\'d like to speak to customer support'),
'gather: succeeds when using deepgram credentials');
disconnect();
} catch (err) {
@@ -272,7 +216,7 @@ test('\'gather\' test - deepgram', async(t) => {
});
test('\'gather\' test - soniox', async(t) => {
if (!SONIOX_API_KEY ) {
if (!process.env.SONIOX_API_KEY ) {
t.pass('skipping soniox tests');
return t.end();
}
@@ -290,7 +234,7 @@ test('\'gather\' test - soniox', async(t) => {
"vendor": "deepgram",
"hints": ["customer support", "sales", "human resources", "HR"],
"deepgramOptions": {
"apiKey": SONIOX_API_KEY
"apiKey": process.env.SONIOX_API_KEY
}
},
"timeout": 10,
@@ -298,7 +242,7 @@ test('\'gather\' test - soniox', async(t) => {
}
];
let from = "gather_success";
await provisionCallHook(from, verbs);
provisionCallHook(from, verbs);
// THEN
await sippUac('uac-gather-account-creds-success.xml', '172.38.0.10', from);
let obj = await getJSON(`http://127.0.0.1:3100/lastRequest/${from}_actionHook`);

View File

@@ -1,53 +0,0 @@
const test = require('tape');
const { sippUac } = require('./sipp')('test_fs');
const clearModule = require('clear-module');
const {provisionCallHook, provisionCustomHook} = require('./utils')
const bent = require('bent');
const getJSON = bent('json')
process.on('unhandledRejection', (reason, p) => {
console.log('Unhandled Rejection at: Promise', p, 'reason:', reason);
});
function connect(connectable) {
return new Promise((resolve, reject) => {
connectable.on('connect', () => {
return resolve();
});
});
}
test('\'hangup\' custom headers', async(t) => {
clearModule.all();
const {srf, disconnect} = require('../app');
try {
await connect(srf);
// GIVEN
const verbs = [
{
verb: 'play',
url: 'https://example.com/example.mp3'
},
{
"verb": "hangup",
"headers": {
"X-Reason" : "maximum call duration exceeded"
}
}
];
const from = 'hangup_custom_headers';
await provisionCallHook(from, verbs)
// THEN
await sippUac('uac-success-received-bye.xml', '172.38.0.10', from);
t.pass('play: succeeds when using single link');
disconnect();
} catch (err) {
console.log(`error received: ${err}`);
disconnect();
t.error(err);
}
});

View File

@@ -1,72 +0,0 @@
const test = require('tape');
const { sippUac } = require('./sipp')('test_fs');
const clearModule = require('clear-module');
const {provisionCallHook, provisionCustomHook} = require('./utils')
const bent = require('bent');
const getJSON = bent('json')
process.on('unhandledRejection', (reason, p) => {
console.log('Unhandled Rejection at: Promise', p, 'reason:', reason);
});
function connect(connectable) {
return new Promise((resolve, reject) => {
connectable.on('connect', () => {
return resolve();
});
});
}
test('\'HTTP proxy\' test Info', async(t) => {
clearModule.all();
process.env.JAMBONES_HTTP_PROXY_IP = "127.0.0.1";
process.env.JAMBONES_HTTP_PROXY_PROTOCOL = "http";
process.env.JAMBONES_HTTP_PROXY_PORT = 3128;
const {srf, disconnect} = require('../app');
try {
await connect(srf);
// GIVEN
const verbs = [
{
verb: 'config',
sipRequestWithinDialogHook: '/customHook'
},
{
verb: 'play',
url: 'silence_stream://5000',
}
];
const waitHookVerbs = [
{
verb: 'hangup'
}
];
const from = 'http_proxy_info';
await provisionCustomHook(from, waitHookVerbs)
await provisionCallHook(from, verbs);
// THEN
await sippUac('uac-success-info-received-bye.xml', '172.38.0.10', from, "16174000015");
t.pass('sip Info: success send Info');
// Make sure that sipRequestWithinDialogHook is called and success
const json = await getJSON(`http:127.0.0.1:3100/lastRequest/${from}_customHook`)
t.pass(json.body.sip_method === 'INFO', 'sipRequestWithinDialogHook contains sip_method')
t.pass(json.body.sip_body === 'hello jambonz\r\n', 'sipRequestWithinDialogHook contains sip_method')
disconnect();
} catch (err) {
console.log(`error received: ${err}`);
disconnect();
t.error(err);
} finally {
process.env.JAMBONES_HTTP_PROXY_IP = null;
process.env.JAMBONES_HTTP_PROXY_PROTOCOL = null;
process.env.JAMBONES_HTTP_PROXY_PORT = null;
}
});

View File

@@ -1,65 +0,0 @@
const test = require('tape');
const { sippUac } = require('./sipp')('test_fs');
const clearModule = require('clear-module');
const {provisionCallHook, provisionCustomHook} = require('./utils')
const bent = require('bent');
const getJSON = bent('json')
process.on('unhandledRejection', (reason, p) => {
console.log('Unhandled Rejection at: Promise', p, 'reason:', reason);
});
function connect(connectable) {
return new Promise((resolve, reject) => {
connectable.on('connect', () => {
return resolve();
});
});
}
test('\'sip Indialog\' test Info', async(t) => {
clearModule.all();
const {srf, disconnect} = require('../app');
try {
await connect(srf);
// GIVEN
const verbs = [
{
verb: 'config',
sipRequestWithinDialogHook: '/customHook'
},
{
verb: 'play',
url: 'silence_stream://5000',
}
];
const waitHookVerbs = [
{
verb: 'hangup'
}
];
const from = 'sip_indialog_info';
await provisionCustomHook(from, waitHookVerbs)
await provisionCallHook(from, verbs);
// THEN
await sippUac('uac-success-info-received-bye.xml', '172.38.0.10', from);
t.pass('sip Info: success send Info');
// Make sure that sipRequestWithinDialogHook is called and success
const json = await getJSON(`http:127.0.0.1:3100/lastRequest/${from}_customHook`)
t.pass(json.body.sip_method === 'INFO', 'sipRequestWithinDialogHook contains sip_method')
t.pass(json.body.sip_body === 'hello jambonz\r\n', 'sipRequestWithinDialogHook contains sip_method')
disconnect();
} catch (err) {
console.log(`error received: ${err}`);
disconnect();
t.error(err);
}
});

View File

@@ -14,10 +14,5 @@ require('./play-tests');
require('./sip-refer-tests');
require('./listen-tests');
require('./config-test');
require('./queue-test');
require('./in-dialog-test');
require('./hangup-test');
require('./sdp-utils-test');
require('./http-proxy-test');
require('./remove-test-db');
require('./docker_stop');

View File

@@ -35,7 +35,7 @@ test('\'listen-success\'', async(t) => {
}
];
await provisionCallHook(from, verbs);
provisionCallHook(from, verbs);
// THEN
await sippUac('uac-gather-account-creds-success-send-bye.xml', '172.38.0.10', from);
@@ -57,7 +57,7 @@ test('\'listen-success\'', async(t) => {
}
});
test.skip('\'listen-maxLength\'', async(t) => {
test('\'listen-maxLength\'', async(t) => {
clearModule.all();
const {srf, disconnect} = require('../app');
try {
@@ -69,13 +69,13 @@ test.skip('\'listen-maxLength\'', async(t) => {
{
"verb": "listen",
"url": `ws://172.38.0.60:3000/${from}`,
"mixType" : "stereo",
"mixType" : "mixed",
"timeout": 2,
"maxLength": 2
}
];
await provisionCallHook(from, verbs);
provisionCallHook(from, verbs);
// THEN
await sippUac('uac-gather-account-creds-success.xml', '172.38.0.10', from);
@@ -109,7 +109,7 @@ test('\'listen-pause-resume\'', async(t) => {
}
];
await provisionCallHook(from, verbs);
provisionCallHook(from, verbs);
// THEN
const p = sippUac('uac-gather-account-creds-success.xml', '172.38.0.10', from);

View File

@@ -33,7 +33,7 @@ test('\'play\' tests single link in plain text', async(t) => {
];
const from = 'play_single_link';
await provisionCallHook(from, verbs)
provisionCallHook(from, verbs)
// THEN
await sippUac('uac-success-received-bye.xml', '172.38.0.10', from);
@@ -62,7 +62,7 @@ test('\'play\' tests multi links in array', async(t) => {
];
const from = 'play_multi_links_in_array';
await provisionCallHook(from, verbs)
provisionCallHook(from, verbs)
// THEN
await sippUac('uac-success-received-bye.xml', '172.38.0.10', from);
@@ -100,8 +100,8 @@ test('\'play\' tests single link in conference', async(t) => {
waitHook: `/customHook`
}
];
await provisionCustomHook(from, waitHookVerbs)
await provisionCallHook(from, verbs)
provisionCustomHook(from, waitHookVerbs)
provisionCallHook(from, verbs)
// THEN
await sippUac('uac-success-send-bye.xml', '172.38.0.10', from);
@@ -141,8 +141,8 @@ test('\'play\' tests multi links in array in conference', async(t) => {
waitHook: `/customHook`
}
];
await provisionCustomHook(from, waitHookVerbs)
await provisionCallHook(from, verbs)
provisionCustomHook(from, waitHookVerbs)
provisionCallHook(from, verbs)
// THEN
await sippUac('uac-success-send-bye.xml', '172.38.0.10', from);
@@ -178,8 +178,8 @@ test('\'play\' tests with seekOffset and actionHook', async(t) => {
const waitHookVerbs = [];
const from = 'play_action_hook';
await provisionCallHook(from, verbs)
await provisionCustomHook(from, waitHookVerbs)
provisionCallHook(from, verbs)
provisionCustomHook(from, waitHookVerbs)
// THEN
await sippUac('uac-success-received-bye.xml', '172.38.0.10', from);
@@ -188,7 +188,7 @@ test('\'play\' tests with seekOffset and actionHook', async(t) => {
const seconds = parseInt(obj.body.playback_seconds);
const milliseconds = parseInt(obj.body.playback_milliseconds);
const lastOffsetPos = parseInt(obj.body.playback_last_offset_pos);
console.log({obj}, 'lastRequest');
//console.log({obj}, 'lastRequest');
t.ok(obj.body.reason === "playCompleted", "play: actionHook success received");
t.ok(seconds === 2, "playback_seconds: actionHook success received");
t.ok(milliseconds === 2048, "playback_milliseconds: actionHook success received");
@@ -218,7 +218,7 @@ test('\'play\' tests with earlymedia', async(t) => {
];
const from = 'play_early_media';
await provisionCallHook(from, verbs)
provisionCallHook(from, verbs)
// THEN
await sippUac('uac-invite-expect-183-cancel.xml', '172.38.0.10', from);

View File

@@ -1,126 +0,0 @@
const test = require('tape');
const { sippUac } = require('./sipp')('test_fs');
const clearModule = require('clear-module');
const {provisionCallHook, provisionActionHook, provisionAnyHook} = require('./utils');
const bent = require('bent');
const { sleepFor } = require('../lib/utils/helpers');
const getJSON = bent('json');
process.on('unhandledRejection', (reason, p) => {
console.log('Unhandled Rejection at: Promise', p, 'reason:', reason);
});
function connect(connectable) {
return new Promise((resolve, reject) => {
connectable.on('connect', () => {
return resolve();
});
});
}
test('\'enqueue-dequeue\' tests', async(t) => {
clearModule.all();
const {srf, disconnect} = require('../app');
try {
await connect(srf);
// GIVEN
const verbs = [
{
verb: 'enqueue',
name: 'support',
actionHook: '/actionHook'
}
];
const verbs2 = [
{
verb: 'dequeue',
name: 'support'
}
];
const actionVerbs = [
{
verb: 'play',
url: 'silence_stream://1000',
earlyMedia: true
}
];
const from = 'enqueue_success';
await provisionCallHook(from, verbs);
await provisionActionHook(from, actionVerbs)
const from2 = 'dequeue_success';
await provisionCallHook(from2, verbs2);
// THEN
const p1 = sippUac('uac-success-received-bye.xml', '172.38.0.10', from);
await sleepFor(1000);
const p2 = sippUac('uac-success-send-bye.xml', '172.38.0.11', from2);
await Promise.all([p1, p2]);
const obj = await getJSON(`http:127.0.0.1:3100/lastRequest/${from}_actionHook`);
t.ok(obj.body.queue_result === 'bridged');
t.pass('enqueue-dequeue: succeeds connect');
disconnect();
} catch (err) {
console.log(`error received: ${err}`);
disconnect();
t.error(err);
}
});
test('\leave\' tests', async(t) => {
clearModule.all();
const {srf, disconnect} = require('../app');
try {
await connect(srf);
// GIVEN
const verbs = [
{
verb: 'enqueue',
name: 'support1',
waitHook: '/anyHook/enqueue_success_leave',
actionHook: '/actionHook'
}
];
const anyHookVerbs = [
{
verb: 'leave'
}
];
const actionVerbs = [
{
verb: 'play',
url: 'silence_stream://1000',
earlyMedia: true
}
];
const from = 'enqueue_success_leave';
await provisionCallHook(from, verbs);
await provisionAnyHook(from, anyHookVerbs);
await provisionActionHook(from, actionVerbs)
// THEN
await sippUac('uac-success-received-bye.xml', '172.38.0.10', from);
const obj = await getJSON(`http:127.0.0.1:3100/lastRequest/enqueue_success_leave`);
t.ok(obj.body.queue_position === 0);
const obj1 = await getJSON(`http:127.0.0.1:3100/lastRequest/${from}_actionHook`);
t.ok(obj1.body.queue_result === 'leave');
t.pass('enqueue-dequeue: succeeds connect');
disconnect();
} catch (err) {
console.log(`error received: ${err}`);
disconnect();
t.error(err);
}
});

View File

@@ -31,7 +31,7 @@ test('\'say\' tests', async(t) => {
];
const from = 'say_test_success';
await provisionCallHook(from, verbs)
provisionCallHook(from, verbs)
// THEN
await sippUac('uac-success-received-bye.xml', '172.38.0.10', from);
@@ -43,124 +43,3 @@ test('\'say\' tests', async(t) => {
t.error(err);
}
});
test('\'config\' reset synthesizer tests', async(t) => {
clearModule.all();
const {srf, disconnect} = require('../app');
try {
await connect(srf);
// GIVEN
const verbs = [
{
"verb": "config",
"synthesizer": {
"vendor": "microsft",
"voice": "foobar"
},
},
{
"verb": "config",
"reset": 'synthesizer',
},
{
verb: 'say',
text: 'hello'
}
];
const from = 'say_test_success';
await provisionCallHook(from, verbs)
// THEN
await sippUac('uac-success-received-bye.xml', '172.38.0.10', from);
t.pass('say: succeeds when using using account credentials');
disconnect();
} catch (err) {
console.log(`error received: ${err}`);
disconnect();
t.error(err);
}
});
test('Say verb array test', async(t) => {
clearModule.all();
const {srf, disconnect} = require('../app');
try {
await connect(srf);
// GIVEN
const verbs = [
{
"verb": "config",
"synthesizer": {
"vendor": "microsft",
"voice": "foobar"
},
},
{
"verb": "config",
"reset": 'synthesizer',
},
{
verb: 'say',
text: ['hello', 'https://samplelib.com/lib/preview/mp3/sample-3s.mp3']
}
];
const from = 'say_test_success';
await provisionCallHook(from, verbs)
// THEN
await sippUac('uac-success-received-bye.xml', '172.38.0.10', from);
t.pass('say: succeeds when using using account credentials');
disconnect();
} catch (err) {
console.log(`error received: ${err}`);
disconnect();
t.error(err);
}
});
const {MICROSOFT_CUSTOM_API_KEY, MICROSOFT_DEPLOYMENT_ID, MICROSOFT_CUSTOM_REGION, MICROSOFT_CUSTOM_VOICE} = process.env;
if (MICROSOFT_CUSTOM_API_KEY && MICROSOFT_DEPLOYMENT_ID && MICROSOFT_CUSTOM_REGION && MICROSOFT_CUSTOM_VOICE) {
test('\'say\' tests - microsoft custom voice', async(t) => {
clearModule.all();
const {srf, disconnect} = require('../app');
try {
await connect(srf);
// GIVEN
const verbs = [
{
verb: 'say',
text: 'hello',
synthesizer: {
vendor: 'microsoft',
voice: MICROSOFT_CUSTOM_VOICE,
options: {
deploymentId: MICROSOFT_DEPLOYMENT_ID,
apiKey: MICROSOFT_CUSTOM_API_KEY,
region: MICROSOFT_CUSTOM_REGION,
}
}
}
];
const from = 'say_test_success';
await provisionCallHook(from, verbs)
// THEN
await sippUac('uac-success-received-bye.xml', '172.38.0.10', from);
t.pass('say: succeeds when using microsoft custom voice');
disconnect();
} catch (err) {
console.log(`error received: ${err}`);
disconnect();
t.error(err);
}
});
}

View File

@@ -1,114 +0,0 @@
<?xml version="1.0" encoding="ISO-8859-1" ?>
<!DOCTYPE scenario SYSTEM "sipp.dtd">
<scenario name="Basic Sipstone UAC">
<!-- In client mode (sipp placing calls), the Call-ID MUST be -->
<!-- generated by sipp. To do so, use [call_id] keyword. -->
<send retrans="500">
<![CDATA[
INVITE sip:[to]@[remote_ip]:[remote_port] SIP/2.0
Via: SIP/2.0/[transport] [local_ip]:[local_port];branch=[branch]
From: [from] <sip:[from]@[local_ip]:[local_port]>;tag=[pid]SIPpTag00[call_number]
To: <sip:[to]@[remote_ip]:[remote_port]>
Call-ID: [call_id]
CSeq: 1 INVITE
Contact: sip:[from]@[local_ip]:[local_port]
Max-Forwards: 70
X-Account-Sid: bb845d4b-83a9-4cde-a6e9-50f3743bab3f
Subject: uac-say
Content-Type: application/sdp
Content-Length: [len]
v=0
o=user1 53655765 2353687637 IN IP[local_ip_type] [local_ip]
s=-
c=IN IP[media_ip_type] [media_ip]
t=0 0
m=audio [media_port] RTP/AVP 0
a=rtpmap:0 PCMU/8000
]]>
</send>
<recv response="100"
optional="true">
</recv>
<recv response="180" optional="true">
</recv>
<recv response="183" optional="true">
</recv>
<!-- By adding rrs="true" (Record Route Sets), the route sets -->
<!-- are saved and used for following messages sent. Useful to test -->
<!-- against stateful SIP proxies/B2BUAs. -->
<recv response="200" rtd="true">
</recv>
<!-- Packet lost can be simulated in any send/recv message by -->
<!-- by adding the 'lost = "10"'. Value can be [1-100] percent. -->
<send>
<![CDATA[
ACK sip:[to]@[remote_ip]:[remote_port] SIP/2.0
Via: SIP/2.0/[transport] [local_ip]:[local_port];branch=[branch]
From: [from] <sip:[from]@[local_ip]:[local_port]>;tag=[pid]SIPpTag00[call_number]
To: [to] <sip:[to]@[remote_ip]:[remote_port]>[peer_tag_param]
Call-ID: [call_id]
CSeq: 1 ACK
Contact: sip:[from]@[local_ip]:[local_port]
Max-Forwards: 70
Subject: uac-say
Content-Length: 0
]]>
</send>
<pause milliseconds="2000"/>
<!-- Send an INFO message -->
<send>
<![CDATA[
INFO sip:[service]@[remote_ip]:[remote_port] SIP/2.0
Via: SIP/2.0/[transport] [local_ip]:[local_port];branch=[branch]
From: [from] <sip:[from]@[local_ip]:[local_port]>;tag=[pid]SIPpTag00[call_number]
To: [to] <sip:[to]@[remote_ip]:[remote_port]>[peer_tag_param]
Call-ID: [call_id]
CSeq: 2 INFO
Contact: sip:[from]@[local_ip]:[local_port]
Max-Forwards: 70
Subject: Performance Test
Content-Type: text/plain
Content-Length: [len]
hello jambonz
]]>
</send>
<!-- Receive 200 OK -->
<recv response="200">
</recv>
<recv request="BYE">
</recv>
<send>
<![CDATA[
SIP/2.0 200 OK
[last_Via:]
[last_From:]
[last_To:]
[last_Call-ID:]
[last_CSeq:]
Contact: <sip:[local_ip]:[local_port];transport=[transport]>
Content-Length: 0
]]>
</send>
</scenario>

Some files were not shown because too many files have changed in this diff Show More