add sip_reason_header to status Callback- #178 (#1579)

* feat: surface the SIP Reason header on call status events

Carriers fronting ISDN/E1 PRI trunks put the authoritative disconnect cause in
an RFC 3326 Reason header rather than in the SIP status line, e.g.

    SIP/2.0 408 Request Timeout
    Reason: Q.850 ;cause=18

Q.850 cause 18 is "no user responding" - nobody answered, not a platform fault.
Different causes also arrive under the same SIP status (503 with cause=38
network out of order, or cause=41 temporary failure), so the status code alone
cannot classify the outcome of the call.

drachtio relays the header intact and it is present on the response object, but
the feature server only read the status line from it, so the cause was lost at
the application boundary and never reached the call status webhook.

Rather than extract the header at each emit site, carry the SIP message that
caused the status change on the callStatusChange event and derive from it in one
place, so provisional responses, the 200, final failures, BYE and CANCEL are all
covered by the same code and future headers cost one line.

Note this partly overlaps the existing _extractCustomHeaders/sip_headers
passthrough: that already exposes a Reason header arriving on a BYE, but it only
runs on the hangup path, so nothing covered the outbound INVITE failure
responses where a Q.850 cause matters most.  sip_reason_header is a dedicated,
documented field that behaves the same on every status event.

sip_reason keeps meaning the status line phrase, and no key is added when there
is no Reason header, so existing consumers see an unchanged payload.

Adds a test:unit script so the smoke test runs without the docker testbed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: note that the Reason header arrives re-serialized, not verbatim

Verified live end-to-end against a cluster, capturing both the external leg and
the leg into the feature server:

    14.226.234.142 -> 10.0.197.31:5060   Reason: Q.850 ;cause=31   (as sent)
    10.0.197.31:5060 -> :5070            Reason: Q.850;cause=31    (to fs)

Proxying re-serializes the header, normalizing the optional whitespace RFC 3326
permits around ';'. The same normalization appears in customer captures from an
unrelated deployment, so this is drachtio behaviour, not cluster-specific.

sip_reason_header is therefore the header as this process received it, not the
carrier's exact bytes - worth stating outright, since the spacing inconsistency
is exactly what consumers ask about, and a consumer who string-matches on the
spaced form would silently never match.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: clear a stale Reason header in redis, pass it on the alloc path, run the tests in CI

Three problems found reviewing the earlier commits.

1. The redis call record kept a stale header forever. updateCallStatus assigns
   unconditionally, but toJSON only emits truthy values, and the same object is
   written to the redis hash with hmset - a MERGE. An absent key therefore left
   the previous status change's header in place, readable via GET /Calls/:sid:
   a leg that got 183 + "Reason: Q.850;cause=31", then answered on a clean 200
   and completed on a plain BYE, reported a Q.850 temporary-failure cause for a
   call that ended normally. The webhooks were right; only the call record was
   wrong. Storing absence as '' overwrites it. The existing "does not linger"
   test passed throughout because it only exercised toJSON in memory, so this
   adds one that pins what survives the redis filter (verified: it fails
   without the fix).

2. The endpoint-allocation failure path already extracted the Reason header and
   relayed it to the SBC, but called _notifyCallStatusChange without it - so a
   FreeSWITCH 488 with "Reason: Q.850;cause=88 INCOMPATIBLE_DESTINATION" told
   the SBC the cause and the application nothing, which is exactly the case
   this feature exists to expose. That error is an fsmrf object rather than a
   SipMessage, so it cannot go through msg (the guard in
   reasonHeaderFromSipMessage would silently return undefined); the event now
   takes an explicit sipReasonHeader for callers holding the value already.

3. The test:unit script added with these tests was never wired into CI - the
   workflow runs jslint and npm test, and npm test enumerates its files
   explicitly and skips test/unit entirely, so the suite would have rotted
   unnoticed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: clear the stale header on the CallSession path too, and keep the CANCEL

Both from PR review.

1. The previous fix only worked for SingleDialer. Storing '' on the instance is
   enough for callers that write the instance itself, but CallSession writes
   toJSON(), and toJSON() drops falsy values on purpose so the key stays out of
   the webhook payload - so '' never reached hmset and the stale header
   survived. Reproduced:

       after 183, toJSON has: Q.850;cause=31
       raw instance value:    ""
       instance   -> redis has key: true    (SingleDialer, clears)
       toJSON()   -> redis has key: false   (CallSession, stale persists)

   The split is by session class, not call direction: place-outdial is required
   only by dial.js, so SingleDialer covers dial-verb child legs while inbound,
   REST-created and adulting all inherit CallSession._notifyCallStatusChange -
   including the REST outdial path this feature was written for.

   Adds CallInfo.toRedisJSON(), a named projection for the merge-semantics
   store, so the divergence from toJSON() lives in one documented place rather
   than being rediscovered at each call site. The unit test asserted against the
   raw instance, which is why it passed while the real path was broken; it now
   goes through both writers and fails if either regresses.

2. The caller-abandoned race dropped the CANCEL. middleware.js had it in hand
   and discarded it, so the constructor's _onCancel() passed nothing whenever
   the CANCEL beat the application fetch - the header landed or not depending on
   timing. Worth closing because the 487 and its 'Request Terminated' phrase are
   both ours, so every abandoned inbound call looks identical: a
   Reason: SIP;cause=200;text="Call completed elsewhere" is what separates a
   forked branch losing the race from a caller who gave up, and today both are
   just no-answer.

   Confirmed on a deployed srf (5.0.27) that this is not a no-op:
   copyUASHeaderToUACForOnlyCancel forwards a hardcoded ['Reason', 'X-Reason']
   when proxying a CANCEL, so the header does reach us.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: keep the Reason header out of the redis call record

Reversing the earlier approach after a closer look at who reads what.

The header was reaching the redis call record simply because CallInfo feeds two
sinks with opposite semantics: the status webhook is an EVENT (each POST is
independent, an absent key means "not in this event") while the redis record is
STATE written with hmset, a MERGE (an absent key means "keep what was there").
sipReasonHeader is the first field here that can legitimately go from set back
to unset - sipStatus and sipReason are only ever overwritten - so it was the
first to expose the mismatch, and two rounds of fixes had to chase it because
the two writers project from different bases.

Rather than keep managing that, exclude it: redis holds calls that are still
live, where there is usually no interesting cause yet, and by the time there is
one the call is over. Call history is served from RecentCalls (the CDR), which
is what the webapp reads - not GET /Calls. So the field earned very little there
while costing an invariant that has to be remembered forever.

Worth being explicit that this is NOT simply a revert: dropping the '' would
only have stopped the empty value being written. When the header is PRESENT it
still reached redis through both writers - toJSON includes it, and SingleDialer
writes the instance's own properties - and was then never cleared. Keeping it out
takes the same machinery as clearing it, just inverted, so this is a choice about
where the field belongs rather than a saving.

WEBHOOK_ONLY_FIELDS names that intent in one place, with the reasoning, so the
next field with the same property has somewhere obvious to go. The test asserts
both writers, since they project from different bases and checking one would
pass while the other still wrote the field (verified: it fails if the exclusion
is removed).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: apply the redis projection at the boundary, and drop dead adulting plumbing

From PR review.

1. There was a THIRD call-record writer. Asking each call site to remember the
   projection was the wrong shape, and review found the proof: this class writes
   the record from the status change, the recording flag AND (private only)
   _persistConferenceState, and the last one still passed toJSON() straight
   through. A conferenced caller whose BYE carried Reason: Q.850;cause=16 would
   have that written into the call hash on the next conference-state persist,
   where hmset merges and nothing can ever clear it.

   Fixed by wrapping updateCallStatus once where it is bound, so every write is
   projected and a newly added writer cannot bypass it by forgetting to ask. The
   call sites go back to passing plain shapes. This is a class of miss that a
   unit test cannot catch - the contract test passed the whole time - so the fix
   is structural rather than another assertion.

2. The msg/byeReq parameters added to AdultingCallSession were dead code. The
   only path that reacts to the far-end BYE there is the inline
   sd.dlg.on('destroy') handler, which calls _callReleased() and discards the
   request; _hangup is reachable only from CallSession.hangup() (LCC), which
   passes nothing. The Reason header on that leg does reach the webhook, via
   SingleDialer's own destroy handler - so the plumbing was not just unused but
   misleading about which path carries it. Removed, with a comment at the handler
   recording where the event actually comes from so it does not get re-added.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: correct the projection comment in SingleDialer

Copied verbatim from CallSession, where the list of writers (status change,
recording flag, conference state) is accurate. SingleDialer has one writer, so
the comment described code that is not there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

.github/workflows/build.yml

@@ -12,6 +12,7 @@ jobs:
           node-version: 20
       - run: npm ci
       - run: npm run jslint
+      - run: npm run test:unit
       - name: Install Docker Compose
         run: |
           sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

lib/http-routes/api/create-call.js

@@ -339,7 +339,7 @@ router.post('/',
             cs.callInfo.sbcCallid = prov.get('X-CID');
             if ([180, 183].includes(prov.status) && prov.body) connectStream(prov.body);
             restDial.emit('callStatus', prov.status, !!prov.body);
-            cs.emit('callStatusChange', {callStatus, sipStatus: prov.status});
+            cs.emit('callStatusChange', {callStatus, sipStatus: prov.status, msg: prov});
           }
         });
         connectStream(dlg.remote.sdp);
@@ -352,7 +352,8 @@ router.post('/',
         cs.emit('callStatusChange', {
           callStatus: CallStatus.InProgress,
           sipStatus: 200,
-          sipReason: 'OK'
+          sipReason: 'OK',
+          msg: dlg.res
         });
         restDial.emit('callStatus', 200);
         restDial.emit('connect', dlg);
@@ -367,7 +368,8 @@ router.post('/',
           if (cs) cs.emit('callStatusChange', {
             callStatus,
             sipStatus: err.status,
-            sipReason: err.reason
+            sipReason: err.reason,
+            msg: err.res
           });
           cs.callGone = true;
         }

lib/middleware.js

@@ -120,6 +120,11 @@ module.exports = function(srf, logger) {
     req.once('cancel', (sipMsg) => {
       logger.info(`${callId} got CANCEL request`);
       req.locals.canceled = true;
+      /* keep the CANCEL: it may carry an RFC 3326 Reason (e.g. SIP;cause=200
+         ;text="Call completed elsewhere"), which is the only thing distinguishing a
+         forked branch losing the race from a caller who simply gave up - the 487 and
+         its reason phrase are ours, so both look identical without it */
+      req.locals.cancelReq = sipMsg;
     });
     next();
   }

lib/session/adulting-call-session.js

@@ -23,6 +23,10 @@ class AdultingCallSession extends CallSession {
     this.sd = singleDialer;
     this.req = callInfo.req;
 
+    /* The BYE is deliberately not threaded on from here: the Completed status event for
+       this leg - and with it any Reason header on the BYE - is emitted by SingleDialer's
+       own 'destroy' handler, which already carries the request. Emitting it here too would
+       double-notify. */
     this.sd.dlg.on('destroy', () => {
       this.logger.info('AdultingCallSession: called party hung up');
       this._callReleased();

lib/session/call-info.js

@@ -2,6 +2,21 @@ const {CallDirection, CallStatus} = require('../utils/constants');
 const parseUri = require('drachtio-srf').parseUri;
 const crypto = require('crypto');
 const {JAMBONES_API_BASE_URL} = require('../config');
+/**
+ * Fields that belong on the status webhook but must never enter the redis call record.
+ *
+ * That record is written with hmset, which MERGES, so a field able to go from set back to
+ * unset - as sipReasonHeader does, unlike sipStatus/sipReason which are only ever
+ * overwritten - would strand a value from an earlier status change where GET /Calls/:sid
+ * reports it: a leg that saw "183 + Reason: Q.850;cause=31" then answered cleanly and ended
+ * on a plain BYE would report a temporary-failure cause for a call that completed normally.
+ *
+ * Excluding it removes that problem instead of managing it.  Redis holds calls that are
+ * still live, where there is usually no interesting cause yet; by the time there is one the
+ * call is over and the history belongs in the CDR.  Add any future webhook-only field here.
+ */
+const WEBHOOK_ONLY_FIELDS = ['sipReasonHeader'];
+
 /**
  * @classdesc Represents the common information for all calls
  * that is provided in call status webhooks
@@ -105,11 +120,26 @@ class CallInfo {
    * update the status of the call
    * @param {string} callStatus - current call status
    * @param {number} sipStatus - current sip status
+   * @param {string} [sipReason] - reason phrase from the SIP status line
+   * @param {string} [sipReasonHeader] - RFC 3326 Reason header of the SIP message that caused
+   * this status change, if it carried one.  Unlike the fields above this is assigned
+   * unconditionally, so that it always describes the current change rather than lingering
+   * from an earlier one.
+   *
+   * This is a webhook-only field, deliberately kept OUT of the redis call record: that
+   * record is written with hmset, which MERGES, so a field that can legitimately go from
+   * set back to unset - as this one does, unlike sipStatus/sipReason - would strand a cause
+   * from an earlier status change where GET /Calls/:sid reports it.  Keeping it undefined
+   * when absent means realtimedb-helpers filters it out and it is never written at all,
+   * which removes the problem rather than managing it.  Redis holds calls that are still
+   * live, where there is usually no interesting cause yet; by the time there is one the
+   * call is over and the history belongs in the CDR.
    */
-  updateCallStatus(callStatus, sipStatus, sipReason) {
+  updateCallStatus(callStatus, sipStatus, sipReason, sipReasonHeader) {
     this.callStatus = callStatus;
     if (sipStatus) this.sipStatus = sipStatus;
     if (sipReason) this.sipReason = sipReason;
+    this.sipReasonHeader = sipReasonHeader;
   }
 
   /**
@@ -132,6 +162,17 @@ class CallInfo {
     return this._sipHeaders;
   }
 
+  /**
+   * Project a call-status shape into what may be written to the redis call record.
+   * Both writers go through this, from different bases: CallSession writes the webhook
+   * payload, SingleDialer writes the CallInfo instance itself.
+   */
+  static toRedisRecord(obj) {
+    const record = Object.assign({}, obj);
+    WEBHOOK_ONLY_FIELDS.forEach((f) => delete record[f]);
+    return record;
+  }
+
   toJSON() {
     const obj = {
       callSid: this.callSid,
@@ -149,7 +190,8 @@ class CallInfo {
       applicationSid: this.applicationSid,
       fsSipAddress: this.localSipAddress
     };
-    ['parentCallSid', 'originatingSipIp', 'originatingSipTrunkName', 'callTerminationBy'].forEach((prop) => {
+    ['parentCallSid', 'originatingSipIp', 'originatingSipTrunkName', 'callTerminationBy',
+      'sipReasonHeader'].forEach((prop) => {
       if (this[prop]) obj[prop] = this[prop];
     });
     if (typeof this.duration === 'number') obj.duration = this.duration;

lib/session/call-session.js

@@ -41,6 +41,8 @@ const { NonFatalTaskError} = require('../utils/error');
 const { createMediaEndpoint } = require('../utils/media-endpoint');
 const { isOnhold } = require('../utils/sdp-utils');
 const SttLatencyCalculator = require('../utils/stt-latency-calculator');
+const {reasonHeaderFromSipMessage} = require('../utils/sip-reason');
+const CallInfo = require('./call-info');
 const sqlRetrieveQueueEventHook = `SELECT * FROM webhooks 
 WHERE webhook_sid = 
 (
@@ -109,7 +111,12 @@ class CallSession extends Emitter {
     this.tmpFiles = new Set();
 
     if (!this.isSmsCallSession) {
-      this.updateCallStatus = srf.locals.dbHelpers.updateCallStatus;
+      /* Route every call-record write through the redis projection here rather than at each
+         call site: this class writes the record from more than one place (status change,
+         recording flag, conference state) and a new one must not be able to leak a
+         webhook-only field into a store with merge semantics by forgetting to ask. */
+      const {updateCallStatus} = srf.locals.dbHelpers;
+      this.updateCallStatus = (obj, serviceUrl) => updateCallStatus(CallInfo.toRedisRecord(obj), serviceUrl);
       this.serviceUrl = srf.locals.serviceUrl;
     }
 
@@ -2541,7 +2548,8 @@ Duration=${duration} `
             this._notifyCallStatusChange({
               callStatus: CallStatus.Failed,
               sipStatus: err.status,
-              sipReason: err.reason || 'Endpoint Allocation Failed'
+              sipReason: err.reason || 'Endpoint Allocation Failed',
+              sipReasonHeader
             });
             this._callReleased();
           }
@@ -3105,7 +3113,7 @@ Duration=${duration} `
    * @param {number} sipStatus - current sip status
    * @param {number} [duration] - duration of a completed call, in seconds
    */
-  async _notifyCallStatusChange({callStatus, sipStatus, sipReason, duration, headers}) {
+  async _notifyCallStatusChange({callStatus, sipStatus, sipReason, duration, headers, msg, sipReasonHeader}) {
     if (this.callMoved) return;
 
     // manage record all call.
@@ -3127,7 +3135,10 @@ Duration=${duration} `
       (!duration && callStatus !== CallStatus.Completed),
     'duration MUST be supplied when call completed AND ONLY when call completed');
 
-    this.callInfo.updateCallStatus(callStatus, sipStatus, sipReason);
+    /* a caller that already extracted the header (endpoint allocation failure, where the
+       error is an fsmrf object rather than a SipMessage) passes it directly */
+    this.callInfo.updateCallStatus(callStatus, sipStatus, sipReason,
+      sipReasonHeader ?? reasonHeaderFromSipMessage(msg));
     if (typeof duration === 'number') this.callInfo.duration = duration;
     if (headers) this.callInfo.sipHeaders = headers;
     this.executeStatusCallback(callStatus, sipStatus);

lib/session/inbound-call-session.js

@@ -25,7 +25,10 @@ class InboundCallSession extends CallSession {
     // if the call was canceled before we got here, handle it
     if (this.req.locals.canceled) {
       req.locals.logger.info('InboundCallSession: constructor - call was already canceled');
-      this._onCancel();
+      /* the CANCEL landed before we got here, so it came in via middleware rather than
+         the listener below; without this the header would survive only when the CANCEL
+         lost the race with the application fetch */
+      this._onCancel(req.locals.cancelReq);
     }
 
     req.once('cancel', this._onCancel.bind(this));
@@ -38,13 +41,14 @@ class InboundCallSession extends CallSession {
     });
   }
 
-  _onCancel() {
+  _onCancel(cancelReq) {
     this.rootSpan.setAttributes({'call.termination': 'caller abandoned'});
     this.callInfo.callTerminationBy = 'caller';
     this._notifyCallStatusChange({
       callStatus: CallStatus.NoAnswer,
       sipStatus: 487,
-      sipReason: 'Request Terminated'
+      sipReason: 'Request Terminated',
+      msg: cancelReq
     });
     this._callReleased();
   }
@@ -113,6 +117,7 @@ class InboundCallSession extends CallSession {
     this.emit('callStatusChange', {
       callStatus: CallStatus.Completed,
       duration,
+      msg: req,
       ...(headers && {headers})
     });
     this._callReleased();

lib/session/rest-call-session.js

@@ -66,7 +66,7 @@ class RestCallSession extends CallSession {
     this.callInfo.callTerminationBy = terminatedBy;
     const duration = moment().diff(this.dlg.connectTime, 'seconds');
     const headers = this._extractCustomHeaders(req);
-    this.emit('callStatusChange', {callStatus: CallStatus.Completed, duration, ...(headers && {headers})});
+    this.emit('callStatusChange', {callStatus: CallStatus.Completed, duration, msg: req, ...(headers && {headers})});
     this.logger.info(`RestCallSession: called party hung up by ${terminatedBy}`);
     this._callReleased();
   }

lib/utils/place-outdial.js

@@ -17,6 +17,7 @@ const HttpRequestor = require('./http-requestor');
 const WsRequestor = require('./ws-requestor');
 const {makeOpusFirst, removeVideoSdp} = require('./sdp-utils');
 const { createMediaEndpoint } = require('./media-endpoint');
+const {reasonHeaderFromSipMessage} = require('./sip-reason');
 
 class SingleDialer extends Emitter {
   constructor({logger, sbcAddress, target, opts, application, callInfo, accountInfo, rootSpan, startSpan, dialTask,
@@ -136,7 +137,12 @@ class SingleDialer extends Emitter {
           assert(false, `invalid dial type ${this.target.type}: must be phone, user, or sip`);
       }
 
-      this.updateCallStatus = srf.locals.dbHelpers.updateCallStatus;
+      /* Route the call-record write through the redis projection here rather than at the
+         call site, matching CallSession: this class has only one writer today, but the
+         redis record has merge semantics and a second one must not be able to leak a
+         webhook-only field into it by forgetting to ask. */
+      const {updateCallStatus} = srf.locals.dbHelpers;
+      this.updateCallStatus = (obj, serviceUrl) => updateCallStatus(CallInfo.toRedisRecord(obj), serviceUrl);
       this.serviceUrl = srf.locals.serviceUrl;
 
       this.ep = await this._createMediaEndpoint();
@@ -221,7 +227,7 @@ class SingleDialer extends Emitter {
           });
         },
         cbProvisional: (prov) => {
-          const status = {sipStatus: prov.status, sipReason: prov.reason};
+          const status = {sipStatus: prov.status, sipReason: prov.reason, msg: prov};
           // Update call-id for sbc outbound INVITE
           this.callInfo.sbcCallid = prov.get('X-CID');
           if ([180, 183].includes(prov.status) && prov.body) {
@@ -246,7 +252,8 @@ class SingleDialer extends Emitter {
       this.emit('callStatusChange', {
         sipStatus: 200,
         sipReason: 'OK',
-        callStatus: CallStatus.InProgress
+        callStatus: CallStatus.InProgress,
+        msg: this.dlg.res
       });
       this.logger.debug(`SingleDialer:exec call connected: ${this.callSid}`);
       const connectTime = this.dlg.connectTime = moment();
@@ -273,7 +280,9 @@ class SingleDialer extends Emitter {
           const duration = moment().diff(connectTime, 'seconds');
           const headers = this._extractCustomHeaders(req);
           this.logger.debug('SingleDialer:exec called party hung up');
-          this.emit('callStatusChange', {callStatus: CallStatus.Completed, duration, ...(headers && {headers})});
+          this.emit('callStatusChange', {
+            callStatus: CallStatus.Completed, duration, msg: req, ...(headers && {headers})
+          });
           this.ep && this.ep.destroy();
         })
         .on('refresh', () => this.logger.info('SingleDialer:exec - dialog refreshed by uas'))
@@ -315,6 +324,7 @@ class SingleDialer extends Emitter {
       if (err instanceof SipError) {
         status.sipStatus = err.status;
         status.sipReason = err.reason;
+        status.msg = err.res;
         if (err.status === 487) status.callStatus = CallStatus.NoAnswer;
         else if ([486, 600].includes(err.status)) status.callStatus = CallStatus.Busy;
         this.logger.info(`SingleDialer:exec outdial failure ${err.status}`);
@@ -547,13 +557,14 @@ class SingleDialer extends Emitter {
     return Object.keys(headers).length ? headers : null;
   }
 
-  _notifyCallStatusChange({callStatus, sipStatus, sipReason, duration, headers}) {
+  _notifyCallStatusChange({callStatus, sipStatus, sipReason, duration, headers, msg, sipReasonHeader}) {
     assert((typeof duration === 'number' && callStatus === CallStatus.Completed) ||
       (!duration && callStatus !== CallStatus.Completed),
     'duration MUST be supplied when call completed AND ONLY when call completed');
 
     if (this.callInfo) {
-      this.callInfo.updateCallStatus(callStatus, sipStatus, sipReason);
+      this.callInfo.updateCallStatus(callStatus, sipStatus, sipReason,
+        sipReasonHeader ?? reasonHeaderFromSipMessage(msg));
       if (typeof duration === 'number') this.callInfo.duration = duration;
       if (headers) this.callInfo.sipHeaders = headers;
       try {
@@ -562,7 +573,8 @@ class SingleDialer extends Emitter {
         this.logger.info(err, `SingleDialer:_notifyCallStatusChange error sending ${callStatus} ${sipStatus}`);
       }
       // update calls db
-      this.updateCallStatus(this.callInfo, this.serviceUrl).catch((err) => this.logger.error(err, 'redis error'));
+      this.updateCallStatus(this.callInfo, this.serviceUrl)
+        .catch((err) => this.logger.error(err, 'redis error'));
     }
     else {
       this.logger.info('SingleDialer:_notifyCallStatusChange: call status change before sending the outbound INVITE!!');

lib/utils/sip-reason.js

@@ -0,0 +1,41 @@
+/**
+ * RFC 3326 Reason header support.
+ *
+ * Carriers fronting ISDN/E1 PRI trunks put the authoritative disconnect cause in
+ * a Reason header rather than in the SIP status line, e.g.
+ *
+ *    SIP/2.0 408 Request Timeout
+ *    Reason: Q.850 ;cause=18
+ *
+ * (Q.850 cause 18 is "no user responding" - i.e. nobody answered, not a fault.)
+ * Different Q.850 causes can arrive under the same SIP status - 503 may carry
+ * cause=38 (network out of order) or cause=41 (temporary failure) - so the status
+ * code on its own is not enough to classify the outcome of the call.  We surface
+ * the header verbatim on call status events and leave interpretation to the
+ * application.
+ */
+
+/**
+ * Return the Reason header of a SIP message, or undefined if it has none.
+ *
+ * A message may legally carry more than one Reason header (RFC 3326), and trunks
+ * that report both a SIP and a Q.850 cause commonly do.  The drachtio parser joins
+ * repeated headers into a single comma-separated string; we pass that through
+ * unchanged rather than picking one of them.
+ *
+ * What reaches us is the header as the SBC relayed it, NOT necessarily the
+ * carrier's exact bytes: proxying re-serializes the header, which normalizes the
+ * optional whitespace RFC 3326 permits around ';'.  A carrier's
+ * "Q.850 ;cause=18" therefore arrives here as "Q.850;cause=18" - confirmed on
+ * the wire (external leg vs the leg into this process) and visible in customer
+ * captures too.  Consumers should parse tolerantly rather than string-match.
+ *
+ * @param {object} [msg] - a drachtio SipMessage (request or response), if we have one
+ * @returns {string|undefined} the Reason header value, or undefined
+ */
+const reasonHeaderFromSipMessage = (msg) => {
+  if (!msg || typeof msg.get !== 'function') return;
+  return msg.get('Reason') || undefined;
+};
+
+module.exports = {reasonHeaderFromSipMessage};

package.json

@@ -20,6 +20,7 @@
   "scripts": {
     "start": "node app",
     "test": "NODE_ENV=test JAMBONES_HOSTING=1 HTTP_POOL=1 JAMBONES_TTS_TRIM_SILENCE=1 ENCRYPTION_SECRET=foobar DRACHTIO_HOST=127.0.0.1 DRACHTIO_PORT=9060 DRACHTIO_SECRET=cymru JAMBONES_MYSQL_HOST=127.0.0.1 JAMBONES_MYSQL_PORT=3360 JAMBONES_MYSQL_USER=jambones_test JAMBONES_MYSQL_PASSWORD=jambones_test JAMBONES_MYSQL_DATABASE=jambones_test JAMBONES_REDIS_HOST=127.0.0.1 JAMBONES_REDIS_PORT=16379 JAMBONES_LOGLEVEL=error ENABLE_METRICS=0 HTTP_PORT=3000 JAMBONES_SBCS=172.38.0.10 JAMBONES_FREESWITCH=127.0.0.1:8022:JambonzR0ck$:docker-host JAMBONES_TIME_SERIES_HOST=127.0.0.1 JAMBONES_NETWORK_CIDR=172.38.0.0/16 node test/ ",
+    "test:unit": "node --test test/unit/*.test.js",
     "coverage": "./node_modules/.bin/nyc --reporter html --report-dir ./coverage npm run test",
     "jslint": "eslint app.js tracer.js lib",
     "jslint:fix": "eslint app.js tracer.js lib --fix"

test/unit/sip-reason-header.test.js

@@ -0,0 +1,163 @@
+const test = require('node:test');
+const assert = require('node:assert');
+const SipMessage = require('drachtio-srf/lib/sip-parser/message');
+const CallInfo = require('../../lib/session/call-info');
+const snakeCaseKeys = require('../../lib/utils/snakecase-keys');
+const {reasonHeaderFromSipMessage} = require('../../lib/utils/sip-reason');
+const {CallDirection, CallStatus} = require('../../lib/utils/constants');
+
+/* build the outbound INVITE that CallInfo is constructed from */
+const makeReq = () => {
+  const req = new SipMessage([
+    'INVITE sip:+971555551234@example.com SIP/2.0',
+    'Call-ID: daa1269b-0b91-1240-9db3-022758ab7fff',
+    'From: <sip:+971455550000@example.com>;tag=abc123',
+    'To: <sip:+971555551234@example.com>',
+    'Content-Length: 0',
+    '', ''
+  ].join('\r\n'));
+  req.srf = {locals: {localSipAddress: '172.30.29.123:5060'}};
+  return req;
+};
+
+const makeSipMessage = (startLine, headers = []) => new SipMessage([
+  startLine,
+  'Call-ID: daa24f5d-0b91-1240-14ab-0ec7040a32ad',
+  ...headers,
+  'Content-Length: 0',
+  '', ''
+].join('\r\n'));
+
+const makeCallInfo = () => new CallInfo({
+  direction: CallDirection.Outbound,
+  req: makeReq(),
+  to: '+971555551234',
+  callSid: '9921be00-ced0-45cb-add1-e02f9ce555ab',
+  accountSid: 'e43117dc-4b91-430c-82ad-74d2725f3026',
+  applicationSid: '72c5c38f-9bba-40ce-aa83-aaa6be55e1b5',
+  traceId: '615e314ac26241863b905931d9aad440'
+});
+
+
+/* mirrors filterNullsAndObjects in realtimedb-helpers, which decides what actually
+   reaches the redis call hash via hmset */
+const redisFields = (callInfo) => Object.keys(callInfo)
+  .filter((k) => callInfo[k] !== null && typeof callInfo[k] !== 'undefined' && typeof callInfo[k] !== 'object');
+
+/* the payload a call status webhook consumer actually receives */
+const statusPayload = (callInfo) => snakeCaseKeys(callInfo.toJSON(), ['customerData', 'sip', 'env_vars', 'args']);
+
+test('Reason header on a final failure response is surfaced as sip_reason_header', () => {
+  const callInfo = makeCallInfo();
+  const res = makeSipMessage('SIP/2.0 408 Request Timeout', ['Reason: Q.850 ;cause=18']);
+
+  callInfo.updateCallStatus(CallStatus.Failed, 408, 'Request Timeout', reasonHeaderFromSipMessage(res));
+  const payload = statusPayload(callInfo);
+
+  assert.strictEqual(payload.sip_reason_header, 'Q.850 ;cause=18');
+  /* sip_reason must keep meaning the status-line phrase - existing consumers depend on it */
+  assert.strictEqual(payload.sip_reason, 'Request Timeout');
+  assert.strictEqual(payload.sip_status, 408);
+});
+
+test('spacing variants of the Reason header are passed through verbatim', () => {
+  for (const raw of ['Q.850 ;cause=31', 'Q.850;cause=31', 'Q.850 ; cause=31']) {
+    const callInfo = makeCallInfo();
+    const res = makeSipMessage('SIP/2.0 480 Temporarily Unavailable', [`Reason: ${raw}`]);
+    callInfo.updateCallStatus(CallStatus.Failed, 480, 'Temporarily Unavailable', reasonHeaderFromSipMessage(res));
+    assert.strictEqual(statusPayload(callInfo).sip_reason_header, raw);
+  }
+});
+
+test('a response with no Reason header adds no key to the payload', () => {
+  const callInfo = makeCallInfo();
+  const res = makeSipMessage('SIP/2.0 503 Service Unavailable');
+
+  callInfo.updateCallStatus(CallStatus.Failed, 503, 'Service Unavailable', reasonHeaderFromSipMessage(res));
+  const payload = statusPayload(callInfo);
+
+  assert.ok(!('sip_reason_header' in payload), 'payload must be unchanged for carriers that send no Reason');
+});
+
+test('repeated Reason headers are preserved rather than one being dropped', () => {
+  const callInfo = makeCallInfo();
+  const res = makeSipMessage('SIP/2.0 486 Busy Here', [
+    'Reason: SIP ;cause=486 ;text="busy"',
+    'Reason: Q.850 ;cause=17'
+  ]);
+
+  callInfo.updateCallStatus(CallStatus.Busy, 486, 'Busy Here', reasonHeaderFromSipMessage(res));
+  const header = statusPayload(callInfo).sip_reason_header;
+
+  assert.match(header, /SIP ;cause=486/);
+  assert.match(header, /Q\.850 ;cause=17/);
+});
+
+test('a Reason header on a BYE is surfaced on the completed event', () => {
+  const callInfo = makeCallInfo();
+  const bye = new SipMessage([
+    'BYE sip:+971555551234@example.com SIP/2.0',
+    'Call-ID: daa24f5d-0b91-1240-14ab-0ec7040a32ad',
+    'Reason: Q.850 ;cause=16',
+    'Content-Length: 0',
+    '', ''
+  ].join('\r\n'));
+
+  callInfo.duration = 42;
+  callInfo.updateCallStatus(CallStatus.Completed, 200, 'OK', reasonHeaderFromSipMessage(bye));
+
+  assert.strictEqual(statusPayload(callInfo).sip_reason_header, 'Q.850 ;cause=16');
+});
+
+test('a Reason header does not linger onto a later status change that has none', () => {
+  const callInfo = makeCallInfo();
+  const prov = makeSipMessage('SIP/2.0 183 Session Progress', ['Reason: Q.850 ;cause=31']);
+  const ok = makeSipMessage('SIP/2.0 200 OK');
+
+  callInfo.updateCallStatus(CallStatus.EarlyMedia, 183, 'Session Progress', reasonHeaderFromSipMessage(prov));
+  assert.strictEqual(statusPayload(callInfo).sip_reason_header, 'Q.850 ;cause=31');
+
+  callInfo.updateCallStatus(CallStatus.InProgress, 200, 'OK', reasonHeaderFromSipMessage(ok));
+  assert.ok(!('sip_reason_header' in statusPayload(callInfo)),
+    'each status event must report the Reason of the message that caused it');
+});
+
+test('status changes with no SIP message at all are handled', () => {
+  /* e.g. jambonz hanging up the call itself, or a media timeout */
+  assert.strictEqual(reasonHeaderFromSipMessage(undefined), undefined);
+  assert.strictEqual(reasonHeaderFromSipMessage(null), undefined);
+  assert.strictEqual(reasonHeaderFromSipMessage({}), undefined);
+
+  const callInfo = makeCallInfo();
+  callInfo.duration = 7;
+  callInfo.updateCallStatus(CallStatus.Completed, 200, 'OK', reasonHeaderFromSipMessage(undefined));
+  assert.ok(!('sip_reason_header' in statusPayload(callInfo)));
+});
+
+
+test('the Reason header never enters the redis call record', () => {
+  const callInfo = makeCallInfo();
+  const res = makeSipMessage('SIP/2.0 480 Temporarily Unavailable', ['Reason: Q.850 ;cause=31']);
+
+  callInfo.updateCallStatus(CallStatus.Failed, 480, 'Temporarily Unavailable', reasonHeaderFromSipMessage(res));
+
+  /* It belongs on the webhook... */
+  assert.strictEqual(statusPayload(callInfo).sip_reason_header, 'Q.850 ;cause=31');
+
+  /* ...and must be kept out of the redis call record, which is written with hmset - a
+     MERGE. A field that can go from set back to unset would otherwise strand a cause from
+     an earlier status change where GET /Calls/:sid reports it.
+     There is more than one writer and they project from DIFFERENT bases - the status
+     change and recording-flag writes send the webhook payload, SingleDialer sends the
+     CallInfo instance - so assert the projection holds for both shapes. The sessions apply
+     it by wrapping updateCallStatus at the boundary rather than at each call site, so a
+     newly added writer cannot bypass it by forgetting to ask. */
+  assert.ok(!redisFields(CallInfo.toRedisRecord(callInfo.toJSON())).includes('sipReasonHeader'),
+    'CallSession must not write sipReasonHeader to the call record');
+  assert.ok(!redisFields(CallInfo.toRedisRecord(callInfo)).includes('sipReasonHeader'),
+    'SingleDialer must not write sipReasonHeader to the call record');
+
+  /* the exclusion must not take anything else with it */
+  assert.ok(redisFields(CallInfo.toRedisRecord(callInfo.toJSON())).includes('sipReason'));
+  assert.ok(redisFields(CallInfo.toRedisRecord(callInfo.toJSON())).includes('callStatus'));
+});