Channels
Channel is a component which connects and enables exchange of verbal as well as other conversation related data between conversation-capable client software to the Flowstorm NLP pipeline.
Flowstorm provides set of channel implementation in order to support integration of various client software (web and mobile apps, phone calls, voice asisstants and even other various conversation mediating systems).
There are two basic types of channel implementations
- 1.External API implementations used to provide endpoint for existing 3rd party conversation/verbal communication systems like IVA (Intelligent Voice Assistant) platforms or IVR (Interactive Voice Response) systems, including
- 2.Flowstorm Channel API implementations leveraged by Flowstorm client apps and libraries as well as specific client implementations (e.g. web, mobile or 3D/VR apps)
Flowstorm Channel API socket implementations leverage set of events declared in platform independent module channel/api so that they can be used on both client and server side
Name | Protocol Version | Emittor | Purpose |
---|---|---|---|
InputEvent | V1, V2 | Client | Input object passed from client to server (e.g. when user is using text input OR ASR is being done on the client side). As soon as this event is emitted client should go to Processing state. |
RequestEvent | V1 only, obsolete | Client | Transport of obsolete request object from client to the server |
InputAudioStreamOpenEvent | V1 only | Client | Indicates beginning of the input audio stream |
InputStreamCloseEvent | V2 only | Client | Used to indicate end of input stream when client user switches to text input |
InputAudioStreamCloseEvent | V1 only | Client | Indicates end of the input audio stream if client has received finally recognized input OR client user has switched to the text input |
RecognizedInputEvent | V2 only | Server | Used to pass client interim or final input object recognized from input stream |
RecognizedEvent | V1 only | Server | Used to pass client interim or final input text recognized from input audio stream |
ResponseItemEvent | V1, V2 | Server | Response item can contain text and/or audio, video or command code (payload) to be interpreted by the client immediately. |
ResponseItem | V1, V2 | Server | Indicates completion of the response (and optionally end of session). Client should go to and stay in the Responding state until it has finished all response item processing, then go to Listening state) or go to sleep (Sleeping state) depending on whether session has ended or not. |
InitEvent | V1 only | Client | Used to initiate connection to the server |
ReadyEvent | V1 only | Server | Acknowledge of initiated connection from the server |
ErrorEvent | V1, V2 | Server | Non-recoverable server error text and source name. Client should display or otherwise indicate (e.g. by prerecorded voice message) server error and end session instantly by going to the Sleeping or Failed state. |
SessionStartEvent | V1 only | Server | Acknowledge of new session start (transport of session ID; V2 socket is using cookie for that) |
SessionEndEvent | V1 only | Server | Indication of session end (client should discard session ID; obsolete, replaced with sessionEnded property of Response object transported via ResponseEvent ) |
LogEvent | V1, V2 | Client | Used to transport client's log entries to the server where they are stored to the current session |
BinaryEvent | V2 only | Client
Server potentially in the future | Used to transport block of audio/video data from client to the server |
Last modified 1yr ago