Basics of a pub/sub service

Streams is a publish/subscribe (pub/sub) messaging service where the senders of messages are decoupled from the receivers of messages.

There are several key concepts in a pub/sub service.

Concept Description
Message The data that transits through the service.
Topic An entity that represents a feed of messages.
Publisher Creates messages and send them in a topic.
Subscriber Subscribes to a topic to receive published messages.

Publisher

Streams supports one type of publisher, which polls an API and publishes the response to a topic. Once a new version of the content is published, Streams identifies the differences and pushes corresponding incremental updates to the subscribers.

Polling frequency

The frequency at which Streams will query the target API can be defined under Settings > Polling Frequency.

Streams attempts to fetch data from the target URL at this interval. This configuration only applies to newly created APIs. If you change the value, the polling frequency of existing APIs remains unchanged.

You can also define a specific polling frequency for each of your APIs by editing it in My APIs.

Query params and headers injection

Streams injects the following headers to all API requests: Forwarded, X-Forwarded & Via headers. The Accept header is forced to application/json.

You can inject additional custom query params and headers to the API request by selecting Configure under My APIs:

Subscriber

To receive messages published to a topic, your subscribing application must initiate a connection with Streams proxy server. Once established, this persistent connection enables Streams to push incremental updates as soon as a new data is published to the topic. If the connection is successfully established, Streams responds with a 200 OK and a Content-Type: text/event-stream.

Related Links