Skip to main content

Sending Messages to Kafka from a Production

InterSystems IRIS becomes a Kafka Producer when an interoperability production includes a business operation that uses the Kafka outbound adapter. This adapter allows the business operation to send messages to a Kafka topic. You have two options when using a production as a Kafka Producer: use a built-in business operation that leverages the outbound adapter or build your own business operation that uses the adapter.

Adapter Details

The Kafka outbound adapter is the class EnsLib.Kafka.OutboundAdapterOpens in a new tab. It includes the following settings, all of which appear in the Management Portal as settings for the business operation that uses the adapter:

  • Servers defines a comma-separated list of IP address:port entries that identify servers in the Kafka broker cluster.

  • ClientID defines the Kafka client ID of the Producer.

  • Credentials defines the InterSystems credentials that correspond to the username and password of a Kafka client. For details on creating credentials, see Defining Reusable Items for Use in Settings.

  • SecurityProtocol specifies the security protocol which secures connections to your Kafka broker cluster. Currently, this property supports two values:

    • SASL_PLAINTEXT, which performs SASL authentication of the client over an unencrypted channel.

    • SASL_SSL, which uses the truststore and keystore information you provide to establish an SSL/TLS connection over which SASL authentication takes place.

  • SASLMechanism specifies the SASL authentication mechanism used to authenticate the Producer using the credentials specified by Credentials. Currently, only PLAIN is supported.

  • TrustStoreLocation (optional) specifies the file system path to the truststore which contains the certificate authority certificates necessary to validate a certificate from your Kafka broker cluster and establish an SSL/TLS connection.

  • TrustStoreCredentials (optional) defines the InterSystems credentials which can be used to gain password-protected access to the truststore at the location specified by truststorelocation.

  • KeyStoreLocation (optional) specifies the file system path to the keystore which contains the keys necessary to establish an SSL/TLS connection with your Kafka broker cluster.

  • KeyStoreCredentials (optional) defines the InterSystems credentials which can be used to gain password-protected access to the keystore at the location specified by keystorelocation.

  • KeyCredentials (optional) defines the InterSystems credentials which can be used to gain password-protected access to a private key within the keystore at the location specified by keystorelocation.

For general information about building a custom business operation that uses an outbound adapter, see Defining Business Operations.

Send Kafka Messages

The class of the messages sent by the outbound adapter to Kafka is EnsLib.Kafka.MessageOpens in a new tab, which contains the following properties:

  • topic defines the Kafka topic where the Producer is sending messages.

  • value (a %String) defines the content of the Kafka message. If value is set, you should not set binaryValue.

  • binaryValue (a binary stream of arbitrary length) defines the content of the Kafka message when the length of the message exceeds the maximum length of a %String. If binaryValue is set, you should not set value.

  • key defines an optional tag for the Kafka message.

The outbound adapter's SendMessage() method takes a EnsLib.Kafka.MessageOpens in a new tab object as its sole argument, and sends that message to the Kafka server. For example, a custom business operation could call the adapter’s method by including the following:

Do ..Adapter.SendMessage(pRequest)

Built-in Business Operation

Rather than building a custom business operation that uses the outbound adapter, you can simply add EnsLib.Kafka.OperationOpens in a new tab to the production and define the adapter properties using the Management Portal settings. This business operation calls the adapter's SendMessage() method when it receives a request from another business host in the production. This request should contain a Kafka message of type EnsLib.Kafka.MessageOpens in a new tab.

For basic information about adding a business operation to a production, see Adding Business Hosts.

See Also

FeedbackOpens in a new tab