Class KafkaConsumerEndpoint

Represents a topic to consume from.

Inheritance
System.Object
Endpoint
ConsumerEndpoint
KafkaConsumerEndpoint
Implements
System.IEquatable<KafkaConsumerEndpoint>
Inherited Members
System.Object.Equals(System.Object, System.Object)
System.Object.GetType()
System.Object.MemberwiseClone()
System.Object.ReferenceEquals(System.Object, System.Object)
System.Object.ToString()
Namespace: Silverback.Messaging
Assembly: Silverback.Integration.Kafka.dll
Syntax
public sealed class KafkaConsumerEndpoint : ConsumerEndpoint, IConsumerEndpoint, IEndpoint, IEquatable<KafkaConsumerEndpoint>

Constructors

| Improve this doc View source

KafkaConsumerEndpoint(TopicPartition[])

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration
public KafkaConsumerEndpoint(params TopicPartition[] topicPartitions)
Parameters
Type Name Description
Confluent.Kafka.TopicPartition[] topicPartitions

The topics and partitions to be consumed.
| Improve this doc View source

KafkaConsumerEndpoint(TopicPartitionOffset[])

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration
public KafkaConsumerEndpoint(params TopicPartitionOffset[] topicPartitions)
Parameters
Type Name Description
Confluent.Kafka.TopicPartitionOffset[] topicPartitions

The topics and partitions to be consumed and the starting offset.
| Improve this doc View source

KafkaConsumerEndpoint(IEnumerable<TopicPartition>, KafkaClientConfig)

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration
public KafkaConsumerEndpoint(IEnumerable<TopicPartition> topicPartitions, KafkaClientConfig clientConfig = null)
Parameters
Type Name Description
System.Collections.Generic.IEnumerable<Confluent.Kafka.TopicPartition> topicPartitions

The topics and partitions to be consumed.
KafkaClientConfig clientConfig

The KafkaClientConfig to be used to initialize the KafkaConsumerConfig.
| Improve this doc View source

KafkaConsumerEndpoint(IEnumerable<TopicPartitionOffset>, KafkaClientConfig)

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration
public KafkaConsumerEndpoint(IEnumerable<TopicPartitionOffset> topicPartitions, KafkaClientConfig clientConfig = null)
Parameters
Type Name Description
System.Collections.Generic.IEnumerable<Confluent.Kafka.TopicPartitionOffset> topicPartitions

The topics and partitions to be assigned.
KafkaClientConfig clientConfig

The KafkaClientConfig to be used to initialize the KafkaConsumerConfig.
| Improve this doc View source

KafkaConsumerEndpoint(String, KafkaClientConfig)

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration
public KafkaConsumerEndpoint(string topicName, KafkaClientConfig clientConfig = null)
Parameters
Type Name Description
System.String topicName

The name of the topic.
KafkaClientConfig clientConfig

The KafkaClientConfig to be used to initialize the KafkaConsumerConfig.
| Improve this doc View source

KafkaConsumerEndpoint(String[])

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration
public KafkaConsumerEndpoint(params string[] topicNames)
Parameters
Type Name Description
System.String[] topicNames

The name of the topics.
| Improve this doc View source

KafkaConsumerEndpoint(String[], KafkaClientConfig)

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration
public KafkaConsumerEndpoint(string[] topicNames, KafkaClientConfig clientConfig = null)
Parameters
Type Name Description
System.String[] topicNames

The name of the topics.
KafkaClientConfig clientConfig

The KafkaClientConfig to be used to initialize the KafkaConsumerConfig.

Properties

| Improve this doc View source

BackpressureLimit

Gets or sets the maximum number of messages to be consumed and enqueued waiting to be processed. When ProcessPartitionsIndependently is set to true (default) the limit will be applied per partition. The default is 1.

Declaration
public int BackpressureLimit { get; set; }
Property Value
Type Description
System.Int32
| Improve this doc View source

Configuration

Gets or sets the Kafka client configuration. This is actually an extension of the configuration dictionary provided by the Confluent.Kafka library.

Declaration
public KafkaConsumerConfig Configuration { get; set; }
Property Value
Type Description
KafkaConsumerConfig
| Improve this doc View source

MaxDegreeOfParallelism

Gets or sets the maximum number of incoming message that can be processed concurrently. Up to a message per each subscribed partition can be processed in parallel. The default is 10.

Declaration
public int MaxDegreeOfParallelism { get; set; }
Property Value
Type Description
System.Int32
| Improve this doc View source

Names

Gets the name of the topics.

Declaration
public IReadOnlyCollection<string> Names { get; }
Property Value
Type Description
System.Collections.Generic.IReadOnlyCollection<System.String>
| Improve this doc View source

ProcessPartitionsIndependently

Gets or sets a value indicating whether the partitions must be processed independently. When true a stream will published per each partition and the sequences (ChunkSequence, BatchSequence, ...) cannot span across the partitions. The default is true.

Declaration
public bool ProcessPartitionsIndependently { get; set; }
Property Value
Type Description
System.Boolean
| Improve this doc View source

TopicPartitions

Gets the topic and partitions to be consumed. If the collection is null the topics from the Names property will be subscribed and the partitions will be automatically assigned by the broker. If the collection is empty no partition will be consumed.

Declaration
public IReadOnlyCollection<TopicPartitionOffset> TopicPartitions { get; }
Property Value
Type Description
System.Collections.Generic.IReadOnlyCollection<Confluent.Kafka.TopicPartitionOffset>

Methods

| Improve this doc View source

Equals(KafkaConsumerEndpoint)

Declaration
public bool Equals(KafkaConsumerEndpoint other)
Parameters
Type Name Description
KafkaConsumerEndpoint other
Returns
Type Description
System.Boolean
| Improve this doc View source

Equals(Object)

Declaration
public override bool Equals(object obj)
Parameters
Type Name Description
System.Object obj
Returns
Type Description
System.Boolean
Overrides
System.Object.Equals(System.Object)
| Improve this doc View source

GetHashCode()

Declaration
public override int GetHashCode()
Returns
Type Description
System.Int32
Overrides
System.Object.GetHashCode()
| Improve this doc View source

GetUniqueConsumerGroupName()

Gets a unique name for the consumer group (e.g. Kafka's consumer group id). This value (joint with the endpoint name) will be used for example to ensure the exactly-once delivery.

Declaration
public override string GetUniqueConsumerGroupName()
Returns
Type Description
System.String

Returns the unique name for the consumer group.
Overrides
Remarks

It's not enough to use the endpoint name, since the same topic could be consumed by multiple consumer groups within the same process and/or using the same database to store the information needed to ensure the exactly-once delivery.

| Improve this doc View source

Validate()

Validates the endpoint configuration and throws an EndpointConfigurationException if not valid.

Declaration
public override void Validate()
Overrides

Implements

System.IEquatable<T>