Class KafkaConsumerEndpoint

Represents a topic to consume from.

Inheritance

object

Endpoint

ConsumerEndpoint

KafkaConsumerEndpoint

Implements

IConsumerEndpoint

IEndpoint

IEquatable<KafkaConsumerEndpoint>

Inherited Members

ConsumerEndpoint.ErrorPolicy

ConsumerEndpoint.ExactlyOnceStrategy

ConsumerEndpoint.Batch

ConsumerEndpoint.Sequence

ConsumerEndpoint.NullMessageHandlingStrategy

ConsumerEndpoint.ThrowIfUnhandled

Endpoint.DefaultSerializer

Endpoint.DisplayName

Endpoint.Name

Endpoint.FriendlyName

Endpoint.Serializer

Endpoint.Encryption

Endpoint.MessageValidationMode

object.Equals(object, object)

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Namespace: Silverback.Messaging

Assembly: Silverback.Integration.Kafka.dll

Syntax

public sealed class KafkaConsumerEndpoint : ConsumerEndpoint, IConsumerEndpoint, IEndpoint, IEquatable<KafkaConsumerEndpoint>

Constructors

KafkaConsumerEndpoint(params TopicPartitionOffset[])

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration

public KafkaConsumerEndpoint(params TopicPartitionOffset[] topicPartitions)

Parameters

Type	Name	Description
TopicPartitionOffset[]	topicPartitions	The topics and partitions to be consumed and the starting offset.

KafkaConsumerEndpoint(params TopicPartition[])

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration

public KafkaConsumerEndpoint(params TopicPartition[] topicPartitions)

Parameters

Type	Name	Description
TopicPartition[]	topicPartitions	The topics and partitions to be consumed.

KafkaConsumerEndpoint(IEnumerable<TopicPartitionOffset>, KafkaClientConfig?)

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration

public KafkaConsumerEndpoint(IEnumerable<TopicPartitionOffset> topicPartitions, KafkaClientConfig? clientConfig = null)

Parameters

Type	Name	Description
IEnumerable<TopicPartitionOffset>	topicPartitions	The topics and partitions to be consumed.
KafkaClientConfig	clientConfig	The KafkaClientConfig to be used to initialize the KafkaConsumerConfig.

KafkaConsumerEndpoint(IEnumerable<TopicPartition>, KafkaClientConfig?)

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration

public KafkaConsumerEndpoint(IEnumerable<TopicPartition> topicPartitions, KafkaClientConfig? clientConfig = null)

Parameters

Type	Name	Description
IEnumerable<TopicPartition>	topicPartitions	The topics and partitions to be consumed.
KafkaClientConfig	clientConfig	The KafkaClientConfig to be used to initialize the KafkaConsumerConfig.

KafkaConsumerEndpoint(string, KafkaClientConfig?)

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration

public KafkaConsumerEndpoint(string topicName, KafkaClientConfig? clientConfig = null)

Parameters

Type	Name	Description
string	topicName	The name of the topic.
KafkaClientConfig	clientConfig	The KafkaClientConfig to be used to initialize the KafkaConsumerConfig.

KafkaConsumerEndpoint(string, Func<IReadOnlyCollection<TopicPartition>, IEnumerable<TopicPartitionOffset>>, KafkaClientConfig?)

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration

public KafkaConsumerEndpoint(string topicName, Func<IReadOnlyCollection<TopicPartition>, IEnumerable<TopicPartitionOffset>> topicPartitionsResolver, KafkaClientConfig? clientConfig = null)

Parameters

Type	Name	Description
string	topicName	The name of the topic.
Func<IReadOnlyCollection<TopicPartition>, IEnumerable<TopicPartitionOffset>>	topicPartitionsResolver	The function that returns the topics and partitions to be consumed.
KafkaClientConfig	clientConfig	The KafkaClientConfig to be used to initialize the KafkaConsumerConfig.

KafkaConsumerEndpoint(string, Func<IReadOnlyCollection<TopicPartition>, IEnumerable<TopicPartition>>, KafkaClientConfig?)

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration

public KafkaConsumerEndpoint(string topicName, Func<IReadOnlyCollection<TopicPartition>, IEnumerable<TopicPartition>> topicPartitionsResolver, KafkaClientConfig? clientConfig = null)

Parameters

Type	Name	Description
string	topicName	The name of the topic.
Func<IReadOnlyCollection<TopicPartition>, IEnumerable<TopicPartition>>	topicPartitionsResolver	The function that returns the topics and partitions to be consumed.
KafkaClientConfig	clientConfig	The KafkaClientConfig to be used to initialize the KafkaConsumerConfig.

KafkaConsumerEndpoint(params string[])

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration

public KafkaConsumerEndpoint(params string[] topicNames)

Parameters

Type	Name	Description
string[]	topicNames	The name of the topics.

KafkaConsumerEndpoint(string[], KafkaClientConfig?)

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration

public KafkaConsumerEndpoint(string[] topicNames, KafkaClientConfig? clientConfig = null)

Parameters

Type	Name	Description
string[]	topicNames	The name of the topics.
KafkaClientConfig	clientConfig	The KafkaClientConfig to be used to initialize the KafkaConsumerConfig.

KafkaConsumerEndpoint(string[], Func<IReadOnlyCollection<TopicPartition>, IEnumerable<TopicPartitionOffset>>, KafkaClientConfig?)

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration

public KafkaConsumerEndpoint(string[] topicNames, Func<IReadOnlyCollection<TopicPartition>, IEnumerable<TopicPartitionOffset>> topicPartitionsResolver, KafkaClientConfig? clientConfig = null)

Parameters

Type	Name	Description
string[]	topicNames	The name of the topics.
Func<IReadOnlyCollection<TopicPartition>, IEnumerable<TopicPartitionOffset>>	topicPartitionsResolver	The function that returns the topics and partitions to be consumed.
KafkaClientConfig	clientConfig	The KafkaClientConfig to be used to initialize the KafkaConsumerConfig.

KafkaConsumerEndpoint(string[], Func<IReadOnlyCollection<TopicPartition>, IEnumerable<TopicPartition>>, KafkaClientConfig?)

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration

public KafkaConsumerEndpoint(string[] topicNames, Func<IReadOnlyCollection<TopicPartition>, IEnumerable<TopicPartition>> topicPartitionsResolver, KafkaClientConfig? clientConfig = null)

Parameters

Type	Name	Description
string[]	topicNames	The name of the topics.
Func<IReadOnlyCollection<TopicPartition>, IEnumerable<TopicPartition>>	topicPartitionsResolver	The function that returns the topics and partitions to be consumed.
KafkaClientConfig	clientConfig	The KafkaClientConfig to be used to initialize the KafkaConsumerConfig.

Properties

BackpressureLimit

Gets or sets the maximum number of messages to be consumed and enqueued waiting to be processed. When ProcessPartitionsIndependently is set to true (default) the limit will be applied per partition. The default is 1.

Declaration

public int BackpressureLimit { get; set; }

Property Value

Type	Description
int

Configuration

Gets or sets the Kafka client configuration. This is actually an extension of the configuration dictionary provided by the Confluent.Kafka library.

Declaration

public KafkaConsumerConfig Configuration { get; set; }

Property Value

Type	Description
KafkaConsumerConfig

MaxDegreeOfParallelism

Gets or sets the maximum number of incoming message that can be processed concurrently. Up to a message per each subscribed partition can be processed in parallel. The default is 10.

Declaration

public int MaxDegreeOfParallelism { get; set; }

Property Value

Type	Description
int

Names

Gets the name of the topics.

Declaration

public IReadOnlyCollection<string> Names { get; }

Property Value

Type	Description
IReadOnlyCollection<string>

ProcessPartitionsIndependently

Gets or sets a value indicating whether the partitions must be processed independently. When true a stream will published per each partition and the sequences (ChunkSequence, BatchSequence, ...) cannot span across the partitions. The default is true.

Declaration

public bool ProcessPartitionsIndependently { get; set; }

Property Value

Type	Description
bool

TopicPartitions

Gets the topics and partitions to be consumed.

If both TopicPartitions and TopicPartitionsResolver are null the topics from the Names property will be subscribed and the partitions will be automatically assigned by the broker.

If the collection is empty no partition will be consumed.

Declaration

public IReadOnlyCollection<TopicPartitionOffset>? TopicPartitions { get; }

Property Value

Type	Description
IReadOnlyCollection<TopicPartitionOffset>

TopicPartitionsResolver

Gets the function that returns the topics and partitions to be consumed.

If both TopicPartitions and TopicPartitionsResolver are null the topics from the Names property will be subscribed and the partitions will be automatically assigned by the broker.

If an empty collection is returned by the function no partition will be consumed.

Declaration

public Func<IReadOnlyCollection<TopicPartition>, IEnumerable<TopicPartitionOffset>>? TopicPartitionsResolver { get; }

Property Value

Type	Description
Func<IReadOnlyCollection<TopicPartition>, IEnumerable<TopicPartitionOffset>>

Methods

Equals(KafkaConsumerEndpoint?)

Indicates whether the current object is equal to another object of the same type.

Declaration

public bool Equals(KafkaConsumerEndpoint? other)

Parameters

Type	Name	Description
KafkaConsumerEndpoint	other	An object to compare with this object.

Returns

Type	Description
bool	true if the current object is equal to the other parameter; otherwise, false.

Equals(object?)

Determines whether the specified object is equal to the current object.

Declaration

public override bool Equals(object? obj)

Parameters

Type	Name	Description
object	obj	The object to compare with the current object.

Returns

Type	Description
bool	true if the specified object is equal to the current object; otherwise, false.

Overrides

object.Equals(object)

GetHashCode()

Serves as the default hash function.

Declaration

public override int GetHashCode()

Returns

Type	Description
int	A hash code for the current object.

Overrides

object.GetHashCode()

GetUniqueConsumerGroupName()

Gets a unique name for the consumer group (e.g. Kafka's consumer group id). This value (joint with the endpoint name) will be used for example to ensure the exactly-once delivery.

Declaration

public override string GetUniqueConsumerGroupName()

Returns

Type	Description
string	Returns the unique name for the consumer group.

Overrides

ConsumerEndpoint.GetUniqueConsumerGroupName()

Remarks

It's not enough to use the endpoint name, since the same topic could be consumed by multiple consumer groups within the same process and/or using the same database to store the information needed to ensure the exactly-once delivery.

Validate()

Validates the endpoint configuration and throws an EndpointConfigurationException if not valid.

Declaration

public override void Validate()

Overrides

ConsumerEndpoint.Validate()

Implements

IConsumerEndpoint

IEndpoint

IEquatable<T>