Class KafkaConsumerEndpoint

Represents a topic to consume from.

Inheritance

System.Object

Endpoint

ConsumerEndpoint

KafkaConsumerEndpoint

Implements

IConsumerEndpoint

IEndpoint

System.IEquatable<KafkaConsumerEndpoint>

Inherited Members

ConsumerEndpoint.ErrorPolicy

ConsumerEndpoint.ExactlyOnceStrategy

ConsumerEndpoint.Batch

ConsumerEndpoint.Sequence

ConsumerEndpoint.NullMessageHandlingStrategy

ConsumerEndpoint.ThrowIfUnhandled

Endpoint.DefaultSerializer

Endpoint.DisplayName

Endpoint.Name

Endpoint.FriendlyName

Endpoint.Serializer

Endpoint.Encryption

Endpoint.MessageValidationMode

Endpoint.BaseEquals(Endpoint)

System.Object.Equals(System.Object, System.Object)

System.Object.GetType()

System.Object.MemberwiseClone()

System.Object.ReferenceEquals(System.Object, System.Object)

System.Object.ToString()

Namespace: Silverback.Messaging

Assembly: Silverback.Integration.Kafka.dll

Syntax

public sealed class KafkaConsumerEndpoint : ConsumerEndpoint, IConsumerEndpoint, IEndpoint, IEquatable<KafkaConsumerEndpoint>

Constructors

| Improve this doc View source

KafkaConsumerEndpoint(TopicPartition[])

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration

public KafkaConsumerEndpoint(params TopicPartition[] topicPartitions)

Parameters

Type	Name	Description
Confluent.Kafka.TopicPartition[]	topicPartitions	The topics and partitions to be consumed.

| Improve this doc View source

KafkaConsumerEndpoint(TopicPartitionOffset[])

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration

public KafkaConsumerEndpoint(params TopicPartitionOffset[] topicPartitions)

Parameters

Type	Name	Description
Confluent.Kafka.TopicPartitionOffset[]	topicPartitions	The topics and partitions to be consumed and the starting offset.

| Improve this doc View source

KafkaConsumerEndpoint(IEnumerable<TopicPartition>, KafkaClientConfig)

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration

public KafkaConsumerEndpoint(IEnumerable<TopicPartition> topicPartitions, KafkaClientConfig clientConfig = null)

Parameters

Type	Name	Description
System.Collections.Generic.IEnumerable<Confluent.Kafka.TopicPartition>	topicPartitions	The topics and partitions to be consumed.
KafkaClientConfig	clientConfig	The KafkaClientConfig to be used to initialize the KafkaConsumerConfig.

| Improve this doc View source

KafkaConsumerEndpoint(IEnumerable<TopicPartitionOffset>, KafkaClientConfig)

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration

public KafkaConsumerEndpoint(IEnumerable<TopicPartitionOffset> topicPartitions, KafkaClientConfig clientConfig = null)

Parameters

Type	Name	Description
System.Collections.Generic.IEnumerable<Confluent.Kafka.TopicPartitionOffset>	topicPartitions	The topics and partitions to be consumed.
KafkaClientConfig	clientConfig	The KafkaClientConfig to be used to initialize the KafkaConsumerConfig.

| Improve this doc View source

KafkaConsumerEndpoint(String, KafkaClientConfig)

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration

public KafkaConsumerEndpoint(string topicName, KafkaClientConfig clientConfig = null)

Parameters

Type	Name	Description
System.String	topicName	The name of the topic.
KafkaClientConfig	clientConfig	The KafkaClientConfig to be used to initialize the KafkaConsumerConfig.

| Improve this doc View source

KafkaConsumerEndpoint(String, Func<IReadOnlyCollection<TopicPartition>, IEnumerable<TopicPartition>>, KafkaClientConfig)

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration

public KafkaConsumerEndpoint(string topicName, Func<IReadOnlyCollection<TopicPartition>, IEnumerable<TopicPartition>> topicPartitionsResolver, KafkaClientConfig clientConfig = null)

Parameters

Type	Name	Description
System.String	topicName	The name of the topic.
System.Func<System.Collections.Generic.IReadOnlyCollection<Confluent.Kafka.TopicPartition>, System.Collections.Generic.IEnumerable<Confluent.Kafka.TopicPartition>>	topicPartitionsResolver	The function that returns the topics and partitions to be consumed.
KafkaClientConfig	clientConfig	The KafkaClientConfig to be used to initialize the KafkaConsumerConfig.

| Improve this doc View source

KafkaConsumerEndpoint(String, Func<IReadOnlyCollection<TopicPartition>, IEnumerable<TopicPartitionOffset>>, KafkaClientConfig)

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration

public KafkaConsumerEndpoint(string topicName, Func<IReadOnlyCollection<TopicPartition>, IEnumerable<TopicPartitionOffset>> topicPartitionsResolver, KafkaClientConfig clientConfig = null)

Parameters

Type	Name	Description
System.String	topicName	The name of the topic.
System.Func<System.Collections.Generic.IReadOnlyCollection<Confluent.Kafka.TopicPartition>, System.Collections.Generic.IEnumerable<Confluent.Kafka.TopicPartitionOffset>>	topicPartitionsResolver	The function that returns the topics and partitions to be consumed.
KafkaClientConfig	clientConfig	The KafkaClientConfig to be used to initialize the KafkaConsumerConfig.

| Improve this doc View source

KafkaConsumerEndpoint(String[])

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration

public KafkaConsumerEndpoint(params string[] topicNames)

Parameters

Type	Name	Description
System.String[]	topicNames	The name of the topics.

| Improve this doc View source

KafkaConsumerEndpoint(String[], KafkaClientConfig)

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration

public KafkaConsumerEndpoint(string[] topicNames, KafkaClientConfig clientConfig = null)

Parameters

Type	Name	Description
System.String[]	topicNames	The name of the topics.
KafkaClientConfig	clientConfig	The KafkaClientConfig to be used to initialize the KafkaConsumerConfig.

| Improve this doc View source

KafkaConsumerEndpoint(String[], Func<IReadOnlyCollection<TopicPartition>, IEnumerable<TopicPartition>>, KafkaClientConfig)

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration

public KafkaConsumerEndpoint(string[] topicNames, Func<IReadOnlyCollection<TopicPartition>, IEnumerable<TopicPartition>> topicPartitionsResolver, KafkaClientConfig clientConfig = null)

Parameters

Type	Name	Description
System.String[]	topicNames	The name of the topics.
System.Func<System.Collections.Generic.IReadOnlyCollection<Confluent.Kafka.TopicPartition>, System.Collections.Generic.IEnumerable<Confluent.Kafka.TopicPartition>>	topicPartitionsResolver	The function that returns the topics and partitions to be consumed.
KafkaClientConfig	clientConfig	The KafkaClientConfig to be used to initialize the KafkaConsumerConfig.

| Improve this doc View source

KafkaConsumerEndpoint(String[], Func<IReadOnlyCollection<TopicPartition>, IEnumerable<TopicPartitionOffset>>, KafkaClientConfig)

Initializes a new instance of the KafkaConsumerEndpoint class.

Declaration

public KafkaConsumerEndpoint(string[] topicNames, Func<IReadOnlyCollection<TopicPartition>, IEnumerable<TopicPartitionOffset>> topicPartitionsResolver, KafkaClientConfig clientConfig = null)

Parameters

Type	Name	Description
System.String[]	topicNames	The name of the topics.
System.Func<System.Collections.Generic.IReadOnlyCollection<Confluent.Kafka.TopicPartition>, System.Collections.Generic.IEnumerable<Confluent.Kafka.TopicPartitionOffset>>	topicPartitionsResolver	The function that returns the topics and partitions to be consumed.
KafkaClientConfig	clientConfig	The KafkaClientConfig to be used to initialize the KafkaConsumerConfig.

Properties

| Improve this doc View source

BackpressureLimit

Gets or sets the maximum number of messages to be consumed and enqueued waiting to be processed. When ProcessPartitionsIndependently is set to true (default) the limit will be applied per partition. The default is 1.

Declaration

public int BackpressureLimit { get; set; }

Property Value

Type	Description
System.Int32

| Improve this doc View source

Configuration

Gets or sets the Kafka client configuration. This is actually an extension of the configuration dictionary provided by the Confluent.Kafka library.

Declaration

public KafkaConsumerConfig Configuration { get; set; }

Property Value

Type	Description
KafkaConsumerConfig

| Improve this doc View source

MaxDegreeOfParallelism

Gets or sets the maximum number of incoming message that can be processed concurrently. Up to a message per each subscribed partition can be processed in parallel. The default is 10.

Declaration

public int MaxDegreeOfParallelism { get; set; }

Property Value

Type	Description
System.Int32

| Improve this doc View source

Names

Gets the name of the topics.

Declaration

public IReadOnlyCollection<string> Names { get; }

Property Value

Type	Description
System.Collections.Generic.IReadOnlyCollection<System.String>

| Improve this doc View source

ProcessPartitionsIndependently

Gets or sets a value indicating whether the partitions must be processed independently. When true a stream will published per each partition and the sequences (ChunkSequence, BatchSequence, ...) cannot span across the partitions. The default is true.

Declaration

public bool ProcessPartitionsIndependently { get; set; }

Property Value

Type	Description
System.Boolean

| Improve this doc View source

TopicPartitions

Gets the topics and partitions to be consumed.

If both TopicPartitions and TopicPartitionsResolver are null the topics from the Names property will be subscribed and the partitions will be automatically assigned by the broker.

If the collection is empty no partition will be consumed.

Declaration

public IReadOnlyCollection<TopicPartitionOffset> TopicPartitions { get; }

Property Value

Type	Description
System.Collections.Generic.IReadOnlyCollection<Confluent.Kafka.TopicPartitionOffset>

| Improve this doc View source

TopicPartitionsResolver

Gets the function that returns the topics and partitions to be consumed.

If both TopicPartitions and TopicPartitionsResolver are null the topics from the Names property will be subscribed and the partitions will be automatically assigned by the broker.

If an empty collection is returned by the function no partition will be consumed.

Declaration

public Func<IReadOnlyCollection<TopicPartition>, IEnumerable<TopicPartitionOffset>> TopicPartitionsResolver { get; }

Property Value

Type	Description
System.Func<System.Collections.Generic.IReadOnlyCollection<Confluent.Kafka.TopicPartition>, System.Collections.Generic.IEnumerable<Confluent.Kafka.TopicPartitionOffset>>

Methods

| Improve this doc View source

Equals(KafkaConsumerEndpoint)

Declaration

public bool Equals(KafkaConsumerEndpoint other)

Parameters

Type	Name	Description
KafkaConsumerEndpoint	other

Returns

Type	Description
System.Boolean

| Improve this doc View source

Equals(Object)

Declaration

public override bool Equals(object obj)

Parameters

Type	Name	Description
System.Object	obj

Returns

Type	Description
System.Boolean

Overrides

System.Object.Equals(System.Object)

| Improve this doc View source

GetHashCode()

Declaration

public override int GetHashCode()

Returns

Type	Description
System.Int32

Overrides

System.Object.GetHashCode()

| Improve this doc View source

GetUniqueConsumerGroupName()

Gets a unique name for the consumer group (e.g. Kafka's consumer group id). This value (joint with the endpoint name) will be used for example to ensure the exactly-once delivery.

Declaration

public override string GetUniqueConsumerGroupName()

Returns

Type	Description
System.String	Returns the unique name for the consumer group.

Overrides

ConsumerEndpoint.GetUniqueConsumerGroupName()

Remarks

It's not enough to use the endpoint name, since the same topic could be consumed by multiple consumer groups within the same process and/or using the same database to store the information needed to ensure the exactly-once delivery.

| Improve this doc View source

Validate()

Validates the endpoint configuration and throws an EndpointConfigurationException if not valid.

Declaration

public override void Validate()

Overrides

ConsumerEndpoint.Validate()

Implements

IConsumerEndpoint

IEndpoint

System.IEquatable<T>