Table of Contents

Kafka - JSON with Schema Registry

This sample implements a producer and consumer which take advantage of the schema registry and serialize the messages as JSON, validated against a JSON schema.

What This Sample Shows

  • Enabling the Confluent schema registry integration with AddConfluentSchemaRegistry.
  • Defining the JSON schema alongside the message class (as a constant excluded from serialization via [JsonIgnore]).
  • Registering the schema for the samples-json-schema-registry-value subject at startup through the IConfluentSchemaRegistryClientFactory.
  • Serializing with SerializeAsJsonUsingSchemaRegistry, disabling the automatic schema registration and using the latest registered version.
  • Deserializing with DeserializeJsonUsingSchemaRegistry.

How to Run

Start the infrastructure as described in Samples (the schema registry is required, listening on localhost:8081), then run the two applications from the repository root.

dotnet run --project ./samples/Kafka/JsonSchemaRegistry.Producer/.
dotnet run --project ./samples/Kafka/JsonSchemaRegistry.Consumer/.

Observe the console output while the messages are produced and consumed.

Common

The message being exchanged and its JSON schema are defined in a common project.

using Newtonsoft.Json;

namespace Silverback.Samples.Kafka.JsonSchemaRegistry.Common;

public class SampleMessage
{
    [JsonIgnore]
    public const string Schema =
        """
        {
          "$schema": "http://json-schema.org/draft-04/schema#",
          "description": "",
          "type": "object",
          "properties": {
            "number": {
              "type": "number"
            }
          },
          "required": [
            "number"
          ]
        }
        """;

    public int Number { get; set; }
}

Full source code: https://github.com/BEagle1984/silverback/tree/main/samples/Kafka/JsonSchemaRegistry.Common

Producer

The producer uses a hosted service to register the schema and then publish some messages in the background.

using Microsoft.Extensions.DependencyInjection;
using Silverback.Configuration;
using Silverback.Messaging.Configuration;

namespace Silverback.Samples.Kafka.JsonSchemaRegistry.Producer;

public class Startup
{
    public void ConfigureServices(IServiceCollection services)
    {
        // Enable Silverback
        services.AddSilverback()

            // Use Apache Kafka as message broker and the Confluent schema registry
            .WithConnectionToMessageBroker(options => options
                .AddKafka()
                .AddConfluentSchemaRegistry())

            // Delegate the broker clients configuration to a separate class
            .AddBrokerClientsConfigurator<BrokerClientsConfigurator>();

        // Add the hosted service that produces the random sample messages
        services.AddHostedService<ProducerBackgroundService>();
    }

    public void Configure()
    {
    }
}

Full source code: https://github.com/BEagle1984/silverback/tree/main/samples/Kafka/JsonSchemaRegistry.Producer

Consumer

The consumer processes the messages and outputs their value to the standard output.

using Microsoft.Extensions.DependencyInjection;
using Silverback.Configuration;
using Silverback.Messaging.Configuration;

namespace Silverback.Samples.Kafka.JsonSchemaRegistry.Consumer;

public class Startup
{
    public void ConfigureServices(IServiceCollection services)
    {
        // Enable Silverback
        services.AddSilverback()

            // Use Apache Kafka as message broker and the Confluent schema registry
            .WithConnectionToMessageBroker(options => options
                .AddKafka()
                .AddConfluentSchemaRegistry())

            // Delegate the broker clients configuration to a separate class
            .AddBrokerClientsConfigurator<BrokerClientsConfigurator>()

            // Register the subscribers
            .AddSingletonSubscriber<SampleMessageSubscriber>();
    }

    public void Configure()
    {
    }
}

Full source code: https://github.com/BEagle1984/silverback/tree/main/samples/Kafka/JsonSchemaRegistry.Consumer

Additional Resources