Hilla and Kafka. Part 1: Consuming messages

René Wilby | Jun 21, 2025 min read
Hilla and Kafka. Part 1: Consuming messages

Article series

This is the first part of a small article series about how to build Hilla web apps for Apache Kafka.

In this blog post, we will learn how to consume Kafka messages using Spring Kafka in the Hilla backend and how to display these messages in the Hilla frontend in a reactive way.

Setup Kafka

Setting up a Kafka Broker is beyond the scope of this article. In this article, we will use a simple Kafka Docker Container with a minimal default configuration. You can use the following Docker Compose file to start a Kafka Docker Container and to create a topic.

name: hilla-kafka-example

services:
  broker:
    image: apache/kafka:latest
    hostname: broker
    container_name: broker
    ports:
      - 9092:9092
    environment:
      KAFKA_NODE_ID: 1
      KAFKA_PROCESS_ROLES: broker,controller
      KAFKA_LISTENERS: EXTERNAL://0.0.0.0:9092,INTERNAL://0.0.0.0:19092,CONTROLLER://0.0.0.0:9093
      KAFKA_ADVERTISED_LISTENERS: EXTERNAL://localhost:9092,INTERNAL://broker:19092,CONTROLLER://broker:9093
      KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: EXTERNAL:PLAINTEXT,INTERNAL:PLAINTEXT,CONTROLLER:PLAINTEXT
      KAFKA_CONTROLLER_QUORUM_VOTERS: 1@broker:9093
      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
      KAFKA_NUM_PARTITIONS: 1

  topic-creator:
    image: apache/kafka:latest
    container_name: topic-creator
    depends_on:
      broker:
        condition: service_started
    volumes:
      - ./kafka-init.sh:/tmp/kafka-init.sh
    command: ['sh', '-c', '/tmp/kafka-init.sh']

You can use the Docker Compose file with the following command.

docker compose -f docker/docker-compose.yaml up|down

Please check out the official documentation of the Kafka Docker Image for more information about the image and how to configure it.

The script kafka-init.sh automatically creates a topic called reservations in our local Kafka Broker, and we will use this later to consume messages from it.

#!/bin/bash
BROKER="broker:19092"

echo "Creating topic..."

/opt/kafka/bin/kafka-topics.sh --if-not-exists --create --topic reservations --bootstrap-server "$BROKER" --partitions 1 --replication-factor 1

echo "Topic created successfully!"

You could also create the topic using a special NewTopic Bean, as described in the Spring Boot docs, for example.

New Hilla project

We can create a new Hilla project, either using start.vaadin.com or npx create-vaadin, for example.

Consuming Kafka messages in Hilla backend

To consume Kafka messages from our topic reservations in the backend of our Hilla app, we will use Spring Kafka. Therefore, we need to add the required dependency to our pom.xml.

<dependency>
  <groupId>org.springframework.kafka</groupId>
  <artifactId>spring-kafka</artifactId>
</dependency>

Next up, we need a data model that represents the reservation messages, we are going to receive.

package com.example.application.order;

import java.time.LocalDate;
import org.jspecify.annotations.NonNull;

public record Reservation(@NonNull String id, @NonNull LocalDate date, @NonNull String customer) { }

In order to be able to consume Kafka messages, we need to add some configuration to our Hilla backend. We could create the required beans as described in the Spring for Apache Kafka documentation, or we could use Spring Boot’s auto-configuration capabilities (as described in the Spring Boot documentation for Apache Kafka) by providing the required configuration in the application.properties of our Hilla backend.

# Set up Kafka:
spring.kafka.bootstrap-servers=localhost:9092

# Configure the consumer:
spring.kafka.consumer.client-id=reservation-consumer-client
spring.kafka.consumer.group-id=reservation-consumer-group
spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.consumer.value-deserializer=org.springframework.kafka.support.serializer.JsonDeserializer
spring.kafka.consumer.properties[spring.json.value.default.type]=com.example.application.reservation.Reservation

Most of the configuration should be self-explaining. When receiving messages, we want to deserialize the message to our Reservation model. For this purpose, we can use the JsonDeserializer that is provided by Spring. In addition, we configure our Reservation to be the default type for incoming messages. This way we don’t have to care about message headers and their mapping.

Now, we create a KafkaConsumerService that will be responsible for receiving the Kafka messages and for providing the resulting reservations.

package com.example.application.kafka;

import org.jspecify.annotations.NonNull;
import org.springframework.kafka.annotation.KafkaListener;
import com.example.application.reservation.Reservation;
import com.vaadin.flow.server.auth.AnonymousAllowed;
import com.vaadin.hilla.BrowserCallable;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Sinks;
import reactor.core.publisher.Sinks.EmitResult;
import reactor.core.publisher.Sinks.Many;

@BrowserCallable
@AnonymousAllowed
public class KafkaConsumerService {

    private final Many<Reservation> reservationSink;

    public KafkaConsumerService() {
        this.reservationSink = Sinks.many().replay().all();
    }

    @KafkaListener(topics = "reservations", groupId = "reservation-consumer-group")
    private void consume(Reservation reservation) {
        reservationSink.emitNext(reservation, (signalType, emitResult) -> emitResult == EmitResult.FAIL_NON_SERIALIZED);
    }

    public Flux<@NonNull Reservation> getLatestReservation() {
        return reservationSink.asFlux();
    }
}

We use the @KafkaListener annotation as a simple message listener. Every received message in our topic reservations will be deserialized to a Reservation and emitted to the reservation sink. We configure the reservation sink to remember all reservations and to replay all elements pushed to this sink to new subscribers. In order to enable our Hilla frontend to subscribe to the sink and to get updates when new messages are received, we need to convert the sink into a Flux. This is done in the getLatestReservation method. Hilla’s @BrowserCallable annotation takes care of the rest. It automatically creates a reactive browser callable service to push messages from the backend to the frontend as described in the corresponding guide in the docs.

Display Kafka messages in Hilla frontend

Now, we can create a simple React view in our Hilla frontend to subscribe to the Flux returned by the method getLatestReservation.

import { useSignal } from '@vaadin/hilla-react-signals';
import { Grid, GridColumn, VerticalLayout } from '@vaadin/react-components';
import { ViewToolbar } from 'Frontend/components/ViewToolbar';
import Reservation from 'Frontend/generated/com/example/application/reservation/Reservation';
import { KafkaConsumerService } from 'Frontend/generated/endpoints';
import { useEffect } from 'react';

export default function ReservationsView() {
  const reservations = useSignal<Reservation[]>();

  useEffect(() => {
    const reservationSubscription = KafkaConsumerService.getLatestReservation().onNext((reservation: Reservation) => {
      reservations.value = [...(reservations.value ?? []), reservation];
    });
    return () => reservationSubscription.cancel();
  }, []);

  return (
    <VerticalLayout theme="margin">
      <ViewToolbar title="Reservations" />
      <Grid items={reservations.value}>
        <GridColumn path={'id'} />
        <GridColumn path={'date'} />
        <GridColumn path={'customer'} />
      </Grid>
    </VerticalLayout>
  );
}

We subscribe to the Flux using a useEffect hook. Every received reservation will be added to the local reservations state that is used by the Grid component to show all available reservations.

The cleanup function of the useEffect hook will call the cancel function of the subscription once the view has been removed from the DOM.

We can extend the subscription by providing a simple error handling.

import { ActionOnLostSubscription } from '@vaadin/hilla-frontend';
import { useSignal } from '@vaadin/hilla-react-signals';
import {
  Button,
  Grid,
  GridColumn,
  HorizontalLayout,
  Icon,
  Notification,
  VerticalLayout,
} from '@vaadin/react-components';
import { ViewToolbar } from 'Frontend/components/ViewToolbar';
import Reservation from 'Frontend/generated/com/example/application/reservation/Reservation';
import { KafkaConsumerService } from 'Frontend/generated/endpoints';
import { useEffect } from 'react';

export default function ReservationsView() {
  const reservations = useSignal<Reservation[]>();
  const subscriptionNotificationOpened = useSignal<boolean>(false);
  const subscriptionNotificationMessage = useSignal<string | undefined>(undefined);

  useEffect(() => {
    const reservationSubscription = KafkaConsumerService.getLatestReservation()
      .onNext((reservation: Reservation) => {
        reservations.value = [...(reservations.value ?? []), reservation];
      })
      .onError((message: string) => {
        subscriptionNotificationOpened.value = true;
        subscriptionNotificationMessage.value = message;
      })
      .onSubscriptionLost(() => ActionOnLostSubscription.RESUBSCRIBE);
    return () => reservationSubscription.cancel();
  }, []);

  const closeSubscriptionNotification = () => {
    subscriptionNotificationOpened.value = false;
    subscriptionNotificationMessage.value = undefined;
  };

  const retrySubscription = () => {
    window.location.reload();
  };

  return (
    <VerticalLayout theme="margin">
      <ViewToolbar title="Reservations" />
      <Grid items={reservations.value}>
        <GridColumn path={'id'} />
        <GridColumn path={'date'} />
        <GridColumn path={'customer'} />
      </Grid>
      <Notification
        theme="warning"
        duration={0}
        position="middle"
        opened={subscriptionNotificationOpened.value}
        onOpenedChanged={(event) => {
          subscriptionNotificationOpened.value = event.detail.value;
        }}>
        <HorizontalLayout theme="spacing" style={{ alignItems: 'center' }}>
          <div>{subscriptionNotificationMessage.value ?? 'Failed to subscribe to reactive reservation endpoint'}</div>
          <Button theme="tertiary-inline" style={{ marginLeft: 'var(--lumo-space-xl)' }} onClick={retrySubscription}>
            Retry
          </Button>
          <Button theme="tertiary-inline icon" onClick={closeSubscriptionNotification} aria-label="Close">
            <Icon icon="lumo:cross" />
          </Button>
        </HorizontalLayout>
      </Notification>
    </VerticalLayout>
  );
}

In case of a subscription error, we remember the error message and open a notification dialog to inform the user about the error. We also provide a simple retry mechanism that reloads the page to enforce a new subscription.

In case of a lost subscription (e.g. a short network interruption), we tell Hilla to re-subscribe by calling the same server method again. Please check out the documentation for more information about this kind of error handling.

Manual testing

Now it’s time to put it all together and to try the Hilla app. Ensure that the Kafka broker is (still) running, start the Hilla app (for example with ./mvnw spring-boot:run) and visit http://localhost:8080 in your browser.

The next step is to create some Kafka messages in our reservations topic. One way to do it, is by using the Kafka CLI. It’s part of the Kafka binaries that you can download here: https://kafka.apache.org/downloads. After downloading and extracting it, you can call the kafka-console-producer.sh script and add messages to the topic like this:

./bin/kafka-console-producer.sh --topic reservations --bootstrap-server localhost:9092 --property "parse.key=true" --property "key.separator=="
>R00000001={"id":"R00000001", "date": "2025-06-16", "customer":"Leif"}
>R00000002={"id":"R00000002", "date": "2025-06-17", "customer":"Miikka"}
>R00000003={"id":"R00000003", "date": "2025-06-18", "customer":"Marcus"}
>R00000004={"id":"R00000004", "date": "2025-06-19", "customer":"Sami"}
>R00000005={"id":"R00000005", "date": "2025-06-20", "customer":"Matti"}
>R00000006={"id":"R00000006", "date": "2025-06-21", "customer":"Rolf"}
>R00000007={"id":"R00000007", "date": "2025-06-22", "customer":"Petter"}
>R00000008={"id":"R00000008", "date": "2025-06-23", "customer":"Seb"}
>R00000009={"id":"R00000009", "date": "2025-06-24", "customer":"Joonas"}
>R00000010={"id":"R00000010", "date": "2025-06-25", "customer":"Vesa"}

Every message that you add to the topic should immediately appear in the Grid component in our ReserverationsView.

Reservations

Summary

Using Spring Kafka and Spring Boot’s auto-configuration capabilities makes it easy to subscribe to a Kafka topic. Displaying the topic messages in a UI is also easy thanks to Hilla’s excellent support for reactive browser callable services and the shown subscription mechanism in the frontend.

The next article of this series will show how to create a new Kafka message from a user input in a Hilla frontend.

You can find the source code of the shown example Hilla app at GitHub.

Image Credits: