How to integrate Redpanda with PHP

In this blog, I’m going to show you how to set up Redpanda and use it with PHP.

We will build a very simple producer and consumer example in a Docker setup.

This is an introduction to Redpanda, so if you already have experience with it or similar streaming platforms, you can check out some of our posts on more advanced topics, like this one on consistency, or this one on autotuning.

As you may have heard by now, Redpanda is API-compatible with APache Kafka®, which means that you can leverage the countless client libraries created for Kafka (if you find something that is not supported, reach out to our team on our Slack community).

In this case we will use the PHP extension simple_kafka_client.

Prepare Docker setup

First create a Dockerfile for our PHP container:

FROM php:8.0-cli-alpine3.13

Install packages
RUN apk --no-cache add bash gcc g++ make autoconf && \
    apk add librdkafka librdkafka-dev \
                    --update-cache --repository http://dl-3.alpinelinux.org/alpine/edge/community && \
    pecl install simple_kafka_client && \
    docker-php-ext-enable simple_kafka_client

Then let’s create our docker-compose.yml like follows:

version: '3.7'
services:
  php:
    build:
      context: ./
    tty: true
    working_dir: /src
    volumes:
      - ./src:/src
  redpanda:
    entrypoint:
      - /usr/bin/rpk
      - redpanda
      - start
      - --smp
      - '1'
      - --reserve-memory
      - 0M
      - --overprovisioned
      - --node-id
      - '0'
      - --kafka-addr
      - PLAINTEXT://0.0.0.0:29097,OUTSIDE://0.0.0.0:9097
      - --advertise-kafka-addr
      - PLAINTEXT://redpanda:29097,OUTSIDE://redpanda:9097
      - --check=false
    image: vectorized/redpanda:v21.4.12
    ports:
      - 9097:9097
      - 29097:29097

Create the PHP code examples

Next we create the src folder with our examples. This folder will be mounted into our Docker setup.

mkdir src;cd src

Then let’s create our producer.php in the src folder with the following content:

<?php

declare(strict_types=1);

use SimpleKafkaClient\Configuration;
use SimpleKafkaClient\Message;
use SimpleKafkaClient\Producer;

$conf = new Configuration();
$conf->set('metadata.broker.list', 'redpanda:9097');
$conf->set('compression.codec', 'zstd');

$producer = new Producer($conf);
$topic = $producer->getTopicHandle('php-test-topic');

for ($i = 0; $i < 10; ++$i) {
    $topic->producev(
        RD_KAFKA_PARTITION_UA,
        RD_KAFKA_MSG_F_BLOCK, // will block produce if queue is full
        sprintf('test message-%d',$i),
        sprintf('test-key-%d', $i),
        [
            'some' => sprintf('header value %d', $i)
        ]
    );

    $producer->poll(0);
}

$result = $producer->flush(20000);

echo sprintf('Produced %d messages', $i) . PHP_EOL;

if (RD_KAFKA_RESP_ERR_NO_ERROR !== $result) {
    echo 'Was not able to shutdown within 20s. Messages might be lost!' . PHP_EOL;
}

Next let’s create our consumer.php in the src folder with the following content:

<?php

declare(strict_types=1);

use SimpleKafkaClient\Configuration;
use SimpleKafkaClient\Consumer;

$conf = new Configuration();
$conf->set('group.id', 'php-consumer');
$conf->set('metadata.broker.list', 'redpanda:9097');
$conf->set('auto.offset.reset', 'earliest');
$conf->set('enable.partition.eof', 'true');

$consumer = new Consumer($conf);
$consumer->subscribe(['php-test-topic']);

while (true) {
    $message = $consumer->consume(20000);

    if (RD_KAFKA_RESP_ERR__PARTITION_EOF === $message->err) {
        echo 'Reached end of partition, shutting down' . PHP_EOL;
        break;
    } else if (RD_KAFKA_RESP_ERR__TIMED_OUT === $message->err) {
        echo 'Timed out without receiving a new message, waiting for more messages...' . PHP_EOL;
        continue;
    } else if (RD_KAFKA_RESP_ERR_NO_ERROR !== $message->err) {
        echo kafka_err2str($message->err) . PHP_EOL;
        continue;
    }

    echo sprintf(
            'Read message with key:%s payload:%s topic:%s partition:%d offset:%d',
            $message->key,
            $message->payload,
            $message->topic_name,
            $message->partition,
            $message->offset
        ) . PHP_EOL;
}

Running the examples

Now let’s put it all together and run our examples:

docker-compose up -d
docker-compose exec php php producer.php
docker-compose exec php php consumer.php

What will you build next?

As you can see, it is a breeze to setup PHP integration with Redpanda. Try it yourself, there are endless use cases - what we built here was just the simplest of examples. To learn more about Redpanda and what you can do with it, check out our other tutorials here or join our Slack community to interact directly with our engineers.

This blog was contributed by Nick Chiu.