Using GitHub Actions to automate your Redpanda-based development

This post was written by Celso Fernandes and originally published on the Redpanda blog.

If you’re already using Redpanda, then you know one of its most alluring draws is its intention to make data streaming development work as simple as possible.

Having scorned complexity, the logical next step was to create an easy and efficient way to automate and test builds that depend on Redpanda. In this post, we’ll show you how to do just that, using the Redpanda GitHub Action.

What are GitHub Actions?

Before digging into how to test our code, it’s worth understanding what a GitHub Action is. According to GitHub’s website, “GitHub Actions makes it easy to automate all your software workflows.”

In our case, we'll focus specifically on the continuous integration workflow so that we can run automated tests on GitHub CI. After you get your code running and working with Redpanda on your local development environment, how do you ensure that your teammates won’t introduce breaking changes to code? Running the automated tests on the CI will ensure it's working as expected.

Sometimes it makes sense to isolate third-party dependencies in the software architecture, but what if we want to test using a real Redpanda instance? That's the main reason why the Action was created: to bring all the Redpanda power to the GitHub CI environment. It makes the test suite faster and more reliable.

Even if you aren’t using Redpanda on your production environment, you can benefit from its drop-in replacement for other Apache Kafka^Ⓡ distributions and take advantage of faster boot times and less RAM usage. The Redpanda GitHub Action means saving build minutes (and CI costs) by using it.

So how do you use it? Let's configure it together.

Local development

Disclaimer: The Ruby language is an entirely arbitrary choice. You can write the code in any language you prefer.

All the code discussed here is available on the Redpanda-action-demo repository.

Here we have a test suite that performs two tests:

1. It publishes to a Redpanda topic
2. It fetches the message

Both tests are supported by a setup that connects to Redpanda using the ruby-kafka gem.

Note: Always use localhost:9092 as your Redpanda address. It works locally, and GitHub CI will spin up the Docker image and bind its port to 9092.

Continuous Integration

To use the Redpanda GitHub Action, you’ll need to configure GitHub CI to run the test. This is configured in .github/workflows/ci.yml. Here are its contents, which we will talk more about below:

# the name of our job
name: CI

# yes, we want to run for all branches and pull requests
on:
  push:
    branches: "*"
  pull_request:
    branches: "*"

# we have just our job `test`
jobs:
  test:
    runs-on: ubuntu-latest
    # here is the main section for us, where we spin up the Redpanda instance
    # using the Redpanda GitHub action,pay attention on the `.with.version` key, we are using _latest_ but you can use any Redpanda version
    # tip: version is exactly the same as the Redpanda docker image
    steps:
      - name: start Redpanda
        uses: Redpanda-data/github-action@v0.1.3
        with:
          version: "latest"
      - uses: actions/checkout@v2
      # below is how we setup ruby and run the tests using `rake`
      - name: Set up Ruby
        uses: ruby/setup-ruby@359bebbc29cbe6c87da6bc9ea3bc930432750108
        with:
          ruby-version: '3.0'
          bundler-cache: true
      - name: Install dependencies
        run: bundle install
      - name: Run tests
        run: bundle exec rake

After committing this file and pushing it to a branch, GitHub CI will run the test suite automatically and give you feedback about the changes on your pull request, showing you if the test failed or was successful.

Using the Redpanda GitHub Action is a straightforward way to get Kafka-based projects tested. It's simple, using just one Docker image with everything you need. There’s no JVM, no Zookeeper^Ⓡ: it's an all-in-one solution. And it's super fast! Redpanda will boot up and be ready to work in a few seconds. It's a huge win in the developer experience.

When compared to a popular Kafka Docker image, the Redpanda GitHub Action offers smaller RAM usage (47MB vs. 465MB) and smaller Docker image size (only 130MB vs. 465MB). Considering GitHub Actions is billed by minutes usage, the smaller the footprint, the cheaper it's to run the tests.

After getting set up and running the tests locally, we now know that everything is working properly and can now publish to our repo and collaborate.

How will you use the Redpanda GitHub Action?

We want to hear how you use your newfound knowledge about the Redpanda Action on Github CI. Join the community Slack to share your ideas and experience, and check out Redpanda’s documentation for information on the other things you can do with Redpanda and your applications.

For more information on the Redpanda GitHub Action, or to report an issue, please see the GitHub repository for the project.

You can also visit the repository for the demo in this article, where you’ll find the code from this article and a sample test suite that uses the Github Action in a real-world scenario.

Happy testing!