Getting Started

Prerequisites¶

To verify that Apache Kafka is running, run the following command:
```
docker ps | grep kafka
```
If it is not running, then set up Kafka. For instructions, see Kafka Setup.

For Existing Applications¶

Add Karafka to your Gemfile:

# Make sure to install Karafka 2.6, as older lines are no longer actively maintained
bundle add karafka --version ">= 2.6.0"

To install Karafka for both Rails and standalone applications, run the following command:
```
bundle exec karafka install
```
Result: All necessary files and directories are generated:

karafka.rb - the main file where you configure Karafka and define which consumers should consume what topics
app/consumers/example_consumer.rb - an example consumer
app/consumers/application_consumer.rb - the base consumer from which all consumers should inherit

To produce test messages, open the development console (bundle exec karafka console) and enter:

# Works from any place in your code and is thread-safe
# You usually want to produce async. Make sure Kafka is running: if it is not reachable,
# this call blocks for up to `message.timeout.ms` (150 seconds by default) and then raises
Karafka.producer.produce_sync(topic: 'example', payload: { 'ping' => 'pong' }.to_json)

To start consuming messages, run the Karafka server:

bundle exec karafka server

# example outcome
[7616dc24-505a-417f-b87b-6bf8fc2d98c5] Polled 2 messages in 1000ms
[dcf3a8d8-0bd9-433a-8f63-b70a0cdb0732] Consume job for ExampleConsumer on example-0 started
{"ping" => "pong"}
{"ping" => "pong"}
[dcf3a8d8-0bd9-433a-8f63-b70a0cdb0732] Consume job for ExampleConsumer on example-0 finished in 0 ms

Below is the demo of the installation process:

(Optional) To install and configure the Web UI, follow this documentation section.

For New Applications (Starting From Scratch)¶

Create a Gemfile:

# Gemfile
source "https://rubygems.org"

gem "karafka", ">= 2.6.0"
# Needed by the Karafka console on Ruby 4.0+, where irb is a bundled gem
gem "irb"

Run: bundle install
To install Karafka for both Rails and standalone applications, run the following command:
```
bundle exec karafka install
```
Result: All necessary files and directories are generated:
- karafka.rb - the main file where you configure Karafka and where you define which consumers should consume what topics.
- app/consumers/example_consumer.rb - an example consumer.
- app/consumers/application_consumer.rb - the base consumer from which all consumers should inherit.

Run a development console (bundle exec karafka console) to produce messages to this example topic:

# Works from any place in your code and is thread-safe
# You usually want to produce async. Make sure Kafka is running: if it is not reachable,
# this call blocks for up to `message.timeout.ms` (150 seconds by default) and then raises
Karafka.producer.produce_sync(topic: 'example', payload: { 'ping' => 'pong' }.to_json)

To start consuming messages, run the karafka server:

bundle exec karafka server

# example outcome
[7616dc24-505a-417f-b87b-6bf8fc2d98c5] Polled 2 messages in 1000ms
[dcf3a8d8-0bd9-433a-8f63-b70a0cdb0732] Consume job for ExampleConsumer on example-0 started
{"ping" => "pong"}
{"ping" => "pong"}
[dcf3a8d8-0bd9-433a-8f63-b70a0cdb0732] Consume job for ExampleConsumer on example-0 finished in 0 ms

(Optional) To install and configure the Web UI, see Getting Started with the Web UI.

Example applications¶

If you have any problems setting up Karafka or need a ready application to play with, clone our examples repository:

git clone https://github.com/karafka/example-apps ./example_apps

For instructions, see Karafka Example Applications Wiki.

Use cases, edge cases, and usage examples¶

Karafka ships with a full integration test suite that illustrates various use cases and edge cases of working with Karafka and Kafka. For comprehensive understanding of our framework, visit Integrations directory of the Karafka repository.

Sunbelt Computer Software

PL/B Language Development and Support