8000 GitHub - zioka/librdkafka: The Apache Kafka C/C++ library · GitHub
[go: up one dir, main page]

Skip to content
  • About

    The Apache Kafka C/C++ library

    Resources

    Code of conduct

    Contributing

    Stars

    0 stars

    Watchers

    0 watching

    Forks

     
     

    Latest commit

     

    History

    3,242 Commits

    Folders and files

    NameName
    Last commit message
    Last commit date
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     

    librdkafka - the Apache Kafka C/C++ client library

    Copyright (c) 2012-2019, Magnus Edenhill.

    https://github.com/edenhill/librdkafka

    librdkafka is a C library implementation of the Apache Kafka protocol, providing Producer, Consumer and Admin clients. It was designed with message delivery reliability and high performance in mind, current figures exceed 1 million msgs/second for the producer and 3 million msgs/second for the consumer.

    librdkafka is licensed under the 2-clause BSD license.

    Features

    • High-level producer
    • High-level balanced KafkaConsumer (requires broker >= 0.9)
    • Simple (legacy) consumer
    • Admin client
    • Compression: snappy, gzip, lz4, zstd
    • SSL support
    • SASL (GSSAPI/Kerberos/SSPI, PLAIN, SCRAM, OAUTHBEARER) support
    • Full list of supported KIPs
    • Broker version support: >=0.8 (see Broker version compatibility)
    • Guaranteed API stability for C & C++ APIs (ABI safety guaranteed for C)
    • Statistics metrics
    • Debian package: librdkafka1 and librdkafka-dev in Debian and Ubuntu
    • RPM package: librdkafka and librdkafka-devel
    • Gentoo package: dev-libs/librdkafka
    • Portable: runs on Linux, OSX, Win32, Solaris, FreeBSD, AIX, ...

    Documentation

    NOTE: The master branch is actively developed, use latest release for production use.

    Installation

    Installing prebuilt packages

    On Mac OSX, install librdkafka with homebrew:

    $ brew install librdkafka

    On Debian and Ubuntu, install librdkafka from the Confluent APT repositories, see instructions here and then install librdkafka:

    $ apt install librdkafka-dev

    On RedHat, CentOS, Fedora, install librdkafka from the Confluent YUM repositories, instructions here and then install librdkafka:

    $ yum install librdkafka-devel

    On Windows, reference librdkafka.redist NuGet package in your Visual Studio project.

    For other platforms, follow the source building instructions below.

    Build from source

    Requirements

    The GNU toolchain
    GNU make
    pthreads
    zlib-dev (optional, for gzip compression support)
    libssl-dev (optional, for SSL and SASL SCRAM support)
    libsasl2-dev (optional, for SASL GSSAPI support)
    libzstd-dev (optional, for ZStd compression support)
    

    NOTE: Static linking of ZStd (requires zstd >= 1.2.1) in the producer enables encoding the original size in the compression frame header, which will speed up the consumer. Use STATIC_LIB_zstd=/path/to/libzstd.a ./configure --enable-static to enable static ZStd linking. MacOSX example: STATIC_LIB_zstd=$(brew ls -v zstd | grep libzstd.a$) ./configure --enable-static

    Building

      ./configure
      # Or, to automatically install dependencies using the system's package manager:
      # ./configure --install-deps
      # Or, build dependencies from source:
      # ./configure --install-deps --source-deps-only
    
      make
      sudo make install
    

    NOTE: See README.win32 for instructions how to build on Windows with Microsoft Visual Studio.

    NOTE: See CMake instructions for experimental CMake build (unsupported).

    Usage in code

    1. Refer to the examples directory for code using:
    • Producers: basic producers, idempotent producers
    • Consumers: basic consumers, reading batches of messages
    • Performance tester
    1. Refer to the examples GitHub repo for code connecting to a cloud streaming data service based on Apache Kafka

    2. Link your program with -lrdkafka (C) or -lrdkafka++ (C++).

    Commercial support

    Commercial support is available from Confluent Inc

    Community support

    Only the last official release is supported for community members.

    File bug reports, feature requests and questions using GitHub Issues

    Questions and discussions are also welcome on the Confluent Community slack #clients channel, or irc.freenode.org #apache-kafka channel.

    Language bindings

    Users of librdkafka

    About

    The Apache Kafka C/C++ library

    Resources

    Code of conduct

    Contributing

    Stars

    0 stars

    Watchers

    0 watching

    Forks

    Releases

    Packages

    Contributors

    Languages

    0