Dynamic Buffers

This section introduces dynamic buffers—growable storage that adapts to data flow between producers and consumers.

Prerequisites

The Producer/Consumer Model

Dynamic buffers serve as intermediate storage between a producer (typically network I/O) and a consumer (your application code).

The flow:

  1. Producer writes data into the buffer

  2. Buffer grows as needed to accommodate data

  3. Consumer reads and processes data

  4. Buffer releases consumed data

This model decouples production rate from consumption rate—the buffer absorbs variations.

The DynamicBuffer Concept

template<typename T>
concept DynamicBuffer = requires(T& t, std::size_t n) {
    // Producer side
    { t.prepare(n) } -> MutableBufferSequence;
    { t.commit(n) };

    // Consumer side
    { t.data() } -> ConstBufferSequence;
    { t.consume(n) };

    // Capacity
    { t.size() } -> std::same_as<std::size_t>;
    { t.max_size() } -> std::same_as<std::size_t>;
    { t.capacity() } -> std::same_as<std::size_t>;
};

Producer Interface

prepare(n)

Returns mutable buffer space for writing up to n bytes:

auto buffers = dynamic_buf.prepare(1024);  // Space for up to 1024 bytes

The returned space may be larger than requested. The data is not yet part of the readable sequence.

commit(n)

Marks n bytes of prepared space as written and readable:

// After writing data:
dynamic_buf.commit(bytes_written);
// Data is now visible via data()

Typical Producer Pattern

task<> read_into_buffer(Stream& stream, DynamicBuffer auto& buffer)
{
    // Prepare space
    auto space = buffer.prepare(1024);

    // Read into prepared space
    auto [ec, n] = co_await stream.read_some(space);

    if (!ec.failed())
        buffer.commit(n);  // Make data readable
}

Consumer Interface

data()

Returns the readable data as a const buffer sequence:

auto readable = dynamic_buf.data();
// Process readable bytes

consume(n)

Removes n bytes from the front of readable data:

dynamic_buf.consume(processed_bytes);
// Those bytes are no longer in data()

Typical Consumer Pattern

void process_buffer(DynamicBuffer auto& buffer)
{
    auto data = buffer.data();

    while (buffer_size(data) >= message_header_size)
    {
        auto msg_size = parse_header(data);
        if (buffer_size(data) < msg_size)
            break;  // Need more data

        process_message(data, msg_size);
        buffer.consume(msg_size);
        data = buffer.data();  // Refresh after consume
    }
}

Capacity Management

size()

Current number of readable bytes (the length of data()).

max_size()

Maximum allowed size. Attempts to grow beyond this throw or fail.

capacity()

Current allocated capacity. May be larger than size().

DynamicBufferParam

When passing dynamic buffers to coroutines, use DynamicBufferParam for safe parameter handling:

template<typename DB>
concept DynamicBufferParam = DynamicBuffer<std::remove_reference_t<DB>>;

template<DynamicBufferParam Buf>
task<std::size_t> read_until(Stream& stream, Buf&& buffer, char delimiter);

This concept ensures proper handling of lvalues and rvalues, preventing dangling references across suspension points.

Provided Implementations

flat_dynamic_buffer

Linear storage with single-buffer sequences:

#include <boost/capy/buffers/flat_dynamic_buffer.hpp>

flat_dynamic_buffer buffer;
buffer.prepare(1024);
// ... write data ...
buffer.commit(n);

// data() returns a single const_buffer

Advantages:

  • Contiguous memory—good for parsing that needs contiguous data

  • Cache-friendly

Disadvantages:

  • May require copying when buffer wraps or grows

circular_dynamic_buffer

Ring buffer implementation:

#include <boost/capy/buffers/circular_dynamic_buffer.hpp>

circular_dynamic_buffer<1024> buffer;  // Fixed capacity

Advantages:

  • No copying on wrap—head/tail pointers move

  • Fixed memory footprint

Disadvantages:

  • data() may return two buffers (wrapped around end)

  • Fixed capacity

vector_dynamic_buffer

Backed by std::vector<char>:

#include <boost/capy/buffers/vector_dynamic_buffer.hpp>

std::vector<char> storage;
vector_dynamic_buffer buffer(storage);

Adapts an existing vector for use as a dynamic buffer.

string_dynamic_buffer

Backed by std::string:

#include <boost/capy/buffers/string_dynamic_buffer.hpp>

std::string storage;
string_dynamic_buffer buffer(storage);

Useful when you want the final data as a string.

Example: Line-Based Protocol

task<std::string> read_line(Stream& stream)
{
    flat_dynamic_buffer buffer;

    while (true)
    {
        // Prepare space and read
        auto space = buffer.prepare(256);
        auto [ec, n] = co_await stream.read_some(space);
        if (ec.failed())
            throw std::system_error(ec);
        buffer.commit(n);

        // Search for newline in readable data
        auto data = buffer.data();
        std::string_view sv(
            static_cast<char const*>(data.data()), data.size());

        auto pos = sv.find('\n');
        if (pos != std::string_view::npos)
        {
            std::string line(sv.substr(0, pos));
            buffer.consume(pos + 1);  // Include newline
            co_return line;
        }
    }
}

Reference

Header Description

<boost/capy/concept/dynamic_buffer.hpp>

DynamicBuffer concept definition

<boost/capy/buffers/flat_dynamic_buffer.hpp>

Linear dynamic buffer

<boost/capy/buffers/circular_dynamic_buffer.hpp>

Ring buffer implementation

<boost/capy/buffers/vector_dynamic_buffer.hpp>

Vector-backed adapter

<boost/capy/buffers/string_dynamic_buffer.hpp>

String-backed adapter

You have now learned about dynamic buffers for producer/consumer patterns. This completes the Buffer Sequences section. Continue to Stream Concepts to learn about Capy’s stream abstractions.

Edit this Page