Libosmium  2.12.2
Fast and flexible C++ library for working with OpenStreetMap data
Public Types | Public Member Functions | Static Private Member Functions | Private Attributes | List of all members
osmium::memory::Buffer Class Reference

#include <buffer.hpp>

Public Types

enum  auto_grow : bool { auto_grow::yes = true, auto_grow::no = false }
 
using value_type = Item
 
template<typename T >
using t_iterator = osmium::memory::ItemIterator< T >
 
template<typename T >
using t_const_iterator = osmium::memory::ItemIterator< const T >
 
using iterator = t_iterator< osmium::OSMEntity >
 
using const_iterator = t_const_iterator< osmium::OSMEntity >
 

Public Member Functions

 Buffer () noexcept
 
 Buffer (unsigned char *data, size_t size)
 
 Buffer (unsigned char *data, size_t capacity, size_t committed)
 
 Buffer (size_t capacity, auto_grow auto_grow=auto_grow::yes)
 
 Buffer (const Buffer &)=delete
 
Bufferoperator= (const Buffer &)=delete
 
 Buffer (Buffer &&)=default
 
Bufferoperator= (Buffer &&)=default
 
 ~Buffer ()=default
 
void increment_builder_count () noexcept
 
void decrement_builder_count () noexcept
 
uint8_t builder_count () const noexcept
 
unsigned char * data () const noexcept
 
size_t capacity () const noexcept
 
size_t committed () const noexcept
 
size_t written () const noexcept
 
bool is_aligned () const noexcept
 
OSMIUM_DEPRECATED void set_full_callback (std::function< void(Buffer &)> full)
 
void grow (size_t size)
 
size_t commit ()
 
void rollback ()
 
size_t clear ()
 
template<typename T >
T & get (const size_t offset) const
 
unsigned char * reserve_space (const size_t size)
 
template<typename T >
T & add_item (const T &item)
 
void add_buffer (const Buffer &buffer)
 
void push_back (const osmium::memory::Item &item)
 
template<typename T >
ItemIteratorRange< T > select ()
 
template<typename T >
ItemIteratorRange< const T > select () const
 
template<typename T >
t_iterator< T > begin ()
 
iterator begin ()
 
template<typename T >
t_iterator< T > get_iterator (size_t offset)
 
iterator get_iterator (size_t offset)
 
template<typename T >
t_iterator< T > end ()
 
iterator end ()
 
template<typename T >
t_const_iterator< T > cbegin () const
 
const_iterator cbegin () const
 
template<typename T >
t_const_iterator< T > get_iterator (size_t offset) const
 
const_iterator get_iterator (size_t offset) const
 
template<typename T >
t_const_iterator< T > cend () const
 
const_iterator cend () const
 
template<typename T >
t_const_iterator< T > begin () const
 
const_iterator begin () const
 
template<typename T >
t_const_iterator< T > end () const
 
const_iterator end () const
 
 operator bool () const noexcept
 
void swap (Buffer &other)
 
template<typename TCallbackClass >
void purge_removed (TCallbackClass *callback)
 

Static Private Member Functions

static size_t calculate_capacity (size_t capacity) noexcept
 

Private Attributes

std::unique_ptr< unsigned char[]> m_memory
 
unsigned char * m_data
 
size_t m_capacity
 
size_t m_written
 
size_t m_committed
 
uint8_t m_builder_count {0}
 
auto_grow m_auto_grow {auto_grow::no}
 
std::function< void(Buffer &)> m_full
 

Detailed Description

A memory area for storing OSM objects and other items. Each item stored has a type and a length. See the Item class for details.

Data can be added to a buffer piece by piece using reserve_space() and add_item(). After all data that together forms an item is added, it must be committed using the commit() call. Usually this is done through the Builder class and its derived classes.

You can iterate over all items in a buffer using the iterators returned by begin(), end(), cbegin(), and cend().

Buffers exist in two flavours, those with external memory management and those with internal memory management. If you already have some memory with data in it (for instance read from disk), you create a Buffer with external memory management. It is your job then to free the memory once the buffer isn't used any more. If you don't have memory already, you can create a Buffer object and have it manage the memory internally. It will dynamically allocate memory and free it again after use.

By default, if a buffer gets full it will throw a buffer_is_full exception. You can use the set_full_callback() method to set a callback functor which will be called instead of throwing an exception. The full callback functionality is deprecated and will be removed in the future. See the documentation for set_full_callback() for alternatives.

Member Typedef Documentation

◆ const_iterator

A const iterator that can be used to iterate over all OSMEntity objects in a buffer.

◆ iterator

An iterator that can be used to iterate over all OSMEntity objects in a buffer.

◆ t_const_iterator

A const iterator that can be used to iterate over all items of type T in a buffer.

◆ t_iterator

An iterator that can be used to iterate over all items of type T in a buffer.

◆ value_type

Member Enumeration Documentation

◆ auto_grow

Enumerator
yes 
no 

Constructor & Destructor Documentation

◆ Buffer() [1/6]

osmium::memory::Buffer::Buffer ( )
inlinenoexcept

The constructor without any parameters creates an invalid, buffer, ie an empty hull of a buffer that has no actual memory associated with it. It can be used to signify end-of-data.

Most methods of the Buffer class will not work with an invalid buffer.

◆ Buffer() [2/6]

osmium::memory::Buffer::Buffer ( unsigned char *  data,
size_t  size 
)
inlineexplicit

Constructs a valid externally memory-managed buffer using the given memory and size.

Parameters
dataA pointer to some already initialized data.
sizeThe size of the initialized data.
Exceptions
std::invalid_argumentif the size isn't a multiple of the alignment.

◆ Buffer() [3/6]

osmium::memory::Buffer::Buffer ( unsigned char *  data,
size_t  capacity,
size_t  committed 
)
inlineexplicit

Constructs a valid externally memory-managed buffer with the given capacity that already contains 'committed' bytes of data.

Parameters
dataA pointer to some (possibly initialized) data.
capacityThe size of the memory for this buffer.
committedThe size of the initialized data. If this is 0, the buffer startes out empty.
Exceptions
std::invalid_argumentif the capacity or committed isn't a multiple of the alignment.

◆ Buffer() [4/6]

osmium::memory::Buffer::Buffer ( size_t  capacity,
auto_grow  auto_grow = auto_grow::yes 
)
inlineexplicit

Constructs a valid internally memory-managed buffer with the given capacity. Will internally get dynamic memory of the required size. The dynamic memory will be automatically freed when the Buffer is destroyed.

Parameters
capacityThe (initial) size of the memory for this buffer. Actual capacity might be larger tue to alignment.
auto_growShould this buffer automatically grow when it becomes to small?

◆ Buffer() [5/6]

osmium::memory::Buffer::Buffer ( const Buffer )
delete

◆ Buffer() [6/6]

osmium::memory::Buffer::Buffer ( Buffer &&  )
default

◆ ~Buffer()

osmium::memory::Buffer::~Buffer ( )
default

Member Function Documentation

◆ add_buffer()

void osmium::memory::Buffer::add_buffer ( const Buffer buffer)
inline

Add committed contents of the given buffer to this buffer.

Precondition
The buffer must be valid.
No builder can be open on this buffer.

Note that you have to eventually call commit() to actually commit this data.

Parameters
bufferThe source of the copy. Must be valid.

◆ add_item()

template<typename T >
T& osmium::memory::Buffer::add_item ( const T &  item)
inline

Add an item to the buffer. The size of the item is stored inside the item, so we know how much memory to copy.

Note that you have to eventually call commit() to actually commit this data.

Precondition
The buffer must be valid.
Template Parameters
TClass of the item to be copied.
Parameters
itemReference to the item to be copied.
Returns
Reference to newly copied data in the buffer.

◆ begin() [1/4]

template<typename T >
t_iterator<T> osmium::memory::Buffer::begin ( )
inline

Get iterator for iterating over all items of type T in the buffer.

Precondition
The buffer must be valid.
Returns
Iterator to first item of type T in the buffer.

◆ begin() [2/4]

iterator osmium::memory::Buffer::begin ( )
inline

Get iterator for iterating over all objects of class OSMEntity in the buffer.

Precondition
The buffer must be valid.
Returns
Iterator to first OSMEntity in the buffer.

◆ begin() [3/4]

template<typename T >
t_const_iterator<T> osmium::memory::Buffer::begin ( ) const
inline

◆ begin() [4/4]

const_iterator osmium::memory::Buffer::begin ( ) const
inline

◆ builder_count()

uint8_t osmium::memory::Buffer::builder_count ( ) const
inlinenoexcept

◆ calculate_capacity()

static size_t osmium::memory::Buffer::calculate_capacity ( size_t  capacity)
inlinestaticprivatenoexcept

◆ capacity()

size_t osmium::memory::Buffer::capacity ( ) const
inlinenoexcept

Returns the capacity of the buffer, ie how many bytes it can contain. Always returns 0 on invalid buffers.

◆ cbegin() [1/2]

template<typename T >
t_const_iterator<T> osmium::memory::Buffer::cbegin ( ) const
inline

◆ cbegin() [2/2]

const_iterator osmium::memory::Buffer::cbegin ( ) const
inline

◆ cend() [1/2]

template<typename T >
t_const_iterator<T> osmium::memory::Buffer::cend ( ) const
inline

◆ cend() [2/2]

const_iterator osmium::memory::Buffer::cend ( ) const
inline

◆ clear()

size_t osmium::memory::Buffer::clear ( )
inline

Clear the buffer.

No-op on an invalid buffer.

Precondition
No builder can be open on this buffer.
Returns
Number of bytes in the buffer before it was cleared.

◆ commit()

size_t osmium::memory::Buffer::commit ( )
inline

Mark currently written bytes in the buffer as committed.

Precondition
The buffer must be valid.
The buffer must be aligned properly (as indicated by is_aligned().
No builder can be open on this buffer.
Returns
Number of committed bytes before this commit. Can be used as an offset into the buffer to get to the object being committed by this call.

◆ committed()

size_t osmium::memory::Buffer::committed ( ) const
inlinenoexcept

Returns the number of bytes already filled in this buffer. Always returns 0 on invalid buffers.

◆ data()

unsigned char* osmium::memory::Buffer::data ( ) const
inlinenoexcept

Return a pointer to data inside the buffer.

Precondition
The buffer must be valid.

◆ decrement_builder_count()

void osmium::memory::Buffer::decrement_builder_count ( )
inlinenoexcept

◆ end() [1/4]

template<typename T >
t_iterator<T> osmium::memory::Buffer::end ( )
inline

Get iterator for iterating over all items of type T in the buffer.

Precondition
The buffer must be valid.
Returns
End iterator.

◆ end() [2/4]

iterator osmium::memory::Buffer::end ( )
inline

Get iterator for iterating over all objects of class OSMEntity in the buffer.

Precondition
The buffer must be valid.
Returns
End iterator.

◆ end() [3/4]

template<typename T >
t_const_iterator<T> osmium::memory::Buffer::end ( ) const
inline

◆ end() [4/4]

const_iterator osmium::memory::Buffer::end ( ) const
inline

◆ get()

template<typename T >
T& osmium::memory::Buffer::get ( const size_t  offset) const
inline

Get the data in the buffer at the given offset.

Precondition
The buffer must be valid.
Template Parameters
TType we want to the data to be interpreted as.
Returns
Reference of given type pointing to the data in the buffer.

◆ get_iterator() [1/4]

template<typename T >
t_iterator<T> osmium::memory::Buffer::get_iterator ( size_t  offset)
inline

Get iterator for iterating over all items of type T in the buffer.

Precondition
The buffer must be valid.
Returns
Iterator to first item of type T after given offset in the buffer.

◆ get_iterator() [2/4]

iterator osmium::memory::Buffer::get_iterator ( size_t  offset)
inline

Get iterator for iterating over all objects of class OSMEntity in the buffer.

Precondition
The buffer must be valid.
Returns
Iterator to first OSMEntity after given offset in the buffer.

◆ get_iterator() [3/4]

template<typename T >
t_const_iterator<T> osmium::memory::Buffer::get_iterator ( size_t  offset) const
inline

◆ get_iterator() [4/4]

const_iterator osmium::memory::Buffer::get_iterator ( size_t  offset) const
inline

◆ grow()

void osmium::memory::Buffer::grow ( size_t  size)
inline

Grow capacity of this buffer to the given size. This works only with internally memory-managed buffers. If the given size is not larger than the current capacity, nothing is done.

Precondition
The buffer must be valid.
Parameters
sizeNew capacity.
Exceptions
std::logic_errorif the buffer doesn't use internal memory management.
std::invalid_argumentif the size isn't a multiple of the alignment.
std::bad_allocif there isn't enough memory available.

◆ increment_builder_count()

void osmium::memory::Buffer::increment_builder_count ( )
inlinenoexcept

◆ is_aligned()

bool osmium::memory::Buffer::is_aligned ( ) const
inlinenoexcept

This tests if the current state of the buffer is aligned properly. Can be used for asserts.

Precondition
The buffer must be valid.

◆ operator bool()

osmium::memory::Buffer::operator bool ( ) const
inlineexplicitnoexcept

In a bool context any valid buffer is true.

◆ operator=() [1/2]

Buffer& osmium::memory::Buffer::operator= ( const Buffer )
delete

◆ operator=() [2/2]

Buffer& osmium::memory::Buffer::operator= ( Buffer &&  )
default

◆ purge_removed()

template<typename TCallbackClass >
void osmium::memory::Buffer::purge_removed ( TCallbackClass *  callback)
inline

Purge removed items from the buffer. This is done by moving all non-removed items forward in the buffer overwriting removed items and then correcting the m_written and m_committed numbers.

Note that calling this function invalidates all iterators on this buffer and all offsets in this buffer.

For every non-removed item that moves its position, the function 'moving_in_buffer' is called on the given callback object with the old and new offsets in the buffer where the object used to be and is now, respectively. This call can be used to update any indexes.

Precondition
The buffer must be valid.

◆ push_back()

void osmium::memory::Buffer::push_back ( const osmium::memory::Item item)
inline

Add an item to the buffer. This function is provided so that you can use std::back_inserter.

Precondition
The buffer must be valid.
No builder can be open on this buffer.
Parameters
itemThe item to be added.

◆ reserve_space()

unsigned char* osmium::memory::Buffer::reserve_space ( const size_t  size)
inline

Reserve space of given size in buffer and return pointer to it. This is the only way of adding data to the buffer. You reserve the space and then fill it.

Note that you have to eventually call commit() to actually commit this data.

If there isn't enough space in the buffer, one of three things can happen:

  • If you have set a callback with set_full_callback(), it is called. After the call returns, you must have either grown the buffer or cleared it by calling buffer.clear(). (Usage of the full callback is deprecated and this functionality will be removed in the future. See the documentation for set_full_callback() for alternatives.
  • If no callback is defined and this buffer uses internal memory management, the buffers capacity is grown, so that the new data will fit.
  • Else the buffer_is_full exception is thrown.
Precondition
The buffer must be valid.
Parameters
sizeNumber of bytes to reserve.
Returns
Pointer to reserved space. Note that this pointer is only guaranteed to be valid until the next call to reserve_space().
Exceptions
osmium::buffer_is_fullif the buffer is full there is no callback defined and the buffer isn't auto-growing.

◆ rollback()

void osmium::memory::Buffer::rollback ( )
inline

Roll back changes in buffer to last committed state.

Precondition
The buffer must be valid.
No builder can be open on this buffer.

◆ select() [1/2]

template<typename T >
ItemIteratorRange<T> osmium::memory::Buffer::select ( )
inline

◆ select() [2/2]

template<typename T >
ItemIteratorRange<const T> osmium::memory::Buffer::select ( ) const
inline

◆ set_full_callback()

OSMIUM_DEPRECATED void osmium::memory::Buffer::set_full_callback ( std::function< void(Buffer &)>  full)
inline

Set functor to be called whenever the buffer is full instead of throwing buffer_is_full.

The behaviour is undefined if you call this on an invalid buffer.

Precondition
The buffer must be valid.
Deprecated:
Callback functionality will be removed in the future. Either detect the buffer_is_full exception or use a buffer with auto_grow::yes. If you want to avoid growing buffers, check that the used size of the buffer (committed()) is small enough compared to the capacity (for instance small than 90% of the capacity) before adding anything to the Buffer. If the buffer is initialized with auto_grow::yes, it will still grow in the rare case that a very large object will be added taking more than the difference between committed() and capacity().

◆ swap()

void osmium::memory::Buffer::swap ( Buffer other)
inline

◆ written()

size_t osmium::memory::Buffer::written ( ) const
inlinenoexcept

Returns the number of bytes currently filled in this buffer that are not yet committed. Always returns 0 on invalid buffers.

Member Data Documentation

◆ m_auto_grow

auto_grow osmium::memory::Buffer::m_auto_grow {auto_grow::no}
private

◆ m_builder_count

uint8_t osmium::memory::Buffer::m_builder_count {0}
private

◆ m_capacity

size_t osmium::memory::Buffer::m_capacity
private

◆ m_committed

size_t osmium::memory::Buffer::m_committed
private

◆ m_data

unsigned char* osmium::memory::Buffer::m_data
private

◆ m_full

std::function<void(Buffer&)> osmium::memory::Buffer::m_full
private

◆ m_memory

std::unique_ptr<unsigned char[]> osmium::memory::Buffer::m_memory
private

◆ m_written

size_t osmium::memory::Buffer::m_written
private

The documentation for this class was generated from the following file: