![]() |
ROOT 6.08/07 Reference Guide |
BidirMMapPipe creates a bidirectional channel between the current process and a child it forks.
This class creates a bidirectional channel between this process and a child it creates with fork().
The channel is comrised of a small shared pool of buffer memory mmapped into both process spaces, and two pipes to synchronise the exchange of data. The idea behind using the pipes at all is to have some primitive which we can block on without having to worry about atomic operations or polling, leaving these tasks to the OS. In case the anonymous mmap cannot be performed on the OS the code is running on (for whatever reason), the code falls back to mmapping /dev/zero, mmapping a temporary file, or (if those all fail), a dynamically allocated buffer which is then transmitted through the pipe(s), a slightly slower alternative (because the data is copied more often).
The channel supports five major operations: read(), write(), flush(), purge() and close(). Reading and writing may block until the required buffer space is available. Writes may queue up data to be sent to the other end until either enough pages are full, or the user calls flush which forces any unsent buffers to be sent to the other end. flush forces any data that is to be sent to be sent. purge discards any buffered data waiting to be read and/or sent. Closing the channel on the child returns zero, closing it on the parent returns the child's exit status.
The class also provides operator<< and operator>> for C++-style I/O for basic data types (bool, char, short, int, long, long long, float, double and their unsigned counterparts). Data is transmitted binary (i.e. no formatting to strings like std::cout does). There are also overloads to support C-style zero terminated strings and std::string. In terms of performance, the former is to be preferred.
If the caller needs to multiplex input and output to/from several pipes, the class provides the poll() method which allows to block until an event occurs on any of the polled pipes.
After the BidirMMapPipe is closed, no further operations may be performed on that object, save for the destructor which may still be called.
If the BidirMMapPipe has not properly been closed, the destructor will call close. However, the exit code of the child is lost in that case.
Closing the object causes the mmapped memory to be unmapped and the two pipes to be closed. We also install an atexit handler in the process of creating BidirMMapPipes. This ensures that when the current process terminates, a SIGTERM signal is sent to the child processes created for all unclosed pipes to avoid leaving zombie processes in the OS's process table.
BidirMMapPipe creation, closing and destruction are thread safe. If the BidirMMapPipe is used in more than one thread, the other operations have to be protected with a mutex (or something similar), though.
End of file (other end closed its pipe, or died) is indicated with the eof() method, serious I/O errors set a flags (bad(), fail()), and also throw exceptions. For normal read/write operations, they can be suppressed (i.e. error reporting only using flags) with a constructor argument.
Technicalities:
Here is a trivial example of a parent and a child talking to each other over a BidirMMapPipe:
When designing your own protocols to use over the pipe, there are a few things to bear in mind:
Definition at line 379 of file BidirMMapPipe.h.
Classes | |
class | PollEntry |
for poll() interface More... | |
Public Types | |
enum | { eofbit = 1, failbit = 2, rderrbit = 4, wrerrbit = 8, badbit = rderrbit | wrerrbit, exceptionsbit = 16 } |
flag bits for partial C++ iostream compatibility More... | |
typedef BidirMMapPipe_impl::BidirMMapPipeException | Exception |
convenience typedef for BidirMMapPipeException More... | |
enum | PollFlags { None = 0, Readable = 1, Writable = 2, ReadError = 4, WriteError = 8, Error = ReadError | WriteError, ReadEndOfFile = 32, WriteEndOfFile = 64, EndOfFile = ReadEndOfFile | WriteEndOfFile, ReadInvalid = 64, WriteInvalid = 128, Invalid = ReadInvalid | WriteInvalid } |
condition flags for poll More... | |
typedef std::vector< PollEntry > | PollVector |
convenience typedef for poll() interface More... | |
typedef std::size_t | size_type |
type used to represent sizes More... | |
Public Member Functions | |
BidirMMapPipe (bool useExceptions=true, bool useSocketpair=false) | |
constructor (forks!) More... | |
~BidirMMapPipe () | |
destructor More... | |
bool | bad () const |
true on I/O error More... | |
size_type | bytesReadableNonBlocking () |
number of bytes that can be read without blocking More... | |
size_type | bytesWritableNonBlocking () |
number of bytes that can be written without blocking More... | |
int | close () |
flush buffers, close pipe More... | |
bool | closed () const |
true if closed More... | |
bool | eof () const |
true if end-of-file More... | |
bool | fail () const |
logical failure (e.g. More... | |
void | flush () |
flush buffers with unwritten data More... | |
bool | good () const |
status of stream is good More... | |
bool | isChild () const |
return if this end of the pipe is the child end More... | |
bool | isParent () const |
return if this end of the pipe is the parent end More... | |
operator bool () const | |
return true if not serious error (fail/bad) More... | |
bool | operator! () const |
return true if serious error (fail/bad) More... | |
BidirMMapPipe & | operator<< (const char *str) |
write a C-style string More... | |
BidirMMapPipe & | operator<< (const std::string &str) |
write a std::string object More... | |
template<class T > | |
BidirMMapPipe & | operator<< (const T *tptr) |
write raw pointer to T to other side More... | |
BidirMMapPipe & | operator<< (BidirMMapPipe &(*manip)(BidirMMapPipe &)) |
I/O manipulator support. More... | |
BidirMMapPipe & | operator>> (char *(&str)) |
read a C-style string More... | |
BidirMMapPipe & | operator>> (std::string &str) |
read a std::string object More... | |
template<class T > | |
BidirMMapPipe & | operator>> (T *&tptr) |
read raw pointer to T from other side More... | |
BidirMMapPipe & | operator>> (BidirMMapPipe &(*manip)(BidirMMapPipe &)) |
I/O manipulator support. More... | |
pid_t | pidOtherEnd () const |
return PID of the process on the other end of the pipe More... | |
void | purge () |
purge buffered data waiting to be read and/or written More... | |
int | rdstate () const |
return flags (end of file, BidirMMapPipe closed, ...) More... | |
size_type | read (void *addr, size_type sz) |
read from pipe More... | |
STREAMOP (bool) | |
C++ style stream operators for bool. More... | |
STREAMOP (char) | |
C++ style stream operators for char. More... | |
STREAMOP (short) | |
C++ style stream operators for short. More... | |
STREAMOP (int) | |
C++ style stream operators for int. More... | |
STREAMOP (long) | |
C++ style stream operators for long. More... | |
STREAMOP (long long) | |
C++ style stream operators for long long. More... | |
STREAMOP (unsigned char) | |
C++ style stream operators for unsigned char. More... | |
STREAMOP (unsigned short) | |
C++ style stream operators for unsigned short. More... | |
STREAMOP (unsigned int) | |
C++ style stream operators for unsigned int. More... | |
STREAMOP (unsigned long) | |
C++ style stream operators for unsigned long. More... | |
STREAMOP (unsigned long long) | |
C++ style stream operators for unsigned long long. More... | |
STREAMOP (float) | |
C++ style stream operators for float. More... | |
STREAMOP (double) | |
C++ style stream operators for double. More... | |
bool | usesPipepair () const |
if BidirMMapPipe uses a pipe pair for communications More... | |
bool | usesSocketpair () const |
if BidirMMapPipe uses a socketpair for communications More... | |
size_type | write (const void *addr, size_type sz) |
wirte to pipe More... | |
Static Public Member Functions | |
static int | debugflag () |
return the current setting of the debug flag More... | |
static BidirMMapPipe & | flush (BidirMMapPipe &pipe) |
for usage a la "pipe << flush;" More... | |
static int | poll (PollVector &pipes, int timeout) |
poll a set of pipes for events (ready to read from, ready to write to, error) More... | |
static BidirMMapPipe & | purge (BidirMMapPipe &pipe) |
for usage a la "pipe << purge;" More... | |
static void | setDebugflag (int flag) |
set the debug flags More... | |
Private Types | |
enum | { TotPages = 16, PagesPerEnd = TotPages / 2, FlushThresh = (3 * PagesPerEnd) / 4 } |
tuning constants More... | |
typedef BidirMMapPipe_impl::Page | Page |
convenience typedef for Page More... | |
Private Member Functions | |
BidirMMapPipe (const BidirMMapPipe &) | |
copy-construction forbidden More... | |
Page * | busypage () |
get a busy page to read data from (may block) More... | |
Page * | dirtypage () |
get a dirty page to write data to (may block) More... | |
int | doClose (bool force, bool holdlock=false) |
close the pipe (no flush if forced) More... | |
void | doFlush (bool forcePartialPages=true) |
perform the flush More... | |
void | feedPageLists (Page *plist) |
"feed" the busy and free lists with a list of pages More... | |
void | markPageDirty (Page *p) |
put on dirty pages list More... | |
BidirMMapPipe & | operator= (const BidirMMapPipe &) |
assignment forbidden More... | |
unsigned | recvpages () |
receive a pages from the other end (may block), queue them More... | |
unsigned | recvpages_nonblock () |
receive pages from other end (non-blocking) More... | |
void | sendpages (Page *plist) |
send page(s) to the other end (may block) More... | |
Static Private Member Functions | |
static unsigned | lenPageList (const Page *list) |
return length of a page list More... | |
static BidirMMapPipe_impl::PagePool & | pagepool () |
return page pool More... | |
static void | teardownall (void) |
cleanup routine - at exit, we want our children to get a SIGTERM... More... | |
static size_type | xferraw (int fd, void *addr, size_type len, ssize_t(*xferfn)(int, void *, std::size_t)) |
transfer bytes through the pipe (reading, writing, may block) More... | |
static size_type | xferraw (int fd, void *addr, const size_type len, ssize_t(*xferfn)(int, const void *, std::size_t)) |
transfer bytes through the pipe (reading, writing, may block) More... | |
Private Attributes | |
Page * | m_busylist |
linked list: busy pages (data to be read) More... | |
pid_t | m_childPid |
pid of the child (zero if we're child) More... | |
Page * | m_dirtylist |
linked list: dirty pages (data to be sent) More... | |
int | m_flags |
flags (e.g. end of file) More... | |
Page * | m_freelist |
linked list: free pages More... | |
int | m_inpipe |
pipe end from which data may be read More... | |
int | m_outpipe |
pipe end to which data may be written More... | |
BidirMMapPipe_impl::Pages | m_pages |
mmapped pages More... | |
pid_t | m_parentPid |
pid of the parent More... | |
Static Private Attributes | |
static int | s_debugflag = 0 |
debug flag More... | |
static std::list< BidirMMapPipe * > | s_openpipes |
list of open BidirMMapPipes More... | |
static pthread_mutex_t | s_openpipesmutex = PTHREAD_MUTEX_INITIALIZER |
protects s_openpipes More... | |
static BidirMMapPipe_impl::PagePool * | s_pagepool = 0 |
pool of mmapped pages More... | |
static unsigned | s_pagepoolrefcnt = 0 |
page pool reference counter More... | |
Friends | |
class | BidirMMapPipe_impl::Page |
page is our friend More... | |
typedef BidirMMapPipe_impl::BidirMMapPipeException BidirMMapPipe::Exception |
convenience typedef for BidirMMapPipeException
Definition at line 385 of file BidirMMapPipe.h.
|
private |
convenience typedef for Page
Definition at line 869 of file BidirMMapPipe.h.
typedef std::vector<PollEntry> BidirMMapPipe::PollVector |
convenience typedef for poll() interface
Definition at line 536 of file BidirMMapPipe.h.
typedef std::size_t BidirMMapPipe::size_type |
type used to represent sizes
Definition at line 383 of file BidirMMapPipe.h.
anonymous enum |
flag bits for partial C++ iostream compatibility
Enumerator | |
---|---|
eofbit | end of file reached |
failbit | logical failure (e.g. pipe closed) |
rderrbit | read error |
wrerrbit | write error |
badbit | general I/O error |
exceptionsbit | error reporting with exceptions |
Definition at line 387 of file BidirMMapPipe.h.
|
private |
tuning constants
Enumerator | |
---|---|
TotPages | pages shared (child + parent) |
PagesPerEnd | pages per pipe end |
FlushThresh | flush threshold |
Definition at line 872 of file BidirMMapPipe.h.
condition flags for poll
Definition at line 507 of file BidirMMapPipe.h.
BidirMMapPipe::BidirMMapPipe | ( | bool | useExceptions = true , |
bool | useSocketpair = false |
||
) |
constructor (forks!)
Creates a bidirectional communications channel between this process and a child the constructor forks. On return from the constructor, isParent() and isChild() can be used to tell the parent end from the child end of the pipe. In the child, all other open BidirMMapPipes are closed.
useExceptions | read()/write() error reporting also done using exceptions |
useSocketpair | use a socketpair instead of a pair or pipes |
Normally, exceptions are thrown for all serious I/O errors (apart from end of file). Setting useExceptions to false will force the read() and write() methods to only report serious I/O errors using flags.
When useSocketpair is true, use a pair of Unix domain sockets created using socketpair instead a pair of pipes. The advantage is that only one pair of file descriptors is needed instead of two pairs which are needed for the pipe pair. Performance should very similar on most platforms, especially if mmap works, since only very little data is sent through the pipe(s)/socketpair.
Definition at line 750 of file BidirMMapPipe.cxx.
BidirMMapPipe::~BidirMMapPipe | ( | ) |
|
private |
copy-construction forbidden
Definition at line 739 of file BidirMMapPipe.cxx.
|
inline |
|
private |
get a busy page to read data from (may block)
Definition at line 1241 of file BidirMMapPipe.cxx.
BidirMMapPipe::size_type BidirMMapPipe::bytesReadableNonBlocking | ( | ) |
number of bytes that can be read without blocking
Definition at line 1306 of file BidirMMapPipe.cxx.
BidirMMapPipe::size_type BidirMMapPipe::bytesWritableNonBlocking | ( | ) |
number of bytes that can be written without blocking
Definition at line 1317 of file BidirMMapPipe.cxx.
int BidirMMapPipe::close | ( | ) |
flush buffers, close pipe
Flush buffers, discard unread data, closes the pipe. If the pipe is in the parent process, it waits for the child.
Definition at line 910 of file BidirMMapPipe.cxx.
|
inline |
|
inlinestatic |
return the current setting of the debug flag
Definition at line 432 of file BidirMMapPipe.h.
|
private |
get a dirty page to write data to (may block)
Definition at line 1252 of file BidirMMapPipe.cxx.
|
private |
close the pipe (no flush if forced)
Definition at line 916 of file BidirMMapPipe.cxx.
|
private |
perform the flush
Definition at line 1270 of file BidirMMapPipe.cxx.
|
inline |
|
inline |
logical failure (e.g.
I/O on closed BidirMMapPipe)
Definition at line 718 of file BidirMMapPipe.h.
"feed" the busy and free lists with a list of pages
plist | linked list of pages |
goes through plist, puts free pages from plist onto the freelist (or sends them to the remote end if they belong there), and puts non-empty pages on plist onto the busy list
Definition at line 1131 of file BidirMMapPipe.cxx.
void BidirMMapPipe::flush | ( | ) |
flush buffers with unwritten data
This forces unwritten data to be written to the other end. The call will block until this has been done (or the attempt failed with an error).
Definition at line 1267 of file BidirMMapPipe.cxx.
|
inlinestatic |
for usage a la "pipe << flush;"
Definition at line 856 of file BidirMMapPipe.h.
|
inline |
status of stream is good
Definition at line 730 of file BidirMMapPipe.h.
|
inline |
return if this end of the pipe is the child end
Definition at line 688 of file BidirMMapPipe.h.
|
inline |
return if this end of the pipe is the parent end
Definition at line 682 of file BidirMMapPipe.h.
|
staticprivate |
return length of a page list
Definition at line 1124 of file BidirMMapPipe.cxx.
put on dirty pages list
Definition at line 1227 of file BidirMMapPipe.cxx.
|
inline |
return true if not serious error (fail/bad)
(if EOF, this is still true)
Definition at line 744 of file BidirMMapPipe.h.
|
inline |
return true if serious error (fail/bad)
Definition at line 750 of file BidirMMapPipe.h.
BidirMMapPipe & BidirMMapPipe::operator<< | ( | const char * | str | ) |
write a C-style string
str | C-style string |
Definition at line 1564 of file BidirMMapPipe.cxx.
BidirMMapPipe & BidirMMapPipe::operator<< | ( | const std::string & | str | ) |
write a std::string object
str | string to write |
Definition at line 1585 of file BidirMMapPipe.cxx.
|
inline |
write raw pointer to T to other side
NOTE: This will not write the pointee! Only the value of the pointer is transferred.
tptr | pointer to be written |
Definition at line 815 of file BidirMMapPipe.h.
|
inline |
I/O manipulator support.
manip | manipulator |
example:
Definition at line 839 of file BidirMMapPipe.h.
|
inlineprivate |
assignment forbidden
Definition at line 864 of file BidirMMapPipe.h.
BidirMMapPipe & BidirMMapPipe::operator>> | ( | char *& | str | ) |
read a C-style string
str | pointer to string (space allocated with malloc!) |
since this is for C-style strings, we use malloc/realloc/free for strings. passing in a NULL pointer is valid here, and the routine will use realloc to allocate a chunk of memory of the right size.
Definition at line 1572 of file BidirMMapPipe.cxx.
BidirMMapPipe & BidirMMapPipe::operator>> | ( | std::string & | str | ) |
read a std::string object
str | string to be read |
Definition at line 1593 of file BidirMMapPipe.cxx.
|
inline |
read raw pointer to T from other side
NOTE: This will not read the pointee! Only the value of the pointer is transferred.
tptr | pointer to be read |
Definition at line 826 of file BidirMMapPipe.h.
|
inline |
I/O manipulator support.
manip | manipulator |
example:
Definition at line 852 of file BidirMMapPipe.h.
|
staticprivate |
return page pool
Definition at line 719 of file BidirMMapPipe.cxx.
|
inline |
return PID of the process on the other end of the pipe
Definition at line 503 of file BidirMMapPipe.h.
|
static |
poll a set of pipes for events (ready to read from, ready to write to, error)
pipes | set of pipes to check |
timeout | timeout in milliseconds |
Timeout can be zero (check for specified events, and return), finite (wait at most timeout milliseconds before returning), or -1 (infinite). The poll method returns when the timeout has elapsed, or if an event occurs on one of the pipes being polled, whichever happens earlier.
Pipes is a vector of one or more PollEntries, which each list a pipe and events to poll for. If events is left empty (zero), all conditions are polled for, otherwise only the indicated ones. On return, the revents fields contain the events that occurred for each pipe; error Error, EndOfFile or Invalid events are always set, regardless of wether they were in the set of requested events.
poll may block slightly longer than specified by timeout due to OS timer granularity and OS scheduling. Due to its implementation, the poll call can also return early if the remote end of the page sends a free page while polling (which is put on that pipe's freelist), while that pipe is polled for e.g Reading. The status of the pipe is indicated correctly in revents, and the caller can simply poll again. (The reason this is done this way is because it helps to replenish the pool of free pages and queue busy pages without blocking.)
Here's a piece of example code waiting on two pipes; if they become readable they are read:
Definition at line 1434 of file BidirMMapPipe.cxx.
void BidirMMapPipe::purge | ( | ) |
purge buffered data waiting to be read and/or written
Discards all internal buffers.
Definition at line 1289 of file BidirMMapPipe.cxx.
|
inlinestatic |
for usage a la "pipe << purge;"
Definition at line 858 of file BidirMMapPipe.h.
|
inline |
return flags (end of file, BidirMMapPipe closed, ...)
Definition at line 706 of file BidirMMapPipe.h.
BidirMMapPipe::size_type BidirMMapPipe::read | ( | void * | addr, |
size_type | sz | ||
) |
read from pipe
addr | where to put read data |
sz | size of data to read (in bytes) |
read may block until data from other end is available. It will return 0 if the other end closed the pipe.
Definition at line 1363 of file BidirMMapPipe.cxx.
|
private |
receive a pages from the other end (may block), queue them
this is an application-level scatter read, which gets the list of pages to read from the pipe. if mmap works, it needs only one read call (to get the head of the list of pages transferred). if we need to copy pages through the pipe, we need to add one read for each empty page, and two reads for each non-empty page.
Definition at line 1066 of file BidirMMapPipe.cxx.
|
private |
receive pages from other end (non-blocking)
like recvpages(), but does not block if nothing is available for reading
Definition at line 1099 of file BidirMMapPipe.cxx.
send page(s) to the other end (may block)
plist | linked list of pages to send |
the implementation gathers the different write(s) whereever possible; if mmap works, this results in a single write to transfer the list of pages sent, if we need to copy things through the pipe, we have one write to transfer which pages are sent, and then one write per page.
Definition at line 1044 of file BidirMMapPipe.cxx.
|
inlinestatic |
set the debug flags
flag | debug flags (if zero, no messages are printed) |
Definition at line 438 of file BidirMMapPipe.h.
BidirMMapPipe::STREAMOP | ( | bool | ) |
C++ style stream operators for bool.
BidirMMapPipe::STREAMOP | ( | char | ) |
C++ style stream operators for char.
BidirMMapPipe::STREAMOP | ( | short | ) |
C++ style stream operators for short.
BidirMMapPipe::STREAMOP | ( | int | ) |
C++ style stream operators for int.
BidirMMapPipe::STREAMOP | ( | long | ) |
C++ style stream operators for long.
BidirMMapPipe::STREAMOP | ( | long | long | ) |
C++ style stream operators for long long.
BidirMMapPipe::STREAMOP | ( | unsigned | char | ) |
C++ style stream operators for unsigned char.
BidirMMapPipe::STREAMOP | ( | unsigned | short | ) |
C++ style stream operators for unsigned short.
BidirMMapPipe::STREAMOP | ( | unsigned | int | ) |
C++ style stream operators for unsigned int.
BidirMMapPipe::STREAMOP | ( | unsigned | long | ) |
C++ style stream operators for unsigned long.
BidirMMapPipe::STREAMOP | ( | unsigned long | long | ) |
C++ style stream operators for unsigned long long.
BidirMMapPipe::STREAMOP | ( | float | ) |
C++ style stream operators for float.
BidirMMapPipe::STREAMOP | ( | double | ) |
C++ style stream operators for double.
cleanup routine - at exit, we want our children to get a SIGTERM...
Definition at line 726 of file BidirMMapPipe.cxx.
|
inline |
if BidirMMapPipe uses a pipe pair for communications
Definition at line 700 of file BidirMMapPipe.h.
|
inline |
if BidirMMapPipe uses a socketpair for communications
Definition at line 694 of file BidirMMapPipe.h.
BidirMMapPipe::size_type BidirMMapPipe::write | ( | const void * | addr, |
size_type | sz | ||
) |
wirte to pipe
addr | where to get data to write from |
sz | size of data to write (in bytes) |
write may block until data can be written to other end (depends a bit on available buffer space). It will return 0 if the other end closed the pipe. The data is queued to be written on the next convenient occasion, or it can be forced out with flush().
Definition at line 1399 of file BidirMMapPipe.cxx.
|
staticprivate |
transfer bytes through the pipe (reading, writing, may block)
Definition at line 999 of file BidirMMapPipe.cxx.
|
inlinestaticprivate |
transfer bytes through the pipe (reading, writing, may block)
Definition at line 933 of file BidirMMapPipe.h.
|
friend |
page is our friend
Definition at line 867 of file BidirMMapPipe.h.
|
private |
linked list: busy pages (data to be read)
Definition at line 901 of file BidirMMapPipe.h.
|
private |
pid of the child (zero if we're child)
Definition at line 907 of file BidirMMapPipe.h.
|
private |
linked list: dirty pages (data to be sent)
Definition at line 903 of file BidirMMapPipe.h.
|
private |
flags (e.g. end of file)
Definition at line 906 of file BidirMMapPipe.h.
|
private |
linked list: free pages
Definition at line 902 of file BidirMMapPipe.h.
|
private |
pipe end from which data may be read
Definition at line 904 of file BidirMMapPipe.h.
|
private |
pipe end to which data may be written
Definition at line 905 of file BidirMMapPipe.h.
|
private |
mmapped pages
Definition at line 900 of file BidirMMapPipe.h.
|
private |
pid of the parent
Definition at line 908 of file BidirMMapPipe.h.
|
staticprivate |
debug flag
Definition at line 894 of file BidirMMapPipe.h.
|
staticprivate |
list of open BidirMMapPipes
Definition at line 888 of file BidirMMapPipe.h.
|
staticprivate |
protects s_openpipes
Definition at line 886 of file BidirMMapPipe.h.
|
staticprivate |
pool of mmapped pages
Definition at line 890 of file BidirMMapPipe.h.
|
staticprivate |
page pool reference counter
Definition at line 892 of file BidirMMapPipe.h.