• LibioVtables

libio vtables

libio is the glibc component which implements the foundations of FILE * streams support (printf and scanf are part of the stdio-common subdirectory.)

Although libio is written in C, its streams are extensible with C++ classes. This document explains how this mechanism works, and outlines how it constrains future libio changes. Note that this mechanism is no longer used by current GCC versions (and their libstdc++ library), and has not been the libstdc++ default since GCC 3.0.

Rationale

In GCC 2.95, you could write code like this:

   1 #include <iostream>
   2 #include <sstream>
   3 
   4 #include <stdio.h>
   5 
   6 int
   7 main()
   8 {
   9   std::stringbuf buf;
  10   fprintf(&buf, "Hello, %s!\n", "world");
  11   std::cout << buf.str();
  12 }

That is, you could pass a pointer to a C++ stream buffer class to a function which expected a FILE *, and the C stdio functions would read from or write to the C++ stream buffer.

There are additional benefits to this approach:

• The old libstdc++ documentation calls this **the cool way**.
• Reduce footprint with code sharing.

• Implement the synchronization between std::cout etc. on the C++ side and stdout etc. on the C side, as required by the C++ standard.

The downside is a strong coupling between glibc and libstdc++, and the loss of optimization opportunities within the glibc stdio implementation.

Implementation details

The following description is based on the libstdc++ implementation in GCC 2.95.1. Later GCC versions added additional complexity because they allowed different libstdc++ implementations which are not based on libio. Current versions of libstdc++ cannot use libio at all (libio support was completely removed from GCC 3.3, and not enabled by default since GCC 3.0).

Starting with GCC 3.0, there is a new C++ ABI for all architectures. The location of the vtable pointer changes. libio does not have support code for this, so libstdc++ compiled with GCC 3.0 and with libio enabled is incompatible with glibc. This means that for vtable compatibility, only architectures which were supported by GCC 2.95 need to be considered.

In addition, since wide/wchar_t support was added to the GCC-internal copy of libio in GCC 3.0, it means that for the wide streams in glibc, there is no requirement to provide vtable layout compatibility. No released GCC version can use this because there is either no support for wide streams, or the C++ ABI is incompatible.

glibc/libio side

struct _IO_FILE is defined in <libio.h>, which is provided by glibc or a separate libio library (and a copy was included in libstdc++ as part of GCC):

struct _IO_FILE {
  int _flags;
  char* _IO_read_ptr;
  char* _IO_read_end;
  char* _IO_read_base;
  char* _IO_write_base;
  char* _IO_write_ptr;
  char* _IO_write_end;
  char* _IO_buf_base;
  char* _IO_buf_end;
  char *_IO_save_base;
  char *_IO_backup_base;
  char *_IO_save_end;
  struct _IO_marker *_markers;
  struct _IO_FILE *_chain;
  int _fileno;
  int _flags2;
  _IO_off_t _old_offset;
  unsigned short _cur_column;
  signed char _vtable_offset;
  char _shortbuf[1];
  _IO_lock_t *_lock;
};

The vtable pointer is part of struct _IO_FILE_plus, defined in libio/libioP.h inside the glibc sources (i.e., not exposed in a public header file):

struct _IO_FILE_plus
{
  _IO_FILE file;
  const struct _IO_jump_t *vtable;
};

The vtable pointer is not at the beginning because struct _IO_FILE is a POD, and the virtual member functions are introduced only in a derived class, struct streambuf (see below).

struct _IO_jump_t is defined as:

struct _IO_jump_t
{
    size_t __dummy;
    size_t __dummy2;
    _IO_finish_t __finish;
    _IO_overflow_t __overflow;
    _IO_underflow_t __underflow;
    _IO_underflow_t __uflow;
    _IO_pbackfail_t __pbackfail;
    _IO_xsputn_t __xsputn;
    _IO_xsgetn_t __xsgetn;
    _IO_seekoff_t __seekoff;
    _IO_seekpos_t __seekpos;
    _IO_setbuf_t __setbuf;
    _IO_sync_t __sync;
    _IO_doallocate_t __doallocate;
    _IO_read_t __read;
    _IO_write_t __write;
    _IO_seek_t __seek;
    _IO_close_t __close;
    _IO_stat_t __stat;
    _IO_showmanyc_t __showmanyc;
    _IO_imbue_t __imbue;
};

The types of most of these members are function pointers. For example, _IO_overflow_t is defined as:

typedef int (*_IO_overflow_t) (_IO_FILE *, int);

This matches the list of virtual functions in the definition of struct streambuf below, so that the C struct above is layout-compatible with vtables generated by the compiler. libio/libioP.h contains further comments about this mechanism. C implementations of _IO_FILE classes define an explicit vtable (for example, _IO_file_jumps). C++ implementations will provide an implicit, compiler-generated vtable object.

To construct _IO_FILE objects, a C constructor function has to be called. For struct _IO_FILE itself, this is the function _IO_init. For struct _IO_FILE_plus (which backs on-disk FILE * streams), the function is called _IO_file_init, and so on. Unlike C++ constructors, they do not overwrite the vtable pointer.

libstdc++ side

In GCC and libstdc++, libio/streambuf.h defines struct streambuf like this:

struct streambuf : public _IO_FILE { // protected??
…
    virtual ~streambuf();
    virtual int overflow(int c = EOF); // Leave public for now
    virtual int underflow(); // Leave public for now
    virtual int uflow(); // Leave public for now
    virtual int pbackfail(int c);
    virtual streamsize xsputn(const char* s, streamsize n);
    virtual streamsize xsgetn(char* s, streamsize n);
    virtual streampos seekoff(streamoff, _seek_dir, int mode=ios::in|ios::out);
    virtual streampos seekpos(streampos pos, int mode = ios::in|ios::out);
    virtual streambuf* setbuf(char* p, int len);
    virtual int sync();
    virtual int doallocate();
    virtual streamsize sys_read(char* buf, streamsize size);
    virtual streamsize sys_write(const char*, streamsize);
    virtual streampos sys_seek(streamoff, _seek_dir);
    virtual int sys_close();
    virtual int sys_stat(void*); // Actually, a (struct stat*)
    virtual int showmanyc();
    virtual void imbue(void *);
…
};

(Later GCC versions involve a typedef __c_cfile.)

The streambuf::streambuf constructor calls _IO_init. The filebuf::filebuf constructor calls _IO_file_init, and so on.

History

libstdc++ used libio from glibc by default in GCC 2.95 (and Red Hat's custom 2.96 version). Starting with GCC 3.0, libio-from-glibc was still available as an option, but no longer the default. There have been multiple soname bumps for libstdc++ since GCC 2.95, so running old programs requires compatibility packages (such as compat-libstdc++-296, which is part of Red Hat Enterprise Linux 6).

ABI exposure

<libio.h> is an installed header file. Some applications use struct _IO_FILE * instead of FILE * to refer to stdio streams. Inline versions (implement as macros) for getc, putc, and similar functions access fields of struct _IO_FILE. The struct offset is compiled directly into applications and is invisible at the ELF layer.

The vtables in libstdc++ are created from scratch. The only ELF symbols reference which are visible from applications (or libraries) which derive directly from _IO_FILE is _IO_init, and from _IO_FILE_plus, _IO_file_init, and so on. These functions are not declared in <libio.h>, and are presumably only exported for the benefit of libstdc++. The libstdc++ implementation calls additional internal, libio-related glibc functions as well, see below.

For libstdc++ versions which use libio from glibc, additional vtables for _IO_FILE can be introduced by deriving classes from the streambuf-related classes provided by libstdc++ (which are part of the C++ standard and intended for this purpose).

Symbols used by `compat-libstdc++-296-2.96-144.el6.i686`

NB: List contains public internal functions (such as those resulting from expansion of macros in <stdio.h>).

_IO_adjust_column@GLIBC_2.0
_IO_default_doallocate@GLIBC_2.0
_IO_default_finish@GLIBC_2.0
_IO_default_pbackfail@GLIBC_2.0
_IO_default_uflow@GLIBC_2.0
_IO_default_xsgetn@GLIBC_2.0
_IO_default_xsputn@GLIBC_2.0
_IO_doallocbuf@GLIBC_2.0
_IO_do_write@GLIBC_2.1
_IO_file_attach@GLIBC_2.1
_IO_file_close@GLIBC_2.0
_IO_file_close_it@GLIBC_2.1
_IO_file_doallocate@GLIBC_2.0
_IO_file_fopen@GLIBC_2.1
_IO_file_init@GLIBC_2.1
_IO_file_open@GLIBC_2.0
_IO_file_overflow@GLIBC_2.1
_IO_file_read@GLIBC_2.0
_IO_file_seek@GLIBC_2.0
_IO_file_seekoff@GLIBC_2.1
_IO_file_setbuf@GLIBC_2.1
_IO_file_stat@GLIBC_2.0
_IO_file_sync@GLIBC_2.1
_IO_file_underflow@GLIBC_2.1
_IO_file_write@GLIBC_2.1
_IO_file_xsputn@GLIBC_2.1
_IO_flockfile@GLIBC_2.0
_IO_flush_all@GLIBC_2.0
_IO_flush_all_linebuffered@GLIBC_2.0
_IO_free_backup_area@GLIBC_2.0
_IO_funlockfile@GLIBC_2.0
_IO_getc@GLIBC_2.0
_IO_getline_info@GLIBC_2.1
_IO_init@GLIBC_2.0
_IO_init_marker@GLIBC_2.0
_IO_link_in@GLIBC_2.0
_IO_marker_delta@GLIBC_2.0
_IO_marker_difference@GLIBC_2.0
_IO_padn@GLIBC_2.0
_IO_peekc_locked@GLIBC_2.0
_IO_proc_close@GLIBC_2.1
_IO_proc_open@GLIBC_2.1
_IO_putc@GLIBC_2.0
_IO_remove_marker@GLIBC_2.0
_IO_seekmark@GLIBC_2.0
_IO_seekoff@GLIBC_2.0
_IO_seekpos@GLIBC_2.0
_IO_setb@GLIBC_2.0
_IO_sgetn@GLIBC_2.0
_IO_sputbackc@GLIBC_2.0
_IO_str_init_readonly@GLIBC_2.0
_IO_str_init_static@GLIBC_2.0
_IO_str_overflow@GLIBC_2.0
_IO_str_pbackfail@GLIBC_2.0
_IO_str_seekoff@GLIBC_2.0
_IO_str_underflow@GLIBC_2.0
_IO_sungetc@GLIBC_2.0
_IO_switch_to_get_mode@GLIBC_2.0
_IO_un_link@GLIBC_2.0
_IO_unsave_markers@GLIBC_2.0
_IO_vfprintf@GLIBC_2.0
_IO_vfscanf@GLIBC_2.0

Symbols referenced by libstdc++2.10-glibc2.2_2.95.4-11woody1_i386

The ABI status column uses the following tags:

• public The symbol is part of the public API (referenced in an installed header file).

• legacy public The symbol was formerly used by a public API in the installed headers, but currently installed headers do not use it.

• private The symbol provides some additional way to access stream functionality; public alternatives are available.

• private helper The symbol is an internal building block for the implementation of stream objects.

• private method The symbol is an exported function implementing a vtable method. In a sense, it is a private helper symbol used in a very specific way.

• private constructor The symbol is a building block for a stream constructor (both for C and C++ implementations). Again, this is a private helper symbol with a specific purpose. Private constructor symbols are used to enable vtable compatibility.

The Task column shows what to do with this symbol.

• compat symbol Turn the symbol into a compat symbol. The implementation may have to be preserved for internal use.

• compat wrapper Turn the symbol into a compat symbol, implement it as a thin wrapper around the internal implementation (which is used within glibc libio). This strategy is used for the private constructor symbols mentioned above, to enable vtable compatibility.

Note that symbols exclusively used for wide-oriented streams are absent from this list.

Interposed symbols

libstdc++ interposes the following libio symbols

• _IO_list_all

• _IO_stderr_, _IO_stdin_, _IO_stdout_

• _IO_stdin_used

• _IO_2_1_stdin_, _IO_2_1_stdout_, _IO_2_1_stderr_

The data structure backing the stdin/stdout/stderr global variables changed size between glibc 2.0 and 2.1, and due to the way ELF binding of global variables works, new variables had to be introduced, with the presence of the _IO_stdin_used symbol as a flag to indicated that the new definitions are referenced by the process image.

_IO_list_all is initialized by libio to refer to _IO_stderr_buf.s.file (instead of one of the stderr symbols).

Other extension mechanisms

The fopencookie function remains fully supported, and is not deprecated.