libio vtables

libio is the glibc component which implements the foundations of FILE * streams support (printf and scanf are part of the stdio-common subdirectory.)

Although libio is written in C, its streams are extensible with C++ classes. This document explains how this mechanism works, and outlines how it constrains future libio changes. Note that this mechanism is no longer used by current GCC versions (and their libstdc++ library), and has not been the libstdc++ default since GCC 3.0.

Rationale

In GCC 2.95, you could write code like this:

   1 #include <iostream>
   2 #include <sstream>
   3 
   4 #include <stdio.h>
   5 
   6 int
   7 main()
   8 {
   9   std::stringbuf buf;
  10   fprintf(&buf, "Hello, %s!\n", "world");
  11   std::cout << buf.str();
  12 }

That is, you could pass a pointer to a C++ stream buffer class to a function which expected a FILE *, and the C stdio functions would read from or write to the C++ stream buffer.

There are additional benefits to this approach:

The downside is a strong coupling between glibc and libstdc++, and the loss of optimization opportunities within the glibc stdio implementation.

Implementation details

The following description is based on the libstdc++ implementation in GCC 2.95.1. Later GCC versions added additional complexity because they allowed different libstdc++ implementations which are not based on libio. Current versions of libstdc++ cannot use libio at all (libio support was completely removed from GCC 3.3, and not enabled by default since GCC 3.0).

Starting with GCC 3.0, there is a new C++ ABI for all architectures. The location of the vtable pointer changes. libio does not have support code for this, so libstdc++ compiled with GCC 3.0 and with libio enabled is incompatible with glibc. This means that for vtable compatibility, only architectures which were supported by GCC 2.95 need to be considered.

glibc/libio side

struct _IO_FILE is defined in <libio.h>, which is provided by glibc or a separate libio library (and a copy was included in libstdc++ as part of GCC):

struct _IO_FILE {
  int _flags;
  char* _IO_read_ptr;
  char* _IO_read_end;
  char* _IO_read_base;
  char* _IO_write_base;
  char* _IO_write_ptr;
  char* _IO_write_end;
  char* _IO_buf_base;
  char* _IO_buf_end;
  char *_IO_save_base;
  char *_IO_backup_base;
  char *_IO_save_end;
  struct _IO_marker *_markers;
  struct _IO_FILE *_chain;
  int _fileno;
  int _flags2;
  _IO_off_t _old_offset;
  unsigned short _cur_column;
  signed char _vtable_offset;
  char _shortbuf[1];
  _IO_lock_t *_lock;
};

The vtable pointer is part of struct _IO_FILE_plus, defined in libio/libioP.h inside the glibc sources (i.e., not exposed in a public header file):

struct _IO_FILE_plus
{
  _IO_FILE file;
  const struct _IO_jump_t *vtable;
};

The vtable pointer is not at the beginning because struct _IO_FILE is a POD, and the virtual member functions are introduced only in a derived class, struct streambuf (see below).

struct _IO_jump_t is defined as:

struct _IO_jump_t
{
    size_t __dummy;
    size_t __dummy2;
    _IO_finish_t __finish;
    _IO_overflow_t __overflow;
    _IO_underflow_t __underflow;
    _IO_underflow_t __uflow;
    _IO_pbackfail_t __pbackfail;
    _IO_xsputn_t __xsputn;
    _IO_xsgetn_t __xsgetn;
    _IO_seekoff_t __seekoff;
    _IO_seekpos_t __seekpos;
    _IO_setbuf_t __setbuf;
    _IO_sync_t __sync;
    _IO_doallocate_t __doallocate;
    _IO_read_t __read;
    _IO_write_t __write;
    _IO_seek_t __seek;
    _IO_close_t __close;
    _IO_stat_t __stat;
    _IO_showmanyc_t __showmanyc;
    _IO_imbue_t __imbue;
};

The types of most of these members are function pointers. For example, _IO_overflow_t is defined as:

typedef int (*_IO_overflow_t) (_IO_FILE *, int);

This matches the list of virtual functions in the definition of struct streambuf below, so that the C struct above is layout-compatible with vtables generated by the compiler. libio/libioP.h contains further comments about this mechanism. C implementations of _IO_FILE classes define an explicit vtable (for example, _IO_file_jumps). C++ implementations will provide an implicit, compiler-generated vtable object.

To construct _IO_FILE objects, a C constructor function has to be called. For struct _IO_FILE itself, this is the function _IO_init. For struct _IO_FILE_plus (which backs on-disk FILE * streams), the function is called _IO_file_init, and so on. Unlike C++ constructors, they do not overwrite the vtable pointer.

libstdc++ side

In GCC and libstdc++, libio/streambuf.h defines struct streambuf like this:

struct streambuf : public _IO_FILE { // protected??

    virtual ~streambuf();
    virtual int overflow(int c = EOF); // Leave public for now
    virtual int underflow(); // Leave public for now
    virtual int uflow(); // Leave public for now
    virtual int pbackfail(int c);
    virtual streamsize xsputn(const char* s, streamsize n);
    virtual streamsize xsgetn(char* s, streamsize n);
    virtual streampos seekoff(streamoff, _seek_dir, int mode=ios::in|ios::out);
    virtual streampos seekpos(streampos pos, int mode = ios::in|ios::out);
    virtual streambuf* setbuf(char* p, int len);
    virtual int sync();
    virtual int doallocate();
    virtual streamsize sys_read(char* buf, streamsize size);
    virtual streamsize sys_write(const char*, streamsize);
    virtual streampos sys_seek(streamoff, _seek_dir);
    virtual int sys_close();
    virtual int sys_stat(void*); // Actually, a (struct stat*)
    virtual int showmanyc();
    virtual void imbue(void *);

};

(Later GCC versions involve a typedef __c_cfile.)

The streambuf::streambuf constructor calls _IO_init. The filebuf::filebuf constructor calls _IO_file_init, and so on.

History

libstdc++ used libio from glibc by default in GCC 2.95 (and Red Hat's custom 2.96 version). Starting with GCC 3.0, libio-from-glibc was still available as an option, but no longer the default. There have been multiple soname bumps for libstdc++ since GCC 2.95, so running old programs requires compatibility packages (such as compat-libstdc++-296, which is part of Red Hat Enterprise Linux 6).

ABI exposure

<libio.h> is an installed header file. Some applications use struct _IO_FILE * instead of FILE * to refer to stdio streams. Inline versions (implement as macros) for getc, putc, and similar functions access fields of struct _IO_FILE. The struct offset is compiled directly into applications and is invisible at the ELF layer.

The vtables in libstdc++ are created from scratch. The only ELF symbols reference which are visible from applications (or libraries) which derive directly from _IO_FILE is _IO_init, and from _IO_FILE_plus, _IO_file_init, and so on. These functions are not declared in <libio.h>, and are presumably only exported for the benefit of libstdc++. The libstdc++ implementation calls additional internal, libio-related glibc functions as well, see below.

For libstdc++ versions which use libio from glibc, additional vtables for _IO_FILE can be introduced by deriving classes from the streambuf-related classes provided by libstdc++ (which are part of the C++ standard and intended for this purpose).

Symbols used by `compat-libstdc++-296-2.96-144.el6.i686`

NB: List contains public internal functions (such as those resulting from expansion of macros in <stdio.h>).

_IO_adjust_column@GLIBC_2.0
_IO_default_doallocate@GLIBC_2.0
_IO_default_finish@GLIBC_2.0
_IO_default_pbackfail@GLIBC_2.0
_IO_default_uflow@GLIBC_2.0
_IO_default_xsgetn@GLIBC_2.0
_IO_default_xsputn@GLIBC_2.0
_IO_doallocbuf@GLIBC_2.0
_IO_do_write@GLIBC_2.1
_IO_file_attach@GLIBC_2.1
_IO_file_close@GLIBC_2.0
_IO_file_close_it@GLIBC_2.1
_IO_file_doallocate@GLIBC_2.0
_IO_file_fopen@GLIBC_2.1
_IO_file_init@GLIBC_2.1
_IO_file_open@GLIBC_2.0
_IO_file_overflow@GLIBC_2.1
_IO_file_read@GLIBC_2.0
_IO_file_seek@GLIBC_2.0
_IO_file_seekoff@GLIBC_2.1
_IO_file_setbuf@GLIBC_2.1
_IO_file_stat@GLIBC_2.0
_IO_file_sync@GLIBC_2.1
_IO_file_underflow@GLIBC_2.1
_IO_file_write@GLIBC_2.1
_IO_file_xsputn@GLIBC_2.1
_IO_flockfile@GLIBC_2.0
_IO_flush_all@GLIBC_2.0
_IO_flush_all_linebuffered@GLIBC_2.0
_IO_free_backup_area@GLIBC_2.0
_IO_funlockfile@GLIBC_2.0
_IO_getc@GLIBC_2.0
_IO_getline_info@GLIBC_2.1
_IO_init@GLIBC_2.0
_IO_init_marker@GLIBC_2.0
_IO_link_in@GLIBC_2.0
_IO_marker_delta@GLIBC_2.0
_IO_marker_difference@GLIBC_2.0
_IO_padn@GLIBC_2.0
_IO_peekc_locked@GLIBC_2.0
_IO_proc_close@GLIBC_2.1
_IO_proc_open@GLIBC_2.1
_IO_putc@GLIBC_2.0
_IO_remove_marker@GLIBC_2.0
_IO_seekmark@GLIBC_2.0
_IO_seekoff@GLIBC_2.0
_IO_seekpos@GLIBC_2.0
_IO_setb@GLIBC_2.0
_IO_sgetn@GLIBC_2.0
_IO_sputbackc@GLIBC_2.0
_IO_str_init_readonly@GLIBC_2.0
_IO_str_init_static@GLIBC_2.0
_IO_str_overflow@GLIBC_2.0
_IO_str_pbackfail@GLIBC_2.0
_IO_str_seekoff@GLIBC_2.0
_IO_str_underflow@GLIBC_2.0
_IO_sungetc@GLIBC_2.0
_IO_switch_to_get_mode@GLIBC_2.0
_IO_un_link@GLIBC_2.0
_IO_unsave_markers@GLIBC_2.0
_IO_vfprintf@GLIBC_2.0
_IO_vfscanf@GLIBC_2.0

Symbols referenced by libstdc++2.10-glibc2.2_2.95.4-11woody1_i386

The ABI status column uses the following tags:

The Task column shows what to do with this symbol.

Symbol

Subsystem

ABI status

Task

_IO_adjust_column@GLIBC_2.0

private helper

compat symbol

_IO_default_doallocate@GLIBC_2.0

private method

compat symbol

_IO_default_finish@GLIBC_2.0

private method

compat symbol

_IO_default_pbackfail@GLIBC_2.0

private method

compat symbol

_IO_default_uflow@GLIBC_2.0

private method

compat symbol

_IO_default_xsgetn@GLIBC_2.0

private method

compat symbol

_IO_default_xsputn@GLIBC_2.0

private method

compat symbol

_IO_do_write@GLIBC_2.1

private helper

compat symbol

_IO_doallocbuf@GLIBC_2.0

private helper

compat symbol

_IO_file_attach@GLIBC_2.1

private helper

compat symbol

_IO_file_close@GLIBC_2.0

private method

compat symbol

_IO_file_close_it@GLIBC_2.1

private helper

compat symbol

_IO_file_doallocate@GLIBC_2.0

private method

compat symbol

_IO_file_fopen@GLIBC_2.1

private helper

compat symbol

_IO_file_init@GLIBC_2.1

private constructor

compat wrapper

_IO_file_open@GLIBC_2.0

private helper

compat symbol

_IO_file_overflow@GLIBC_2.1

private method

compat symbol

_IO_file_read@GLIBC_2.0

private method

compat symbol

_IO_file_seek@GLIBC_2.0

private method

compat symbol

_IO_file_seekoff@GLIBC_2.1

private method

compat symbol

_IO_file_setbuf@GLIBC_2.1

private method

compat symbol

_IO_file_stat@GLIBC_2.0

private method

compat symbol

_IO_file_sync@GLIBC_2.1

private method

compat symbol

_IO_file_underflow@GLIBC_2.1

private method

compat symbol

_IO_file_write@GLIBC_2.1

private helper

compat symbol

_IO_file_xsputn@GLIBC_2.1

private helper

compat symbol

_IO_flockfile@GLIBC_2.0

private

compat symbol

_IO_flush_all@GLIBC_2.0

private

compat symbol

_IO_flush_all_linebuffered@GLIBC_2.0

private

compat symbol

_IO_free_backup_area@GLIBC_2.0

private helper

compat symbol

_IO_funlockfile@GLIBC_2.0

private

compat symbol

_IO_getc@GLIBC_2.0

public

_IO_getline_info@GLIBC_2.1

private helper

compat symbol

_IO_init@GLIBC_2.0

private constructor

compat wrapper

_IO_init_marker@GLIBC_2.0

private helper

compat symbol

_IO_link_in@GLIBC_2.0

private helper

compat symbol

_IO_marker_delta@GLIBC_2.0

private helper

compat symbol

_IO_marker_difference@GLIBC_2.0

private helper

compat symbol

_IO_padn@GLIBC_2.0

private helper

compat symbol

_IO_peekc_locked@GLIBC_2.0

private

compat symbol

_IO_proc_close@GLIBC_2.1

private method

compat symbol

_IO_proc_open@GLIBC_2.1

private helper

compat symbol

_IO_putc@GLIBC_2.0

public

_IO_remove_marker@GLIBC_2.0

private helper

compat symbol

_IO_seekmark@GLIBC_2.0

private helper

compat symbol

_IO_seekoff@GLIBC_2.0

private helper

compat symbol

_IO_seekpos@GLIBC_2.0

private helper

compat symbol

_IO_setb@GLIBC_2.0

private helper

compat symbol

_IO_sgetn@GLIBC_2.0

private helper

compat symbol

_IO_sputbackc@GLIBC_2.0

private helper

compat symbol

_IO_str_init_readonly@GLIBC_2.0

private constructor

compat wrapper

_IO_str_init_static@GLIBC_2.0

private constructor

compat wrapper

_IO_str_overflow@GLIBC_2.0

private method

compat symbol

_IO_str_pbackfail@GLIBC_2.0

private method

compat symbol

_IO_str_seekoff@GLIBC_2.0

private method

compat symbol

_IO_str_underflow@GLIBC_2.0

private method

compat symbol

_IO_sungetc@GLIBC_2.0

private

compat symbol

_IO_switch_to_get_mode@GLIBC_2.0

private helper

compat symbol

_IO_un_link@GLIBC_2.0

private helper

compat symbol

_IO_unsave_markers@GLIBC_2.0

private helper

compat symbol

_IO_vfprintf@GLIBC_2.0

private

compat symbol

_IO_vfscanf@GLIBC_2.0

private

compat symbol

__ctype_b@GLIBC_2.0

(not libio)

__cxa_finalize@GLIBC_2.1.3

(not libio)

__errno_location@GLIBC_2.0

(not libio)

__gmon_start__

(not libio)

__printf_fp@GLIBC_2.0

private

compat symbol

__underflow@GLIBC_2.0

legacy public

compat symbol

_pthread_cleanup_pop_restore

(not libio)

_pthread_cleanup_push_defer

(not libio)

abort@GLIBC_2.0

(not libio)

cos@GLIBC_2.0

(not libio)

cosh@GLIBC_2.0

(not libio)

exp@GLIBC_2.0

(not libio)

fclose@GLIBC_2.1

(public libio)

fflush@GLIBC_2.0

(public libio)

fileno@GLIBC_2.0

(public libio)

fprintf@GLIBC_2.0

(public libio)

fread@GLIBC_2.0

(public libio)

free@GLIBC_2.0

(not libio)

fseek@GLIBC_2.0

(public libio)

fwrite@GLIBC_2.0

(public libio)

log@GLIBC_2.0

(not libio)

malloc@GLIBC_2.0

(not libio)

memchr@GLIBC_2.0

(not libio)

memcpy@GLIBC_2.0

(not libio)

memmove@GLIBC_2.0

(not libio)

memset@GLIBC_2.0

(not libio)

pthread_create

(not libio)

pthread_getspecific

(not libio)

pthread_key_create

(not libio)

pthread_mutex_lock@GLIBC_2.0

(not libio)

pthread_mutex_unlock@GLIBC_2.0

(not libio)

pthread_once

(not libio)

pthread_setspecific

(not libio)

realloc@GLIBC_2.0

(not libio)

sin@GLIBC_2.0

(not libio)

sinh@GLIBC_2.0

(not libio)

sprintf@GLIBC_2.0

(public libio)

stderr@GLIBC_2.0

public

stdout@GLIBC_2.0

public

strcmp@GLIBC_2.0

(not libio)

sys_nerr@GLIBC_2.1

(not libio)

Note that symbols exclusively used for wide-oriented streams are absent from this list.

Interposed symbols

libstdc++ interposes the following libio symbols

The data structure backing the stdin/stdout/stderr global variables changed size between glibc 2.0 and 2.1, and due to the way ELF binding of global variables works, new variables had to be introduced, with the presence of the _IO_stdin_used symbol as a flag to indicated that the new definitions are referenced by the process image.

_IO_list_all is initialized by libio to refer to _IO_stderr_buf.s.file (instead of one of the stderr symbols).

Other extension mechanisms

The fopencookie function remains fully supported, and is not deprecated.

None: LibioVtables (last edited 2016-06-10 20:35:08 by CarlosODonell)