Today's long path name patch

[Corinna Vinschen <corinna-cygwin at cygwin dot com>]

Hi,

as I threatened the community with today, I've now checked in my patch
which finally enables long path names in Cygwin.  I tested it on XP,
2008, and 2008 x64.  What this patch does:

- It allows to use long path names as arguments to applications.

- It allows to chdir into directories with pathnames > 260 chars.

- It allows to run applications in directories with pathnames > 260
  chars.

- It creates the Windows environment for child processes as Unicode
  environment, thus allowing to define an environment > 32K.  Together
  with an earlier patch to environ.cc, Cygwin processes can read big
  environments on all NT OSes.

- Yesterday I stumbled over a problem with service applications.
  Apparently the default stack for service apps is smaller than for
  desktop apps.  Given the number of 32K or 64K buffers on the stack (in
  contrast to the former 260 byte buffers), it was not possible anymore
  to start service apps using cygrunsrv.  With Chris' helpful input I
  created a new class, tmp_pathbuf, which allows to request thread local
  storage for temporary buffers which replace the stack based buffers by
  buffers created through malloc.  The usage is really simple:

    #include <tls_pbuf.h>

    function_which_needs_path_buffer ()
    {
      tmp_pathbuf tp;
      char *path_buffer = tp.c_get ();
      PWCHAR wide_char_buffer = tp.w_get ();
    }

  The tmp buffers are organized as stack themselves.  Every new buffer
  is only malloced once and reused the next time another buffer is
  requested.  At the end of the function, the destructor of tmp_pathbuf
  allows to reuse the buffers requested in this function.  When the
  thread exits, all these TLS buffers are free'd.  If you ever consider
  a patch which needs path buffers, use the above described method.

There's still an awful lot to do before we can call it a release, and
I'd still be grateful for help.  What needs to be done includes:

- Plain stupid bug fixing.

- Input path arguments from the application are not yet checked for
  PATH_MAX length most of the time.

- A couple of place still use CYG_MAX_PATH.  These need review and
  CYG_MAX_PATH must be converted to NT_MAX_PATH, MAX_PATH or PATH_MAX,
  whatever is the logical choice at that point.

- A couple of places still use path_conv::get_win32() or
  fhandler_base::get_win32_name() functions.  These places should be
  converted to using WCHAR paths and the path_conv::get_nt_native_path
  or path_conv::get_wide_win32_path_len/path_conv::get_wide_win32_path
  as much as possible.
  In the long run, the get_win32 method will add an extra hit for
  conversion from WCHAR to char.

- Create the new path conversion API, convert the old API so that it
  only calls the new API with max output path length set to MAX_PATH.

- The path_conv class still needs to be rewritten so that it
  - stores and handles the native path only as native NT WCHAR or
    UNICODE_STRING path,
  - evaluates paths from left to right per POSIX, to avoid mistreating
    paths crossing directory symlinks in certain cases.
  - can take a directory handle and a relative path name as arguments
    to allow implementing the openat() family of functions with OS
    support (OBJECT_ATTRIBUTES w/ handle and rel-path),

- The POSIX path of mount points is restricted to 256 characters.
  That's because it's used as the value name of a registry value and
  value names are restricted to (surprise!) 256 chars.  A decision
  and, perhaps, coding has to be done:

  - Do we stick to this restriction?
  - Do we introduce a new mount point storage in the registry?
  - Do we drop registry mount points and store mount points in future
    in fstab-like files as, say, /etc/fstab (system mount points) and
    ~/.fstab (user mount points)?

- Creating the openat family of functions.

- Review and fix the utilities  in the utils directory to remove
  path length restrictions.

- Review and change documentation accordingly.  Well, review and
  *fix* documentation is a task which needs a helping hand for a
  long time already.  It contains a lot of really old cruft which
  is partly just wrong.

- Set up a new net release package area for the 1.7 times.  Since
  PATH_MAX changes, a couple of applications which naively use
  static buffers of that size will be broken in future.  We start
  to need the setup tool which uses the new package area.  As we
  already discussed in 2006 and/or 2007, we keep the old package
  area for 9x users for a few years.  Other than that, we should
  start to slowly migrate to the new package area which is supposed
  to be used with 1.7.0 and on NT systems only.


I'm sure I missed something in this list, but the tasks at hand
should already be enough to be busy for quite some time.


Have fun,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat