Mark Wielaard
Sat Aug 13 14:14:03 GMT 2022


It looks like our public-inbox experiment at is starting to work out.

Currently only I and Simon have access to the inbox account through
ssh, but I think we can automate it enough to not need any manual
intervention unless now lists are added. But please ask if you want to
help with the setup.

I have setup sourceware-vhost-inbox.conf with corresponding
letsencrypt certificate. And public-inbox-nntpd, public-inbox-imapd
and public-inbox-httpd through systemd socket and service files.

So you should be able to access the mailboxes through git mirroring,
https, mbox downloads, atom feeds, nntp and imap.

You can already look at the experimental setup per list, e.g.
individual messages and mbox per thread instructions:
git mirror instructions:
atom feed:
  imap:// (readonly, port 143, any user/pass)
  nntp:// (readonly, port 119)

Note that nntp group names and imap folder names might still change.
All current mailboxes are imported/mirrored as public-inbox-v1-format
but for scalability we will want to import them into
public-inbox-v2-format (this also parallelizes xapian indexing and
uses an sqlite database).

It looks like the inbox user can access the original emails to the
lists before mailman mangles the headers, but it cannot easily see for
which domain (sourceware, gcc, cygwin, ecos, etc.) they are. It would
be nice if we could name the news groups/folders after the primary
domain e.g. inbox.sourceware.elfutils-devel, inbox.gcc.gcc-patches,

The inbox.sourceware.test group at
is a simple mirror of and I will
remove it soon (plus the cronjob that does the mirroring).

Looking at the script I have a few suggestions (I can
make them to the script myself, but don't know if you are currently
editing/running it):

- public-inbox-init should probably use -V2 (see above). You can then
  also use -j JOBS to speed up the import.

- --indexlevel shuld be full to make the Xapian searching more useful
  (this is the default, so you can also not set it). Note that this
  also affects the incremental updating done by public-inbox-mda.

- You want to kill public-inbox-httpd using -SIGHUP so it just reloads
  the new config files. Yo also want to kill the other daemons,
  public-inbox-imapd and public-inbox-nntpd

- The --ng name should be based on the primary domain name (see
  above). I don't know how to determine that easily though. Maybe
  mailman knows, then we can also set the initial ADDRESS properly.

The formail -s public-inbox-mda seems to work well for batch
importing, but is it efficient enough for keeping the importing up to
date? It looks like the last .mbox file is just really big and new
messages are appended at the end, so we would be trying to import all
messages all the ime. And how do we make sure it is triggered when new
messages come in?



More information about the Overseers mailing list