Moving Email

WARNING

Be careful, especially if moving from mail.cs. I tried the command as listed below, and it assumed EVERYTHING in the file system of my home directory was a mail folder.

So, I ended up with Testing/.login Testing/.ssh/ etc, etc until it finally alerted on a bad name, and I could Ctrl C out of it.

Moving email to Exchange

Here are some notes sent to me from Giles Malet in IST:

Subject:    Fwd: Transferring mail to Exchange
Hi Lawrence,,

Here's an e-mail, minus the attachment, that I sent to Keith McGowan,
when he was doing similar things. I used the `root' accounts in this
case, but otherwise it's similar.

So, if your user is `fred', I'd try a command such as this (replace
artsservices with your server name) on some xhier Solaris box:

$ mailutil transfer -m suffix=-Dup -v \
'{artsservices/tls/novalidate-cert/norsh/authuser=fred/user=fred}'\
'{connect/tls/novalidate-cert/norsh/authuser=fred/user=fred}' 

Hopefully you can test & play around a bit before stepping up to the
line! Please call if you need more info.

gdm


-------- Original Message --------
Subject: Transferring mail from cyrus to Exchange
Date: Wed, 02 Dec 2009 13:59:45 -0500
From: Giles Malet 
To: Keith C McGowan 

Hi Keith,

The command I use to do it is something like this:

$ mailutil transfer -m suffix=-Dup -v \
'{artsservices/tls/novalidate-cert/norsh/authuser=cyrus/user=friedman}'\
'{connect/tls/novalidate-cert/norsh/authuser=!exemadmin/user=friedman}'

This works from a Linux box, and will ask for the source & dest passords
in turn. The mailutil in xhier imap2006 will work too: it can read the
password from a file, but it would use the same password in each case.

The -m tries to work around the stupidity of this command: you can't get
it to create folders that don't exist, and just merge into ones that do.
It has to distinguish between those two cases, and requires separate
invocations to deal with them: the creating new, hopefully non-existent,
folders is a compromise.

The last two fields are source and destination mailboxes. So I auth'd as
your cyrus to get the mail, and as a special service account on Exchange
to put it in place, although auth'ing as the user, and using their
password is another option.

I test by using a destination such as:

'{connect/tls/novalidate-cert/norsh/authuser=!exemadmin/user=gdmalet}Testing/'

.... so then all the mail is put under a sub-tree on my account, which I
can delete if it worked.

Anyhow, we should be able to do something similar if we need to move
students. You could use it to move others to connect too, although I'm
happy to get involved.

I attached the output from the above for the friedman import (use -d -v
to get copious output). I ran the following on it to produce a more
readable report:

$ grep '^{.*=>' /tmp/friedman.dbg | sed -e s'/{[^}]*}//g'

[...]

Cheers,
gdm 

Using mailutil on cscf.cs for our accounts (dump of notes from ST#66259)

@cscf[104]% which mailutil
/software/.admin/bins/maintenance/mailutil


Yup

@cscf[105]% mailutil -v
mailutil version 2007b.13

usage: mailutil command [switches] arguments

Standard switches valid with any command:
        [-d[ebug]] [-v[erbose]] [-u[ser] userid] [--]

Commands:
 check [MAILBOX]
   ;; report number of messages and new messages
 create MAILBOX
   ;; create new mailbox
 delete MAILBOX
   ;; delete existing mailbox
 rename SOURCE DESTINATION
   ;; rename mailbox to a new name
 copy [-rw[copy]] [-kw[copy]] [-ig[nore]] SOURCE DESTINATION
 move [-rw[copy]] [-kw[copy]] [-ig[nore]] SOURCE DESTINATION
   ;; create new mailbox and copy/move messages
 append [-rw[copy]] [-kw[copy]] [-ig[nore]] SOURCE DESTINATION
 appenddelete [-rw[copy]] [-kw[copy]] [-ig[nore]] SOURCE DESTINATION
   ;; copy/move messages to existing mailbox
 prune mailbox SEARCH_CRITERIA
   ;; prune mailbox of messages matching criteria
 transfer [-rw[copy]] [-kw[copy]] [-ig[nore]] [-m[erge] m] SOURCE DEST
   ;; copy source hierarchy to destination
   ;;  -merge modes are prompt, append, or suffix=xxxx


From the man page:
     mailutil transfer copies an entire hierarchy of mailboxes
     from the named source to the named destination.  Mailboxes
     are created on the destination as needed.  Any error in
     copying messages will cause the transfer to stop. 

In the following, I'm going to try copying my "Vendors" mailbox from my maildir account to my Connect account:

$ mailutil transfer -m suffix=-Dup -v \\

'{maildir.cs.uwaterloo.ca/tls/novalidate-cert/norsh/authuser=lfolland/user=lfolland}Vendors/'\\

'{connect.uwaterloo.ca/tls/novalidate-cert/norsh/authuser=lfolland/user=lfolland}Testing/'

Let's see if that works:

@cscf[107]% mailutil transfer -m suffix=-Dup -v \\

'{maildir.cs.uwaterloo.ca/tls/novalidate-cert/norsh/authuser=lfolland/user=lfolland}Vendors/'\\

'{connect.uwaterloo.ca/tls/novalidate-cert/norsh/authuser=lfolland/user=lfolland}Testing/'

[Trying IP address [129.97.152.133]]
{maildir.cs.uwaterloo.ca/imap/user=lfolland/authuser=lfolland} password: *enter maildir password*

[Trying IP address [129.97.128.123]]
[The Microsoft Exchange IMAP4 service is ready.]
warning: Kerberos error: No credentials cache found (try running kinit) for autodiscover.connect.uwaterloo.ca
[Retrying using PLAIN authentication after The AUTH protocol exchange was canceled by the client.]
{autodiscover.connect.uwaterloo.ca/imap/user=lfolland/authuser=lfolland} password: ***enter ADS password ***
Listing mailboxes...
Copying {maildir.cs.uwaterloo.ca:143/imap/tls/novalidate-cert/user="lfolland"}Vendors/SGI
  => {connect.uwaterloo.ca:143/imap/tls/novalidate-cert/user="lfolland"}Testing/Vendors/SGI
[Reusing connection to maildir.cs.uwaterloo.ca/user="lfolland"]
[[UNSEEN 37] First unseen.]
[[HIGHESTMODSEQ 1] Highest]
[CREATE completed.]
{maildir.cs.uwaterloo.ca:143/imap/tls/novalidate-cert/readonly/user="lfolland"}Vendors/SGI [76 message(s)] => {connect.uwaterloo.ca:143/imap/tls/novalidate-cert/user="lfolland"}Testing/Vendors/SGI
warning: Keywords are not supported!
... repeated a total of 76 times ...
warning: Keywords are not supported!
[Ok 76 messages(s)]
[Logging out]
[Microsoft Exchange Server 2007 IMAP4 server signing off.] 

After it was done, I checked both accounts.  The original (maildir) account was untouched - so it is a copy.  The Connect account did not display the new mailbox until I subscribed to it.  Once I did that (and all the subfolders) everything was there with the original status (read/unread) and dates.

So it looks good! 

Example copying email from maildir to Exchange account

Here is the script I setup for Tamer to transfer all of his mail on maildir to a mailbox called "Temporary" on Connect:
(root)[337]% cat copymail
mailutil transfer -m suffix=-Dup -v '{maildir.cs.uwaterloo.ca/tls/novalidate-cert/norsh/authuser=cs-direc/user=cs-direc}' '{connect.uwaterloo.ca/tls/novalidate-cert/norsh/authuser=cs-direc/user=cs-direc}Temporary/' 

An alternate copy uses ImapSync as described in MaildirToConnect

Example copying a folder from mail.cs to maildir.cs

Copying the mail folder "Vendors" which is in the "mail" directory on mail.cs to a folder called "Temporary" on maildir.cs. User is "lfolland" in both cases.

Note that you will be prompted for lfolland's password for both servers

@cscf[108]% cat copymail.sh
#!/bin/sh
mailutil transfer -m suffix=-Dup -v '{mail.cs.uwaterloo.ca/tls/novalidate-cert/norsh/authuser=lfolland/user=lfolland}mail/Vendors' '{maildir.cs.uwaterloo.ca/tls/novalidate-cert/norsh/authuser=lfolland/user=lfolland}Temporary/' 

Example copying all of a user's email from mail.cs to maildir.cs

In the example below, we are copying all of the email for "lfolland" from mail.cs to the folder "oldmail" on "maildir.cs" (all of the mail will still be on mail.cs. They will need to delete it manually when they're ready)

Note that you will be prompted for lfolland's password for both servers

@cscf[108]% cat copymail.sh
#!/bin/sh
mailutil transfer -m suffix=-Dup -v '{mail.cs.uwaterloo.ca/tls/novalidate-cert/norsh/authuser=lfolland/user=lfolland}' '{maildir.cs.uwaterloo.ca/tls/novalidate-cert/norsh/authuser=lfolland/user=lfolland}oldmail/'