Gmail to Google Apps Email Migration (part 2)

I promised an Amazon EC2 AMI with imapsync and a few Gmail migration scripts. See the Gmail to Google Apps Email Migration post for background information. I’ll jump right in with a procedure to get you started with your own migration:

Run the instance: ami-01789d68

ec2-run-instances ami-01789d68 -k your-own-keypair

Use your key to authenticate through SSH using the username “gimapsync”. See Running an Instance in the Amazon EC2 Getting Started Guide for more information.
Edit the ‘run-imapsync-justfoldersizes’ script:
```
nano -w run-imapsync-justfoldersizes
```
- Change alias@gmail.com to your ‘from’ email address
- Change alias@google-apps-domain.com to your ‘to’ email address
Edit the ‘run-imapsync-all’ script the same way
Edit the ‘run-imapsync-folder’ script the same way
Create passfile1, which should contain the password for your ‘from’ email account
```
nano -w passfile1
```
Create passfile2, which should contain the password for your ‘to’ email account
```
nano -w passfile2
```
Test it out with
```
./run-imapsync-justfoldersizes
```
- It should connect to both accounts and list each imap folder and its size
If you are feeling lucky, just run
```
./run-imapsync-all
```
Otherwise, go one folder at a time with commands like:
```
./run-imapsync-folder "[Gmail]/Sent Mail"
```

Feel free to edit the run-imapsync* scripts to fit your needs. There are many command-line features of imapsync that I have not made use of here. You may be interested in the ‘--maxage N’ parameter, for example. It only transfers email that are newer than N days old.

Let me know how it goes!

Comments

10:36 PM, Apr 25, 2008 clark

Wow. Thanks for writing this up and setting up the EC2 image. This was a life saver. Still running the conversion, but it seems to be going off without a hitch!

5:09 PM, Apr 28, 2008 "Cowboy" Ben Alman

You need to be sure that when editing the command-lines, you not only change "alias@gmail.com" to your old email and "alias@google-apps-domain.com" to your new email, but that you also change "alias\@gmail\.com" and "alias\@google-apps-domain\.com" as well. It might be a good idea, in the future, to handle the escaped and non-escaped versions programmatically, and store the email address in shell variables (to make editing them easier) Awesome job though, thanks!! - Cowboy

12:16 PM, Jun 8, 2008 Scott Mottonen

Thanks for setting up this image. Other than the problem of imapsync dying periodically during a long transfer, it works very well! Getting imapsync to run on my own Mac wasn't going well. The ec2 image saved me a lot of time.

8:36 PM, Jun 28, 2008 ff

Hi, would you mind sharing the scripts and your version of imapsync? I have the problem that if I sync my Gmail account with my GApps account some of the attachments get broken. Somehow there seem to be some missing bytes at the end of an email... Thanks in advance

4:09 AM, Jul 4, 2008 Sam

ff, the script is here - http://www.thamtech.com/blog/2008/03/29/gmail-to-google-apps-email-migration I'm having a similar problem syncing my Gmail with a local IMAP server, or my ISP's IMAP server. The end of messages are truncated.

3:41 PM, Aug 3, 2008 George Davis

Great tool - thanks for sharing! Have you done any other interesting AMI work recently? G

8:12 PM, Aug 13, 2008 Bruce

Thanks for the tool, but when I try to log in using Putty, I get a message that there is no Authentication Method. Is there a way to set it up (without a key pair), but using a default username and password? please?

7:37 AM, Aug 14, 2008 Tyler

Bruce, I'm sorry to hear you are having trouble using a key pair. Setting up the AMI and publishing a default username and password is a bad security practice and could result in someone finding, publishing, and/or using YOUR gmail passwords. Take a look at the "Running an Instance" section (http://docs.amazonwebservices.com/AWSEC2/2008-02-01/GettingStartedGuide/index.html?running-an-instance.html) of Amazon's Getting Started Guide. It explains how to set up a key pair and has a note about making it work with Putty. Tyler

3:22 PM, Aug 14, 2008 Bruce

Tyler, thanks for the reply. I'm using Elasticfox (a Firefox Extension for Amazon EC2, no need to use command line tools, or java), so in 1 click I run the instance, but I can't SSH. What I meant with defalt username/password was to access the instance. Only I know the DNS address of my instance, and it's only for a few minutes/hours. I would login with a default root/pass, run the scripts, the delete the instance. By the way, I have about 800 MB in Gmail.

4:08 AM, Aug 16, 2008 Ami

FWIW, http://google.com/support/a/bin/answer.py?answer=61369 seems to be working fine for me migrating from an @gmail.com account to a GAFYD account with no external programs/processes (all google-hosted).

3:50 PM, Aug 25, 2008 Bruce

I ended up following these steps http://glomerate.wordpress.com/2008/08/20/migrating-from-gmail-to-google-apps/

2:33 AM, Oct 14, 2008 Waldo Jaquith

Well, that was very useful. (And my first foray into AWS.) Thank you for providing this.

6:41 PM, Dec 2, 2008 Larry McJunkin

Any ideas for moving back from Google Apps Gmail to Regular Gmail and keeping all these things intact? I'm tired of waiting for Google Apps to include Google Labs and Themes and they're more important to me than Google Apps. Thanks. Larry

11:12 PM, Dec 2, 2008 Tyler

Larry, I think the same procedure should in reverse without any changes. Just use your Google apps login as the source and your @gmail.com login as the destination.

7:07 PM, Dec 25, 2008 Hamden

Awesome job!!

2:28 AM, Feb 23, 2009 Oscar

Thank you for sharing this, the main thing keeping me from switching my email to my domain name was leaving all the old emails behind, and this has solved my problem big time!

10:42 AM, Apr 8, 2009 Stuart

Thanks for this, I had similar requirements for email import and this was almost spot on. I imported label by label, which meant that about 400 out of 7500 emails weren't imported.. however I think it's a reasonable trade off given how little I go back through all that old mail. :)

1:19 PM, Apr 9, 2009 Dave C

Tyler, Thanks! I also had to run the script several times in order for it to migrate ALL my emails. And also had trouble getting PuTTY to work properly, until I realized I had to log in with the "GIMAPSYNC" login. Once I changed the login (and key pair, just to be consistant), it all worked fine. Thanks again, Dave