Migrate Google Drive Across Domains

Intro

Have you ever needed to migrate files or an entire drive structure from one domain to another? Many people have. Whether you rebrand or just want to share some files with another company quickly, there is a growing need to do this; however, Google does not provide a way to automatically migrate files across domains. I spent the last two years brainstorming and testing methods to do this after the company I worked for rebranded and we suddenly had a need to move 4+ terrabytes of data and thousands of folders across domains. Here is the journey we took to get there.

What didn't work

• Google Takeout: only allows 50gb/day limit and breaks backups into small 2gb-50gb zip files. This would take us months to migrate everything
• Browser: too slow and inevitably will timeout due to web server constraints
• Drive Migrator: had issues with file structures and eventually timed out
• Export Data: converts forms to zip files and export Google Files to their closest Microsoft alternative. This creates issues when uploading to the new domain as it will require each file to be copied (creating thousands of duplicates) back to its Google File version
• Transfer ownership and/or make copies: Too complicated at our scale and would require hundreds to thousands of man hours
• Wifi & File Stream: too slow over wifi (obviously)

We tried everything I could think of to make this work and eventually landed on Google File Stream via Ethernet. When we started this project, Google File Stream was limited to the point it was not a viable option; however, in the last year they've added the missing features we needed. File Stream isn't perfect though. This guide assumes you're using the commercial Google Drive File Stream and not the consumer Backup and Sync solution. Let's dive into how to do this:

The Great Drive Migration

Requirements

1) A dedicated Windows computer to perform the migration. More ram and more CPU cores will help handle all the files coming and going. An SSD is preferred to a HD. The more space available, the more files you can migrate without needing to wait for File Stream to purge local cache. (macOS had issues with file structure, untested on Linux) 2) Download Google File Stream 3) Fast fiber internet, preferrably 100mb up/down or more. Try doing this with anything less and you'll be at it for awhile 4) Ethernet connection directly to router, no throttling, and if possible, preferential network treatment to your device

How to Migrate

1) Share file/folder permissions to a single user. Typically this means a user from your current domain would receive permissions for the destination folder and permissions from your old domain for all files/folders that need to be migrated. Add these files/folders to your My Drive for easy reference 2) Install Google File Stream and login with the account that has access to both domain's files/folders 3) Turn off all Antivirus software (including Windows Defender) 4) Start copying files over in batches. It's recommended to verify a few folders at a time. 5) If all went well and you give File Stream time to finish uploading, the files/folders from the 1st domain should now reside in the 2nd domain! Congrats.

Lessons Learned

• G Suite Admins: You'll need to allow File Stream from your domains. Additionally, you'll need to allow files/folders to be shared externally.
• Antivirus: Antivirus MUST be turned off during this entire process. The antivirus software will try and scan every single incoming file which grinds the migration to a halt or completely stops it.
• Google Files: Google Files cannot be moved via File Stream meaning that your Google Docs, Slides, Sheets, and Forms cannot be moved via this method. Instead, you'll need to move them manually via the browser. I'm assuming this is because File Stream doesn't know how to inherit permissions properly and therefore doesn't allow Google Files to be copied in this manner. When Windows says it can't copy these files, simply click skip and select the checkbox to apply to the following files. Take down a list of failed files via Windows notification center and manually move those via Google Drive in a browser.
• Invalid MS-DOS Function: It's possible you'll run into an issue on large files where Windows will give you the Invalid MS-DOS Function error. If this happens, follow these instructions to resolve. Pairing this fix with disabling antivirus should get rid of this error. You may continue getting this error after fixing these two issues. If so, usually your Antivirus came back on. You can simply click try again and select the checkbox to apply to all following items.
• Keep Checking: Keep checking in on big batches. Windows may come across files and ask what you'd like to do (skip, try again, cancel) to various criteria (size, replace, errors, etc).
• How does File Stream work?: File Stream first downloads files to the computer's HD, then re-uploads them to the new location. These files take up precious space on your computer's HD during transite. Once File Drive verifies the files have correctly been copied over, it will slowly purge the cache from your computer's HD. This means if you have 100's of GB of data to migrate, you may need to take small breaks to allow File Stream to finish uploading and purging files from the HD before you can start moving more to be downloaded.
• How can I make this process faster?: You can fast track your migration if you have faster internet, SSD's instead of HD's, and multiple machines migrating files. Remember that your machine(s) could monopolize your network depending on network rules while you do this. If you are migrating items in a commercial environment, make sure to keep an eye on network utilization.
• Team Drive Sharing: If you are a business using Team Drives, you may be concerned about Team Drive sharing. There is a "trick" to sharing Team Drive folders with others found on this blog.