This past Friday, I went to a MongoDB meetup on backup strategies -- both current and newly available through MongoDB’s new backup service, MMS Backup. The speaker was Kelly Stirman from MongoDB. Andrew from Crashlytics was hosting. I thought the content was quite helpful, so I’m posting my notes here, for all you Mongo users (mongoids? mongods?).
Backup strategy and cost is derived from your Recovery Point Objective (RPO) and Recovery Time Objective (RTO). The RPO is the maximum amount of data that may be lost due to a service interruption. The RTO is the amount of time it takes to recover from a service interruption. These two topics are part of a bigger idea known as business continuity.
There are 3 ways to backup MongoDB data:
MMS Backup service
Backup live or offline
Utilizes DB API
This thrashes the database because everything is flushed to disk, backed up, then the working set is loaded back into memory.
Filterable - you can query a subset of the data to save
Usable on shards, but not recommended.
Generally a filesystem or block level storage snapshot
Loose collection-level granularity - it's an all-db-or-nothing backup.
MMS is a collection of tools and services offered for db-level monitoring and backup. You can use it in the cloud for free, or you can run it on-premise (it’s hardened on the cloud first) if you have a valid enterprise-level subscription.
How it works:
A backup agent is installed on the database server
The backup agent ships the oplog to a hidden replica set member in the MongoDB datacenter
The initial sync rebuilds data in their datacenter then tails the oplog.
Multiple redundant copies of backup data in multiple datacenters (Not AWS)
Transit is encrypted.
Requires 2 factor auth to get the data out (restored)
Point in time restores
Restores are free; you pay for the amount of data stored.
Very close to realtime backup (think replicaset)
Filesystem level snapshots are taken every 6 hours. 6 hour snapshots are stored for 2 days. Daily snapshots are stored for a week. Weekly snapshots are stored for a month. Monthly snapshot is stored for a year.
Oplog is stored for 48 hours. This is what allows the custom point in time recovery by applying the most recent snapshot and then replaying the oplog.
File System snapshots and oplog storage lengths are not configurable today, but will be in the future,
To restore the data, you download the actual db files from a URL or can scp a tar.gz backup
MongoDB recommends using SSD for heavy write performance databases because documents are updated in place. Unless document updates are sequential, then writes will be random.