You can create a backup of a basic Looker installation simply by making a copy of the Looker user’s home directory (including all normal and hidden subdirectories). This may be accomplished via scp, rsync, or any standard backup application. Similarly, restoring a basic Looker installation just requires restoring the files and starting up Looker.
In some configurations, including clustered environments, Looker uses an external MySQL database for application settings, user accounts, and other data. In this case we recommend creating a backup of the MySQL database, in addition to the Looker home directory.
We highly recommend creating these backups on a daily basis. We also recommend performing a test restoration once per quarter.
As an added convenience, Looker uploads a backup of its internal database to an Amazon S3 bucket on a daily basis, if possible (see below). However, this is not a replacement for creating your own, local backups.
The standard subdirectories under the Looker user’s home directory (usually /home/looker) are described below.
|Directory||Backup Required||Change Frequency||Description|
|.ssh||Yes||Rare||ssh keys used for authenticating to Git|
|looker/.backup||No||Daily||Temporary directory for S3 backup files|
|looker/.cache||No||Frequent||Temporary cache files|
|looker/.db||Yes||Frequent||Looker internal database|
|looker/.snapshots||No||On Upgrade||A backup copy of the Looker jar and .db directory are stored here during upgrades|
|looker/.ssl||Maybe||Rare||Self-signed SSL certificates (see note below)|
|looker/log||Maybe||Frequent||Log files; only needed if required by your retention policies|
|looker/models||No||Variable||Production models, copied from the source repository (usually GitHub)|
|looker/models-user-*||Yes||Variable||Each user’s development models are stored in separate directories with the user’s ID number|
SSL Note: By default the SSL directory only contains a self-signed SSL cert, which does not need to be retained. However, if you store any additional files in this directory - such as SSL certificates signed by a certificate authority - this directory should be added to your backup.
Files outside of the Looker home directory, which should be added to your backup, are:
|Directory||Backup Required||Change Frequency||Description|
|/etc/init.d/looker||Yes||Rare||System startup script for Looker|
|SSL Certificates||Yes||Rare||If you are using SSL certificates, ensure all required files are included|
Although it doesn’t typically cause problems, some customers have reported issues if they include the
looker/.db/looker.lckfile in their backups. You can safely exclude this file if necessary.
Creating a Looker Backup
You can create a Looker backup with any standard backup application, as well as with command line tools such as rsync.
It is best for the backup process to run when the application is as lightly used as possible. In addition to normal user interaction, consider the times that scheduled Looks might be running, derived tables might be re-building, etc.
Clustered Lookers store their application configuration, user accounts, and other data in an external MySQL database. We recommend creating a backup of this database at the same time as the Looker application. See the MySQL documentation for more details on how to backup MySQL databases.
Automatic Amazon S3 Backups
Looker cannot create S3 backups if it is using an external database for storing its configuration (such as in clustered environments), or if it is unable to send network traffic to S3 and the Looker license server (see outbound port requirements).
Looker automatically creates an encrypted backup, once per day, of all the information stored in its internal database. This backup includes:
- User accounts
- User roles
- Application settings
This backup does NOT include:
- LookML files. LookML files which have been commited and pushed to production will be stored in the source repository (usually GitHub). LookML files which have not been pushed are stored only on the Looker host’s local filesystem.
- The Looker startup script
- SSL certificates
- SSH keys
- Files in the .tmp directory
- The Looker cache
Using a Custom Bucket for Backup
By default, all Looker instances send their backups to a Looker S3 bucket. You can configure Looker to back up to your own S3 bucket instead.
Create an S3 bucket in your own AWS account, and then create an account that has access to that bucket. Looker recommends that each user with access to the bucket be given the minimum set of permissions they need.
Looker needs the
PutObject permission for the bucket. Here is a sample IAM policy with the minimum set of permissions:
There is no need to add a bucket policy or CORS configuration to the S3 bucket. However, we highly recommend that the bucket permissions not be set to
To configure Looker to use a custom S3 bucket, go to the Looker Admin section and select the Backup page. Choose Custom S3 bucket, and provide the bucket name, bucket region, S3 key, and S3 secret.
Looker does not support the forward slash character (
/) in custom S3 bucket names. Looker backups stored in a custom S3 bucket must be organized in a flat structure.
Generating a Keystore-Independent Backup
A customer-hosted installation that has migrated to AES-256 GCM encryption and is not using AWS KMS can use this procedure to create a backup of their Looker instance that is independent of their local Customer Master Key (CMK). This provides a method for a self-hosted customer to move to a Looker-hosted installation without having to provide their CMK, or to move a customer-hosted installation to a new host that does not have access to the same local keystore.
To create a keystore-independent backup:
cd looker ./looker stop
If Looker is clustered, make sure to stop every node before proceeding.
Ensure the environment variable
LKR_MASTER_KEY_FILEis set to the path of your CMK file:
Generate a new key file that will be used to recrypt the Key Encryption Key (KEK):
./looker generate_keyfile_for_backup <key_file_name>
<key_file_name>is the name you want to use for the file that Looker will create and then use to write the new key.
The contents of the new key file will look like:
Where the value for
dbmkis a Base64 encoding 256 bit encryption key and
backup_uidis a unique name used when saving the key to the database.
Use the new key file to create a new key entry in Looker’s internal database:
./looker keystore_independent_recrypt <key_file_name>
<key_file_name>is the key file created above.
This decrypts the KEK in the internal database using the CMK, then re-encrypts the KEK with the new key and persists that encrypted value to the database.
Create a backup of Looker using your regular backup method.
To restore this keystore-independent backup, you will need the new key file created above.
If you need to restore a Looker backup, see our Restoring Backups page.
After you have set up backups you’re ready to ensure Looker can access necessary services.