How to Secure S3 Objects with Amazon S3 Encryption
Many companies store data in cloud storage. Cloud storage services are popular today due to their great reliability and high availability, two very important factors for business. Amazon is one of the top players providing public cloud services on the market. These services are called Amazon Web Services (AWS). Amazon provides a high level of security, including an encrypted network connection used to access files and services. Encryption reduces the risk of data interception over the network. Data is crucial for every company and every user. Nobody wants their data to be lost, corrupted or stolen.
For this reason, Amazon provides encryption options for storing data on its different cloud storage services. These cloud storage options include EBS volumes, a high-performance storage for virtual machines (instances), and Amazon S3, a cloud storage service developed to store backups, archives, application files, and other data. You can encrypt Amazon S3 buckets and the files stored in the buckets by using AWS encryption options. This blog post covers Amazon S3 encryption including encryption types and configuration.
About Data Encryption
Data encryption is a process for securing data by encoding information. Data is encoded using a password or an encryption (cypher) key and special encryption algorithms. The encrypted data can then be accessed by using the correct password or encryption (decryption) key. Data encryption is used to protect digital data confidentiality even if an unauthorized person gains logical or physical access to that data. If an unauthorized person gets access to the encrypted data, the data is unreadable without the key or password. Modern encryption algorithms make it difficult and almost impossible to crack a long encryption key or a complex password. Different encryption algorithms can be used, for example, AES, 3DES, RSA, Blowfish, and so on. The main types of cryptography are symmetric-key cryptography and asymmetric-key cryptography.
Amazon recommends the use of S3 encryption when storing data in Amazon S3 buckets. The first reason for this recommendation is security. Encryption increases the level of security and privacy. However, there is another reason for why data stored in the cloud should be encrypted. Amazon stores data of users from different countries. Sometimes a country can request data be submitted for an investigation if a client or an organization is suspected of violating the law. However, Amazon must respect the license agreement and laws of other countries (countries whose citizens are Amazon customers) and a conflict can occur.
If a user’s data is encrypted and Amazon doesn’t have the encryption keys, the user’s data cannot be provided to third party organizations or persons (even if the encrypted data is provided, it is a useless and unreadable set of bits). Imagine a situation in which the USA requests data from a European Amazon customer for investigation. What to do in this case? As you may already know, the personal data of European citizens is protected by the General Data Protection Regulation (GDPR). Read more about it in the white paper EU General Data Protection Regulation.
Amazon S3 Encryption Types
How does S3 encryption work? Amazon provides several encryption types for data stored in Amazon S3. Is S3 encrypted? By default, data stored in an S3 bucket is not encrypted, but you can configure the AWS S3 encryption settings.
You should define which encryption method to use after answering the following questions:
- Who encrypts and decrypts the data?
- Who stores the secret key?
- Who manages the secret key?
Let’s look at the available AWS encryption methods for S3 objects stored in a bucket.
Server-side encryption
Server-Side Encryption (SSE) is the simplest data encryption option. All heavy encryption operations are performed on the server side in the AWS cloud. You send raw (unencrypted) data to AWS and then data is encrypted on the AWS side when recorded on the cloud storage. When you need to get your data back, Amazon reads the encrypted data, decrypts the needed data on the Amazon server side, and then sends the unencrypted data to you over the network. This process is transparent for end-users.
SSE-S3 is the simplest method – the keys are managed and handled by AWS to encrypt the data you have selected. You cannot see the key directly or use this key manually to encrypt or decrypt the data. AES-256 is used as the encryption algorithm. AES (Advanced Encryption Standard) is a symmetric block cypher, with 256 bit being the cryptographic key length. If you fully trust AWS, use this S3 encryption method.
SSE-KMS is a slightly different method from SSE-S3. AWS Key Management Service (KMS) is used to encrypt S3 data on the Amazon server side. The data key is managed by AWS, but a user manages the customer master key (CMK) in AWS KMS. The advantages of using the SSE-KMS encryption type are user control and audit trail.
With SSE-C, keys are provided by a customer and AWS doesn’t store the encryption keys. The provided key is passed in to AWS to handle each request related to data encryption or decryption. A user must ensure the safety of the keys. S3 data encryption is performed on the AWS server side. Only an HTTPS connection can be used (not HTTP).
Client-side encryption
When using S3 client-side encryption, the client is responsible for all encryption operations. In this case, data is not encrypted by AWS but rather it is encrypted on the user’s side. Data encrypted in the user’s datacenter is uploaded directly to AWS. Two options are provided for S3 client-side encryption – a master key can be stored on the client side or on the server side. If a master key is stored on the client side, the client takes full responsibility for encryption. The advantage of this approach is that Amazon never knows the encryption keys of the user and data is never stored on Amazon servers in an unencrypted state. A user encrypts data before sending data to Amazon S3 and decrypts data after retrieving it from Amazon S3.
What is Amazon S3 encryption client?
The Amazon S3 encryption client is used to encrypt data on the user’s side. A one-time encryption key is randomly generated and is used for data encryption on a per-object level, meaning that there can be encrypted and unencrypted objects in the same Amazon S3 bucket. AmazonS3EncryptionClient is a public class for AWS SDK. You can use AWS SDK for Java, C++, Python, .NET and other supported programming languages to create your own applications that work with Amazon S3 and can be used to encrypt data sent to S3 and to decrypt data received from S3 on the client side.
How to Configure AWS S3 Encryption?
- Log into the web interface of AWS. Your account must have enough permissions to edit S3 settings.
- Go to the Amazon S3 page (the link may differ depending on your region and account):
https://s3.console.aws.amazon.com/s3/home
- Select your bucket or create a new bucket for which you want to configure encryption settings.
- On the page with the bucket settings, click the Properties tab and then click Default encryption.
- The encryption settings are now open. By default, S3 bucket encryption option is disabled.
- Select the needed option, for example, AES-256. This is server-side encryption with Amazon S3-managed keys (SSE-S3).You can view the bucket policy. Click Save to save the encryption settings for the bucket. The settings will be used as the default S3 encryption settings for objects added to this bucket in the future.
- Click Save.
- Now default encryption is set. All new objects stored in the S3 bucket will be encrypted according to the set configuration. It is recommended that you enable encryption when creating a bucket. You can also enable encryption later at the bucket level. However, if you configure encryption settings later, these setting won’t affect unencrypted files that have been already uploaded to the bucket.
- If you want to select the AWS-KMS encryption, click the appropriate option. In this case, select a key from the drop-down list.
Encryption of custom objects in the bucket
Let’s explore how to encrypt custom objects that have already been uploaded to a bucket and for which encryption settings are set to None.
Open your bucket in the web interface of AWS. You can see your S3 objects in the Overview tab. Click the object (a file or directory) to see the current encryption settings applied to this object. Object properties and permissions are displayed in the pop-up window. As you can see on the screenshot below, the uploaded file is not encrypted (Encryption: None).
Select one object or multiple objects, click Actions and then click Change encryption to change encryption settings for custom objects in your S3 bucket.
In the window that opens, select the needed encryption type, for example, AES-256, and click Save.
In the confirmation window, check the selected objects that will be affected by the new encryption settings and click Change.
Now encryption is set for the selected objects.
Note that after you set the encryption settings for the entire bucket, the files that have been uploaded to the bucket before enabling encryption are left unencrypted. If there is a large number of objects in your S3 bucket, it may be difficult to find and select all unencrypted objects that must be encrypted. To select the unencrypted objects in a bucket with enabled encryption, you can use Amazon S3 Inventory or AWS CLI.
You should identify the unencrypted objects and then you can re-upload those objects to encrypt them with the default S3 bucket encryption level set for the entire bucket. If you don’t want to search for unencrypted S3 objects in your bucket (bucket1, for example), you can create a new bucket (bucket2), copy all files from bucket1 to bucket2, and then copy all files back from bucket2 to bucket1.
You can copy unencrypted objects by rewriting them with the AWS CLI copy command by defining the encryption method, for example, –sse enables SSE-S3 128-bit encryption without creating a new bucket:
aws s3 cp s3://mybucket/myfile.zip s3://mybucket/myfile.zip –sse
After rewriting, the file becomes encrypted. Note that file attributes such as time of file creation and modification are changed after rewriting. Your applications that can read these attributes (date and time of file creation/last modified) and use them to work with files stored in Amazon S3 buckets may not behave as expected after rewriting the files.
If lifecycle management options are enabled for your AWS S3 bucket for cost efficiency, some issues may occur. For example, you can set lifecycle configuration to delete files created or modified 6 months ago. Old files are automatically deleted, less storage space is used in the cloud and you pay less money for cloud storage. When the LastModified timestamp (last modified date/time) is rewritten for an old file that is about to be deleted soon, the lifecycle management feature detects this file as a recently created file that should not be deleted for a long time (for example, for 6 months, as mentioned above). As a result, more files are stored in the bucket leading to higher costs.
If versioning is enabled, a new encrypted version of an object is created. Previous versions of the object are left unencrypted.
Uploading objects via AWS CLI
AWS CLI can be used in other cases when you need to copy files to Amazon S3 or from Amazon S3 with encryption options. You can also use this command line interface to copy objects within one S3 bucket and from one bucket to another. You can set the encryption options for the files that are being uploaded by using the CLI to the bucket in cases when default encryption settings of the bucket and encryption settings that must be used for the files being uploaded are different. Let’s see examples of using AWS CLI commands in bash.
Copy a file from the local machine to the AWS S3 bucket and set server-side encryption (SSE-S3 encryption):
aws s3 cp /directory/file-name s3://bucket-name/file-encrypted –sse AES256
Copy and decrypt a file from AWS S3 to a local disk:
aws s3 cp s3://bucket-name/file-encrypted /directory/file-name
Upload and encrypt the file from a local disk to an S3 bucket by using the SSE-KMS encryption:
aws s3 cp /directory/file-name s3://bucket-name/file-encrypted –sse aws:kms
Download and decrypt a file from an S3 bucket to a local disk:
aws s3 cp s3://bucket-name/file-encrypted /directory/file-name
Conclusion
Amazon S3 encryption helps you protect your data stored in AWS S3 buckets in the cloud, and this is especially important for sensitive data. AWS S3 encryption can be performed on the server side of Amazon and on the client side of a customer. Secret keys can be stored on the server side and client side. If you are thinking about using Amazon S3 encryption for objects you are going to store in S3 buckets, enable encryption when creating a bucket. If you are already using a bucket and objects stored in that bucket are unencrypted, you can enable encryption for those objects. However, in that case, there are a few issues that you need to keep in mind.
Using encryption is a good idea to increase the security level and protect your data against access by third parties. In addition to using AWS encryption, consider performing AWS S3 backup and AWS EC2 backup to enhance the safety of your data.
