¶ What is Object Lock?
Object Lock is a framework that was developed by AWS that allows objects from being deleted or overwritten for a fixed amount of time or indefinitely. Object Lock uses a write-once-read-many (WORM) model to store objects. You can use Object Lock to help meet regulatory requirements that require WORM storage, or to add another layer of protection against object changes or deletion.
As of today, the object-lock framework within OSE is currently integrated with AWS, Cloudian HyperStore, and Dell ECS (supported from ECS version 3.6).
¶ How Does VMware Cloud Director Integrate the Object-Lock Framework?
In this guide and example, we will illustrate the object-lock integration in relation to Cloudian Hyperstore.
- The Object Storage Extension (OSE) leverages the object-lock API
PutObjectLockConfigurationwhich allows OSE to translate the API call directly to the underlying object storage provider to create an object-lock bucket. - OSE leverages (2) types of retentions modes with configurable retention periods (
daysoryears).- Governance Mode = Governance mode can be deactivated by storage accounts that have specific IAM permissions.
- With this mode, you as the bucket owner can remove the lock from a locked object before the end of its retention period, if you wish. Once you've removed the lock from an object, you can delete the object. This mode also allows you to shorten the retention period for individual locked objects if you wish.
- Compliance Mode = Compliance mode cannot be deactivated by any user, including the root account.
- With this mode, you cannot remove the lock from a locked object before the end of its retention period. Nobody, including you as the bucket owner, can delete locked objects through the S3 interface until the end of their retention period. Also with this mode, nobody can shorten the retention period for a locked object.
- Governance Mode = Governance mode can be deactivated by storage accounts that have specific IAM permissions.
¶ Cloudian Object Storage Licensing Requirements
HyperStore can implement WORM (Write Once Read Many) protection for stored objects by supporting the standard AWS S3 "Object Lock" functionality. To use the Object Lock feature you must have a HyperStore license that activates this feature. Two different types of HyperStore licensing are available for Object Lock functionality:
- Certified Object Lock. With this type of Object Lock:
- HyperStore is fully compliant with the AWS S3 APIs relating to Object Lock, and the safeguards they provide against deletion or modification of locked objects through the S3 API.
- HyperStore also implements safeguards against deletion or modification of locked objects through means outside of the S3 API. Specifically:
- To use the Certified Object Lock feature you must disable password-based root access to HyperStore nodes and enable the HyperStore Shell (HSH). The HSH allows administrators to log into HyperStore nodes in a manner that is much more restrictive than
rootaccess. - With Certified Object Lock there is no mechanism to delete or modify locked objects through HyperStore administrative commands or the HyperStore Admin API.
- To use the Certified Object Lock feature you must disable password-based root access to HyperStore nodes and enable the HyperStore Shell (HSH). The HSH allows administrators to log into HyperStore nodes in a manner that is much more restrictive than
Certified Object Lock is appropriate for HyperStore customers who are subject to the data protection mandates of U.S. SEC-17a or a comparable regulatory regime.
- Compatible Object Lock. With this type of Object Lock:
- HyperStore is fully compliant with the AWS S3 APIs relating to Object Lock, and the safeguards they provide against deletion or modification of locked objects through the S3 API. In this respect Compatible Object Lock operates like Certified Object Lock.
- Unlike Certified Object Lock, with Compatible Object Lock no additional data protection measures are enforced beyond those enforced by the S3 API. Specifically:
- With Compatible Object Lock you are not required to disable password-based root access toHyperStore nodes and you are not required to enable the restrictive HyperStore Shell.
- With Compatible Object Lock you can use a HyperStore Admin API call to delete all objects from an Object Lock enabled bucket -- including objects that are still within their lock retention period -- and (optionally) to delete the bucket itself. To use this Admin API call on an Object Lock enabled bucket you must obtain a temporary security token from Cloudian Support.
Compatible Object Lock is appropriate for HyperStore customers who want to utilize the WORM functionality provided by the AWS S3 Object Lock APIs, but who are not subject to the data protection mandates of U.S. SEC-17a or a comparable regulatory regime.
¶ Protections for Locked Objects
The table below shows the key differences between Governance mode retention, Compliance mode retention, and Legal Hold in terms of how they protect locked objects against premature deletion through the S3 API. Recall that for all forms of Object Lock, the lock is applied to each individual version of an object. The scenarios are only applicable to Cloudian HyperStore.
| Scenario | Governance Mode Retention | Compliance Mode Retention | Legal Hold |
| Delete a locked object version? | The bucket owner and users who have been granted both s3:DeleteObjectVersion and s3:BypassGovernanceRetention permission can use the DeleteObject request with an x-amz-bypass-governance-retention: true request header to delete a locked object version. |
No user can delete a locked object version | No user can delete a locked object version |
| Remove the lock on an object version? | The bucket owner and users who have been granted both s3:PutObjectRetention and s3:BypassGovernanceRetention permission can use the PutObjectRetention request with an x-amz-bypass-governance-retention: true request header to remove the retention lock on a object version. |
No user can remove the retention lock on an object version | The bucket owner and users who have been granted s3:PutObjectLegalHold permission can use the PutObjectLegalHold request to remove the legal hold on an object version. |
| Reduce the retention period for an object version? | The bucket owner and users who have been granted both s3:PutObjectRetention and s3:BypassGovernanceRetention permission can use the PutObjectRetention request with an x-amz-bypass-governance-retention: true request header to reduce the retention period for a locked object version. |
No user can reduce the retention period for an object version | The bucket owner and users who have been granted s3:PutObjectLegalHold permission can use the PutObjectLegalHold request to remove the legal hold on an object version. |
¶ Object Storage Extension Object-Lock User Interface
1. Using the OSE UI, Select + NEW BUCKET
2. Ensure to Activate Versioning (Required) and toggle Object Lock
- Optional: You can select the Retention Mode to
Governance ModeorCompliance Mode
3. Inspect the properties of the newly created bucket ose-ui
5. Optional: You can add a Retention Policy (after the bucket has been created) to Governance Mode or Compliance Mode and set Retention Period
¶ Object Storage Extension Object-Lock API
Object Storage Extension API supports VMware Cloud Director Bearer token authentication. In addition, S3 API supports AWS Signature V4 authentication.
¶ Establishing an API Access Token for Tenant Access
- Generate an API Access Token Using Your VMware Cloud Director Tenant Portal
- Leveraging tools like Postman can facilitate the process.
- Download the Postman JSON file = demo-oauth-tenant-example.postman_collection.json
- Replace
{{FQDN}}+{{TENANT}}+{{REFRESH_TOKEN}}
¶ Example of a 200 Response Using Postman
¶ Access and Configure the APl Explorer Authorization
1. In OSE 2.2.3 – API Explorer is accessible from the the OSE Extension menu.
2. Locate the Authorize menu button:
3. In the API Explorer, you must add Bearer before the refresh_token
4. For example: Bearer example123456example123456 (In OSE 3.0, we will automatically add the Bearer variable)
¶ OSE S3 API Calls
¶ Create a Bucket with Object Lock (No Retention Example)
PUT /api/v1/s3/{bucketName}
curl -X 'PUT' \
'https://vose.philbendeck.com/api/v1/s3/vmware-example' \
-H 'accept: */*' \
-H 'x-amz-bucket-object-lock-enabled: true' \
-H 'Authorization: Bearer 'TOKEN' \
-H 'Content-Type: application/json' \
-d '{
"locationConstraint": "vcd"
}'Response Headers
connection: keep-alive
content-length: 0
date: Tue,27 Feb 2024 09:20:12 GMT
location: https://vose.philbendeck.com/api/v1/s3/vmware-example
strict-transport-security: max-age=31536000 ; includeSubDomains
x-amz-request-id: ed8fccca-bcf2-451e-bf15-d8ef73e3dd85
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block ¶ S3 Get Object Lock Configuration
GET /api/v1/s3/{bucketName}?object-lock
curl -X 'GET' \
'https://vose.philbendeck.com/api/v1/s3/vmware-example?object-lock=' \
-H 'accept: application/xml' \
-H 'Authorization: Bearer 'TOKEN'Response Headers
<ObjectLockConfiguration>
<ObjectLockEnabled>Enabled</ObjectLockEnabled>
</ObjectLockConfiguration>
¶ Using the AWS S3 API to Create an Object Lock Bucket
The OSE S3 API supports the AWS S3 Object Lock API and a tenant administrator/user can leverage creating object locked enabled buckets.
Essentially, the same supported API calls that you can invoke on the AWS S3 API can be executed on the Object Storage Extension's API leveraging Cloudian HyperStore.
¶ S3 API Syntax
s3api create-bucket --bucket {BUCKET} --object-lock-enabled-for-bucket --no-verify-ssl --endpoint-url=https://{FQDN}:{PORT}/api/v1/s3¶ S3 API Object Lock Example
# Create Object Lock Bucket
aws --profile=msp2admin s3api create-bucket --bucket objectlockbucket1 --object-lock-enabled-for-bucket --no-verify-ssl --endpoint-url=https://vose.philbendeck.com:443/api/v1/s3
{
"Location": "https://vose.philbendeck.com/api/v1/s3/objectlockbucket1"
}¶ S3 API Object Lock Configuration Verification
aws --profile=msp2admin s3api get-object-lock-configuration --bucket objectlockbucket1 --no-verify-ssl --endpoint-url=https://vose.philbendeck.com:443/api/v1/s3
{
"ObjectLockConfiguration": {
"ObjectLockEnabled": "Enabled"
}
}¶ S3 API Syntax – Adding Retention (Optional)
s3api put-object-lock-configuration --bucket {BUCKET} --object-lock-configuration='{ "ObjectLockEnabled": "Enabled", "Rule": { "DefaultRetention": { "Mode": "{MODE}", "Days": X }}}'¶ S3 API Object Lock PUT Retention Example
aws --profile=msp2admin --no-verify-ssl --endpoint-url=https://vose.philbendeck.com s3api put-object-lock-configuration --bucket objectlockbucket1 --object-lock-configuration='{ "ObjectLockEnabled": "Enabled", "Rule": { "DefaultRetention": { "Mode": "COMPLIANCE", "Days": 1 }}}'¶ S3 API Object Lock GET Retention Example
aws --profile=msp2admin s3api get-object-lock-configuration --bucket objectlockbucket1 --no-verify-ssl --endpoint-url=https://vose.philbendeck.com:443/api/v1/s3
{
"ObjectLockConfiguration": {
"ObjectLockEnabled": "Enabled",
"Rule": {
"DefaultRetention": {
"Mode": "COMPLIANCE",
"Days": 1
}
}
}
}