Onboarding: How to Use DigitalOcean Spaces

Introduction

Welcome to DigitalOcean! We’re excited to have you on board. DigitalOcean offers a variety of products and services. From our simple-to-use Droplets to our managed Kubernetes services, you can run a blog, a business, and everything between and beyond.

After creating a DigitalOcean account, you’ll enter the Control Panel. This is the central hub for all of your projects on the DigitalOcean platform. We’re here to help you get comfortable with the different menus and workflows that will become important to you as you begin your next project.

In this tutorial, you will learn how to use DigitalOcean’s Spaces, an object storage service that can store and serve your data in a bucket. Spaces Object Storage is S3-compatible and includes a built-in Content Delivery Network (CDN) to decrease page load time, improve performance, and reduce the cost of bandwidth and infrastructure. By the end of this tutorial, you will be able to access and work with files stored in your Spaces bucket.

Getting Started with DigitalOcean Spaces

You can create a DigitalOcean Space by following the process in this section. The tasks are detailed in the embedded video and the text that follows.

When you sign in to your DigitalOcean account, you are directed to your main project page. If you have more than one project, select it from the list under Projects in the left navigation panel. You can also create a new project by selecting the + New Project option. Learn more about starting a new project in our tutorial on First-Time Users.

To view your current Spaces or create a new one, you can navigate to Spaces from the options on the left-hand bar or you can select the button Start using Spaces from the main project page:

Once you’ve navigated to the Spaces dashboard, you will have a list of current Spaces and their respective sizes, creation date, and a drop-down menu under the ellipses … with various options such as settings, moving to a different project, and deletion.

For the purposes of this tutorial, you will focus on creating a Space. If you don’t have an existing Space set up, you can press the button Create a Space from the main page. Alternatively, you can navigate to the Create button in the top right-hand corner and select Spaces from the drop-down menu:

This action redirects you to the Spaces Object Storage page, where you can create and customize your Space.

First, choose your preferred data center; in this example, we selected NYC3. The next option is to choose whether or not to enable the built-in Content Delivery Network (CDN), which can help deliver files faster and with a custom domain. In this example, we won’t enable the CDN, however, you may want to enable the CDN for lower latency:

Next, choose a unique name for your Space. You can name it whatever you prefer, as long as that meets the length requirement of between 3 and 63 characters long. We’ve named this Space spaces-demo-2023-01.

Once you’ve provided a unique name for your Space, select the project you want your Space to belong to. Here, we’ve selected the first-project project from the list. When you’re ready, press the Create a Spaces Bucket button:

When your Space is created, you will be redirected to your new Space file listing. There are two tabs: a Files and a Settings tab.

Within the Files tab, you can upload single or multiple files and set the permission levels for those files. To demonstrate, we will upload two text files named public.txt and private.txt, but you can upload whatever files you prefer. Once you’ve selected the files you want to upload, a prompt with the name, size, and type of file(s) appears. You can select the permission settings you want for the files. For now, set these permissions to Private. When you’re ready to upload your files, press the Upload button:

Note: To access private files, you will need a Spaces Access Key and Secret Key. We will discuss how to generate these keys in the next section.

Your Space file listing now contains the two files you’ve uploaded. Navigate to the ellipses … option for one of the files to review the options. This drop-down menu offers various options, such as managing metadata and permissions, renaming the file, moving the file to a folder, obtaining a link to quick share, copying endpoints, downloading the file, and deleting the file:

You may want to Manage Metadata so that you can create custom tags for your files. Once you select this option, a modal prompt will appear with the option to add a tag.

In this example, you will create two tags for the private.txt file: one for project and one for cost center. Naming the tags by project and cost center is a practice to aggregate bill and usage reports, but you can name them according to your needs. You can add these tags by selecting the Add Custom Pairing option, which autogenerates the key and allows you to add your name. The two tags used in this example are named x-amz-meta-project with a value of spaces-walkthrough and x-amz-meta-costcenter with a value of marketing.

In addition to recommended best practices for naming your tags, you must include an x-amz-meta prefix to add tags to your Space. This metadata prefix is part of the S3 API specification to identify tags; otherwise, the API may read tag metadata as HTTP headers.

Once you’re done adding metadata tags, press Save:

Within the Space file listing, you can also build a folder structure. To start, select the Create Folder option next to Upload. You can name the folder whatever you prefer; here, we’ve named it assets. Once it’s been added, it appears on the list as in the following:

Now that you have this folder available, select Move to… under the ellipses … dropdown menu for one of your files. From there, a new window prompt will appear, asking you to confirm you’re moving the file to the appropriate folder. You can also create a new folder with the Create new folder option in the upper right of this prompt. Select the assets folder and press the Move item button:

Your file is now stored in the folder you’ve moved it to. If you navigate to that folder, you can view your file:

Next, you will learn how to clean up your resources. For example, you can delete a specific file or folder by selecting Delete from the ellipses … drop-down menu:

Confirm the deletion by pressing Delete in the prompt that appears. Please note the warning that this deletion is irreversible:

Deletion will also delete all the contents in the folder. Double check there are no contents in this folder that you do not want to delete before confirming. Remember to move any files you wish to keep to a different folder before deletion.

In addition to the Files options, you can access the Space Settings via the tab at the top of the Spaces dashboard.

Within the Settings tab, you can delete your Space. First, navigate to the Settings tab next to the Files tab. On this page, scroll to the Delete this Spaces Bucket section and press the Delete button:

A new prompt will open for you to confirm deletion. A warning states that a Space can take several days to delete, depending on the amount of data you have stored in your Space. If there’s a small amount of data, it won’t take very long to delete. If there’s a lot of data, keep in mind that it may take more time. To continue with deleting your Space, you will write the name of the Space into the empty text box and then press Delete, as in the following :

Once you have confirmed your Spaces deletion, it will be listed as scheduled for deletion at a certain date and time. If you decide you do not want to delete your Space before that time, you can press the Cancel Deletion button:

Now that you know how to create and manage your Space, you’ll next learn how to access files with an access key and secret key for your Space.

Accessing Files on DigitalOcean Space

In this section, you will create an access key for your DigitalOcean Space. An access key provides more security to your Space by generating a pair of random tokens, called the access key and secret. These tokens serve as the username and password for authorized users to access the Spaces bucket and its contents.

You can access files in a DigitalOcean Space by following the process in this section. The tasks are detailed in the embedded video and the text that follows.

To get started, select API from the left panel. This selection takes you to Applications & API in the Control Panel, where you can manage your various tokens and keys, including API keys and OAuth Tokens:

In the Applications & API dashboard, scroll to the Spaces Keys section and press the Generate New Key button. This action will open the Spaces access keys page and load a new line where you will add a key name. Once you have added a name, press the check mark symbol to the right. To cancel this process, press the X symbol.

After you have pressed the check mark, two keys will appear on the list: one for your Spaces key and one for your secret key. If you want to make changes, you can press on the More drop-down menu on the right to rename, regenerate, or delete the key.

Remember to save these credentials somewhere because they will be needed later on. Make sure to save your secret key information because if you return to this menu later on, you will only be able to view your access key information, but not the secret key information:

Next, navigate to Spaces via the left navigation panel to review the Space you want to connect with this key. In this example, we’ve used the same spaces-demo-2023-01 Space created earlier. Toggle to the Settings tab, where you will have several options available. Select File Listing and press Edit to manage your file permissions:

A new prompt will appear, providing the following options for managing your file permissions: Restrict File Listing and Enable File Listing. If you choose Restrict File Listing, the contents of your Spaces bucket are privately accessible only by those with an active Spaces access key. Selecting Enable File Listing, however, allows anyone to access the contents of the Space.

For the purposes of this tutorial, select Restrict File Listing and press the Save button:

After you save the file settings, toggle to the Files tab to view your files and folders.

Press the ellipses … button on the right-hand side of a file to access the drop-down menu. In this example, we’ve selected Manage Permissions for the public.txt file to view its current permissions:

Once you press Manage Permissions, a prompt will load with the option to change the permission settings. In this case, the permissions for public.txt are set to Private. Set the file’s permissions to public access by pressing the Public option and then the Update button:

With publicly accessible files, such as public.txt, you only need a link to access it. To obtain this link, press the ellipses … option next to the file and choose Quick Share from the drop-down menu. The Quick Share public.txt prompt will load with the link for your file. You’ll also have the option to choose a certain sharing duration of hours or days, where anyone with the link can view a private file within that time:

Insert this link in a browser to access the public file, displaying its contents as well.

To fetch a public file, you only need the file’s URL, which you can get via the Quick Share feature. If you replace public.txt with private.txt in this URL, you will receive the following error that access is denied. This error occurs because the permissions for private.txt are still private because you have not changed the file permissions in your dashboard:

You can always download your private files directly from within the Spaces Control Panel. However, to access private files outside the Spaces Control Panel, you will need to use your Spaces Access Key and the S3 Authorization.

You’ll now test your access to the privately accessible private.txt file with the API platform Postman.

Once you have opened your Postman app, select the Authorization tab, then select AWS Signature from the Type drop-down menu:

This action will generate a form where you can add information about your access key, secret key, AWS region, and service name. Once you have provided the necessary information for each of these fields, press the Send button in the upper right corner:

Adding this information ensures that you have provided the authorization needed to access your files. After you’ve sent this information, you can now access the file. Keep in mind that Postman is only fetching your files, while any permissions you desire must be set within the Control Panel for your DigitalOcean Space. When accessing the file in Postman, the contents of this file appear in the lower panel within the Body tab, as demonstrated in the following example:

You now understand how to create a DigitalOcean Spaces access key and secret key. Next, you’ll use those credentials to work with S3-compatible tools and libraries like FileZilla Pro and the AWS S3 CLI.

Accessing Files on DigitalOcean Spaces with an S3-Compatible Tools and Libraries

Managing your DigitalOcean space is simple with the console, but you can also use S3-compatible Tools and Libraries. One example of a tool that can be used to visualize a DigitalOcean space is FileZilla Pro, a commonly used FTP browser. To follow along in this section, you need to download a FileZilla Pro FTP Client and generate a DigitalOcean Spaces Access Key and Secret. You can read the following FileZilla Pro Download instructions for your specific operating system, and review the previous Accessing Files on DigitalOcean Space to obtain the Access Key and Secret.

You can access files on DigitalOcean Spaces with S3-Compatible Tools and Libraries by following the process in this section. The tasks are detailed in the embedded video and the text that follows.

To get started, log in to FileZilla Pro and add DigitalOcean as a provider:

Navigate to the Edit tab in the top menu and then choose Settings to open a new prompt. Press the Transfers option and choose S3: Providers from the drop-down list. Then press the Add button in the Providers section at the top and write DigitalOcean to add it to the list. Next, add your information for Catch All and Format toward the bottom. Finally, press the Add button for the Regions section to add information about all the available region(s) or just the region(s) you’re working with. For the example in this section, we’re working with the SFO3 region. When everything is completed, press the OK button in the lower left of the prompt:

You can now set your credentials.

Navigate to the File tab in the top menu and then select Site Manager. This action opens up a new prompt, where you can press the New site button in the bottom left under Select entry. Once you do this, name the site DigitalOcean and press Enter.

Navigate to the General tab and press the drop-down menu for Protocol. From this list, choose S3 - Amazon Simple Storage Service and set the Host to the address for your Space. If you’re uncertain what your Space address is, you can find it by navigating to your DigitalOcean Space in the Control Panel and locating it at the top right where it says Origin Endpoint.

After that, put in the Access key ID and Secret Access Key information that you set up for your DigitalOcean account. Verify everything is correct, and then press the Connect button when you’re ready:

Now you can view all the buckets associated with this team. If you select a folder in the FileZilla app, you can view the contents and efficiently move files between other locations you desire. You can do this with multiple files and folders as well. Here’s an example of the layout:

FileZilla Pro is one way you can use your DigitalOcean Space with S3-compatible tools. In the next section, you will achieve the same task using a Command Line Interface (CLI).

Working with Files on DigitalOcean Spaces via AWS S3 CLI

Another way to access the materials in your DigitalOcean Space is through the command line with S3-compatible Tools and Libraries. This approach is a useful way for developers to automate and version-control various storage-related tasks. In this section, we’ll demonstrate how to use AWS CLI with a DigitalOcean Spaces Access Key.

You can work with files on DigitalOcean Spaces via the command line by following the process in this section. The tasks are detailed in the embedded video and the text that follows.

You need to set up your credentials to get started.

To begin, run the aws configure command:

aws configure

Complete the prompts with your DigitalOcean Spaces Key and Secret information:

[secondary_label Output]
AWS Access Key ID [None]: your_spaces_access_key
AWS Secret Access Key [None]: your_spaces_secret_access_key
Default region name [us-east-1]: us-east-1
Default output format [json]: json

For this example, add the us-east-1 region because this AWS region is required by the software development kit (SDK). The DigitalOcean region will be parsed out from the endpoint.

The prompt’s final question asks your preferred output format. In this example, we’ve selected json, but you can also return your output as YAML or tab-separated text. If you were chaining commands, you could typically pass the output back in as a parameter without changing the format:

Once your credentials are set up, you can test the connection.

The S3 commands are an abstracted set that only works with AWS. For this reason, we’ll use the S3 API, or aws s3api command.

Start by listing your buckets. Include your endpoint-url information and append the name of your bucket to the profile argument. Remember your endpoint-url is specific to the datacenter region you selected. In this example, it’s within the sfo3 datacenter region. The profile is the name of your DigitalOcean Space:

s3api list-buckets --endpoint-url https://sfo3.digitaloceanspaces.com --profile spaces-demo

This command returns an output of your existing buckets, or the line will be blank if no buckets have been created yet:

{
  "Buckets": [],
  "Owner": {
    "DisplayName": "bucket_display_name",
    "ID": "bucket_id"
  }
}

Whether you have existing buckets or have not yet created buckets, you will receive numbered responses where this output block displays ”bucket_display_name” and ”bucket_id”.

You can create a bucket in the CLI, as well as in the Cloud Panel dashboard. You’ll run the same S3 API with the create-bucket command. You also need to include a name; here, we’ve called it spaces-walkthrough-2022-cli:

aws s3api create-bucket --bucket spaces-walkthrough-2022-cli --acl public-read --profile spaces-demo --endpoint-url https://sfo3.digitaloceanspaces.com

In this command, you include the --bucket option to signify the name of your bucket you want to create. Then set the --acl option to public-read to make files publicly accessible. This is because DigitalOcean sets all new files uploaded to the bucket as private by default. You also include the credential information you set up earlier and set the endpoint that will target DigitalOcean in the CLI.

You can run the list-buckets command again to confirm this new bucket was added:

aws s3api list-buckets --endpoint-url https://sfo3.digitaloceanspaces.com --profile spaces-demo

The output returns information about your recently created bucket, including the name, when it was created, the owners, display name, and ID:

{
  "Buckets": [
    {
      "Name": "spaces-walkthrough-2022-cli",
      "CreationDate": "2023-01-09T23:16:15.312000+00:00"
    }
  ],
  "Owner": {
    "DisplayName": "bucket_display_name",
    "ID": "bucket_id"
  }
}

Keep in mind that the list-buckets command will return all of the Spaces buckets you have set up and authorized. Your output may differ from the sample output shown above.

With your bucket created, you’ll now add a file to the new bucket with the put-object command:

aws s3api put-object --bucket spaces-walkthrough-2022-cli --key newfile.md --body ./new-file.md --profile spaces-demo --endpoint-url https://sfo3.digitaloceanspaces.com

In this command, you define your target buckets and the filename to be used as their key. Keep in mind that AWS does not have a true folder system but can give a prefix such as folder_name / file_name to achieve the same result. For this example, we’re adding a file called newfile.md to the root of the bucket. Then set the body as whatever your source file is; in this case, use new-file.md. To finish, add your DigitalOcean profile and endpoint information.

You will receive a response similar to this once you have run the put-object command:

{
  "ETag": "\"etag_information\""
}

Confirm this file was added by running list-objects-v2 with declarations for the bucket, profile, and endpoint URL:

aws s3api list-objects-v2 --bucket spaces-walkthrough-2022-cli --profile spaces-demo --endpoint-url https://sfo3.digitaloceanspaces.com

The output returns the "Contents" of your newly added file:

{
  "Contents": [
    {
      "Key": "newfile.md",
      "LastModified": "2023-01-09T23:20:15.980000+00:00",
      "ETag": "\"etag_information\"",
      "Size": 660,
      "StorageClass": "STANDARD"
    }
  ]
}

If you want to download this file, use get-object with the bucket name, file name, profile, and endpoint URL. You also need to include the path to the output file in the last part of the argument of the command:

aws s3api get-object --bucket spaces-walkthrough-2022-cli --key newfile.md --profile spaces-demo --endpoint-url https://sfo3.digitaloceanspaces.com newfile.md

You will receive a response with metadata information about the file:

{
  "AcceptRanges": "bytes",
  "LastModified": "2023-01-09T23:20:15+00:00",
  "ContentLength": 660,
  "ETag": "\"etag_information\"",
  "ContentType": "binary/octet-stream",
  "Metadata": {}
}

You can confirm that this file exists by running the same aws s3api list-buckets command as before.

To delete a file in your bucket, run delete-bucketwith the bucket name, profile, and endpoint URL in the command:

aws s3api delete-object --bucket spaces-walkthrough-2022-cli --key newfile.md --profile spaces-demo --endpoint-url https://sfo3.digitaloceanspaces.com

If the deletion was successful, you will not receive any output. However, to verify deletion, you can run list-objects:

aws s3api list-objects-v2 --bucket spaces-walkthrough-2022-cli --profile spaces-demo --endpoint-url https://sfo3.digitaloceanspaces.com

After running this command, an empty string appears. This empty string response confirms that the newfile.md file was deleted from the bucket.

If you want to delete a bucket, run the delete-bucket command with the bucket name, profile, and endpoint URL:

aws s3api delete-bucket --bucket spaces-walkthrough-2022-cli --profile spaces-demo --endpoint-url https://sfo3.digitaloceanspaces.com

You won’t receive an output response, so confirm the bucket was deleted by running list-buckets again:

aws s3api list-buckets --profile spaces-demo --endpoint-url https://sfo3.digitaloceanspaces.com

The output confirms that the bucket used throughout this section is no longer available:

{
  "Buckets": [],
  "Owner": {
    "DisplayName": "bucket_display_name",
    "ID": "bucket_id"
  }
}

You can now manage your DigitalOcean Space in the terminal with the S3 API. You can learn more about other options available with our reference documentation on Spaces API.

Conclusion

In this tutorial, you learned how to meet your object storage needs by creating and managing a DigitalOcean Space. You also generated an Access Key and Secret Key so that you can access your files on DigitalOcean Spaces with S3 compatible tools and libraries. We encourage you to explore how Spaces can provide solutions for your object storage needs and get started building with DigitalOcean Spaces today.

Check out this DigitalOcean Onboarding Series video playlist with even more guidance on how to optimize your DigitalOcean products and services.