1. Overview
Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance.
With the GoInsight AWS S3 node, you can seamlessly integrate S3 storage operations into your automated workflows. This allows you to perform a full range of object, folder, and bucket management tasks, including:
- Bucket Management: Create, delete, and list all your S3 buckets.
- Folder Management: Create, delete, and list folders within your buckets.
- File Management: Upload, download, delete, copy, and search for files (objects) within your S3 buckets.
2. Prerequisites
Before using this node, you need to have a valid AWS account with programmatic access enabled. You may also need appropriate IAM (Identity and Access Management) permissions to create and manage S3 buckets and objects.
3. Credentials
For a detailed guide on how to obtain and configure your credentials, please refer to our official documentation: Credentials Configuration Guide.
4. Supported Operations
Summary
This node provides operations centered around the management of S3 Buckets, Folders, and Files. The table below summarizes the available actions.
| Resource | Operation | Description |
|---|---|---|
| Bucket | Create Bucket | Creates a new S3 bucket in a specified region. |
| Bucket | Delete Bucket | Deletes an empty S3 bucket. |
| Bucket | Get Buckets | Retrieves a list of all S3 buckets associated with the credentials. |
| Folder | Create Folder | Creates a new folder within a specified bucket. |
| Folder | Delete Folder | Deletes a folder and all its contents. |
| Folder | Get Folders | Retrieves a list of all folders within a bucket. |
| File | Upload File | Uploads a file from a Base64-encoded string to an S3 bucket. |
| File | Download File | Downloads a file from an S3 bucket and returns its content as a Base64-encoded string. |
| File | Copy File | Copies a file from one location to another within S3. |
| File | Delete Object | Deletes a specific file (object) from an S3 bucket. |
| File | Get Files | Retrieves a list of all files within an S3 bucket. |
| File | Search Objects | Searches for objects within a bucket based on various criteria. |
Operation Details
Create Bucket
Creates an AWS S3 bucket in the specified region with optional ACL and custom endpoint support.
Input Parameters:
- BucketName: S3 bucket name to create.
Options:
- SessionToken: Optional session token for temporary credentials.
- EndpointUrl: Optional custom S3-compatible endpoint.
- Acl: Optional canned ACL: private, public-read, etc.
Output:
- BucketCreated (bool): Whether the bucket was created.
- BucketData (object): Created bucket details.
- StatusCode (number): Operation status code.
- ErrorMessage (string): Error message if any.
Delete Bucket
Deletes an empty AWS S3 bucket in the specified region.
Input Parameters:
- BucketName: S3 bucket name to delete.
Options:
- SessionToken: Optional session token for temporary credentials.
- EndpointUrl: Optional custom S3-compatible endpoint.
Output:
- BucketDeleted (bool): Whether the bucket was deleted.
- StatusCode (number): Operation status code.
- ErrorMessage (string): Error message if any.
Get Buckets
Retrieves a list of all AWS S3 buckets associated with the provided credentials.
Options:
- SessionToken: Optional session token for temporary credentials.
- EndpointUrl: Optional custom S3-compatible endpoint.
Output:
- BucketsRetrieved (bool): Whether the buckets list was retrieved.
- BucketsList (object-array): List of buckets.
- Owner (object): Owner information.
- StatusCode (number): Operation status code.
- ErrorMessage (string): Error message if any.
Create Folder
Creates a folder in an AWS S3 bucket.
Input Parameters:
- BucketName: S3 bucket name where the folder will be created.
- FolderName: Folder name to create (slash will be automatically added).
Options:
- SessionToken: Optional session token for temporary credentials.
- EndpointUrl: Optional custom S3-compatible endpoint.
Output:
- FolderCreated (bool): Whether the folder was created.
- FolderData (object): Object containing created folder details.
- StatusCode (number): Operation status code.
- ErrorMessage (string): Error message if any.
Delete Folder
Deletes a folder and all its contents from an AWS S3 bucket.
Input Parameters:
- BucketName: S3 bucket name where the folder is located.
- FolderName: Folder name to delete along with all its contents.
Options:
- SessionToken: Optional session token for temporary credentials.
- EndpointUrl: Optional custom S3-compatible endpoint.
Output:
- FolderDeleted (bool): Whether the folder was deleted successfully.
- DeletedObjects (object-array): Array of deleted objects with status details.
- TotalObjects (number): Total number of objects that needed deletion.
- SuccessfulCount (number): Number of objects successfully deleted.
- StatusCode (number): Operation status code.
- ErrorMessage (string): Error message if any.
Get Folders
Lists all folders in an AWS S3 bucket by analyzing object keys and extracting folder paths.
Input Parameters:
- BucketName: S3 bucket name to list folders from.
Options:
- Prefix: Optional folder prefix filter, empty to get all folders.
- SessionToken: Optional session token for temporary credentials.
- EndpointUrl: Optional custom S3-compatible endpoint.
Output:
- FoldersFound (bool): Whether folders were found successfully.
- FoldersData (object-array): Array containing folder information.
- TotalFolders (number): Total number of folders found.
- StatusCode (number): Operation status code.
- ErrorMessage (string): Error message if any.
Upload File
Uploads a file to AWS S3 from Base64-encoded content.
Input Parameters:
- BucketName: S3 bucket name where the file will be uploaded.
- ObjectKey: The key (path) for the uploaded file.
- FileContent: Base64-encoded file content to be uploaded.
Options:
- ContentType: MIME type of the file.
- MaxSizeMb: Maximum file size limit in MB.
- SessionToken: Optional session token for temporary credentials.
- EndpointUrl: Optional custom S3-compatible endpoint.
Output:
- UploadSuccessful (bool): Whether the upload operation completed successfully.
- FileLocation (string): The S3 URI of the uploaded file.
- ContentLength (number): The size of the uploaded file in bytes.
- ETag (string): ETag of the uploaded file for verification.
- VersionId (string): File version ID if bucket versioning is enabled.
- StatusCode (number): Operation status code.
- ErrorMessage (string): Error message if any.
Download File
Downloads a file from AWS S3 and returns the file content as a Base64-encoded string.
Input Parameters:
- BucketName: S3 bucket name where the file is located.
- ObjectKey: The key (path) of the file to be downloaded.
Options:
- MaxSizeMb: Maximum file size limit in MB.
- SessionToken: Optional session token for temporary credentials.
- EndpointUrl: Optional custom S3-compatible endpoint.
Output:
- DownloadSuccessful (bool): Whether the download operation completed successfully.
- FileLocation (string): The S3 URI of the downloaded file.
- ContentType (string): The MIME type of the downloaded file.
- ContentLength (number): The size of the file in bytes.
- LastModified (string): The last modified date of the file.
- ETag (string): ETag of the file for verification.
- FileContent (string): Base64-encoded file content.
- StatusCode (number): Operation status code.
- ErrorMessage (string): Error message if any.
Copy File
Copies a file from one location to another within AWS S3, supporting cross-bucket operations.
Input Parameters:
- SourceBucket: Source bucket name where the file is located.
- SourceKey: Source file key (path) to be copied.
- DestBucket: Destination bucket name where the file will be copied to.
- DestKey: Destination file key (path) for the copied file.
Options:
- SessionToken: Optional session token for temporary credentials.
- EndpointUrl: Optional custom S3-compatible endpoint.
Output:
- CopySuccessful (bool): Whether the copy operation completed successfully.
- SourceLocation (string): The S3 URI of the source file.
- DestLocation (string): The S3 URI of the destination file.
- ETag (string): ETag of the copied file for verification.
- StatusCode (number): Operation status code.
- ErrorMessage (string): Error message if any.
Delete Object
Deletes a specific object from an AWS S3 bucket.
Input Parameters:
- BucketName: S3 bucket name containing the object.
- ObjectKey: Object key (path) to delete.
Options:
- SessionToken: Optional session token for temporary credentials.
- EndpointUrl: Optional custom S3-compatible endpoint.
Output:
- ObjectDeleted (bool): Whether the object was deleted.
- StatusCode (number): Operation status code.
- ErrorMessage (string): Error message if any.
Get Files
Retrieves a list of all files in an AWS S3 bucket with optional prefix filtering.
Input Parameters:
- BucketName: S3 bucket name to list files from.
Options:
- MaxKeys: Maximum number of files to return.
- Prefix: Optional prefix filter for file keys.
- SessionToken: Optional session token for temporary credentials.
- EndpointUrl: Optional custom S3-compatible endpoint.
Output:
- ListComplete (bool): Whether the file listing completed successfully.
- FilesCount (number): Number of files found in the bucket.
- FilesData (object-array): Array of file objects containing details.
- StatusCode (number): Operation status code.
- ErrorMessage (string): Error message if any.
Search Objects
Searches for objects within an AWS S3 bucket with optional filtering by prefix, suffix, and contains text.
Input Parameters:
- BucketName: S3 bucket name to search in.
Options:
- Prefix: Optional prefix filter for object keys.
- Suffix: Optional suffix filter for object keys.
- Contains: Optional text that object keys must contain.
- MaxKeys: Maximum number of objects to return.
- SessionToken: Optional session token for temporary credentials.
- EndpointUrl: Optional custom S3-compatible endpoint.
Output:
- SearchComplete (bool): Whether the search operation completed.
- ObjectsFound (number): Number of objects found matching criteria.
- ObjectsData (object-array): Array of objects containing details.
- StatusCode (number): Operation status code.
- ErrorMessage (string): Error message if any.
5. Example Usage
This section will guide you through creating a simple workflow to upload a file to your AWS S3 bucket.
Workflow Overview
The workflow will consist of three nodes: Start -> AWS S3: Upload File -> Answer.
Step-by-Step Guide
- Add the AWS S3 Node:
- In the workflow canvas, click the + icon to add a new node.
- Select the "Tools" tab in the popup panel.
- Find and select Aws S3 from the list of tools.
- From the list of supported operations for Aws S3, click on Upload File to add the node to the canvas.
- Configure the Node:
- Click on the newly added Upload File node to open its configuration panel on the right.
- Configure Credentials: In the credentials field at the top, select your pre-configured Aws S3 credential from the dropdown menu.
- Fill in Parameters:
- BucketName: Enter the name of the S3 bucket where you want to upload the file (e.g., my-test-bucket).
- ObjectKey: Specify the full path and name for the file in the bucket (e.g., reports/monthly-report.txt).
- FileContent: Provide the content of the file, encoded in Base64. For example, to upload a file with the text "Hello, S3!", you would use the Base64 string SGVsbG8sIFMzIQ==.
- Run and Verify:
- Once all required parameters are filled correctly, the error indicator on the top right of the workflow canvas will disappear.
- Click the "Run" button in the top right corner to execute the workflow.
- After a successful run, you can click the log icon to view the detailed inputs and outputs of the node and verify that the operation was successful. You should see an UploadSuccessful output with a value of true.
Final Workflow
After completing these steps, your workflow is fully configured. When you run it, a new file will be created in your specified S3 bucket with the content you provided.
6. FAQs
Q: I'm getting a 403 Forbidden error. What should I do?
A: A 403 error typically indicates a permissions issue. Please check the following:
- IAM Permissions: Ensure the IAM user or role associated with your credentials has the necessary permissions (e.g., s3:PutObject, s3:GetObject, s3:ListBucket) for the bucket and objects you are trying to access.
- Bucket Policy: Review the bucket policy to ensure it doesn't explicitly deny access to your user or role.
Q: Why is my file upload failing with a "File size exceeds maximum limit" error?
A: The Upload File and Download File nodes have a default size limit to prevent excessive memory usage. You can increase this limit by setting the MaxSizeMb optional parameter, but be mindful of your workflow's resource constraints.
7. Official Documentation
For more in-depth information about the AWS S3 API, please refer to the AWS S3 Official API Documentation.
Leave a Reply.