File API

Working with Files in Vendia projects allows for the sharing of large, unstructured data objects across all participants in a project. Vendia’s Files are versioned and each action on a File is captured in the project’s ledger. This means that any actions taken on a File are recorded, providing a lineage for the File. Files are consistent across all the participants in the project and can be shared using Read or Write permissions. The participant ( workspace) adding the File can determine which other workspaces are allowed to read and/or write the File.

To learn more about how to upload/download Files into and out of your project see Getting Started with Files.

Properties

Every Vendia File has the following properties:

• _id: The unique identifier that identifies the specific file. This _id value is consistent across all workspaces in the project.

• sourceBucket: The AWS bucket from which the data was sourced for this version of the File. [only for AWS sources]

• sourceRegion: The AWS region from which the data was sourced for this version of the File. [only for AWS sources]

• sourceKey: The AWS key from which the data was sourced for this version of the File. [only for AWS sources]

• sourceVersion: The version of the AWS key from which data was sourced for this version of the File. [only for AWS sources]

• sourceUri: The unique identifier of the resource from which data was sourced for this version of the File. For Azure, this is a URL which can optionally contain either a versionid or a snapshot. [only for Azure sources]

• destinationKey: The unique name of the file in the project. The name is restricted to alphanumeric characters, periods, underscores, hyphens and spaces. Names cannot start with spaces. Names can include forward slashes to indicate the folder, however, the folder must exist already and names cannot end with a forward slash or spaces.

• copyStrategy: The copy strategy used when managing this file in the project. The default strategy is ALWAYS.

  • The three copy strategy options are:
    • ALWAYS: Every workspace in the project has its own copy of the File.
    • NEVER: Only the owning workspace (the workspace which added the file initially) has a copy. All other workspaces have a “symbolic link” to the file.
  • Note: Only the workspace that adds the File with a NEVER copyStrategy can have write permissions to the File.
  • Note: The NEVER copyStrategy is only available for non Free Tier projects.

• read: List of comma-separated strings that identifies the workspaces that can read the file.

• write: List of comma-separated strings that identifies the workspaces that can write to the file.

• vendia.checksums.algorithm: The hashing algorithm for which a verifiable checksum of the File was computed.

• vendia.checksums.value: The verifiable checksum of the File. The value computed for each algorithm is consistent across all workspaces in the project.

• etag: The workspace-specific value for etag within the workspace’s cloud-specific resource.

• fileVersion: The workspace-specific value for the value within the workspace’s cloud-specific resource.

• createdAt: The time the file was added to the workspace.

• updatedAt: The time the file was last updated.

• temporaryUrl: A URL that provides non-gated access to the File’s contents. The URL is only valid for one hour after generation. This property can be returned by a call to getVendia_File or listVendia_FileItems. Each call will generate a new URL valid for an hour.

  • Note: temporaryUrl is only available for Enterprise Tier projects.

Methods

All File methods use the same GraphQL syntax as the rest of the project’s API. The File API has the standard operations including methods to add, update, remove, list and get.

Add File

The basic GraphQL mutation to add a File to a project:

mutation addFile {

addVendia_File(

input: {

sourceBucket: "<bucket>",

sourceKey: "<key>",

sourceRegion: "<region>",

destinationKey: "<key>",

copyStrategy: "<strategy>",

read: [... List of **workspaces** ...],

write: [... List of **workspaces** ...]

},

syncMode: ASYNC

) {

transaction {

_id

}

}

}

When adding a Vendia_File entity:

• destinationKey is always required
• sourceBucket, sourceRegion, and sourceRegion are required when the operation is performed against an AWS workspace
• sourceUri is required when the operation is performed against an Azure workspace

Other values are optional with the following default values:

• copyStrategy: ALWAYS - Every workspace in the project has their own copy of the File.
• read: All workspaces in the project are able to read this File.
• write: Only the workspace that added the File has write permissions.
• sourceVersion: This parameter is only necessary is you are attempting to copy a specific version of the source object.

Update File

The basic GraphQL mutation to update a File in a project:

mutation updateFile {

updateVendia_File(

id: "my-file-id"

input: {

sourceBucket: "<bucket>"

sourceKey: "<key>"

sourceRegion: "<region>"

destinationKey: "<key>"

}

syncMode: ASYNC

) {

transaction {

_id

}

}

}

You must provide the file-id of the File you are attempting to update as the id in the mutation, and the mutation’s input must not change the value for the File’s destinationKey.

You must have existing Write permission to be able to update a File.

All other parameters can be updated similar to adding a File; however, metadata-only updates are supported and do not require re-populating existing values.

For example, the following GraphQL mutation will only update the read and write permissions on an existing File:

mutation updateFilePermissions {

updateVendia_File(

id: "my-file-id"

input: { read: ["*"], write: ["my-workspace"] }

syncMode: ASYNC

) {

transaction {

_id

}

}

}

Remove File

The basic GraphQL mutation to remove a File from a project:

mutation removeFile {

removeVendia_File(id: "my-file-id", syncMode: ASYNC) {

transaction {

_id

}

}

}

Any workspace can always remove a File.

List Files

The basic GraphQL mutation to list Files in a project:

query listFiles {

listVendia_FileItems(filter: {}) {

Vendia_FileItems {

_id

sourceBucket

sourceKey

sourceRegion

sourceVersion

destinationKey

copyStrategy

read

write

etag

createdTime

updatedTime

temporaryUrl

fileVersion

}

}

}

When retrieving a list of Files you can specify which File properties to return. Further, you can filter the list using standing GraphQL filter syntax on the existing properties. You will only receive those Files you have Read access for.

Get File

The basic GraphQL mutation to get a File from a project:

query getFile {

getVendia_File(id: "my-file-id") {

sourceBucket

sourceKey

sourceRegion

sourceVersion

destinationKey

copyStrategy

read

write

etag

createdTime

updatedTime

temporaryUrl

fileVersion

}

}

You must have Read permissions to get a File. Like listing Files, you can specify which properties you want returned.

Tracking Add, Update, and Put File Operations

addVendia_File, updateVendia_File, and putVendia_File are all executed asynchronously and their status is reported via FileTask objects. The {transaction{_id}} field returned by these operations is the id of the file task tracking the operation. File tasks can be queried by calling listVendia_FileTaskItems and getVendia_FileTask

What is in a Vendia FileTask?

A Vendia File Task contains the following fields:

The following fields in the file task record are erasable:

• sourceUri
• sourceVersion
• sourceRegion
• sourceKey
• sourceBucket

FileTask execution details

When a file task execution completes successfully and updateVendia_FileTask is ledgered alongside the original add, update, or put operation:

mutation m {

file: addVendia_File(

id: "b417d4fc-18c3-b864-d126-69c3abf7bd16-018b01f4-7565-f4ef-616f-a7bb1b4360ae"

input: {

_owner: "WorkspaceTwo"

copyStrategy: ALWAYS

createdTime: "2023-10-05T22:28:27.506506+00:00"

destinationKey: "test-file.txt"

etag: "5e30215c238817405d9a9f689115b089"

fileVersion: "AmlssIaumDZljDAsDIwnNLHSuyM9myfC"

read: ["WorkspaceTwo"]

sourceBucket: "test-bucket"

sourceKey: "test-file.txt"

sourceRegion: "us-west-2"

sourceVersion: "AmlssIaumDZljDAsDIwnNLHSuyM9myfC"

updatedTime: "2023-10-05T22:28:27.506506+00:00"

vendia: {

checksums: [\
\
          { algorithm: md5, value: "5e30215c238817405d9a9f689115b089" }\
\
          {\
\
            algorithm: sha256\
\
            value: "d929a72f1a600f7f64eb704a280e940a9cbfed28ac05c3e3293c4d28b38e9e07"\
\
          }\
\
        ]

}

write: ["WorkspaceTwo"]

}

) {

error

}

fileTask: updateVendia_FileTask(

id: "018b01f4-7674-0dbe-bbbb-1b0278e8a7ac"

input: {

action: ADD

completionTime: "2023-10-05T22:28:31.158164+00:00"

status: SUCCESS

}

) {

error

}

}

Error Message

The File APIs will return the standard GraphQL exceptions if invalid syntax is used. However, errors can be returned in specific instances:

• File already exists: Attempting to add a File with a duplicate destinationKey value.
• File destination key mismatch: Attempting to update the destinationKey value for a File.
• No write permissions: Attempting to update a File without Write permissions.
• Can’t update permissions: Attempting to update permissions on a File you are not the owner of.
• Can’t update copy strategy: Attempting to update copy strategy on a File you are not the owner of.
• File not found: Attempting to remove a file that doesn’t exist.
• Folder not found: Attempting to add a File to a Folder that doesn’t exist.
• No permission to write to folder: No permissions to add Files to the specified folder.

Limits

See Platform Limits and Quotas for information on the limits and quotas for Files in a project.