Vendia | Open Beta Changes

Open Beta Changes

Posted by
Nitesh Arora

Updates

This post has been updated since it was first published.

Overview

Vendia Share was unveiled in the summer of 2020 and adoption has steadily increased ever since. As we reach new customers, address more use cases, and span additional industries, we continuously look for opportunities to improve our product and the development experience for all those using the platform. Our forthcoming Open Beta milestone, anticipated later in Q4, will unveil a collection of new features and enhancements, many of which originated from customer outreach and user feedback.

We value the experience of our current and future customers and, while we strive to avoid backwards incompatible changes at all costs, we also recognize there are times when modifying a platform feature now will save our customers countless hours in the future. In preparation for the forthcoming Open Beta milestone, we’ve made the choice to address several known issues in ways that will not be backwards compatible. We’re confident these changes, once released, will make for a significantly better experience for current and future customers. This includes:

To create a new Uni that has access to these improvements, no further action is required. Create a new Uni as you normally would and you’ll have access to the latest and great features of Vendia Share. Our documentation has been updated to reflect all of the recent changes so please read through carefully as you get reacclimated with Share.

The remainder of this post focuses on the Vendia Share client changes that may be required to access a Uni using the latest and greatest features of Vendia Share. The exact changes required will depend on the Vendia Share features and fields used and referenced by your clients. This post is a comprehensive guide to help you identify the items that require a change.

If you have questions as you prepare for this migration, please reach out to us through our existing customer support channels or at support@vendia.com.

GraphQL Schema and Type Improvements

Examples in this section were updated on 10/29/2021 to account for additional future proofing type collision updates.

GraphQL is a primary interface for Vendia Share clients – both synchronous and asynchronous – and making that interface as simple, consistent, and descriptive as possible is our goal.

Vendia Share accepts a user-provided data model and generates a GraphQL interface as part of Uni creation. The generated GraphQL API includes GraphQL types that reflect the properties provided in the user-provided data model as well as Vendia-provided types related to Node and Uni settings and features.

The combination of a user-provided data model and Vendia-provided types can, in certain situations, result in type collisions. Further, the risk of type collisions increases as more customers adopt more complex data models.

We have taken a new approach to type naming conventions to eliminate the potential for user/Vendia type collisions, allow users to work effectively with complex schemas, and reduce the potential for customer breaking changes that arise from schema evolution. This change may result in breaking changes to client code that currently references a generated GraphQL type name. However, future type name changes should not be necessary once client code is updated to leverage the new type names. This will remain true even as the user-provided data model or Vendia-provided types evolve in the future.

The schema and type improvements can be categorized across a few key change areas.

Change Areas

The remainder of this section outlines how the change areas above impact existing Vendia and User types.

In addition to these type name changes, equivalent changes have been applied to corresponding Query, Mutation, and Subscription operations. Similar changes have been made to related, Vendia-generated types, such as **Input, **Result, **Connection, **Partial, among others.

Remember, you can use GraphQL introspection against your Uni’s GraphQL endpoint to retrieve a complete GraphQL schema for your Uni at any time.

Change Areas Applied to Blocks

Block Type

The previous _Block type is now modeled as Vendia_Block:

type Vendia_Block {
    _id: ID!
    _owner: String
    blockHash: String!
    blockId: String!
    blockSchemaVersion: String
    commitTime: String!
    previousBlockHash: String!
    previousBlockId: String
    previousRedactedBlockHash: String!
    redactedBlockHash: String!
    status: Vendia_Block_statusEnum
    transactions: [Vendia_Block_transactions_transactionsItem]!
}

In addition to type naming and camelCase field modifications, changes include:

Vendia_Block_statusEnum Type

The new Vendia_Block_statusEnum type is modeled as:

enum Vendia_Block_statusEnum {
    APPLYING
    COMMITTED
    FAILED
}

Vendia_Block_transactions_transactionsItem Type

The previous _TX_element type is now modeled as Vendia_Block_transactions_transactionsItem:

type Vendia_Block_transactions_transactionsItem {
    _id: String!
    _owner: String
    hash: String!
    mutations: [String]!
    redactedHash: String
    signature: String
    submissionTime: String!
    version: String
}

In addition to type naming and camelCase field modifications, changes include:

Vendia Generated Types

As mentioned above, Vendia generated type names have changed. This includes Vendia-generated types, such as **Input, **Result, **Connection, and **Partial types, among others.

For example, the previous Block_Result type is now modeled as Vendia_Block_Result_ (note the trailing _):

type Vendia_Block_Result_ {
    error: String
    result: Vendia_Block
}

Change Areas Applied to Files and Folders

File Type

The previous _File type is now modeled as Vendia_File:

type Vendia_File {
    _id: ID!
    _owner: String
    copyStrategy: Vendia_File_copyStrategyEnum
    createdTime: String
    destinationKey: String!
    etag: String
    fileVersion: String
    read: [String]
    sourceBucket: String!
    sourceKey: String!
    sourceRegion: String!
    sourceVersion: String
    temporaryUrl: String
    updatedTime: String
    write: [String]
}

In addition to type naming and camelCase field modifications, changes include:

Vendia_File_copyStrategyEnum Type

The new Vendia_File_copyStrategyEnum type is modeled as:

enum Vendia_File_copyStrategyEnum {
    ALWAYS
    NEVER
    ON_ACCESS
}

Folder Type

The previous _Folder type is now modeled as Vendia_Folder:

type Vendia_Folder {
    _id: ID!
    _owner: String
    copyStrategy: Vendia_Folder_copyStrategyEnum
    createdTime: String
    name: String!
    parent: String
    read: [String]
    updatedTime: String
    write: [String]
}

In addition to type naming and camelCase field modifications, changes include:

Vendia_Folder_copyStrategyEnum Type

The new Vendia_Folder_copyStrategyEnum type is modeled as:

enum Vendia_Folder_copyStrategyEnum {
    ALWAYS
    NEVER
    ON_ACCESS
}

Change Areas Applied to Transactions

The previous _Transaction type is now modeled as Vendia_Transaction:

type Vendia_Transaction {
    _id: String
    _owner: String!
    submissionTime: String!
    transactionId: String
    version: String!
}

In addition to type naming and camelCase field modifications, changes include:

Vendia Generated Types

For example, the previous File_Result type is now modeled as Vendia_File_Result_ (note the trailing _):

type Vendia_File_Result_ {
    error: String
    result: Vendia_File
}

Changes Applied to DeploymentInfo

DeploymentInfo Type

The previous _DeploymentInfo type is now modeled as Vendia_DeploymentInfo:

type Vendia_DeploymentInfo {
    _id: ID!
    _owner: String
    consensusDefinitionHash: String!
    deploymentTime: String!
    versionTag: String!
}

In addition to type naming and camelCase field modifications, changes include:

Vendia Generated Types

For example, the previous DeploymentInfo_Result type is now modeled as Vendia_DeploymentInfo_Result_ (note the trailing _):

type Vendia_DeploymentInfo_Result_ {
    error: String
    result: Vendia_DeploymentInfo
}

Changes Applied to UniInfo

UniInfo Type

The previous _UniInfo type is now modeled as Vendia_UniInfo:

type Vendia_UniInfo {
    _owner: String
    createdTime: String
    localNodeName: String
    name: String!
    nodes: [Vendia_UniInfo_nodes_nodesItem]!
    schema: String!
    schemaNamespace: String
    sku: String
    status: String
    updatedTime: String
}

In addition to type naming and camelCase field modifications, changes include:

Vendia_UniInfo_nodes_nodesItem Type

The new Vendia_UniInfo_nodes_nodesItem type is modeled as:

type Vendia_UniInfo_nodes_nodesItem {
    bucketName: String
    description: String
    name: String!
    region: String!
    status: String
    temporaryCredentials: Vendia_UniInfo_nodes_nodesItem_temporaryCredentials
    userEmail: String
    userId: String!
    vendiaAccount: Vendia_UniInfo_nodes_nodesItem_vendiaAccount
}

In addition to type naming and camelCase field modifications, changes include:

Vendia_UniInfo_nodes_nodesItem_temporaryCredentials Type

The new Vendia_UniInfo_nodes_nodesItem_temporaryCredentials type is modeled as:

type Vendia_UniInfo_nodes_nodesItem_temporaryCredentials {
    uploadFile: Vendia_UniInfo_nodes_nodesItem_temporaryCredentials_uploadFile
}

In addition to type naming and camelCase field modifications, changes include:

Vendia_UniInfo_nodes_nodesItem_temporaryCredentials_uploadFile Type

The new Vendia_UniInfo_nodes_nodesItem_temporaryCredentials_uploadFile type is modeled as:

type Vendia_UniInfo_nodes_nodesItem_temporaryCredentials_uploadFile {
    accessKeyId: String
    expiration: String
    secretAccessKey: String
    sessionToken: String
}

Vendia_UniInfo_nodes_nodesItem_vendiaAccount Type

The new Vendia_UniInfo_nodes_nodesItem_vendiaAccount type is modeled as:

type Vendia_UniInfo_nodes_nodesItem_vendiaAccount {
    accountId: String!
    csp: String!
    org: String
    userId: String
}

In addition to type naming and camelCase field modifications, changes include:

Vendia Generated Types

For example, the previous UniInfo_Result type is now modeled as Vendia_UniInfo_Result_ (note the trailing _):

type Vendia_UniInfo_Result_ {
    error: String
    result: Vendia_UniInfo
}

Changes Applied to Settings

Settings Type

The previous _Settings type is now modeled as Vendia_Settings:

type Vendia_Settings {
    _owner: String
    apiSettings: Vendia_Settings_apiSettings
    aws: Vendia_Settings_aws
    blockReportEmails: [String]
    blockReportWebhooks: [String]
    deadLetterEmails: [String]
    deadLetterWebhooks: [String]
    resourceMapKeys: [String]
    resourceMapValues: [String]
}

In addition to type naming and camelCase field modifications, changes include:

apiSettings Type

The previous apiSettings type is now modeled as Vendia_Settings_apiSettings:

type Vendia_Settings_apiSettings {
    apiKeys: [Vendia_Settings_apiSettings_apiKeys_apiKeysItem]
    auth: Vendia_Settings_apiSettings_auth
}

In addition to type naming and camelCase field modifications, changes include:

Vendia_Settings_apiSettings_apiKeys_apiKeysItem Type

The new Vendia_Settings_apiSettings_apiKeys_apiKeysItem type is modeled as:

type Vendia_Settings_apiSettings_apiKeys_apiKeysItem {
    usagePlan: Vendia_Settings_apiSettings_apiKeys_apiKeysItem_usagePlan
    value: String
}

In addition to type naming and camelCase field modifications, changes include:

Vendia_Settings_apiSettings_apiKeys_apiKeysItem_usagePlan Type

The new Vendia_Settings_apiSettings_apiKeys_apiKeysItem_usagePlan type is modeled as:

type Vendia_Settings_apiSettings_apiKeys_apiKeysItem_usagePlan {
  quotaSettings: Vendia_Settings_apiSettings_apiKeys_apiKeysItem_usagePlan_quotaSettings
  throttleSettings: Vendia_Settings_apiSettings_apiKeys_apiKeysItem_usagePlan_throttleSettings
}

Vendia_Settings_apiSettings_apiKeys_apiKeysItem_usagePlan_quotaSettings Type

The new Vendia_Settings_apiSettings_apiKeys_apiKeysItem_usagePlan_quotaSettings type is modeled as:

type Vendia_Settings_apiSettings_apiKeys_apiKeysItem_usagePlan_quotaSettings {
  limit: Float
  offset: Float
  period: Vendia_Settings_apiSettings_apiKeys_apiKeysItem_usagePlan_quotaSettings_periodEnum
}

Vendia_Settings_apiSettings_apiKeys_apiKeysItem_usagePlan_quotaSettings_periodEnum Type

The new Vendia_Settings_apiSettings_apiKeys_apiKeysItem_usagePlan_quotaSettings_periodEnum type is modeled as:

enum Vendia_Settings_apiSettings_apiKeys_apiKeysItem_usagePlan_quotaSettings_periodEnum {
  DAY
  MONTH
  WEEK
}

Vendia_Settings_apiSettings_apiKeys_apiKeysItem_usagePlan_throttleSettings Type

The new Vendia_Settings_apiSettings_apiKeys_apiKeysItem_usagePlan_throttleSettings type is modeled as:

type Vendia_Settings_apiSettings_apiKeys_apiKeysItem_usagePlan_throttleSettings {
  burstLimit: Float
  rateLimit: Float
}

Vendia_Settings_apiSettings_auth Type

The new Vendia_Settings_apiSettings_auth type is modeled as:

type Vendia_Settings_apiSettings_auth {
    allowedAccounts: [String]
    authorizerArn: String
    authorizerType: Vendia_Settings_apiSettings_auth_authorizerTypeEnum
}

In addition to type naming and camelCase field modifications, changes include:

Vendia_Settings_apiSettings_auth_authorizerTypeEnum Type

The new Vendia_Settings_apiSettings_auth_authorizerTypeEnum is modeled as:

enum Vendia_Settings_apiSettings_auth_authorizerTypeEnum {
    API_KEY
    COGNITO
    CUSTOM
    IAM
    VENDIA_USER
}

Vendia_Settings_aws Type

The new Vendia_Settings_aws type is modeled as:

type Vendia_Settings_aws {
    blockReportFirehoses: [String]
    blockReportLambdas: [String]
    blockReportSQSQueues: [String]
    dataDogMonitoring: Vendia_Settings_aws_dataDogMonitoring
    deadLetterLambdas: [String]
    deadLetterSQSQueues: [String]
    lambdaIngressAccounts: [String]
    s3ReadAccounts: [String]
    sqsIngressAccounts: [String]
}

Vendia_Settings_aws_dataDogMonitoring Type

The new Vendia_Settings_aws_dataDogMonitoring type is modeled as:

type Vendia_Settings_aws_dataDogMonitoring {
    ddApiKey: String
    ddExternalId: String
    ddLogEndpoint: String
    ddSendLogs: Boolean
}

Vendia Generated Types

For example, the previous Settings_Result type is now modeled as Vendia_Settings_Result_ (note the trailing _):

type Vendia_Settings_Result_ {
    error: String
    result: Vendia_Settings
}

Changes Applied to User Types

JSON Schema types

Types defined in the user’s JSON schema are now prefixed with <UserNamespace>_:

type MyNamespace_MyUserType {
    myStringField: String
    myFloatField: Float
}

The example shown assumes MyNamespace is the value of <UserNamespace>. The value will default to Self is no namespace is provided. User types must adhere to valid GraphQL characters ([a-zA-Z][a-zA-Z0-9]*) and must not include an underscore () themselves.

JSON Schema definitions

Types defined in the user’s JSON schema in the definitions section are now prefixed with <UserNamespace>__definitions_:

type MyNamespace_MyUserType {
    myDefinitionsField: MyNamespace__definitions_MyDefinitionsType
    myStringField: String
}

The example shown assumes MyNamespace is the value of <UserNamespace>. User types must adhere to valid GraphQL characters ([a-zA-Z][a-zA-Z0-9]*) and must not include an underscore () themselves.

GraphQL Type Improvements

Vendia Share GraphQL schema is automatically generated based on the provided JSON Schema when a Uni is first created. GraphQL is a primary interface for clients – both synchronous and asynchronous – and making that interface as simple, consistent, and descriptive as possible is our goal. The schema improvements can be categorized across a few key change areas.

Change Areas

Change Areas Applied to GraphQL Enumerations

GraphQL Enumerations (enums) are a special kind of scalar that is restricted to a specified set of values. Vendia Share customers will continue to model enums in JSON Schema using the enum keyword. We’ve modified our internal JSON Schema 7 to GraphQL Schema compiler to produce GraphQL enums that correspond to those modeled in the JSON Schema 7. This will affect how GraphQL queries and mutations involving enums are written.

For example, the sample schema file in Vendia’s simple product catalog quick start shows how a product’s size can be constrained to a fixed set of values – “S”, “M”, “L”, and “XL”. The way you model your enum in the schema file doesn’t change. However, the way you insert your data when enums are used changes.

Previously, adding a product required a query similar to the following:

addProduct(name: "Awesome shirt", sku: "abc123", price: 10, size: "XL") {id}

Now, adding a product requires changing the size attribute from a string to a valid enum value.

addProduct(name: "Awesome shirt", sku: "abc123", price: 10, size: XL) {id}

Change Areas Applied to GraphQL Filters

Using the same example as above, we can execute a query and filter results based on a product’s size.

Previously, filtering for a product required a query similar to the following:

listProducts( filter: { size: { eq: "XL" } } ) { Products { name sku price size } }

Now, filtering for a product requires changing the size attribute from a string to a valid enum value.

listProducts( filter: { size: { eq: XL } } ) { Products { name sku price size } }

This also addresses a source of confusion as it relates to consistent enumeration support in Share. While Share previously allowed enumerations to be modeled in JSON Schema 7, that modeling did not apply to the generated GraphQL schema. This would often lead users to attempt to filter by size (using the example above) like this {size: XL} only to receive a result they did not expect.

Secure Message Feature Removal

Vendia Share Unis are designed to make it easy to share data and code across companies, regions, and clouds. However, sometimes it’s appropriate to restrict information to a subset of participants. Secure messaging was introduced to address this need. However, over the past year, we heard from customers that they would rather see this capability exposed when writing or updating data through standard GraphQL mutations. We introduced fine-grained data permissions to address this need. This capability allows data producers to limit which participants can take action on data written to a Uni.

Change Areas

Change Areas Applied to Secure Messaging

We’ve removed secure messaging in favor of fine-grained data permissions. There are several advantages to using this capability. Secure messaging only allows for a message to be sent to one or more recipients; the data contained in the message is not written to the shared data model. It is not available to be read, updated, or deleted at a later point. In contrast, data written with fine-grained permission is stored in your model. As such, it is available to be read, updated, or deleted at a later point. In addition, permissions can be updated at any time to change the principal, path, or operations allowed on the data as your data sharing needs evolve.

Please refer to our blog post and product documentation for how to make use of Share’s fine grained data access controls.

Requiring Authorizer Type During Uni Creation

Vendia Share Unis are created via either the Command Line Interface or the Web Console. In either case, a registration.json file provides initial information about the Uni.

Change Area

Change Area Applied to Uni Creation

A registration.json file must now contain an authorizerType element within a Node’s settings

{
    "name": "test-BasicUni",
    "schema": "schema.json",
    "nodes": [
        {
            "name": "TestNode",
            "userId": "user@email.com",
            "region": "us-east-1",
            "settings": {
                "apiSettings": {
                    "auth": {
                        "authorizerType": "VENDIA_USER"
                    }
                }
            }
        }
    ]
}

NOTE: While API_KEY was our previous default when authorizerType was not explicitly set, we do not recommend API_KEY for production Unis. API keys are generally used to track or meter API usage but are not a secure authorization mechanism. For production usage, please consider the other authorization options available.