Working with files and folders (fundamentals)

This page describes how to use the Cocoon Data API to:

Obtain a cryptographic key from the Cocoon Data Platform’s Access Service, which is required for encrypting or decrypting a file’s/Secure Object’s data.
Initialize a new ‘Incomplete’ file object within an organisation by associating the cryptographic key with this new file.
Complete a file object by uploading its locally-encrypted data, which involves your client application encrypting the file object’s data first, then uploading this encrypted data to the Cocoon Data Platform’s Content Service for storage.
Complete a file object by uploading its unencrypted data to the Content Service for encryption, then storage.
Download a file object’s data for local decryption if required.
Create a folder to manage files and other folders.
Move items to other folders.

Before creating and/or manipulating either a file or a folder, your application must have a valid access token obtained by authenticating and authorizing a Cocoon Data user to the Cocoon Data Platform, where this user has the appropriate role and/or permissions within their organisation to create and/or manipulate either:
- their own items, or
- items owned by or located within another Cocoon Data user’s folder.
For more information about authenticating and authorizing Cocoon Data users to the Cocoon Data Platform, see Initiating authentication and authorization on the Authentication and authorization page of this guide and the Configuring client applications page of the SafeShare Administrator’s Guide.
The Supported roles (and conditions) section of each API endpoint page (accessible from the relevant procedures throughout this Developer’s Guide) contains details about the roles a Cocoon Data user requires and the conditions they must meet, for requests to these endpoints (which would include a valid access token representing the user’s authenticated session) to succeed.

Definitions

Secure Object

A Secure Object (in its Created or ‘completed’ state) represents a file and its data/content which has been encrypted according to the Cocoon Data Platform’s encryption requirements - see Completing a file object (uploading its locally-encrypted data) for more information - along with the file’s associated properties, which are recorded by the Cocoon Data Platform’s Access Service. These properties define details such as the file’s name, location, which people (i.e. collaborators) can access the file’s data and the level of access these people have to the file’s data. A Secure Object’s content is only handled either by the Cocoon Data Platform’s Content Service or externally by your application. While a Secure Object’s content is not handled by the Access Service, a Secure Object’s associated properties are retrieved from the Access Service, which in turn can assist in:

downloading the Secure Object’s content from storage (either in its encrypted or unencrypted form) from the Content Service,
obtaining the necessary key required to decrypt a Secure Object’s content, which has been encrypted, handled and/or stored externally (e.g. by your application).

Conceptually, within the scope of Cocoon Data’s SafeShare applications, a Secure Object represents a file and throughout Cocoon Data’s product documentation, is referred to as a ‘file object’ (or when the context is unambiguous, simply just a ‘file’).

A file object in an Incomplete state is one that has been initialized (and associated with a key), although is not yet associated with any encrypted data (which would change the file object’s state to Created).
When using Cocoon Data SafeShare applications, an incomplete file object could also result from a failure in either encrypting the file’s data or uploading this content to storage.
In Organisation Administration, a file object’s current state can be seen through the Files page. For more information, see the Organisation Administrator’s Guide.

Collection

A collection:

Represents a folder for logically organizing one or more Secure Objects and/or other collections.
Like a Secure Object, a collection also has its own associated properties.

Collections are:

Analogous to an operating system’s folders/directories on a file system and
Used by the Cocoon Data Platform’s Content Service to logically organize Secure Objects and other collections.

Conceptually, within the scope of Cocoon Data’s SafeShare applications, a collection represents an operating system’s folder or directory and throughout Cocoon Data’s product documentation, is referred to as simply just a ‘folder’.

Item

An item (in the context of Secure Objects and collections) is a generic term that refers to either a file (object) or a folder. Every item, regardless of whether it is a file or a folder has a unique identifier (or ID).

Organisation

All items on the Cocoon Data Platform are managed and handled within the scope of a self-contained unit called an organisation. An organisation is self-contained meaning that the creating and manipulating of items as well as sharing them is handled solely within the scope of an organisation itself. Organisations provide the Cocoon Data Platform with multitenancy capabilities, whereby each organisation represents an individual ‘tenant’. Be aware, however, that a Cocoon Data user can:

be a member of one or more organisations and also
share their content with users in other organisations.

Therefore, a Cocoon Data user’s ownership of content (or permissions to access this content) as well as the roles they have been granted to access SafeShare features are specific to each organisation. However, if a Cocoon Data user is a member of an organisation and this user shares content with another Cocoon Data user (who is not a member of the organisation), then that 2nd user automatically becomes a member of this organisation.

Obtaining a cryptographic key

A cryptographic key is required to encrypt data into a complete file object. To obtain a cryptographic key from the Cocoon Data Platform’s Access Service (which is the first step before creating a new file object), make a POST method request to this API endpoint:

<access-service>/api/v1/keys

with a JSON object in the body of the request that either:

contains the optional keyLength parameter (as a name/value pair) which determines the bit length of the cryptographic key to be returned from the Cocoon Data Platform. Specify a value of 128, 192 or 256.
OR
is empty. Specifying an empty JSON object implicitly sets the keyLength parameter’s value to 256.

Example request:

curl 'https://access-service.xy-company.com/api/v1/keys' \
    -H 'Authorization: Bearer eyJhbGciOiJSUzI1NiJ9.eyJleHAiOjE0NjcwMTY2NjYsInVzZXJfbmFtZSI6ImFsZXgub...' \
    -H 'Content-Type: application/json' \
    -d '{}' \
    -X POST

As indicated in this example, the access token obtained after authentication and authorization to the Cocoon Data Platform, must be included in the header of this request (as the bearer token) for the request to succeed.

If the request was successful, the Cocoon Data Platform responds by sending back a JSON object containing the key’s ID value (required to associate a cryptographic key with a file object), other information about the key’s characteristics and a date and time stamp of when the key was created. Example response:

{
    "keys": [{
        "id": "728882421139116032",
        "keyValue": "o6rwD3MQ4Saq1g1PnSGB6jarZGa26yLW4IRd38EGnVM=",
        "initializationVector": "E2wWB9FuvOXsimgLULDZfA==",
        "cryptographicAlgorithm": "AES",
        "keyLength": 256,
        "createdAt": "2016-07-04T07:58:31.180Z",
        "modifiedAt": "2016-07-04T07:58:31.180Z"
    }]
}

Next step: Now that you have a cryptographic key, you can begin the process of creating a new file object by Initializing a new ‘Incomplete’ file object, from which you can either upload the file object’s unencrypted data (for server-side encryption) then remote storage, or encrypt the file object’s data locally before uploading it to remote storage.

Initializing a new ‘Incomplete’ file object

Initializing a new file object (whose state is Incomplete):

associates the cryptographic key with a new file object in an organisation, which results in an incomplete file object and
is the initial step required to create a new file object.

To do this, make a POST method request to this API endpoint:

<access-service>/api/v1/organisations/{orgGroupId}/objects

with a JSON object in the body of the request that contains the keyId parameter (as a name/value pair) whose value is the ID of the key (obtained above). The value of {orgGroupId} in the URL path of the request is the ID of the organisation in which the file object will be created.

In this request, you may also wish to specify the name and mimeType parameters too.

Example request:

curl 'https://access-service.xy-company.com/api/v1/organisations/722338038193385472/objects' \
    -H 'Authorization: Bearer eyJhbGciOiJSUzI1NiJ9.eyJleHAiOjE0NjcwMTY2NjYsInVzZXJfbmFtZSI6ImFsZXgub...' \
    -H 'Content-Type: application/json' \
    -d '{
          "keyId": "728882421139116032",
          "mimeType": "image/jpeg",
          "name": "paraglider.jpg"
        }' \
    -X POST

If the request was successful, a JSON object is returned in the response containing an id member whose value is the ID of the incomplete file object. Example response:

{
    "id": "732860187958231040",
    "shared": false,
    "name": "paraglider.jpg",
    "sha512": null,
    "hasView": false,
    "canGenerateView": null,
    "organisation": {
        "name": "XY Company",
        "description": "",
        "mfaEnabled": false,
        "id": "732855142835470336"
    },
    "permissions": [],
    "mimeType": "image/jpeg",
    "contentSize": null,
    "parentId": "0",
    "state": "server.object.states.incomplete",
    "modifiedAt": "2016-07-15T07:24:44.693Z",
    "createdAt": "2016-07-15T07:24:44.693Z",
    "labelName": "Unclassified",
    "labelId": "732858991201665024"
}

You may wish that your application (temporarily) stores this file object’s ID value as this value is required to complete the file object (see Next step section below) or when making subsequent modifications to the file object, such as changing its name, location or who it is shared with.
Your application does not need to store any information about the cryptographic key, since this can (and for security reasons should) be retrieved from the Cocoon Data Platform using the file object’s ID.
When making this request, ensure that your application submits an access token with sufficient permissions within the organisation for the request to succeed. See Supported roles and conditions (of <access-service>/api/v1/organisations/{orgGroupId}/objects) for more information.

Next step: Now that you have initialized your file object, you can begin encrypting its data and complete the file object. Depending on your application’s requirements, this involves either:

Completing a file object (uploading its locally-encrypted data), whose data is encrypted locally before being uploaded to remote storage (via the Cocoon Data Platform’s Content Service).
Completing a file object (uploading its unencrypted data), whose unencrypted data is uploaded to the Content Service, which the Content Service then automatically encrypts and then transfers to remote storage.
Deciding which of these approaches is right for you
If the connection between a client’s machine (which stores the locally unencrypted data) and the Cocoon Data Platform’s Content Service is secured using properly implemented TLS, then the 2nd approach above (i.e. uploading unencrypted data) may be suitable for your requirements.
If, however, you do not wish a file object’s locally stored data to be sent from the client’s machine over any type of connection in its unencrypted form, your application can encrypt this data on the client machine first before uploading the encrypted data to remote storage via the Content Service.

Email notifications that inform collaborators about an item being shared with them are triggered by the Cocoon Data Platform at the time when these collaborators are initially shared the item (see Sharing items for more information). Therefore, if creating a file object within a folder (specified by the parentId parameter in the request above) when this folder has already been shared with collaborators, then it is recommended that your application completes the file object by uploading its data to remote storage as soon as practicable. Otherwise, there is a risk that collaborators will receive such an email notification before the file’s content becomes accessible.

Completing a file object (uploading its locally-encrypted data)

Completing a file object by uploading its locally-encrypted data involves the following steps:

Ensuring you obtained a cryptographic key for the file object and initialized the file object with this cryptographic key.
Using this cryptographic key to encrypt the file object’s data locally and obtaining the SHA-512 value of this encrypted data.
Associating this encrypted data with the file object by both:
- uploading the encrypted data to remote storage via the Cocoon Data Platform’s Content Service and
- assigning this SHA-512 value to the file object
using the file object’s ID. This step ‘completes’ the file object, which changes its state from Incomplete to Created.

Requirements for encrypting data locally When encrypting your data with the cryptographic key (obtained above), ensure your client application encrypts this data using:

The AES algorithm (in CBC mode) with PKCS #7 padding (see Optional parameters of <access-service>/api/v1/keys for more information), as well as
Both the cryptographic key and initialization vector (returned by calling the <access-service>/api/v1/keys API endpoint) for this file object.

If you do not follow these requirements, then either your attempt to upload your encrypted data may fail or if successful, any attempt to download the file object’s data in its decrypted form will result in the retrieval of garbled data. Before continuing, your client application should also calculate the SHA-512 value of this encrypted data (in its entirety). Completing the file object Once a file object’s data has been encrypted locally (above), your application can send this encrypted data to remote storage via the Content Service, which changes the file object’s state to Created. To upload a completed file object’s encrypted data to the Content Service, make one or more POST method request/s to this API endpoint:

<content-service>/api/v1/objects/{objectId}/contents

along with the actual encrypted data to be uploaded in the body of the request as multi-part form data elements. The value of {objectId} in the URL path of the request is the ID of the file object that was previously initialized. When making these requests, the value of the format URL parameter must be encrypted and the sha512 parameter’s value (submitted as a multi-part form data element) must be the SHA-512 value of the entire locally-encrypted data. See Required POST-request parameters (of <content-service>/api/v1/objects/{objectId}/contents) for more information. Example request:

curl 'https://content-service.xy-company.com/api/v1/objects/732860187958231040/contents?format=encrypted' \
    -H 'Authorization: Bearer eyJhbGciOiJSUzI1NiJ9.eyJleHAiOjE0NjcwMTY2NjYsInVzZXJfbmFtZSI6ImFsZXgub...' \
    -F 'sha512=mDxrkb0Nj5m44ZPYxgIJOvHckXHr577f8Ww4VGxssQ60yv5pyhtrrlepL2ZNCNytLRdwcY4CrwRQmiEG\/0y5Mw==' \
    -F 'totalFileSizeBytes=549732' \
    -F 'data=@paraglider-encrypted.jpg;type=image/jpeg' \
    -X POST

If the data being uploaded is larger than the upload chunk size permitted by the Cocoon Data Platform (whose default value is 10 mebibytes), then the data must be uploaded in multiple requests to this endpoint. Your client application must break up this data into chunks of this size, each of which must be uploaded in separate calls to this upload endpoint (above). See Required parameters for multi-chunk uploads (of <content-service>/api/v1/objects/{objectId}/contents) for more information.
The sha512 parameter only needs to be submitted once (and for multi-chunk uploads, in the final chunk upload request).

If a request was successful, a JSON object is returned in the response containing a success member whose boolean value is true (and from a single or final successful request, the response also contains the sha512 member whose value was the SHA-512 value submitted in the request). Example response:

{
    "objectId": "732860187958231040",
    "success": true,
    "uploadId": null,
    "sha512": "mDxrkb0Nj5m44ZPYxgIJOvHckXHr577f8Ww4VGxssQ60yv5pyhtrrlepL2ZNCNytLRdwcY4CrwRQmiEG\/0y5Mw==",
    "contentSize": 549732,
    "uploadedAs": "encrypted"
}

Completing a file object (uploading its unencrypted data)

Completing a file object by uploading unencrypted data stored locally involves the following steps:

Ensuring you obtained a cryptographic key for the file object and initialized the file object with this cryptographic key.
Uploading the file object’s unencrypted data to remote storage via the Cocoon Data Platform’s Content Service. This step automatically encrypts and ‘completes’ the file object, which changes its state from Incomplete to Created.

To upload an incomplete file object’s unencrypted data to the Content Service, make one or more POST method request/s to this API endpoint:

<content-service>/api/v1/objects/{objectId}/contents

along with the actual unencrypted data to be uploaded in the body of the request as multi-part form data elements. The value of {objectId} in the URL path of the request is the ID of the file object that was previously initialized. When making these requests, the value of the format URL parameter must be plaintext (and the sha512 parameter should be omitted). See Required POST-request parameters (of <content-service>/api/v1/objects/{objectId}/contents) for more information. Example request:

curl 'https://content-service.xy-company.com/api/v1/objects/732860187958231040/contents?format=plaintext' \
    -H 'Authorization: Bearer eyJhbGciOiJSUzI1NiJ9.eyJleHAiOjE0NjcwMTY2NjYsInVzZXJfbmFtZSI6ImFsZXgub...' \
    -F 'totalFileSizeBytes=549581' \
    -F 'data=@paraglider.jpg;type=image/jpeg' \
    -X POST

If the data being uploaded is larger than the upload chunk size permitted by the Cocoon Data Platform (whose default value is 10 mebibytes), then the data must be uploaded in multiple requests to this endpoint. Your client application must break up this data into chunks of this size, each of which must be uploaded in separate calls to this upload endpoint (above). See Required parameters for multi-chunk uploads (of <content-service>/api/v1/objects/{objectId}/contents) for more information.

After completing all required requests to upload the unencrypted data (above), the Content Service automatically:

Encrypts the data.
Calculates the SHA-512 value of this encrypted data.
Transfers the encrypted data to remote storage.
Associates this SHA-512 value with the file object, which:
- is returned in the response to the last upload request - see Returns (of <content-service>/api/v1/objects/{objectId}/contents) for more information and
- completes the file object (changing its state from Incomplete to Created).

For a multi-chunk upload, steps 1 to 3 above are conducted for each chunk and after the last chunk has been uploaded, the final SHA-512 value (for the entire encrypted data) is calculated and associated with the file object.

If a request was successful, a JSON object is returned in the response containing a success member whose boolean value is true (and from a single or final successful request, the response also contains the sha512 member whose value is the SHA-512 value calculated automatically by the Content Service). Example response:

{
    "objectId": "732860187958231040",
    "success": true,
    "uploadId": null,
    "sha512": "Ii5TdQLLK7s4KjWmAWrHGVf6UFOHR8f3BHnYgzu5w+RVHdDS+hhGT0z2Hxo8xyvVUcfhGBiWi9pXuQa+LHOjag==",
    "contentSize": 549796,
    "uploadedAs": "plaintext"
}

Downloading a file object’s data

A file object’s data can be downloaded in either its encrypted or unencrypted form (regardless of whether it was uploaded in encrypted or unencrypted form). To download (the active version of) a file object’s data, make a GET method request to this API endpoint:

<content-service>/api/v1/objects/{objectId}/contents

with the format URL parameter with one of these values to specify the format in which the file object’s data is downloaded:

encrypted - in encrypted form, or
plaintext - in unencrypted form.

Specify the encoding URL parameter with a value of base64 to download the file object’s data as Base64-encoded data. Otherwise, the data is downloaded in its native or binary form.

The value of {objectId} in the URL path of the request is the ID of the file object to be downloaded. Example request:

curl 'https://content-service.xy-company.com/api/v1/objects/732860187958231040/contents?format=encrypted' \
    -H 'Authorization: Bearer eyJhbGciOiJSUzI1NiJ9.eyJleHAiOjE0NjcwMTY2NjYsInVzZXJfbmFtZSI6ImFsZXgub...' \
    -X GET

If the request was successful, the file object’s data is downloaded locally.

When making this request, ensure that your application submits an access token with sufficient permissions within the organisation for the request to succeed. See Supported roles and conditions (of <content-service>/api/v1/objects/{objectId}/contents) for more information.

Obtaining the key to decrypt the file object’s data locally If the request was made with the format URL parameter value of encrypted, then your application will need to decrypt this data locally. This requires your application to retrieve the cryptographic key (from the Cocoon Data Platform) that was originally used to encrypt this file object’s data when the file object was completed. To obtain this cryptographic key, make a GET method request to this API endpoint:

<access-service>/api/v1/objects/{objectId}/keys

where the value of {objectId} in the URL path of the request is the ID of the file object whose data is to be decrypted locally. Example request:

curl 'https://access-service.xy-company.com/api/v1/objects/732860187958231040/keys' \
    -H 'Authorization: Bearer eyJhbGciOiJSUzI1NiJ9.eyJleHAiOjE0NjcwMTY2NjYsInVzZXJfbmFtZSI6ImFsZXgub...' \
    -X GET

If the request was successful, a JSON object is returned in the response containing the cryptographic key and initialization vector (required to decrypt the file object’s data locally). Example response:

{
    "keyValue": "2VCZkPeYPhvL4VkzrEqIVGaW2AGmRM1uwuXiPRbe8NU=",
    "cryptographicAlgorithm": "AES",
    "keyLength": 256,
    "initializationVector": "fpEIYIk+dnMw7rgFA+Bz8g=="
}

Requirements for decrypting data locally When decrypting your data with the cryptographic key (obtained above), ensure your client application decrypts this data using:

The AES algorithm (in CBC mode) with PKCS #7 padding (see Optional parameters of <access-service>/api/v1/keys for more information), as well as
Both the cryptographic key and initialization vector (returned by calling the <access-service>/api/v1/objects/{objectId}/keys API endpoint) for this file object.

If you do not follow these requirements, then your attempt to decrypt the encrypted data will result in garbled data.

Creating a folder to manage items

The Cocoon Data Platform allows the creation of folders, which are utilized to organise a Cocoon Data user’s files and other folders (i.e. items). To create a folder, make a POST method request to this API endpoint:

<access-service>/api/v1/organisations/{orgId}/collections

with a JSON object in the body of the request that contains the name of the folder and the parentId value, both of which are parameters (each specified as name/value pairs). The parentId value should be either:

0 to specify that the new folder will be created at the logical root level, or
the ID value of an existing folder that will become the parent of this new folder. See Retrieving information about items for details about retrieving information on existing folders.

The value of {orgId} in the URL path of the request is the ID of the organisation in which the folder will be created. Example request:

curl 'https://access-service.xy-company.com/api/v1/organisations/722338038193385472/collections' \
    -H 'Authorization: Bearer eyJhbGciOiJSUzI1NiJ9.eyJleHAiOjE0NjcwMTY2NjYsInVzZXJfbmFtZSI6ImFsZXgub...' \
    -H 'Content-Type: application/json' \
    -d '{
          "name": "Scenery",
          "parentId": "0"
        }' \
    -X POST

If the request was successful, a JSON object is returned in the response containing an id member whose value is the ID of the new folder. Example response:

{
    "id": "876543210191817161",
    "name": "Scenery",
    "createdAt": "2014-10-15T01:19:08.294Z",
    "modifiedAt": "2014-10-15T01:19:08.294Z",
    "shared": false,
    "parentId": "0"
}

A new file object or folder can be created within an existing folder by specifying the ID of a folder as the parentId value when Initializing a new ‘Incomplete’ file object or creating another folder.
When making this request, ensure that your application submits an access token with sufficient permissions within the organisation for the request to succeed. See Supported roles and conditions (of <access-service>/api/v1/organisations/{orgId}/collections) for more information.

Moving items to other folders

To move a Cocoon Data user’s item to another folder, make a PUT method request to either of these API endpoints:

<access-service>/api/v1/objects/{objectId} (if the item being moved is a file), or
<access-service>/api/v1/collections/{collectionId} (if the item being moved is a folder)

with a JSON object in the body of the request that contains the parentId parameter (as a name/value pair), whose value is the ID of the folder to which the item is being moved. The parentId value should be either:

0 to specify that the item will be moved to the logical root level, or
the ID value of an existing folder that will become the parent of the item, bearing in mind that the parentId folder and the item being moved must be within the same organisation. See Retrieving information about items for details about retrieving information on existing folders.

If the request was successful, a JSON object is returned in the response containing an id member whose value is the ID of the item that was moved and the parentId member whose value is the ID of the folder/location to which the item was moved.

When making this request:

Ensure that your application submits an access token with sufficient permissions within the organisation for the request to succeed. See Supported roles and conditions (of <access-service>/api/v1/objects/{objectId}) and Supported roles and conditions (of <access-service>/api/v1/collections/{collectionId}) for more information.
If the Cocoon Data user (whose access token is submitted in the header of the request) is a collaborator (see Sharing items for more information) on both the item being moved and the parentId folder, then this user must have the Move permission on both these items, which have been inherited from a common parent folder. (parentId could also be the common parent folder itself.)
If the Cocoon Data user is a collaborator on the parentId folder but is the owner of the item being moved (or vice versa), then it is not possible to move the item into the parentId folder.
For more information about the parentId parameter on the API endpoints above, see Optional PUT-request parameters (of <access-service>/api/v1/objects/{objectId}) and Optional parameters (of <access-service>/api/v1/collections/{collectionId}).

​Definitions

​Secure Object

​Collection

​Item

​Organisation

​Obtaining a cryptographic key

​Initializing a new ‘Incomplete’ file object

​Completing a file object (uploading its locally-encrypted data)

​Completing a file object (uploading its unencrypted data)

​Downloading a file object’s data

​Creating a folder to manage items

​Moving items to other folders

Definitions

Secure Object

Collection

Item

Organisation

Obtaining a cryptographic key

Initializing a new ‘Incomplete’ file object

Completing a file object (uploading its locally-encrypted data)

Completing a file object (uploading its unencrypted data)

Downloading a file object’s data

Creating a folder to manage items

Moving items to other folders