SDHS Transfer Service - Directory Models

The transfer server is a temporary storage location providing a mechanism to transfer data to and from the SDHS. As such, any data should be quickly removed from the system into the SDHS network to minimise the risk of unauthorised access.

All of the folders that are published through the Transfer Server are configured to operate in a certain mode. This mode defines how data placed in the folder will be managed, including how long it is retained and who has access.

There are 5 models, of which 3 are for incoming data and 2 are for outgoing data.

Incoming Data

1         Incoming Data – Automatic Move Into a Timestamped Folder

This folder would allow external users to upload data to the transfer server. Once uploaded, the file would be moved into the SDHS Study Group Drive and placed in a folder that is named according to the time that the file was moved. Folders are generated in 15 minute intervals, to save a proliferation of folders containing single or small numbers of files.

This option is most suitable for studies that upload large numbers of files at the same time, or that need to preserve the file name exactly as it is.

For example:

1. User uploads a file called ‘ParticipantData.xls’ into a transfer folder called ‘S0049 – StudyShortName – In’
2. CSCS moves the file to the Study Group Drive to S:\S0049-StudyShortName\IncomingTransferData\In-2016-11-15 08-40\ParticipantData.xls

Permissions

2         Incoming Data – Automatic Move Into Folder And Timestamp File

Similar to above, this folder would allow external users to upload data to the transfer server. Once uploaded, the file would be moved into the SDHS Study Group Drive and placed in a folder matching the Transfer Folder Name. A timestamp will be prepended to the file name. This option is most suitable for studies that upload small numbers of files or want to be able to find all of their transferred in data in a single location.

For example:

1. User uploads a file called ‘ParticipantData.xls’ into a transfer folder called ‘S0049 – StudyShortName – In’
2. CSCS moves the file to the Study Group Drive to S:\S0049-StudyShortName\IncomingTransferData\In\2016-11-15 08-40 ParticipantData.xls

Permissions

3         Incoming Data – Monitoring Only with Failsafe Clearout

Some studies may have existing process that are not compatible with either of the preferred methods above, and do not have the time or resources to rewrite their processes to fit the previous 2 models. Therefore this model is available whereby we keep track of all of the files that pass through the transfer server, but leave the study to remove the data.

Notifications will be sent to Data Managers if data sits on the system for 24 hours without being removed. If data is left on the system without being moved for 72 hours, then it will be automatically moved using the same logic as model 2 above.

For example:

1. User uploads a file called ‘ParticipantData.xls’ into a transfer folder called ‘S0049 – StudyShortName – In’
2. CSCS records that the file has been uploaded, but leaves the file for the Data Manager to move.
3. If the file is still present on the system after 72 hours, CSCS will move the file to S:\S0049-StudyShortName\IncomingTransferData\In\2016-11-15 08-40 ParticipantData.xls

 Permissions

Outgoing Data

4         Outgoing Data – Automatic Clearing after 24 hours

The default outgoing folder model . All data that is placed in this folder will be deleted after being at rest on the server for 24 hours.

It is recommended that the procedures outlined for the study include a step for the end user to delete the file once it has been downloaded

Permissions

5         Outgoing Data – Monitoring Only with Failsafe Clearout

The same style as model 3 above for incoming data. Notifications will be sent to Data Managers if data sits on the system for 24 hours without being removed. If data is left on the system without being moved for 72 hours, then it will be automatically removed.

It is recommended that the procedures outlined for the study include a step for the end user to delete the file once it has been downloaded

Permissions