(Data type) File type for data elements

Registered by Lars Helge Øverland on 2014-06-18

Add new data element of type FILE_RESOURCE.

Implement file storage for FILE_RESOURCE data values using a flexible backend (local/remote).

Implement file uploads for FILE_RESOURCE data values in data entry.

Blueprint information

Status:
Complete
Approver:
Lars Helge Øverland
Priority:
High
Drafter:
Lars Helge Øverland
Direction:
Approved
Assignee:
Halvdan Hoem Grelland
Definition:
New
Series goal:
Accepted for trunk
Implementation:
Implemented
Milestone target:
milestone icon 2.21
Started by
Halvdan Hoem Grelland on 2014-10-30
Completed by
Halvdan Hoem Grelland on 2015-10-14

Related branches

Sprints

Whiteboard

Implemented in trunk 2.21. Has a few related issues still:

1) FIXED: ShallowEtagHeaderFilter causes file downloads of a certain size (corresponding to free heap space most likely) to fail miserably with an outofmemoryexception

2) FIXED (GET): Where applicable the GET of a file should be redirected to a signed url (S3 signed urls for example).

3) WONTFIX: Consider removing content digest generation, it is not free for large files

4) FIXED (separate endpoint): Upload files to separate api/fileresources endpoint, not api/datavalues etc and do cleanup of orphaned files (scheduled task?) every now and then.

-----

21. nov - 14:
- Change to using a separate upload (staging) of the file, returning a json representation of the resource to the user. The user can then save this as a data value of type file. Use an object which holds necessary meta-data and the file path relative to storage location (but not added to the model).
- This allows uploading the files before saving the data value, which is alot easier to deal with in client side code (forms etc.).

---

- Constrain to root directory
- Partition files in directories by day
- Data value is relative path on file system + filename
- Filename is UID
- File extension must be retained
- 1tb max file size
- Deny executables
---
- Consider compression
- Consider dedicated location for files
- Consider cloud storage

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.