Object storage with S3
Store uploaded feedback files separately from their structured metadata.
Amazon S3 stores the original feedback files: documents, audio, video, and images. The database stores their identifiers, ownership, processing status, and results rather than the large files themselves.
Choose a storage class
| Storage class | When it fits |
|---|---|
| S3 Standard | Files that need frequent, immediate access |
| S3 Intelligent-Tiering | Files with changing or uncertain access patterns |
| S3 Standard-IA | Less frequently accessed files that still need immediate retrieval |
| S3 One Zone-IA | Re-creatable, infrequently accessed data where a single Availability Zone is acceptable |
| S3 Glacier classes | Archive data, with retrieval behavior depending on the selected class |
Durability, availability, minimum storage duration, and retrieval charges depend on the storage class. S3 Standard is designed for 99.999999999% durability and 99.99% availability. Use the S3 storage class guide when evaluating alternatives.
Control file access
Keep the bucket private and restrict operations with IAM and bucket policies. After checking the user’s permissions, the backend can issue a presigned URL for a specific upload or download. The URL expires, but anyone holding it can use the permitted operation until it expires, so avoid exposing it unnecessarily.
Versioning preserves earlier object versions when enabled. Lifecycle rules can move older files to another storage class or expire them according to the application’s retention policy.
Start processing after upload
An S3 object-created event triggers the upload handler. In the workshop architecture, that Lambda function places a job in SQS and leaves the longer analysis work to a consumer. Store the object key in the job rather than copying the entire file into a queue message.