Queues and notifications
Use SQS for processing jobs and SNS for completion notifications.
Uploading a file and analyzing it do not need to finish in the same request. SQS separates those stages, while SNS distributes notifications when the result is ready.
SQS: buffer processing jobs
The upload handler writes a message containing the file’s location and processing identifier. A Lambda consumer receives queued work and performs the analysis. The queue absorbs upload bursts so processors can work within their capacity.
| Queue type | Behavior |
|---|---|
| Standard | At-least-once delivery and best-effort ordering |
| FIFO | Ordering within message groups and deduplication features |
FIFO features do not make the application’s processing side effects automatically happen exactly once. Lambda’s SQS integration can deliver records more than once, so processing must be idempotent: repeating a job should not duplicate results or notifications. Configure retries, a visibility timeout, and handling for failed jobs. See Using Lambda with SQS.
SNS: notify subscribers
SNS uses a publish-subscribe model. A topic can distribute a completion event to multiple subscribers, including email, SMS, HTTP endpoints, and Lambda functions, depending on the topic and subscription configuration.
Publish the completion event after the result has been stored successfully. Users and organizers can then retrieve the result through the authorized API rather than receiving sensitive feedback directly in a notification.
Keep the roles distinct
SQS holds work until a consumer can process it. SNS fans an event out to subscribers. Using both keeps uploads responsive and lets notification subscribers change independently of the analysis pipeline.