Skip to content

README.md

Latest commit

 

History

History
272 lines (215 loc) · 6.81 KB

README.md

File metadata and controls

272 lines (215 loc) · 6.81 KB
Raw

Images Service

Image upload, processing, and storage microservice using Azure Blob Storage.

Role

The Images Service handles all product image uploads, processing, and storage. It provides synchronous file uploads and asynchronous URL-based uploads with job tracking. Images are optimized (compressed, resized) using Sharp and stored in Azure Blob Storage for scalable, CDN-ready delivery.

Technologies

  • Runtime: Node.js 18+
  • Framework: Express.js 4.18
  • Cloud Storage: Azure Blob Storage
  • Queue System: Azure Queue Storage
  • Caching: Redis 4.6
  • Image Processing: Sharp 0.33 (compression, resizing)
  • File Upload: Multer 1.4 (multipart/form-data)
  • HTTP Client: Axios 1.6.2
  • Testing: Jest 29.7.0, Supertest 6.3.3
  • Containerization: Docker

Description

The Images Service is a microservice built with Express.js that handles image upload, processing, and storage using Azure Blob Storage. It supports uploading images from files or URLs, performs image optimization (compression and resizing using Sharp), and provides asynchronous job processing via Azure Queue Storage and Redis. Images are stored in public Azure Blob containers with unique sanitized filenames.

Key Features

  • Multiple Upload Methods: Upload from file (multipart/form-data) or URL
  • Image Processing: Automatic compression and resizing using Sharp library
  • Azure Integration: Blob Storage for images, Queue Storage for async jobs, Redis for job status
  • Asynchronous Processing: URL uploads processed via background worker
  • Job Status Tracking: Monitor upload progress via job IDs (1-hour TTL)
  • File Validation: MIME type validation, accepts image/* files only
  • Size Limits: 100MB maximum file size
  • Health Check: Azure container connectivity monitoring
  • Background Worker: Separate worker process for queue-based image processing

REST API

POST /upload

Upload image from file (synchronous processing).

Content-Type: multipart/form-data

Form Data:

  • image (file, required): Image file to upload

Response (201):

{
  "success": true,
  "data": {
    "blobName": "sanitized-filename-uuid.jpg",
    "url": "https://storage.blob.core.windows.net/container/sanitized-filename-uuid.jpg",
    "size": 245678,
    "contentType": "image/jpeg"
  }
}

Error (400):

{
  "error": "No file provided"
}

Note: Files are processed synchronously due to Azure Queue 64KB message size limit. For very large files, temporary blob storage would be needed for async processing.

POST /upload-url

Upload image from URL (asynchronous processing via queue).

Content-Type: application/json

Request Body:

{
  "imageUrl": "https://example.com/image.jpg"
}

Validation:

  • URL must start with http:// or https://
  • URL is validated using validator.isURL()

Response (202):

{
  "status": "processing",
  "jobId": "uuid-job-id",
  "message": "Image is being processed",
  "statusUrl": "/images/status/uuid-job-id"
}

Note: Returns immediately with job ID. Use status endpoint to check progress.

PUT /update-url/:blobName

Update an existing image by replacing it with a new image from URL.

Content-Type: application/json

Request Body:

{
  "imageUrl": "https://example.com/new-image.jpg"
}

Response (202):

{
  "status": "processing",
  "jobId": "uuid-job-id",
  "message": "Image update is being processed",
  "statusUrl": "/images/status/uuid-job-id"
}

GET /status/:jobId

Check the status of an asynchronous image upload job.

Response (200) - Processing:

{
  "success": true,
  "data": {
    "status": "processing",
    "jobId": "uuid-job-id"
  }
}

Response (200) - Completed:

{
  "success": true,
  "data": {
    "status": "completed",
    "jobId": "uuid-job-id",
    "result": {
      "blobName": "sanitized-filename-uuid.jpg",
      "url": "https://storage.blob.core.windows.net/container/sanitized-filename-uuid.jpg",
      "size": 245678,
      "contentType": "image/jpeg"
    }
  }
}

Response (200) - Failed:

{
  "success": true,
  "data": {
    "status": "failed",
    "jobId": "uuid-job-id",
    "error": "Error message"
  }
}

Error (404):

{
  "success": false,
  "error": "Job not found or expired. This job may have completed over 1 hour ago or never existed"
}

Note: Job status expires after 1 hour (configurable via Redis TTL).

DELETE /:blobName

Delete an image from Azure Blob Storage.

Response (200):

{
  "success": true,
  "message": "Image deleted successfully"
}

Error (404):

{
  "success": false,
  "error": "Blob not found"
}

GET /health

Health check endpoint for Azure container connectivity.

Response (200):

{
  "success": true,
  "data": {
    "status": "ok",
    "container": "container-name"
  }
}

Error (503):

{
  "status": "error",
  "message": "Connection error"
}

Environment Variables

Key configuration variables (see .env.example):

  • PORT: Server port (default: 80)
  • AZURE_STORAGE_CONNECTION_STRING: Azure Storage account connection string
  • AZURE_STORAGE_CONTAINER_NAME: Blob container name (created if not exists)
  • AZURE_STORAGE_QUEUE_NAME: Queue name for async processing
  • REDIS_HOST, REDIS_PORT: Redis connection for job tracking
  • REDIS_PASSWORD: Redis authentication (optional)

Image Processing

The service uses Sharp library for image optimization:

  • Format Conversion: Converts to JPEG format
  • Quality: 80% compression for optimal size/quality balance
  • Progressive: Generates progressive JPEGs for faster web loading
  • Metadata Stripping: Removes EXIF data to reduce file size

Architecture

  • Main Server (app.js): Handles HTTP requests and immediate uploads
  • Background Worker (worker.js): Processes queued image jobs from Azure Queue
  • Image Service (imageService.js): Business logic for upload/processing
  • Azure Config (azureConfig.js): Azure Blob/Queue client initialization
  • Redis: Job status tracking with automatic expiration

Error Handling

  • 400 Bad Request: Missing file, invalid URL, or Multer errors
  • 404 Not Found: Blob or job not found
  • 503 Service Unavailable: Redis connection failure
  • 500 Internal Server Error: Azure or processing errors

Security Notes

  • File type validation: Only image/* MIME types accepted
  • File size limit: 100MB maximum
  • URL validation: Only HTTP/HTTPS URLs allowed
  • Blob names are sanitized to prevent directory traversal
  • Container is set to public read access for image URLs

Azure Requirements

  • Storage Account with Blob and Queue enabled
  • Public blob container for image storage
  • Connection string in .env file

See main project README for detailed Azure setup instructions.