Skip to content

Update backend architecture for huge traffic

Reputation22 requested to merge varun-s22/gitlab-video-cut-tool:bullmq-patch into master
  • Adds Redis based Queue Bullmq to the tool, for handling multiple connections, by initializing WORKER_LIMIT (defined as env variable inside docker compose.yml files) workers initially, which takes up jobs from the queue, when not busy.
  • Updated the backend architecture, to support multiple videos processing of the users, by maintaining a list of active socket connections mapped against the user id (deleting them on being inactive), and sending payload data of every instance to every socket connection.

Currently this is WIP, and put on hold due to:

  • videoId not being stored on the client, makes it difficult for seperating out the payloads sent from the server

Bug: T342810 Change-Id: Ie504a64d9514566ff61e23e1cb31a81f51ad0dd0

Merge request reports