Architecting a Video Streaming & Encoding Pipeline

An architecture design for a video streaming pipeline using AWS, FFmpeg, HLS, and Cloudflare R2, from upload to adaptive streaming.

August 15, 20265 min read
Architecting a Video Streaming & Encoding Pipeline

Why do we need video streaming

In the early days of the internet, videos were served directly to users as a single file, meaning that whenever someone wanted to watch a video, they had to download the entire file first before playback could even start. This created a major problem: if a video was 1GB in size, the user had to download the full 1GB before watching, even if they only wanted to see the first minute. On top of that, the same video file was served to every device regardless of screen size or network speed, so a phone and a laptop would both receive the exact same quality, even though a laptop screen could easily handle a much higher resolution like 1440p while a phone might only need 1080p. There was also no flexibility for users to choose their preferred quality, and no way to adjust playback if their internet connection slowed down mid-video, the video would simply freeze and buffer with no real-time fix.

This is where Adaptive Bitrate Streaming (ABR) comes in. Instead of relying on one giant file, the video is encoded into multiple quality levels and broken into small chunks, allowing the player to start playing almost instantly without waiting for the whole file to download. It also picks the best quality automatically based on the device being used, serving lower quality to mobile and higher quality to laptops or TVs, and adjusts that quality in real time if the network speed changes, so instead of buffering, the video simply drops to a lower quality and keeps playing smoothly. This combination of instant playback, device-appropriate quality, and real-time adjustment is what makes modern streaming platforms like YouTube and Netflix feel seamless compared to the old-school approach of downloading an entire video before watching it.

Adaptive bitrate streaming (ABR)

Adaptive bitrate streaming improves video playback by adapting video quality to the viewer’s network and device. The video is encoded into multiple qualities and split into small segments.

During playback, the player continuously selects the highest quality the current connection can handle. If the network slows down, it switches to a lower quality to prevent buffering.

How it works

It all starts with encoding, turning a raw video file into a format that works well across different devices. To make adaptive streaming possible, the same video gets encoded into multiple versions, each at a different bitrate (quality level).

Next comes segmentation. The video is chopped into small chunks, usually just a few seconds each, instead of being one giant file. This matters because it lets the video start playing almost instantly. The player only needs to download the first small chunk, not the whole video.

These segments also make the "adaptive" part possible. At the end of every segment, the player checks the viewer's internet speed. If the connection is too slow to keep up, the player switches to a lower quality (smaller) file for the next segment.

When a video first loads, most players start conservatively, often with the lowest quality version. Then they quickly test higher quality versions to see what the connection can handle, settling on the best one that plays smoothly. As long as your internet stays steady, it keeps streaming at that quality.

This whole system, moving up to better quality when you have good bandwidth and dropping down when you don't, is called the bitrate ladder or encoding ladder.

What Are Streaming Protocols?

A protocol is just a set of rules for how data moves across the internet. Not every streaming protocol supports adaptive bitrate streaming, but three popular ones do: HLS, DASH, and HDS.

All three follow the same basic idea: encode the video, chop it into segments, then stream it. But each one has different rules about file formats and which devices they work on.

  1. HLS (HTTP Live Streaming)
  • Works for both live streams and on-demand video (like YouTube or Netflix style playback)
  • Requires H.264 or H.265 encoding
  • Doesn't need any special server setup
  • Was originally made for Apple devices only, but now works everywhere
  • Important: Apple devices only accept HLS. No other format works on iPhones/iPads.
  1. DASH (Dynamic Adaptive Streaming over HTTP)
  • No specific encoding format required, more flexible
  • Runs over regular HTTP, so any normal server can host it
  • Downside: doesn't work on Apple devices at all
  1. HDS (HTTP Dynamic Streaming)
  • Originally built for Adobe Flash (which no longer exists)
  • Works for both live and on-demand streaming over HTTP
  • Requires converting video from MP4 to F4F (fragmented MP4), using H.264 encoding
  • Like DASH, it doesn't work on Apple devices

Architecture Design

hls.png

  1. Client Uploads the Video - The client (user) requests a presigned URL, then uses it to upload the raw .mp4 file directly to an R2 bucket. This avoids sending the file through your own server, saving bandwidth and time.
  2. Cloudflare Worker Gets Notified - Once the upload finishes, R2 triggers an "on upload" event. A Cloudflare Worker picks this up and acts as the messenger, kicking off the next stage.
  3. Job Goes Into a Queue (AWS SQS) - The Cloudflare Worker sends an event to AWS SQS (a message queue). This queue holds "encode this video" jobs until something is ready to process them. If a job keeps failing, it eventually lands in a DLQ (Dead Letter Queue) so it doesn't get stuck retrying forever.
  4. CloudWatch Alarm Watches the Queue - A CloudWatch alarm keeps an eye on the queue using a metric called ApproximateNumberOfMessagesVisible, basically, "how many jobs are waiting?" When the number goes up, it knows more workers are needed.
  5. Auto Scaling Group Spins Up Workers - Based on the alarm, an Auto Scaling Group (ASG) automatically launches more compute instances to handle the load and scales back down when the queue is empty.
  6. EC2 Spot Fleets Do the Encoding - The actual video encoding work happens on EC2 Spot Fleets (cheaper, temporary servers). Multiple instances can run in parallel to process several videos at once.
  7. Processed Files Go Back to Storage - Once encoding is done, the HLS output files (the segmented, multi-bitrate video files) are uploaded back into an R2 bucket for storage and delivery.
  8. Client Gets Notified - The job status is saved in DynamoDB. A webhook then fires to notify the client that their video is ready to stream.

Why this is cheap

  • R2 has zero egress fees, unlike S3, so moving video files in and out doesn't rack up bandwidth costs.
  • EC2 Spot Fleets cost a fraction of regular EC2 pricing since they use spare AWS capacity.
  • The ASG scales down to zero when there's no work, so you're not paying for idle servers.
  • Cloudflare Workers are effectively free at low volume.

This setup is a good middle ground: you get AWS's mature auto-scaling for compute, while avoiding AWS's expensive storage and egress costs by using Cloudflare R2 instead.

References & Resources