Introduction

Recently I built and deployed YTS (YoutubeAlbumSplitter), a website I've built to take in Youtube mixtapes/music compilations and automatically tag and split the video into individual music tracks. This application works by primarily using YTDL-Core & FFMPEG.wasm with the MERN stack (MongoDB, Express, React, & NodeJS).

The Problem

This application was built from the problem of me finding niche artists/music categories and was unable to either download the individual track or search for the individual track on other platforms such as Spotify. Because of this, when I wanted to play the songs on alternative platforms other than YouTube, I would have to manually download the video, extract the specific song I wanted, and then tag it before adding it to my Spotify custom songs list.

This process was very tedious and whilst a lot of playlists/compilations labeled their songs, the newly implemented timestamp feature on YouTube even said what song was playing whilst it was playing. There was no easy-to-use tool that would automatically download the song & split it up on demand, as a result, this application was made.

The Design Process

To start out, I believed this project solved a real problem in the market. I wanted this project to be; clean, elegant, efficient, and most importantly scalable.

Whilst I've built and temporarily deployed many MERN stack projects, such as Summarized. These have usually been local deployments or one-time deployments where the code is manually set up and left there temporarily for 24-48 hours until a demonstration is run. From which the instance would be closed. These projects, whilst fun and demonstrating a great proof of concept are not scalable and indicative of a potentially successful/successful project in the real world. Whilst an application may function well for 1-2 users, when scaled at mass to thousands to hundreds of thousands of users, efficiency and scalability is key. As a result, the planning process for this project took significantly longer than most projects I take on.

The first and most important issue was trying to understand how this would work functionally. Thankfully I was already acquainted with Youtube-DL and FFMPEG. FFMPEG is a very complex, highly efficient, and flexible video/audio processing tool that many applications and companies use globally to render / process video & audio on demand. Likewise, YTDL-Core is a globally recognized library designed to download Youtube videos in a lightweight, efficient and failsafe manner. By combining these two libraries initially I was able to figure out how to split a video into multiple parts based on timestamps.

The second issue was more technical. I had to learn how to parse user-inputted data into a format that would be recognized by both my code and the libraries. This was made more difficult by the fact that there are a variety of ways to input timestamps (ex. 1:30:00, 1h30m, 90m, 5400s, etc). I solved this by coding my own universal timestamp parser for about (90% of use-cases) with the remaining 10% being flagged as incomplete or incorrect values in my validation stages.

The third and final issue was designing the user interface. I wanted something that would be very simple and straightforward to use. I also wanted the application to be usable without the need for an installation. This meant that the user interface had to be web-based. I decided to use the ReactJS and TailWindCSS libraries to design my user interface. ReactJS is a fantastic library that allows for the creation of complex user interfaces while still keeping the codebase very clean and organized. TailWindCSS is a utility-first CSS library that allowed me to quickly create a responsive and modern-looking user interface.

With a plan of attack ready, I begun coding and quickly spun up a command line proof of concept, using REST APIs to communicate with the YouTube Data and Content APIs. I also used the ffmpeg tool to split the video files. This allowed me to quickly verify that my approach would work and that the necessary APIs were available.

Solution

With the proof of concept ready, I moved on to creating the web-based user interface. This was my first time using ReactJS and TailWindCSS, so it took me a little bit of time to get everything setup and working correctly. However, once I got the hang of things, I was able to quickly put together a functional user interface.

The next step was to take my proof of concept code and turn it into a production ready application. This involved adding support for authentication, error handling, and a few other things. I also had to make some changes to how the application interacted with the YouTube APIs. The YouTube Data API has a quota that limits the number of requests that can be made in a day. To work around this, I added a fake user login to bypass this via Cookies, allowing unlimited downloads with the Youtube Data API.

Finally, I had to figure out a way to deploy the application. Whilst my application ran well locally, as it used FFMPEG for some of its processing, it would not scale well on my DigitalOcean droplet cheapily as it was a resource intensive task.

I found a workaround to this by using FFMPEG.wasm, an FFMPEG implementation in pure WebAssembly, which would enable me to move the resource intensive task to the client-side. This would also enable me to keep my application stateless, as I would not have to store any user data or videos on my server, and could therefore scale my application infinitely.

Finally after building the application and ensuring it's scalability, came the finishing touches. To do this I outsourced the UI design to a friend as I am not a designer, and integrated Google Analytics to help me understand how people were using the application and what features they were using the most. I also added some basic error handling, such as displaying an error message if a user tried to split a video that was too short, or if they tried to split a video that had already been split.

Deploying

After finalizing the code of my Youtube Album Splitter tool, I decided to deploy it using DigitalOcean's App Platform so that it could be used by anyone, anytime, anywhere. I chose to use Docker on Linux and Nginx as my web server and reverse proxy. I also wanted to add SSL to my site so I decided to use LetsEncrypt + CertBot for that.

Here's a step-by-step guide on how I did it:

1. I created a new droplet on DigitalOcean and chose the App Platform product.
2. I connected to my droplet using SSH and created a new directory for my project.
3. I then clone my Youtube Album Splitter code into that directory.
4. I created a new file called Dockerfile and added the following code to it:

FROM node:12-alpine as backend-stage

RUN mkdir -p /app

WORKDIR /app

COPY /package*.json ./

RUN npm install

COPY . .

# EXPOSE $PORT

CMD ["npm", "start"]

FROM node:12-alpine as frontend-stage

# CREATE an app directory
RUN mkdir -p /app

# This is the directory that should act as the main app directory
WORKDIR /app

# Copy lock files
COPY /package*.json ./
# COPY youtube-album-splitter/package-lock.json ./

# Install node packages
RUN npm install

# Copy the rest of the project
COPY . .

# Build the app
RUN npm run build

# Expose a port
# EXPOSE $PORT

# Start app
CMD ["npm", "run", "host"]

5. I then created a new file called nginx.conf and added the following code to it:

##
# You should look at the following URL's in order to grasp a solid understanding
# of Nginx configuration files in order to fully unleash the power of Nginx.
# https://www.nginx.com/resources/wiki/start/
# https://www.nginx.com/resources/wiki/start/topics/tutorials/config_pitfalls/
# https://wiki.debian.org/Nginx/DirectoryStructure
#
# In most cases, administrators will remove this file from sites-enabled/ and
# leave it as reference inside of sites-available where it will continue to be
# updated by the nginx packaging team.
#
# This file will automatically load configuration files provided by other
# applications, such as Drupal or Wordpress. These applications will be made
# available underneath a path with that package name, such as /drupal8.
#
# Please see /usr/share/doc/nginx-doc/examples/ for more detailed examples.
##

# Default server configuration
#
server {
    if ($host = youtubealbumsplitter.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    listen 80;
    server_name youtubealbumsplitter.com;
    return 301 https://youtubealbumsplitter.com$request_uri;
}

server {
 root /var/www/html;
 server_name youtubealbumsplitter.com;


 location / {

 proxy_pass http://localhost:3000;
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 proxy_set_header Host $host;
 proxy_set_header X-NginX-Proxy true;
 proxy_http_version 1.1;
 proxy_set_header Upgrade $http_upgrade;
 proxy_set_header Connection “upgrade”;
 proxy_max_temp_file_size 0;
 proxy_redirect off;
 proxy_read_timeout 240s;
 }

 location /api {
    proxy_pass http://localhost:3001;
 }
}


# Virtual Host configuration for example.com
#
# You can move that to a different file under sites-available/ and symlink that
# to sites-enabled/ to enable it.
#
#server {
#       listen 80;
#       listen [::]:80;
#
#       server_name example.com;
#
#       root /var/www/example.com;
#       index index.html;
#
#       location / {
#               try_files $uri $uri/ =404;
#       }
#}

server {

        # SSL configuration
        #
        # listen 443 ssl default_server;
        # listen [::]:443 ssl default_server;
        #
        # Note: You should disable gzip for SSL traffic.
        # See: https://bugs.debian.org/773332
        #
        # Read up on ssl_ciphers to ensure a secure configuration.
        # See: https://bugs.debian.org/765782
        #
        # Self signed certs generated by the ssl-cert package
        # Don't use them in a production server!
        #
        # include snippets/snakeoil.conf;

        root /var/www/html;

        # Add index.php to the list if you are using PHP
        index index.html index.htm index.nginx-debian.html;
        server_name youtubealbumsplitter.com; # managed by Certbot


        location / {
                # First attempt to serve request as file, then
                # as directory, then fall back to displaying a 404.
                #try_files $uri $uri/ =404;

                proxy_pass http://localhost:3000;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Host $host;
                proxy_set_header X-NginX-Proxy true;
                proxy_http_version 1.1;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection “upgrade”;
                proxy_max_temp_file_size 0;
                proxy_redirect off;
                proxy_read_timeout 240s;
        }

        location /api {
                proxy_pass http://localhost:3001;
        }

        # pass PHP scripts to FastCGI server
        #
        #location ~ \.php$ {
        #       include snippets/fastcgi-php.conf;
        #
        #       # With php-fpm (or other unix sockets):
        #       fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
        #       # With php-cgi (or other tcp sockets):
        #       fastcgi_pass 127.0.0.1:9000;
        #}

        # deny access to .htaccess files, if Apache's document root
        # concurs with nginx's one
        #
        #location ~ /\.ht {
        #       deny all;
        #}

    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/youtubealbumsplitter.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/youtubealbumsplitter.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}

6. I then created a new file called .env and added the following code to it:

COOKIE = "YOUTUBE SESSION COOKIE"

This helps me export secret keys & sensitive data outside of the codebase avoiding accidental leaks of API keys as well as ease of customization

7. I then ran the following command to build my Docker image:

docker compose build
docker compose build -d

This created a new Docker image for both the frontend and backend, each running on separate instances.

8. I then installed SSL by using LetsEncrypt + CertBot.

9. I enabled automatic renewals for my SSL certificate by setting up a cron job.

From this I was able to successfully deploy and run my droplet with minimal downtime. Deploying was honestly the hardest part of this project. I had to learn about setting up a server with DigitalOcean (which I'd never done before), configuring DNS settings, and then actually setting up the server to run the code. It was definitely a lot harder than most of my personal projects, but the amount I learned from this project was invaluable.

Conclusion

Whilst this project took a little longer than anticipated to complete, it was worth it in the end. Not only did we get a chance to improve our skills with React and Node.js, but we also ended up with a tool that is actually useful!

Since we first released Youtube Album Splitter, we have had great feedback from the community and have even seen a few people use it in creative ways that we never thought of.

It just goes to show that with a little bit of effort, you can create something that can make a real difference to people's lives. So, if you have an idea for a project, don't be afraid to give it a go - you never know where it might lead.

For those wanting to try the application, please do! The site is accessible at https://youtubealbumsplitter.com/