Current steps to go from pushing to master to bringing to production deployment

Local machine

If changes have been made to bootstrap submodule

# IMPORTANT NOTE: IF YOU MAKE CHANGES TO BOOTSTRAP SUBMODULE YOU NEED TO RUN THE DEV SERVER AND THEN PUSH THE CHANGES TO THE MAIN REPO TOO

Heed important note above and run development server with python3 app.py; make sure to push changes
cd static/bootstrap; git status
npm run css-compile; npm run css-purge
git add .; git commit -m 'latest'; git push origin submodule_branch

First of all, just tons of manual checks to make sure nothing is broken. Obviously not ideal
git add and push

Remote Server

cd to directory with repo
git stash for changes on remote server
git pull

If changes have been made to bootstrap submodule

cd static/bootstrap; git status
git pull origin submodule_branch
# Might need to do a git stash before

# Copy static files to nginx directory
sudo cp -r ~/twitch-clips/static/dist /var/www/html/static
sudo cp -r ~/twitch-clips/static/img /var/www/html/static
sudo cp -r ~/twitch-clips/static/ico /var/www/html/static

# Obfuscate the javascript code
sudo javascript-obfuscator /var/www/html/static/dist/app.js -o /var/www/html/static/dist/app.js

# Restart gunicorn server
cd ~/twitch-clips
pkill gunicorn
gunicorn --bind 127.0.0.1:5000 -w 1 wsgi:app --access-logfile gunicorn-access.log --error-logfile gunicorn-error.log --daemon
There is one last step that is needed but I can't find the solution to. I need some way to tell the clients to reset their cache when I update javascript files or css files.

Currently have the cache set to expire after 1 day but ideally I'd like to set this longer, and instead be able to send a header that informs their browser to clear the cache.

This might be impossible because how would I know which clients to send the header to. Only would work if I could choose to send header for the day and then no longer. But that would mean the cache would have to expire after 1 day.

Issue With Static Assets

Running an npm install from static/bootstrap directory even though static assets aren't served from gunicorn, but rather from nginx

Somehow, I need a way to bundle the static assets with the flask-assets bundle function

How I currently handle it

I currently handle this issue by having to run the development server to get all the assets into the dist folder and then push the changes. While this is not necessarily a horrible thing to do, its another stumbling block and annoyance in which you make mistakes in the pipeline.

Potential Way To Handle It

So I could have a script that runs the server and then also copies the directories afterwards and runs the obfuscator after as well. I'm not sure how I would be able to make sure people don't download the non-obfuscated code. I also need to make sure I run css-purge on the bootstrap css before its added.

Main issue

Commands I had to manually run in order to get the site running after all pulling

git clone --recurse-submodules git@github.com:slgotting/twitch-clips.git regexreducer
pip3 install -r ~/regexreducer/requirements.txt
Copy configuration from config.py from other server. Would be slightly more laborious but I have config on other server already (secret key, etc. not much)
sudo cp -r ~/regexreducer/static/dist /var/www/html/static
sudo cp -r ~/regexreducer/static/img /var/www/html/static
sudo cp -r ~/regexreducer/static/ico /var/www/html/static
sudo javascript-obfuscator /var/www/html/static/dist/app.js -o /var/www/html/static/dist/app.js
sudo mkdir /var/log/usr# This is for exponential backoff log which prob doesnt need to exist there
sudo chown steven:steven /var/log/usr
sudo nginx -s reload
cd ~/regexreducer/static/bootstrap; npm i# this is really unnecessary because assets are already bundled in the nginx static directory

Staging needs with regards to being on same server as production environment

Needs any authorization tokens generated to be synced to both staging and production environments
Needs cronjobs to run in a similar manner to the production server. Not sure whether to separate those or just have one, or have more nested directories in the cron folder, labeled "All", "Production", "Staging", and then recurse from there.

That would seem to add quite a bit of complexity.

Staging/Production Ideas and associated issues

Problem: How to run cronjobs that use relative imports and need to use different locations for said relative imports (ie ~/twitch-clips/ for one and ~/twitch-clips-staging/ for the other)

Idea:

Issues

List of considerations for staging and production deployment automation

        #!/usr/bin/env bash
        
                                ROOT_PASSWORD=$1
        
                                # install latest requirements (git fetch and reset mustve happened in github actions)
                                ~/project-name/venv/bin/pip3 install -r ~/project-name/requirements.txt
        
                                # build the cron jobs using tool
                                ~/project-name/venv/bin/slg-build-cron-jobs -d ~/project-name/cron/staging -u steven -e ~/project-name/.staging_env -v ~/project-name/venv/bin/python3 -t
        
                                # run any necessary commands in submodule directories ( or regular directories for that matter )
                                cd /home/steven/project-name/static/bootstrap; npm run css-compile; npm run css-purge
        
                                # stop the existing running server
                                echo $ROOT_PASSWORD | sudo -S systemctl stop gunicorn_staging.socket
        
                                # start the server up again
                                echo $ROOT_PASSWORD | sudo -S systemctl start gunicorn_staging.service
        
                                # since the server started, it has created distributions of its static assets
                                # therefore in this block we are passing said static assets along to the appropriate nginx served static directory
                                echo $ROOT_PASSWORD | sudo -S cp -r /home/steven/project-name/static/dist /var/www-staging/html/static
                                echo $ROOT_PASSWORD | sudo -S cp -r /home/steven/project-name/static/img /var/www-staging/html/static
                                echo $ROOT_PASSWORD | sudo -S cp -r /home/steven/project-name/static/ico /var/www-staging/html/static
                                echo $ROOT_PASSWORD | sudo -S javascript-obfuscator /var/www-staging/html/static/dist/app.js -o /var/www-staging/html/static/dist/app.js
        
                                # start up the socket again so that in case the server goes down the socket recreates it
                                echo $ROOT_PASSWORD | sudo -S systemctl start gunicorn_staging.socket

List of considerations for init.sh when initially spinning up server

                                #!/usr/bin/env bash
        
                                ROOT_PASSWORD=$1
        
                                PRODUCTION_DIR=/home/user/production-project-name
                                STAGING_DIR=/home/user/staging-project-name
        
                                # Add to the path so we have access to package commands
                                PATH=/home/steven/.local/bin:$PATH
        
                                # Run any initialization scripts needed to get the server up and running
                                $PRODUCTION_DIR/venv/bin/python3 $PRODUCTION_DIR/cron/init_script.py
                                $STAGING_DIR/venv/bin/python3 $STAGING_DIR/cron/init_script.py
        
                                # Run the cronjob build script for cron directories
                                slg-build-cron-jobs -d $PRODUCTION_DIR/cron -u steven -v $PRODUCTION_DIR/venv/bin/python3
                                slg-build-cron-jobs -d $STAGING_DIR/cron/staging -u steven -v $STAGING_DIR/venv/bin/python3
        
                                # Run any css static asset compilation
                                cd $PRODUCTION_DIR/static/bootstrap; npm i; npm run css-compile; npm run css-purge
                                cd $STAGING_DIR/static/bootstrap; npm i; npm run css-compile; npm run css-purge
        
                                # this is just to init the server and create the static file distribution
                                $PRODUCTION_DIR/venv/bin/gunicorn --bind 127.0.0.1:5000 -w 1 wsgi:app --access-logfile gunicorn-access.log --error-logfile gunicorn-error.log --daemon
                                sleep 1
                                pkill gunicorn
        
                                # Copy any static files into the nginx static served directory and run any obfuscation on the javascript files
                                # Making sure to only include distribution files and not every directory from static
                                echo $ROOT_PASSWORD | sudo -S cp -r $PRODUCTION_DIR/static/dist /var/www/html/static
                                echo $ROOT_PASSWORD | sudo -S cp -r $PRODUCTION_DIR/static/img /var/www/html/static
                                echo $ROOT_PASSWORD | sudo -S cp -r $PRODUCTION_DIR/static/ico /var/www/html/static
                                echo $ROOT_PASSWORD | sudo -S javascript-obfuscator /var/www/html/static/dist/app.js -o /var/www/html/static/dist/app.js
        
                                # Do the same thing for the staging site as well
                                # Making sure to only include distribution files and not every directory from static
                                echo $ROOT_PASSWORD | sudo -S cp -r $STAGING_DIR/static/dist /var/www-staging/html/static
                                echo $ROOT_PASSWORD | sudo -S cp -r $STAGING_DIR/static/img /var/www-staging/html/static
                                echo $ROOT_PASSWORD | sudo -S cp -r $STAGING_DIR/static/ico /var/www-staging/html/static
                                echo $ROOT_PASSWORD | sudo -S javascript-obfuscator /var/www-staging/html/static/dist/app.js -o /var/www-staging/html/static/dist/app.js
        
                                # reload the nginx configuration after everythings done
                                echo $ROOT_PASSWORD | sudo -S nginx -s reload
        
                                # boot up the gunicorn staging and production servers
                                echo $ROOT_PASSWORD | sudo -S systemctl enable gunicorn_staging.socket gunicorn_staging.service gunicorn_production.socket gunicorn_production.service
                                echo $ROOT_PASSWORD | sudo -S systemctl daemon-reload
                                echo $ROOT_PASSWORD | sudo -S systemctl start gunicorn_staging.socket gunicorn_staging.service gunicorn_production.socket gunicorn_production.service
                                echo $ROOT_PASSWORD | sudo -S systemctl daemon-reload

Svelte Skills

Recursion using svelte:self
Binding variable within iterable within loop over said iterable
Portal
Multi-Component tree and binding
Implementing example
Implementing modal positioning global