Mastodon on AWS (für Colin.)

Installing Mastodon on AWS (for Colin)

https://github.com/widdix/mastodon-on-aws

Preconditions:
1. I assume you are using Unix. If not, you will have to translate these to your OS.
2. Install GIT:
  1. Because the pattern in the original script referenced in the above articles has a slight problem, you must download the source, modify it, and then manually initiate the process. It is not difficult, but it is not as easy as the blog article envisions it.
  2. https://git-scm.com/book/en/v2/Getting-Started-Installing-Git
3. Install AWS CLI and Session Manager Plugin:
  1. You will need to call the AWS Cloud Formation from your command line. Once installed, you must log into the webserver to run a command, requiring you to install the AWS_CLI and Session Manager Plugin, both of which are available from Amazon for your OS (probably).
  2. AWS _CLI: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html
  3. Session Manager Plugin: https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-install-plugin.html
4. Install Python 3.x and boto3:
  1. You should have Python3 installed on any Unix system (which is why I use it). However, you must also install the boto3 lib, which provides an API for AWS. These commands will create a version of Python for you to use and only install what you need in it (eliminating any lib conflicts with your system Python). Then, install the boto3 lib.
```
python -m venv venv
./venv/bin/python install boto3
```
5. Install Docker
  1. Docker must be installed to create the keys required to install the software.
    1. Install Docker: https://docs.docker.com/engine/install/
6. Some patience
  1. Cloud architecture takes time—sometimes a lot of time. Just be patient. With one exception (getting your SES approved for use), things should not take more than a few hours. But they might take a few hours, so don’t freak out.

I needed to modify the mastodon.yaml template, because it would not work out of the box, it was pointing to an unsupported engine version. So, I tweaked the file to use ANY version 14. The longer time goes this might change. But this was the change I made. You will have to do it as well.

--- a/mastodon.yaml
+++ b/mastodon.yaml
@@ -141,7 +141,7 @@ Resources:
         DBBackupRetentionPeriod: !Ref DatabaseBackupRetentionPeriod
         DBMasterUsername: 'mastodon'
         DBMultiAZ: 'false'
-        EngineVersion: '14.5'
+        EngineVersion: '14'
       TemplateURL: './node_modules/@cfn-modules/rds-postgres/module.yml'
   Cluster:
     Type: 'AWS::CloudFormation::Stack'

The GIT instructions include a hint about deploying this from the repo (not using the tool in AWS), so this is what we are going to do.

$ npm install
$ aws cloudformation package --template-file mastodon.yaml --s3-bucket <S3_BUCKET> --output-template-file packaged.yml
$ aws cloudformation deploy --template-file packaged.yml --stack-name mastodon-on-aws --capabilities CAPABILITY_IAM --parameter-overrides "DomainName=<DOMAIN_NAME>" "SecretKeyBase=<SECRET_KEY_BASE>" "OtpSecret=<OTP_SECRET>" "VapidPrivateKey=<VAPID_PRIVATE_KEY>" "VapidPublicKey=<VAPID_PUBLIC_KEY>"

First, you need local credentials to log into the correct AWS Profile. Keep in mind that this is how I do things. You might do them differently. (Specifically, I use hardcoded credentials, which is probably a really 2015 way to do things.)

~/.aws/config

[default]
region = us-east-2
output = json
[profile rg]
region = eu-west-2
output = json
[profile to]
region = us-east-2
output = json

~/.aws/credentials

[default]
aws_access_key_id = <KEY_ID>
aws_secret_access_key = <PASSWORD>
[rg]
aws_access_key_id = <KEY_ID>
aws_secret_access_key = <PASSWORD>
[to]
aws_access_key_id = <KEY_ID>
aws_secret_access_key = <PASSWORD>

To generate the passwords, we need to bring up a mastodon container on our local machine and produce the required files using the following script.

#!/bin/bash

# Ensure the mastodon_secrets directory exists
mkdir -p mastodon_secrets

# Run the Docker container and capture the output
docker run -it --rm \\
    ghcr.io/mastodon/mastodon:latest sh -c '

    # Generate SECRET_KEY_BASE
    SECRET_KEY_BASE=$(bundle exec rake secret)
    echo "SECRET_KEY_BASE=$SECRET_KEY_BASE"

    # Generate OTP_SECRET
    OTP_SECRET=$(bundle exec rake secret)
    echo "OTP_SECRET=$OTP_SECRET"

    # Generate VAPID keys 
    bundle exec rake mastodon:webpush:generate_vapid_key
' > mastodon_secrets/mastodon_secrets.txt

# Extract the keys and save them to separate files
while read -r line; do
    key_name=$(echo "$line" | cut -d '=' -f1)
    key_value=$(echo "$line" | cut -d '=' -f2-) # Keep everything after the first '='
    echo "$key_value" > "mastodon_secrets/$key_name.txt"
done < mastodon_secrets/mastodon_secrets.txt

Make an S3 bucket in the correct AWS Account. This bucket stores some information when the cloud formation runs. N.B. profile rg is the profile I set up before. If you are not using a profile, you can remove it.
```
aws s3 mb s3://m-artsoqueer --profile rg
```
Make the package required for the deployment. This will take the template and produce a specific cloud formation template for YOUR mastodon deployment. Remember, things take a little while.
```
aws cloudformation package --template-file mastodon.yaml --s3-bucket m-artsoqueer --output-template-file asq-packaged.yml --profile rg
```

Deploy the new package. This is the main thing that does all the work. Remember to be patient and also be aware that it is really likely this will fail (I will talk about that in the next step). But when you run this, go do some other stuff for a while (or you could open the Cloud Platform dashboard and watch as it slowly creates your infrastructure).

aws cloudformation deploy \\
    --template-file asq-packaged.yml \\
    --stack-name m-artsoqueer \\
    --capabilities CAPABILITY_IAM \\
    --parameter-overrides \\
        "DomainName=artsoqueer.com" \\
        "SecretKeyBase=$(cat mastodon_secrets/SECRET_KEY_BASE.txt)" \\
        "OtpSecret=$(cat mastodon_secrets/OTP_SECRET.txt)" \\
        "VapidPrivateKey=$(cat mastodon_secrets/VAPID_PRIVATE_KEY.txt)" \\
        "VapidPublicKey=$(cat mastodon_secrets/VAPID_PUBLIC_KEY.txt)" \\
    --profile rg

Set up your Certificates. Odds are almost 100% this failed if you have your domain registered ANYWHERE except at Amazon. The reason is that the script will create a certificate. This will be used to provide encrypted access to Mastodon. The problem is that AWS won’t use it until you prove you own the domain you are attempting to modify.
1. N.B. Remember that the AWS Dashboard is Region-Sensitive. If you expect to see something but don’t, make sure you have the correct region chosen.
2. It has failed because the status is ‘waiting.’ In this example, they are both approved and working fine.
3. You need to update your domain to delegate its management to Amazon.
  1. Find the NS record:
  2. Update your domain record:
    1. This example is hover.com, but your mileage may vary.
Set up Secure Email Service (SES):
1. You will have an identity set up for your SES, BUT unless you have done this before, you will need to get enrolled. If you got to the SES manager and click on ‘Identities’ you will see your identity there. And if it says ‘waiting’ you will need to open a ticket and explain how you intend on using the email service.
  1. N.B. Amazon wants you not to spam people, Mastodon needs email to manage the passwords.
  2. There will be a setup link that will take you to a support system. You should tell them something like this.
```
I have set up a Mastodon server. It requires email access to maintain user identities. The system will send one email to the user when they create the account, and then if the user forgets their password, they can request that the system send them an email to reset it.
```
Okay, wait a little bit:
1. You are waiting for the cert to be verified (it might take a few hours) and for the human in the loop to approve your email sending.

Creating the initial account:

At some point, your mastodon server will be on the Internet. Check it using your browser and go to it. It will show up if you wait a little bit. If it does not show up after a day, then you will need to find someone who is more experienced in Cloud Formation because this document will not help you debug problems.

Set the stack_name_prefix and profile_name to the correct values and then run this Python program.

import boto3

def find_mastodon_task(stack_name_prefix, profile_name):
    session = boto3.Session(profile_name=profile_name)
    ecs_client = session.client('ecs')

    response = ecs_client.list_clusters()

    # Get cluster ARNs from response
    cluster_arns = response['clusterArns']

    # Filter and describe clusters to get names
    clusters = []
    for arn in cluster_arns:
        cluster_details = ecs_client.describe_clusters(clusters=[arn])
        for cluster in cluster_details['clusters']:
            if cluster['clusterName'].startswith(stack_name_prefix):
                clusters.append(arn)

    if not clusters:
        raise ValueError("No ECS clusters found matching the prefix.")

    for cluster_arn in clusters:
        # 2. List Tasks in Each Cluster
        tasks = ecs_client.list_tasks(
            cluster=cluster_arn,
            desiredStatus='RUNNING'  
        )['taskArns']

        for task_arn in tasks:
            # 3. Describe Task to Get Details
            task = ecs_client.describe_tasks(
                cluster=cluster_arn,
                tasks=[task_arn]
            )['tasks'][0]

            # 4. Check Task Definition for WebService
            if '-WebService-' in task['taskDefinitionArn']:
                return {
                    'cluster_name': cluster_arn.split('/')[-1],
                    'task_id': task_arn.split('/')[-1]
                }

    raise ValueError("No running WebService task found in the clusters.")

if __name__ == '__main__':
    stack_name_prefix = "m-artsoqueer"
    profile_name = "rg"
    result = find_mastodon_task(stack_name_prefix, profile_name)
    print(f"Found Mastodon Task:\\n  Cluster Name: {result['cluster_name']}\\n  Task ID: {result['task_id']}")
    print (f"aws ecs execute-command --cluster {result['cluster_name']} --container app --command /bin/bash --interactive --task {result['task_id']} --profile {profile_name}")

This will produce a command you can run use to log into the Mastodon web server

./venv/bin/python get_machine.py

Found Mastodon Task:
  Cluster Name: m-artsoqueer-Cluster-<junk>-Cluster-<moarjunq>
  Task ID: bb3fdb61c161476387543276e4188f62
aws ecs execute-command --cluster m-artsoqueer-Cluster-<junk>-Cluster-<moarjunq> --container app --command /bin/bash --interactive --task bb3fdb61c161476387543276e4188f62 --profile rg

If you run this last line, it will open a shell command to the Mastodon server. Then, you need to create your initial user. Mastodon is built using Ruby on Rails, so if you know it, you should understand how to use the repl. But here are the commands to create the initial ‘Owner’ account and permission it so you can log in.
```
RAILS_ENV=production bin/tootctl accounts create <ACCOUNT_ID> --email <ACCOUNT EMAIL> --role Owner

New password: <NEWPASSWORD>
```
If you are NOT the Mastodon user, get the user their new password. They should also have received an email from Mastodon stating that their account was created.
Now, there is a bit of a chicken-and-egg problem with Mastodon creating the account. It must be approved by someone, and since this is the first account, it needs to be approved by the account you just created. I am pretty sure you can spot the flaw here.
```
RAILS_ENV=production bin/tootctl accounts approve <ACCOUNT_ID>
```

12: Profit

Untitled