Skip to content

Developer's Guide

Jump to bottom
Hallie edited this page Jan 22, 2024 · 10 revisions

Developer's Guide

Looking to get started developing and improving WiCHacker-Manager? This is where you should start

Developer's Note

The most important priority of the WiCHacker-Manager is to try and future-proof the application. Whenever possible we have chosen the simplest or easiest to understand/operate alternative to solve an issue. Since our application load is relatively small and the turnover of maintainers will be very large, we do not need optimizations that will make things harder to understand for future developers who may not be as familiar with the technology.

Common Gotchas

  • Sometimes UI development differs from production behavior because we are using react dev server vs productionized build

Setup

General Infrastructure

AWS Required Services:

  • EC2 - compute
  • S3 - resume storage
  • ParameterStore - secret storage
  • IAM - permissions
  • RDS - database
  • Cloudwatch - viewing logs
  • KMS - key management service

DNS Updates:

  • Cloudflare

Local Development

Quick Note: Local development still relies on AWS secrets and everything is determined by environment variables to configure local vs production configurations

Notes on Running the Application:

The database is run in a local docker container. You can also run a database outside of a container, but the connection information comes from secrets so make sure your database matches that same configuration

Most local development is dependent on running the applications on your computer. If you are developing the UI, you can run the API in a docker container. If you are developing the API, I would recommend running both locally to speed up development and you won’t have to worry about as many docker networking issues like calling containers by name as opposed to using localhost.

  1. Setup AWS Account
    1. Have AWS Root User Create User Account
    2. Generate AWS Secret Pair through IAM
    3. Root User will have to authorize permissions for new user
    4. Create .aws folder in user’s home directory
      1. Credentials tell the application who you are, while config tells it to look in the correct region for our tools
    5. Create ~/.aws/credentials file
      1. See below for default credentials file
    6. Create ~/.aws/config file
      1. See below for default config file
  2. Clone Repository
    1. Make sure your github account is in the WiCHacks github organization
      1. Lana, Webmaster, WiCHacks Director, Projects Director or Logistics Director should be able to make this happen
    2. Clone the WiCHacker-Manager repository
  3. Install docker and docker-compose
    1. NOTE: This is the only place in this guide where there will be a difference between operating systems. All other commands have only been verified to work on a Mac dev environment
    2. Mac
      1. Navigate to the docker download page and click the download for your applicable Mac. Make sure to choose Intel or Apple Silicon accordingly
      2. Open the .DMG file that was downloaded. Then, like any other Mac app, drag it to the applications folder
      3. Close the DMG, and then open the newly installed Docker app
      4. When you launch it for the first time, Docker will ask you for your administrator password. This is normal, and safe. Docker is a trusted tool.
      5. Docker will load, potentially open another authorization prompt, let it load
      6. Eventually, Docker will settle down, and you should see a window with a green bar in the bottom left corner
      7. NOTE: You will need Docker running while developing on this project
    3. Windows
      1. Use WSL
    4. Install Dependencies
      1. Fronted
        1. Install NPM
        2. Navigate to the /ui directory and run npm install
      2. Backend
        1. Navigate to the /api directory
        2. Run pip install -r requirements.txt
      3. Database (Optional)
        1. Pull Mysql docker image
        2. Run docker pull mysql:8.0.28
    5. Run Applications
      1. NOTE: This portion of the guide assumes you are running the front and backend directly on your computer and the database in a Docker container
      2. Frontend
        1. Navigate to /ui folder
        2. Open package.json
          1. On line 5, add "proxy": "https://localhost:5002",
        3. Open /src/config/axios.js
          1. Change apiDomain to return localApiDomain
          2. Change getAxios to return localAxios
        4. Run npm start
        5. Application should start running on http://localhost:3000
      3. Database
        1. You can lookup the docker commands to run the docker container as a single container or remove all other elements of the docker-compose.yml aside from the database service
      4. Backend
        1. Navigate to /api
        2. Open /src/server.py
        3. All the way at the bottom, change app.run() to include the additional parameter port=5002
        4. Run server.py
        5. NOTE: If the database is not running, or AWS connections are not enabled, this will error

Blank File Templates

./aws/credentials

[default]
aws_access_key_id=AKIAIOSFODNN7EXAMPLE
aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

./aws/config

[default]
region=us-east-1
output=json

Production

Pre-Requisites: Make sure you have the infrastructure up and running before continuing.

  1. SSH into EC2
  2. Clone repository
    1. Create SSH keypair
    2. Add keypair to GitHub account
    3. Run clone
  3. Navigate to root of project directory
  4. Run docker-compose -f prod-docker-compose.yml up -d
  5. Navigate to https://apply.wichacks.io and verify

Permissions

At the moment we do have the unfortunate need to manually add the first "admin" to WiCHackerManager. This admin can then work on adding permissions to other users once we get the permissions portal set up.

Permissions can be added manually by modifying the user permission linking table in the database to link a user id to a specific permission.

Infrastructure

Eventually, we will be creating a terraform (or some other infrastructure tool) script that will create the environment for us. Until then, it's a manual process

Note to Alex: add more detail to this section as it is currently quite vague. Until this is updated the advice for everything is to try and match the previous setup to the best of your ability and reach out if you have any questions

IAM roles

Highly suggested to steal these from the prior year's AWS account as they are very confusing if you have not done it before.

If you cannot copy the IAM roles, settle in because you're going for a ride. Every interaction between users/S3 buckets, as well as Users/Secrets Ec2s/Databses can be handled via an IAM role. You'll have to google around to figure out exactly how to set these up until we save the IAM roles as JSON docs somewhere - I am sorry I cannot be of more help.

EC2

EC2's can be launched with their own security groups which allows us to restrict the requests coming in and out. Please restrict all non-necessary traffic to these instances to prevent unauthorized access.

Database

If we can afford an RDS instance that's great and highly preferred, if not running a docker container with a database on an EC2 is a very much disliked alternative. The database should only be accessible from the EC2 where the API is running. All other requests should be blocked

S3 Buckets

You'll have to create a new bucket for each year of WiCHacks so we can differentiate resumes

Frontend

Overview

The frontend of WiCHacker Manager is built in React. React was chosen as it is one of the largest frameworks available as of the start of development, as well as being taught by the Software Engineering department at RIT.

Higher Order Components

There are currently two higher order components that assist with authentication and authorization of pages. They can be found in the hocs folder, while being utilized in the routes.js file protecting different routes with differing levels of security.

Considerations

Javascript can be hard for backend developers. The triple equals, the non-obvious truthy statements and null checks can look somewhat foreign to those who aren’t super familiar with front end development. I know there are many super cool and very succinct javascript patterns that can be quite efficient and useful, but I would recommend avoiding these when possible as the functionality can be obscure. If it is necessary to use a super concise pattern, please make sure to heavily comment this code to increase readability for future developers.

Our users will likely be very technical. As such, try not to include any console logs to keep as much information as possible hidden from the users. WiCHacks is a big target for malicious actions of all sorts so security and protecting errors / vulnerabilities is very important for us.

Backend

Documentation

Swagger

To make life easier for future developers, please provide swagger docs for all new endpoints and update existing docs as needed. The format can be finicky, try to pattern match as much as possible.

Swagger page can be served up from the backend at the path "/docs/swagger"

Development

All endpoints are within the controller folder, endpoints should be kept relatively small with calls to relevant-named services so future developers can see what the purpose of the code is at a glance. Endpoints call methods in the data directory which setup and process responses from our persistent data storage. If you would like to interact with the database we use the functions held in db/db_utils. If you need to interact with s3, those functions are held in utils/aws.

Endpoint format:

  1. Authenticate payload
  2. Check permissions
  3. Call helper functions
  4. Return

Unit Tests

The current state of the application has very minimal unit tests as development was rushed. Please consider adding unit tests for all future code changes within the tests folder. These will be run as part of our github actions to try and ensure no inadvertant bugs are introduced to the application.

The current controller tests can be used as examples for how to mock objects for your tests

Data Storage

General

We store persistent data in two places: a mysql database and s3 buckets. The s3 buckets are currently just used for resumes with everything else being stored in the database.

Important Dev Note: For identifying users, both user id and auth0 id are unique to users. Auth0 id is considered more sensitive and is typically used to identify a User, while user id is used when linking users to other tables

Database Schema

PDF as of v1.0 can be found here - https://drive.google.com/file/d/13RhU7bKRbpWzV8v6u0zpWvNR_Am8cWPn/view?usp=drive_link Diagram itself for editing can be found here but I can't seem to find exactly where that file lives so good luck with permissions - https://drive.google.com/file/d/1dNOzpJAHBnJuPfcUaCW176DU-cttzI6F/view?usp=drive_link

Database Migrations

When changes are made to the database they need migration files. This involves creating a script to make your changes in an "up" file, as well as a script to remove your changes in a "down" file. This lets us keep our database structure documented and standardizes changes. Migration files can be found in db/migration/scripts and are named with the date and a small title describing the changes

S3 / Resumes

Hacker resumes are stored in S3 based on userID, make sure the bucket corresponds to the current year

Permissions

We break out permissions by data and type of access. Here are how things stand as of right now -

Data:

  • Hackers
  • Permissions
  • CheckIn
  • Statistics
  • Console (the admin side of WiCHackManager)

Type of Access:

  • Read
  • Write

Editing Permissions

To edit permissions, you will modify the UserPermissions table. To make someone an admin, you will need to add 9 lines per admin. Each permission_id (10-18) paired with their user_id(which you can retrieve from the Users table). You may do this for as many users as you would like.

This is how we control users level of access within the application and should be modified very carefully

Authentication

To make life easy on ourselves we use Auth0 for authentication on both the frontend and backend. On the backend we just need to authenticate the token passed with each request from the frontend. We can get a user's auth0_id from this request which will allow us to uniquely identify the user who made each request

GitHub

Actions / Build Pipeline

We have two actions in github: api.yaml - Builds and runs unit tests on api. This will run on every push publishImages.yaml - builds and pushes docker images to the container registry, will run on every push to main (i.e. - approved PR)

Standards

To protect ourselves, we require all code changes to be reviewed by at least 1 person familiar with the application that did not write the code. This includes administrators on the repository along with the initial creators and any subsequent webmasters.

For development, please create feature branches off of main and then open up Pull Requests back to main to get your code into production. All code must be tested locally before the PR is looked at to try and eliminate bugs. Our unit tests are not robust right now so manual testing is very important

Container Registry

We use the Github Container Registry as persistent public storage for our docker images. We hold one image for the ui and one for the api.

AWS / Infrastructure

General Note

Our infrastructure for both production and local development relies on AWS.

Future Changes

If someone wants to switch cloud providers in the future, please investigate very thoroughly to make sure every way AWS is currently utilized is understood. There are a few hidden benefits that most likely exist at all cloud providers but require understanding and additional setup beyond just getting some allocated space on a cloud provider’s machines.

Deployments

WiCHacks 💜 Women in Computing 💜 Rochester Institute of Technology

Clone this wiki locally