diff --git a/README.md b/README.md index 08ba208..d7ca0b9 100644 --- a/README.md +++ b/README.md @@ -1,243 +1,177 @@ - -# Shipment List Demo Application - AWS in PROD and LocalStack on DEV environment - - -| Environment | | -|------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| __Services__ | Amazon S3, Lambda, DynamoDB, SNS, SQS | -| __Integrations__ | AWS SDK, Terraform, AWS CLI | -| __Categories__ | Spring Boot, S3 Trigger | -| __Level__ | Intermediate | -| __Works on__ | LocalStack v3 | - - -### UPDATE - -The Terraform configuration file now randomly generates names for the bucket, in order to avoid conflicts -at a global scale on AWS. This name shall be written out to a properties file, which the app will pick up -and use for the S3 client. Furthermore, the name is also passed as an environment variable to the Lambda function by Terraform, -so there's no need to worry about managing it. - +# Shipment Management Application with Lambda Image Processing + +| Key | Value | +| ------------ | --------------------------------------------------------------------------------------------------------- | +| Environment | LocalStack, AWS | +| Services | S3, Lambda, DynamoDB, SNS, SQS | +| Integrations | Terraform, AWS SDK, Spring Boot, React, Testcontainers | +| Categories | Serverless, Storage, Messaging | +| Level | Intermediate | +| Use Case | IaC Testing, AWS Parity | +| GitHub | [Repository link](https://github.com/localstack-samples/sample-shipment-list-demo-lambda-dynamodb-s3) | ## Introduction -This application was created for demonstration purposes to highlight the ease of switching from -using actual AWS dependencies to having them emulated on LocalStack for your *developer environment* . -Of course this comes with other advantages, but the first focus point is making the transition. - -## Architecture Overview - -![Diagram](app_diagram.png) - -## Prerequisites - -- [Maven 3.8.5](https://maven.apache.org/install.html) & [Java 17](https://www.java.com/en/download/help/download_options.html) -- [AWS free tier account](https://aws.amazon.com/free/) -- [LocalStack](https://localstack.cloud/) -- [Docker](https://docs.docker.com/get-docker/) - for running LocalStack -- [Terraform](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli) (+ Python pip for [tflocal](https://pypi.org/project/terraform-local/)) for creating AWS & LocalStack resources -- [npm](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm) - for running the frontend app - -#### What it does - -*shipment-list-demo* is a Spring Boot application dealing with CRUD operations a person can -execute -on a bunch of shipments that they're allowed to view - think of it like the Post app. -The demo consists of a backend and a frontend implementation, using React to display the -information. -The AWS services involved are: - -- [S3](https://docs.localstack.cloud/user-guide/aws/s3/) for storing pictures -- [DynamoDB](https://docs.localstack.cloud/user-guide/aws/dynamodb/) for the entities -- [Lambda](https://docs.localstack.cloud/user-guide/aws/lambda/) function that will validate the pictures, apply a watermark and replace non-compliant files. -- [SNS](https://docs.localstack.cloud/user-guide/aws/sns/) that receives update notifications -- [SQS](https://docs.localstack.cloud/user-guide/aws/sqs/) that subscribes to a topic and delivers the messages to the Spring Boot app - - -#### How to use it - -We’ll be walking through a few scenarios using the application, and we expect it to maintain the -behavior in both production (AWS) and development (LocalStack) environments. - -We’ll take advantage of one of the core features of the Spring framework that allows us to bind our -beans to different profiles, such as dev, test, and prod. Of course, these beans need to know how to -behave in each environment, so they’ll get that information from their designated configuration -files, `application-prod.yml`, and `application-dev.yml`. - -#### Terraform - -The Terraform configuration file will create the needed S3 bucket, the DynamoDB `shipment` table and populate it with some -sample data, the Lambda function that will help with the picture processing (make sure you create the jar), -the SQS and SNS which will bring back the notification when the processing is finished. - - -## Instructions +This sample demonstrates a full-stack shipment management application that showcases integration between multiple AWS services. The application consists of a Spring Boot backend with a React frontend, implementing CRUD operations on shipments with Lambda image processing capabilities. To test this application sample, we will demonstrate how you use LocalStack to deploy the infrastructure on your developer machine and run the application locally. The demo highlights the ease of switching from actual AWS dependencies to LocalStack emulation for development environments without any code changes. -### Only run once +## Architecture -The following instructions only need to run once, weather you choose to run both cases, on AWS and -LocalStack, or just jump straight to LocalStack. +The following diagram shows the architecture that this sample application builds and deploys: -### Building the validator module +![Architecture Diagram](app_diagram.png) -Step into the `shipment-picture-lambda-validator` module and run `mvn clean package shade:shade`. -This will create an uber-jar by packaging all its dependencies. We'll need this one in the next -steps. We can keep the same jar for both running on AWS and LocalStack. +- [S3](https://docs.localstack.cloud/aws/services/s3/) for storing shipment pictures and Lambda deployment packages. +- [Lambda](https://docs.localstack.cloud/aws/services/lambda/) function that validates uploaded pictures, applies watermarks, and replaces non-compliant files. +- [DynamoDB](https://docs.localstack.cloud/aws/services/dynamodb/) table to store shipment entities with enhanced client mapping. +- [SNS](https://docs.localstack.cloud/aws/services/sns/) topic that receives update notifications from the Lambda function. +- [SQS](https://docs.localstack.cloud/aws/services/sqs/) queue that subscribes to the SNS topic and delivers messages to the Spring Boot application. +## Prerequisites -### Running the GUI - -`cd` into `src/main/shipment-list-frontend` and run `npm install` and `npm start`. -This will spin up the React app that can be accessed on `localhost:3000`. -You'll only see the title, as the backend is not running yet to provide the list of shipments. - -For running it on Windows, there are some -[extra requirements](https://learn.microsoft.com/en-us/windows/dev-environment/javascript/react-on-windows) -, -but no worries, it should be straightforward. +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) with a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/). +- [AWS CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) with the [`awslocal` wrapper](https://docs.localstack.cloud/user-guide/integrations/aws-cli/#localstack-aws-cli-awslocal). +- [Terraform](https://docs.localstack.cloud/user-guide/integrations/terraform/) with the [`tflocal`](https://github.com/localstack/terraform-local) wrapper. +- [Maven 3.8.5+](https://maven.apache.org/install.html) & [Java 17](https://www.java.com/en/download/help/download_options.html) +- [Node.js](https://nodejs.org/en/download/) & [npm](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm) +- [`make`](https://www.gnu.org/software/make/) (**optional**, but recommended for running the sample application) -#### How to use the GUI +## Installation -After starting the backend, refreshing the React app will fetch a list of shipments. -The weight of a shipment is already given, but not the size, that's why we need pictures to -understand it better, using the "banana for scale" measuring unit. How else would we know?? +To run the sample application, you need to install the required dependencies. -Current available actions using the GUI: +First, clone the repository: -- upload a new image -- delete shipment from the list -- create and update shipment are available only via Postman (or any other API platform) +```shell +git clone https://github.com/localstack-samples/sample-shipment-list-demo-lambda-dynamodb-s3.git +``` -Files that are not pictures will be deleted -and the shipment picture will be replaced with a generic icon, because we don't want any trouble. +Then, navigate to the project directory: +```shell +cd sample-shipment-list-demo-lambda-dynamodb-s3 +``` -## Running on AWS +Next, install the project dependencies by running the following command: -Now, we don’t have a real production environment because that’s not the point here, but most likely -an application like this runs on a container orchestration platform, and all the necessary configs -are still provided. Since we’re only simulating a production instance, all the configurations are -kept in the `application-prod.yml` file. +```shell +make install +``` -### User credentials +This will: +- Build the Lambda validator JAR file +- Install frontend dependencies via npm -Before getting started, it's important to note that an IAM user, who's credentials will be used, -needs to be created with the `AdministratorAccess` policy. Of course, working in a company will have more restrictive and fine-grained -permissions defined, for allowing the creation/update of each individual resource. In this case, we will choose an umbrella policy, that -covers all our needs. +## Deployment -For simplicity, we chose to use full access to all the services, so we don't have to add new permissions later on. -We will be using the user's credentials and export them as temporary environment variables with the -`export` (`set` on Windows) command: +Start LocalStack with the `LOCALSTACK_AUTH_TOKEN` pre-configured: -``` -$ export AWS_ACCESS_KEY_ID=[your_aws_access_key_id] -$ export AWS_SECRET_ACCESS_KEY=[your_aws_secret_access_key_id] +```shell +localstack auth set-token +localstack start ``` -### Creating resources - running Terraform +To deploy the sample application, run the following command: -Make sure you have Terraform [installed](https://developer.hashicorp.com/terraform/downloads) +```shell +make deploy +``` -Under `terraform` run: +The deployment will create: -``` -$ terraform init -$ terraform plan -``` +- S3 buckets for shipment pictures and Lambda code +- DynamoDB table pre-populated with sample shipments +- Lambda function for image validation and processing +- SNS topic and SQS queue for messaging +- All necessary IAM roles and permissions -Once these 2 commands run successfully and no errors occur, it's time to run: -``` -$ terraform apply -``` -If everything finishes successfully, the AWS services should be up and running. +## Testing -### Starting the backend +The sample application provides both automated tests and interactive usage through the web interface. -Go back to the root folder and run the backend simply by using +Start the React frontend: -``` -$ mvn spring-boot:run -Dspring-boot.run.profiles=prod +```shell +cd shipment-list-frontend +npm start ``` -Notice the `prod` profile is being set via command line arguments. +The frontend will be available at `http://localhost:3000`. -### Using the application +Start the Spring Boot backend: -At `localhost:3000` you should now be able to see a list of shipments with standard icons, -that means that only the database is populated, the pictures still need to be added from the -`sample-pictures` folder. -You can now interact with the application using the React app. All services used in the backend are -running on the real AWS cloud. +```shell +mvn spring-boot:run -Dspring-boot.run.profiles=dev +``` +The backend will be available at `http://localhost:8081`. -Before moving on, make sure you clean up your AWS resources by running (also in the `terraform` folder): +You can run full end-to-end integration tests using the following command: -``` -$ terraform destroy +```shell +make test ``` -## Running on LocalStack +### Using the Application -To switch to using LocalStack instead of AWS services just run `docker compose up` in the root -folder to spin up a Localstack container. +Once both frontend and backend are running: -### Creating resources on LocalStack +1. Visit `http://localhost:3000` to see the shipment list +2. Upload images to shipments using the web interface +3. Valid images will be processed and watermarked by the Lambda function +4. Invalid files will be rejected and replaced with a placeholder +5. Real-time updates are delivered via Server-Sent Events when image processing completes -To generate the exact same resources on LocalStack, we need `tflocal`, a thin wrapper script around -the terraform command line client. `tflocal` takes care of automatically configuring the local -service -endpoints, which allows you to easily deploy your unmodified Terraform scripts against LocalStack. +Available actions: -You can [install](https://docs.localstack.cloud/user-guide/integrations/terraform/) the `tflocal` -command via pip (requires a local Python installation): +- Upload new images to existing shipments +- Delete shipments from the list +- View processed images with watermarks +- Create and update shipments via API endpoints -``` -$ pip install terraform-local -``` -Once installed, the `tflocal` command should be available, with the same interface as the terraform -command line. Try it out: +## Use Cases -``` -$ tflocal --help -Usage: terraform [global options] [args] -... -``` +### IaC Testing -From here on, it's the same as using AWS. In the `terraform` folder, run the `cleanup` script -to get rid of any files that keep track of the resources' state. Then: +This sample demonstrates Infrastructure as Code (IaC) testing by using identical Terraform configurations for both AWS and LocalStack environments. The application leverages Spring profiles to seamlessly switch between production and development configurations without code changes. -``` -$ tflocal init -$ tflocal plan -$ tflocal apply -``` +The Terraform configuration defines all necessary AWS resources and their relationships, while `tflocal` automatically reconfigures endpoints for LocalStack. This approach enables: -We run the exact same commands for the exact same file. We no longer need to pass any environment -variables, since the bucket name is generated and passed by Terraform. +- Validation of infrastructure changes before AWS deployment +- Consistent development environments across teams +- Faster iteration cycles during development +- Cost-effective testing of AWS integrations -### Starting the backend +### AWS Parity -After that, the Spring Boot application needs to start using the dev profile (make sure you're in -the -root folder): +The sample showcases LocalStack's AWS parity by demonstrating identical behavior between LocalStack and AWS environments: -``` -$ mvn spring-boot:run -Dspring-boot.run.profiles=dev -``` -### Using the application +- S3 trigger configurations work identically in both environments +- Lambda function execution and environment variable handling +- DynamoDB enhanced client operations and table management +- SNS/SQS messaging patterns and subscription handling +- IAM role and policy enforcement -Go back to `localhost:3000` and a new list will be available; notice that the functionalities of -the application have not changed. +The application uses Spring Boot profiles (`dev` for LocalStack, `prod` for AWS) with different endpoint configurations (`application-prod.yml`, `application-dev.yml`), ensuring the same codebase works across both environments. Testcontainers integration provides additional validation that the LocalStack environment accurately emulates AWS behavior. -There you have it, smooth transition from AWS to Localstack, with no code change. 👍🏻 +## Summary -## Contributing +This sample application demonstrates how to build, test, and deploy a full-stack serverless application using AWS services and LocalStack. It showcases the following patterns: -We appreciate your interest in contributing to our project and are always looking for new ways to improve the developer experience. We welcome feedback, bug reports, and even feature ideas from the community. -Please refer to the [contributing file](CONTRIBUTING.md) for more details on how to get started. +- Defining and deploying S3, Lambda, DynamoDB, SNS, and SQS resources using Terraform. +- Building a Lambda function for automated image processing with watermarking capabilities. +- Integrating multiple AWS services in a Spring Boot application with reactive messaging. +- Using Spring Boot profiles to seamlessly switch between LocalStack and AWS environments. +- Implementing real-time updates using Server-Sent Events and SQS message consumption. +- Leveraging Testcontainers for integration testing against LocalStack infrastructure. +- Utilizing `tflocal` and `awslocal` to streamline local development workflows. +## Learn More +- [Smooth transition from AWS to LocalStack for your DEV environment](https://hashnode.localstack.cloud/smooth-transition-from-aws-to-localstack-for-your-dev-environment) (**recommended**) +- [LocalStack and AWS Parity explained](https://blog.localstack.cloud/2022-08-04-parity-explained/) +- [Using AWS SDKs with LocalStack](https://docs.localstack.cloud/aws/integrations/aws-sdks/) +- [Deploying Terraform with LocalStack](https://docs.localstack.cloud/user-guide/integrations/terraform/) +- [Testcontainers with LocalStack](https://docs.localstack.cloud/user-guide/integrations/testcontainers/)