Merge pull request #3 from texpert/shrine-3

Upgrade to work with Shrine version 3

Author: texpert

.github_changelog_generator

@@ -1,3 +1,3 @@
 unreleased=true
-future-release=0.1.2
+future-release=0.2.0
 exclude-labels=duplicate,question,invalid,wontfix,release

.rubocop.yml

@@ -37,7 +37,7 @@ RSpec/ExampleLength:
 RSpec/MultipleExpectations:
   Description: Checks if examples contain too many `expect` calls.
   Enabled: true
-  Max: 5
+  Max: 10
 
 RSpec/MultipleMemoizedHelpers:
   Description: Checks if example groups contain too many `let` and `subject` calls.
@@ -67,6 +67,7 @@ Style/StringLiterals:
 Metrics/AbcSize:
   # The ABC size is a calculated magnitude, so this number can be a Fixnum or
   # a Float.
+  CountRepeatedAttributes: false
   Max: 30
 
 Metrics/ClassLength:

README.md

@@ -1,8 +1,7 @@
 # Shrine::Plugins::AwsLambda
-Provides [AWS Lambda] integration for [Shrine] File Attachment toolkit for Ruby applications
+Provides [AWS Lambda] integration for the [Shrine] File Attachment toolkit for Ruby applications
 
-This is a gem, renamed from initial [shrine-lambda](https://github.com/texpert/shrine-lambda) to [shrine-aws-lambda](https://github.com/texpert/shrine-aws-lambda) 
-for clarity
+This is a gem, renamed from the initial [shrine-lambda](https://github.com/texpert/shrine-lambda) to [shrine-aws-lambda](https://github.com/texpert/shrine-aws-lambda) for clarity
 
 ## Description
 
@@ -53,7 +52,7 @@ either in the [Shrine] initializer, or in [default profile][AWS profiles] in the
                  region:            'your AWS bucket region' }
 ```
 
-Also, for Lamda functions to work, various [AWS Lamda permissions] should be managed on the [Amazon Web Services] side.
+Also, for Lambda functions to work, various [AWS Lambda permissions] should be managed on the [Amazon Web Services] side.
 
 Add to the [Shrine]'s initializer file the Shrine-AWS-Lambda plugin registration with the `:callback_url` parameter, 
 and the [AWS Lambda] functions list retrieval call (which will retrieve the functions list on application initialization 
@@ -71,11 +70,11 @@ and will store the list into the `Shrine.opts[:lambda_function_list]` for furthe
 By default, Shrine-AWS-Lambda is using the S3 bucket named `:cache` for retrieving the original file, and the `:store` 
 named S3 bucket for storing the resulting files.
 
-Srine-Lamda uses the [Shrine backgrounding plugin] for asynchronous operation, so this plugin should be also included
- into the Shrine's initializer.
+Shrine-AWS-Lambda uses the [Shrine backgrounding plugin] for asynchronous operation, so this plugin should be also 
+included into the Shrine's initializer.
 
-Here is a full example of a Shrine initializer of a [Rails] application using [Roda] endpoints for presigned_url's 
-(used for direct file uploads to [AWS S3]) and [AWS Lambda] callbacks:
+Here is a full example of a Shrine initializer of a [Rails] application using a [Roda] endpoint for presigned_url 
+(used for direct file uploads to [AWS S3]) and [AWS Lambda] callbacks):
 
 ```ruby
 # config/initializers/shrine.rb:
@@ -94,72 +93,78 @@ if Rails.env.test?
 else
   require 'shrine/storage/s3'
 
-  secrets = Rails.application.secrets
+  aws_credentials = Rails.application.credentials.aws
 
-  s3_options = { access_key_id:     secrets.aws_access_key_id,
-                 secret_access_key: secrets.aws_secret_access_key,
+  s3_options = { access_key_id:     aws_credentials[:access_key_id],
+                 secret_access_key: aws_credentials[:secret_access_key],
                  region:            'us-east-2' }
 
   if Rails.env.production?
-    cache_bucket = store_bucket = secrets.aws_s3_bucket
+    cache_bucket = store_bucket = aws_credentials.aws_s3_bucket
   else
     cache_bucket = 'texpert-test-cache'
     store_bucket = 'texpert-test-store'
   end
 
-  Shrine.storages = {
-    cache: Shrine::Storage::S3.new(prefix: 'cache', **s3_options.merge(bucket: cache_bucket)),
-    store: Shrine::Storage::S3.new(prefix: 'store', **s3_options.merge(bucket: store_bucket))
-  }
-
-  lambda_callback_url = if Rails.env.development?
-                          "http://#{ENV['USER']}.localtunnel.me/rapi/lambda"
-                        else
-                          "https://#{ENV.fetch('APP_HOST')}/rapi/lambda"
-                        end
+  Shrine.storages = { cache: Shrine::Storage::S3.new(prefix: 'cache', **s3_options.merge!(bucket: cache_bucket)), 
+                      store: Shrine::Storage::S3.new(prefix: 'store', **s3_options.merge!(bucket: store_bucket)) }
 
-  shrine.plugin :aws_lambda, s3_options.merge(callback_url: lambda_callback_url)
-  Shrine.lambda_function_list
-
-  Shrine.plugin :presign_endpoint, presign_options: ->(request) do
-    filename     = request.params['filename']
-    extension    = File.extname(filename)
-    content_type = Rack::Mime.mime_type(extension)
+  ActiveSupport::Reloader.to_prepare do
+   lambda_callback_url = if Rails.env.development? && NGROK_ENABLED
+                          "#{NGROK_URL}/rapi/lambda"
+                         else
+                          "https://#{ENV['APP_HOST'] || 'localhost'}/rapi/lambda"
+                         end
 
-    {
-      content_length_range: 0..1.gigabyte,                         # limit filesize to 1 GB
-      content_disposition: "attachment; filename=\"#{filename}\"", # download with original filename
-      content_type:        content_type,                           # set correct content type
-    }
+   Shrine.plugin :aws_lambda, s3_options.merge!(callback_url: lambda_callback_url)
+   Shrine.lambda_function_list
   end
 end
 
 Shrine.plugin :activerecord
 Shrine.plugin :backgrounding
 Shrine.plugin :cached_attachment_data # for forms
-Shrine.plugin :logging, logger: Rails.logger
+
+Shrine.logger = Rails.logger
+Shrine.plugin :instrumentation
+
+Shrine.plugin :presign_endpoint, presign_options: lambda { |request|
+ filename     = request.params['filename']
+ extension    = File.extname(filename)
+ content_type = Rack::Mime.mime_type(extension)
+
+ { content_length_range: 0..1.gigabyte,                         # limit filesize to 1 GB
+   content_disposition: "attachment; filename=\"#{filename}\"", # download with original filename
+   content_type:        content_type }                          # set correct content type
+}
+
 Shrine.plugin :rack_file # for non-Rails apps
 Shrine.plugin :remote_url, max_size: 1.gigabyte
 
-Shrine::Attacher.promote { |data| PromoteJob.perform_later(data) }
-Shrine::Attacher.delete { |data| DeleteJob.perform_later(data) }
+Shrine::Attacher.promote_block { PromoteJob.enqueue(self.class.name, record.class.name, record.id, name, file_data) }
+Shrine::Attacher.destroy_block { DeleteJob.enqueue(self.class.name, data) }
+```
+
+Take notice that the promote job is a default, not AWS Lambda job: 
 
+```
+Shrine::Attacher.promote_block { PromoteJob.enqueue(self.class.name, record.class.name, record.id, name, file_data) }
 ```
 
-Take notice that the promote job is a default `Shrine::Attacher.promote { |data| PromoteJob.perform_later(data) }`. 
 This is made to be able to use other than AWS storages in the test environment (like Shrine's `FileSystem` storage) 
-and, also, other uploaders which are not using [AWS Lambda]. This job better to be overrided to a `LambdaPromoteJob` 
- directly in the uploaders' classes which will use [AWS Lambda]. 
+and, also, other uploaders which are not using [AWS Lambda]. To use an AWS Lambda job, this job must be overridden 
+to a `LambdaPromoteJob` directly in the uploaders' classes which will use [AWS Lambda] - see below in the **Usage** 
+chapter. 
 
-Another thing used in this initializer is the [localtunnel] application for exposing the localhost to the world for 
+Another thing used in this initializer is the [ngrok-wrapper] gem for exposing the localhost to the world for 
 catching the Lambda callback requests.
 
 #### How it works
 
-Shrine-Lamnda works in such a way that an "assembly" should be created in the `LambdaUploader`, which contains all 
+Shrine-AWS-Lambda works in such a way that an "assembly" should be created in the `LambdaUploader`, which contains all 
 the information about how the file should be processed. A random generated string is appended to the assembly, stored 
 into the cached file metadata, and used by the Lambda function to sign the requests to the `:lambda_callback_url`, 
-along with the `:access_key_id` from the temporary credentials Lambda function is running with.
+along with the AWS `:access_key_id` from the AWS credentials Lambda function is running with.
 
 Processing itself happens asynchronously - the invoked Lambda function will issue a PUT HTTP request to the  
 `:lambda_callback_url`, specified in the Shrine's initializer, with the request's payload containing the processing  
@@ -168,9 +173,9 @@ results.
 The request should be intercepted by a endpoint at the `:lambda_callback_url`, and its payload transferred to the 
 `lambda_save` method on successful request authorization. 
 
-The authorization is calculatating the HTTP request signature using the random string stored in the cached file and 
-the Lambda function's `:access_key_id` received in the request authorization header. Then, the calculated signature is
- compared to the received in the same authorization header Lambda signature.
+The authorization is calculating the HTTP request signature using the random string stored in the cached file and 
+the AWS Lambda function's `:access_key_id` received in the request authorization header. Then, the calculated 
+signature is compared to the received in the same authorization header AWS Lambda signature.
 
 #### Usage
 
@@ -182,7 +187,11 @@ Shrine-AWS-Lambda assemblies are built inside the `#lambda_process_versions` met
 # frozen_string_literal: true
 
 class LambdaUploader < Uploader
-  Attacher.promote { |data| LambdaPromoteJob.perform_later(data) } unless Rails.env.test?
+  unless Rails.env.test?
+    Attacher.promote_block do
+      LambdaPromoteJob.enqueue(self.class.name, record.class.name, record.id, name, file_data)
+    end
+  end
 
   plugin :upload_options, store: ->(_io, context) do
     if %i[avatar logo].include?(context[:name])
@@ -219,7 +228,6 @@ class LambdaUploader < Uploader
     assembly
   end
 end
-
 ```
 
 The above example is built to interact with the [lambda-image-resize] function, which is using the [Sharp] Javascript
@@ -240,14 +248,13 @@ The default options used by Shrine-AWS-Lambda plugin are the following:
     target_storage: :store }
 ```
 
-These options could be overrided in the `LambdaUploader` specifying them as the `assembly` keys:
+These options could be overridden in the `LambdaUploader` specifying them as the `assembly` keys:
 
 ```ruby
-  assembly[:callbackURL]    = some_callback_url]
-  assembly[:copy_original   = false               # If this is `false`, only the processed file versions will be stored     
+  assembly[:callbackURL]    = some_callback_url
+  assembly[:copy_original]  = false               # If this is `false`, only the processed file versions will be stored     
   assembly[:storages]       = Shrine.buckets_to_use(%i[cache store other_store])
   assembly[:target_storage] = :other_store
-
 ```
 
 Any S3 buckets could be specified, as long as the buckets are defined in the Shrine's initializer file.
@@ -294,15 +301,20 @@ module RAPI
     end
   end
 end
-
 ```
 
 #### Backgrounding
 
 Even though submitting a Lambda assembly doesn't require any uploading, it still does a HTTP request, so it is better 
 to put it into a background job. This is configured in the `LambdaUploader` class:
 
-`Attacher.promote { |data| LambdaPromoteJob.perform_later(data) } unless Rails.env.test?`
+```ruby
+unless Rails.env.test?
+  Attacher.promote_block do
+    LambdaPromoteJob.enqueue(self.class.name, record.class.name, record.id, name, file_data)
+  end
+end
+```
 
 Then the job file should be implemented:
 
@@ -311,9 +323,11 @@ Then the job file should be implemented:
 
 # frozen_string_literal: true
 
-class LambdaPromoteJob < ApplicationJob
-  def perform(data)
-    Timeout.timeout(30) { Shrine::Attacher.lambda_process(data) }
+class LambdaPromoteJob < Que::Job
+  def run(...)
+    ActiveRecord::Base.transaction do
+     Shrine::Attacher.lambda_process(...)
+    end
   end
 end
 ```
@@ -360,15 +374,15 @@ by the following command:
 $ gem build shrine-aws-lambda.gemspec
 ```
 
-Assuming the version was set to `0.1.2`, a `shrine-aws-lambda-0.1.2.gem` binary file will be generated at the root of 
+Assuming the version was set to `0.2.0`, a `shrine-aws-lambda-0.2.0.gem` binary file will be generated at the root of 
 the app (repo).
 
 - The binary file shouldn't be added into the `git` tree, it will be pushed into the RubyGems and to the GitHub releases
 
 #### Pushing a new gem release to RubyGems
 
 ```bash
-$ gem push shrine-aws-lambda-0.1.2.gem # don't forget to specify the correct version number
+$ gem push shrine-aws-lambda-0.2.0.gem # don't forget to specify the correct version number
 ```
 
 #### Crafting the new release on GitHub
@@ -403,12 +417,12 @@ article], which pointed me to use the [Sharp] library for image resizing.
 [Amazon Web Services]: https://aws.amazon.com
 [AWS blog article]: https://aws.amazon.com/blogs/compute/resize-images-on-the-fly-with-amazon-s3-aws-lambda-and-amazon-api-gateway/
 [AWS Lambda]: https://aws.amazon.com/lambda
-[AWS Lamda permissions]: https://docs.aws.amazon.com/lambda/latest/dg/intro-permission-model.html
+[AWS Lambda permissions]: https://docs.aws.amazon.com/lambda/latest/dg/intro-permission-model.html
 [AWS profiles]: https://docs.aws.amazon.com/cli/latest/userguide/cli-multiple-profiles.html
 [AWS S3]: https://aws.amazon.com/s3/
 [Janko]: https://github.com/janko-m
 [lambda-image-resize]: https://github.com/texpert/lambda-image-resize.js
-[localtunnel]: https://github.com/localtunnel/localtunnel
+[ngrok-wrapper]: https://github.com/texpert/ngrok-wrapper
 [Rails]: http://rubyonrails.org
 [Roda]: http://roda.jeremyevans.net
 [Sharp]: https://github.com/lovell/sharp