readme improvement

 - remove images too big / no value added
 - polish description in README
add symboize_names option if user want to use string as dictionary key (closer to JSON

Author: jvmvik

.yardopts

@@ -1 +1,2 @@
---markup=markdown
+--markup=markdown
+--exclude LICENSE

README.md

@@ -1,14 +1,12 @@
-<div align="center">
-<h1 align="center">SerpApi Ruby Library</h1>
+# SerpApi Ruby Library
 
-![serpapi ruby library logo](https://user-images.githubusercontent.com/78694043/235409962-7afe3a25-9272-4d56-9678-9972b771453b.png)
+[![serpapi-ruby](https://github.com/serpapi/serpapi-ruby/actions/workflows/ci.yml/badge.svg)](https://github.com/serpapi/serpapi-ruby/actions/workflows/ci.yml) [![Gem Version](https://badge.fury.io/rb/serpapi.svg)](https://badge.fury.io/rb/serpapi) 
 
-[![Gem Version](https://badge.fury.io/rb/serpapi.svg)](https://badge.fury.io/rb/serpapi) [![serpapi-ruby](https://github.com/serpapi/serpapi-ruby/actions/workflows/ci.yml/badge.svg)](https://github.com/serpapi/serpapi-ruby/actions/workflows/ci.yml)  
-</div>
+Integrate search data into your AI workflow, RAG / fine tuning or ruby application using this official wrapper for [SerpApi](https://serpapi.com). 
 
-Integrate search data into your AI workflow or ruby application. This library is the official wrapper for [SerpApi](https://serpapi.com). 
-SerpApi supports Google, Google Maps, Google Shopping, Baidu, Yandex, Yahoo, eBay, App Stores, and more.
-Fast query at scale a vast range of data, including web search results, flight schedule, stock market data, news headlines, and more.
+SerpApi supports Google, Google Maps, Google Shopping, Baidu, Yandex, Yahoo, eBay, App Stores, and [more.](https://serpapi.com). 
+
+Fast query at scale a vast range of data, including web search results, flight schedule, stock market data, news headlines, and [more.](https://serpapi.com). 
 
 ## Features
   * `persistent` → Keep socket connection open to save on SSL handshake / reconnection (2x faster).  [Search at scale](#Search-At-Scale)
@@ -20,7 +18,7 @@ Fast query at scale a vast range of data, including web search results, flight s
 
 To achieve optimal performance, it is essential to have Ruby 3.1+ (preferably version 3.4) installed. 
 
-| Older versions such as Ruby 1.9, 2.x, and JRuby are compatible with [serpapi older library](https://github.com/serpapi/google-search-results-ruby), which continues to function effectively.
+| Older versions such as Ruby 1.9, 2.x, and JRuby are compatible with [serpapi older library](https://github.com/serpapi/google-search-results-ruby), which continues to function effectively. see [migration guide](#Migration-quick-guide) if you are using the older library.
 
 ### Bundler
 ```ruby
@@ -53,7 +51,10 @@ Environment variables are a secure, safe, and easy way to manage secrets.
  Ruby accesses these variables from `ENV['SERPAPI_KEY']`.
 
 
-## Search API advanced Usage
+## Search API advanced usage with Google search engine
+
+This example dives into all the available parameters for the Google search engine.
+ The set of parameters is extensive and depends on the search engine you choose.
 
 ```ruby
 # load gem
@@ -67,6 +68,7 @@ client = SerpApi::Client.new(
   async: false, # non blocking HTTP request see: Search Asynchronous (default: false)
   persistent: true, # leave socket connection open for faster response time see: Search at scale (default: true)
   timeout: 5, # HTTP timeout in seconds on the client side only. (default: 120s)
+  symbolize_names: true # turn on/off JSON keys to symbols (default: on more efficient)
 )
 
 # search query overview (more fields available depending on search engine)
@@ -91,23 +93,20 @@ params = {
   tbs: "custom to be client criteria",
 }
 
-# formatted search results as a Hash
-#  serpapi.com converts HTML -> JSON
+# search results as a symbolized Hash (per performance)
 results = client.search(params)
 
-# raw search engine html as a String
-#  serpapi.com acts a proxy to provide high throughputs, no search limit and more.
+# search results as a raw HTML string
 raw_html = client.html(params) # Corrected parameter reference
 ```
-
-[SerpApi documentation](https://serpapi.com/search-api).
+ → [SerpApi documentation](https://serpapi.com/search-api).
 
 #### Documentations
-
- * [API documentation](https://rubydoc.info/github/serpapi/serpapi-ruby/master)
+This library is well documented, and you can find the following resources:
  * [Full documentation on SerpApi.com](https://serpapi.com)
  * [Library Github page](https://github.com/serpapi/serpapi-ruby)
  * [Library GEM page](https://rubygems.org/gems/serpapi/)
+ * [Library API documentation](https://rubydoc.info/github/serpapi/serpapi-ruby/master)
  * [API health status](https://serpapi.com/status)
 
 ## Advanced search API usage
@@ -130,9 +129,7 @@ client = SerpApi::Client.new(engine: 'google', async: true, persistent: true, ap
 search_queue = Queue.new
 company_list.each do |company|
   result = client.search({ q: company })
-  if result[:search_metadata][:status] =~ /Cached/
-    puts "#{company}: search results found in cache for: #{company}"
-  end
+  puts "#{company}: search results found in cache for: #{company}" if result[:search_metadata][:status] =~ /Cached/
 
   search_queue.push(result[:search_metadata][:id])
 end

README.md.erb

@@ -18,17 +18,15 @@ def snippet(format, path, demo = false)
  %Q(```#{format}\nrequire 'serpapi'\n#{buf}```\n\n * source code: [#{path}](https://github.com/serpapi/serpapi-ruby/blob/master/#{path}))
 end
 -%>
-<div align="center">
-<h1 align="center">SerpApi Ruby Library</h1>
+# SerpApi Ruby Library
 
-![serpapi ruby library logo](https://user-images.githubusercontent.com/78694043/235409962-7afe3a25-9272-4d56-9678-9972b771453b.png)
+[![serpapi-ruby](https://github.com/serpapi/serpapi-ruby/actions/workflows/ci.yml/badge.svg)](https://github.com/serpapi/serpapi-ruby/actions/workflows/ci.yml) [![Gem Version](https://badge.fury.io/rb/serpapi.svg)](https://badge.fury.io/rb/serpapi) 
 
-[![Gem Version](https://badge.fury.io/rb/serpapi.svg)](https://badge.fury.io/rb/serpapi) [![serpapi-ruby](https://github.com/serpapi/serpapi-ruby/actions/workflows/ci.yml/badge.svg)](https://github.com/serpapi/serpapi-ruby/actions/workflows/ci.yml)  
-</div>
+Integrate search data into your AI workflow, RAG / fine tuning or ruby application using this official wrapper for [SerpApi](https://serpapi.com). 
 
-Integrate search data into your AI workflow or ruby application. This library is the official wrapper for [SerpApi](https://serpapi.com). 
-SerpApi supports Google, Google Maps, Google Shopping, Baidu, Yandex, Yahoo, eBay, App Stores, and more.
-Fast query at scale a vast range of data, including web search results, flight schedule, stock market data, news headlines, and more.
+SerpApi supports Google, Google Maps, Google Shopping, Baidu, Yandex, Yahoo, eBay, App Stores, and [more.](https://serpapi.com). 
+
+Fast query at scale a vast range of data, including web search results, flight schedule, stock market data, news headlines, and [more.](https://serpapi.com). 
 
 ## Features
   * `persistent` → Keep socket connection open to save on SSL handshake / reconnection (2x faster).  [Search at scale](#Search-At-Scale)
@@ -40,7 +38,7 @@ Fast query at scale a vast range of data, including web search results, flight s
 
 To achieve optimal performance, it is essential to have Ruby 3.1+ (preferably version 3.4) installed. 
 
-| Older versions such as Ruby 1.9, 2.x, and JRuby are compatible with [serpapi older library](https://github.com/serpapi/google-search-results-ruby), which continues to function effectively.
+| Older versions such as Ruby 1.9, 2.x, and JRuby are compatible with [serpapi older library](https://github.com/serpapi/google-search-results-ruby), which continues to function effectively. see [migration guide](#Migration-quick-guide) if you are using the older library.
 
 ### Bundler
 ```ruby
@@ -73,7 +71,10 @@ Environment variables are a secure, safe, and easy way to manage secrets.
  Ruby accesses these variables from `ENV['SERPAPI_KEY']`.
 
 
-## Search API advanced Usage
+## Search API advanced usage with Google search engine
+
+This example dives into all the available parameters for the Google search engine.
+ The set of parameters is extensive and depends on the search engine you choose.
 
 ```ruby
 # load gem
@@ -87,6 +88,7 @@ client = SerpApi::Client.new(
   async: false, # non blocking HTTP request see: Search Asynchronous (default: false)
   persistent: true, # leave socket connection open for faster response time see: Search at scale (default: true)
   timeout: 5, # HTTP timeout in seconds on the client side only. (default: 120s)
+  symbolize_names: true # turn on/off JSON keys to symbols (default: on more efficient)
 )
 
 # search query overview (more fields available depending on search engine)
@@ -111,23 +113,20 @@ params = {
   tbs: "custom to be client criteria",
 }
 
-# formatted search results as a Hash
-#  serpapi.com converts HTML -> JSON
+# search results as a symbolized Hash (per performance)
 results = client.search(params)
 
-# raw search engine html as a String
-#  serpapi.com acts a proxy to provide high throughputs, no search limit and more.
+# search results as a raw HTML string
 raw_html = client.html(params) # Corrected parameter reference
 ```
-
-[SerpApi documentation](https://serpapi.com/search-api).
+ → [SerpApi documentation](https://serpapi.com/search-api).
 
 #### Documentations
-
- * [API documentation](https://rubydoc.info/github/serpapi/serpapi-ruby/master)
+This library is well documented, and you can find the following resources:
  * [Full documentation on SerpApi.com](https://serpapi.com)
  * [Library Github page](https://github.com/serpapi/serpapi-ruby)
  * [Library GEM page](https://rubygems.org/gems/serpapi/)
+ * [Library API documentation](https://rubydoc.info/github/serpapi/serpapi-ruby/master)
  * [API health status](https://serpapi.com/status)
 
 ## Advanced search API usage

lib/serpapi/client.rb

@@ -4,10 +4,13 @@ module SerpApi
   # Client for SerpApi.com
   # powered by HTTP.rb
   #
-  #  it supports:
+  #  features:
   #  * async non-block search
   #  * persistent HTTP connection
-  #  * many endpoints
+  #  * search API
+  #  * location API
+  #  * account API
+  #  * search archive API
   #
   class Client
     # Backend service URL
@@ -99,7 +102,6 @@ def initialize(params = {})
     #
     # see: https://serpapi.com/search-api
     #
-    #
     # note that the raw response
     #                 from the search engine is converted to JSON by SerpApi.com backend.
     #                 thus, most of the compute power is on the backsdend and not on the client side.
@@ -110,7 +112,10 @@ def search(params = {})
       get('/search', :json, params)
     end
 
-    # html search
+    # html search perform a search using SerpApi.com
+    #  the output is raw HTML from the search engine.
+    #  it is useful for training AI models, RAG, debugging
+    #   or when you need to parse the HTML yourself.
     #
     # @return [String] raw html search results directly from the search engine.
     def html(params = {})
@@ -120,7 +125,7 @@ def html(params = {})
     # Get location using Location API
     #
     # example: spec/serpapi/location_api_spec.rb
-    # doc: https://serpapi.com/search-api
+    # doc: https://serpapi.com/locations-api
     #
     # @param [Hash] params must includes fields: q, limit
     # @return [Array<Hash>] list of matching locations
@@ -129,11 +134,17 @@ def location(params = {})
     end
 
     # Retrieve search result from the Search Archive API
-    #
+    # 
+    # ```ruby
+    # client = SerpApi::Client.new(engine: 'google', api_key: ENV['SERPAPI_KEY'])
+    # results = client.search(q: 'Coffee', location: 'Portland')
+    # search_id = results[:search_metadata][:id]
+    # archive_search = client.search_archive(search_id)
+    # ```
     # example: spec/serpapi/client/search_archive_api_spec.rb
     # doc: https://serpapi.com/search-archive-api
     #
-    # @param [String|Integer] search_id is returned
+    # @param [String|Integer] search_id from original search `results[:search_metadata][:id]`
     # @param [Symbol] format :json or :html [default: json, optional]
     # @return [String|Hash] raw html or JSON / Hash
     def search_archive(search_id, format = :json)
@@ -145,9 +156,9 @@ def search_archive(search_id, format = :json)
     # Get account information using Account API
     #
     # example: spec/serpapi/client/account_api_spec.rb
-    # doc: https://serpapi.com/google-jobs-api
+    # doc: https://serpapi.com/account-api
     #
-    # @param [String] api_key secret key
+    # @param [String] api_key secret key [optional if already provided to the constructor]
     # @return [Hash] account information
     def account(api_key = nil)
       params = (api_key.nil? ? {} : { api_key: api_key })
@@ -177,7 +188,12 @@ def query(params)
       raise SerpApiError, "params must be hash, not: #{params.class}" unless params.instance_of?(Hash)
 
       # merge default params with custom params
-      q = @params.merge(params)
+      q = @params.clone.merge(params)
+
+      # do not pollute default params with custom params
+      if q.key?(:symbolize_names)
+        q.delete(:symbolize_names)
+      end
 
       # delete empty key/value
       q.compact
@@ -188,14 +204,14 @@ def persistent?
       persistent
     end
 
-    # get HTTP query formatted results
+    # Perform HTTP GET request to the SerpApi.com backend endpoint.
     #
-    # @param [String] endpoint HTTP service uri
+    # @param [String] endpoint HTTP service URI
     # @param [Symbol] decoder type :json or :html
     # @param [Hash] params custom search inputs
-    # @param [Boolean] symbolize_names if true, convert JSON keys to symbols
-    # @return decoded response as JSON / Hash or String
-    def get(endpoint, decoder = :json, params = {}, symbolize_names = true)
+    # @param [Boolean] symbolize_names if true, convert JSON keys to symbols more memory efficient.
+    # @return [String|Hash] raw HTML or decoded response as JSON / Hash
+    def get(endpoint, decoder = :json, params = {})
       # execute get via open socket
       response = if persistent?
                    @socket.get(endpoint, params: query(params))
@@ -208,6 +224,11 @@ def get(endpoint, decoder = :json, params = {}, symbolize_names = true)
       when :json
         # read http response
         begin
+          # user can turn on/off JSON keys to symbols
+          # this is more memory efficient, but not always needed
+          symbolize_names = params.key?(:symbolize_names) ? params[:symbolize_names] : true
+
+          # parse JSON response with Ruby standard library
           data = JSON.parse(response.body, symbolize_names: symbolize_names)
           if data.instance_of?(Hash) && data.key?(:error)
             raise SerpApiError, "HTTP request failed with error: #{data[:error]} from url: https://#{BACKEND}#{endpoint}, params: #{params}, decoder: #{decoder}, response status: #{response.status} "

lib/serpapi/error.rb

@@ -3,6 +3,7 @@ module SerpApi
   # SerpApiException wraps any errors related to the SerpApi client.
   class SerpApiError < StandardError
     # List the specific types of errors handled by the Error class.
+    #  - HTTP response errors from SerpApi.com
     #  - Missing API key
     #  - Credit limit
     #  - Incorrect query

spec/serpapi/client/client_spec.rb

@@ -5,16 +5,29 @@
     client = SerpApi::Client.new(engine: 'google', api_key: ENV['SERPAPI_KEY'], timeout: 30)
   end
 
-  it 'search for coffee in Austin, TX and receive json results' do
-    data = client.search(q: 'Coffee', location: 'Austin, TX')
-    expect(data.size).to be > 5
-    expect(data.class).to be Hash
-    expect(data.keys.size).to be > 5
+  it 'search for coffee in Austin, TX, returns symbolized Hash' do
+    results = client.search(q: 'Coffee', location: 'Austin, TX')
+
+    expect(results.size).to be > 5
+    expect(results.class).to be Hash
+    expect(results.keys.size).to be > 5
+
+    expect(results.keys).to include(:search_metadata), ':search_metadata should be present in the results'
+  end
+
+  it 'search for coffee in Austin, TX, returns non-symbolized Hash' do
+    results = client.search(q: 'Coffee', location: 'Austin, TX', symbolize_names: false)
+
+    expect(results.size).to be > 5
+    expect(results.class).to be Hash
+    expect(results.keys.size).to be > 5
+
+    expect(results.keys).to include('search_metadata'), 'search_metadata should be present in the results'
   end
 
-  it 'search fir coffee in Austin, TX and receive raw HTML' do
-    data = client.html(q: 'Coffee', location: 'Austin, TX')
-    expect(data).to match(/coffee/i)
+  it 'search for coffee in Austin, TX and receive raw HTML' do
+    results = client.html(q: 'Coffee', location: 'Austin, TX')
+    expect(results).to match(/coffee/i)
   end
 
   it 'missing query' do
@@ -77,7 +90,7 @@
     begin
       client.send(:get, '/invalid', :html, {})
     rescue SerpApi::SerpApiError => e
-      expect(e.message).to include('HTTP request failed with response status: 404')
+      expect(e.message).to include(/HTTP request failed with response status: 404 Not Found reponse/), "got #{e.message}"
     rescue => e
       raise("wrong exception: #{e}")
     end
@@ -96,11 +109,11 @@
   end
 
   it 'makes a search request with valid parameters' do
-    response = client.search(q: 'Coffee', location: 'Austin, TX')
-    expect(response.size).to be > 5
-    expect(response.class).to be Hash
-    expect(response.keys.size).to be > 5
-    expect(response[:search_metadata][:id]).not_to be_nil
+    results = client.search(q: 'Coffee', location: 'Austin, TX')
+    expect(results.size).to be > 5
+    expect(results.class).to be Hash
+    expect(results.keys.size).to be > 5
+    expect(results[:search_metadata][:id]).not_to be_nil
 
     expect(client.close).to eq(:clean)
   end