8.0.0 doc update.

Author: jcamiel

.github/workflows/publish.yml

@@ -25,13 +25,9 @@ jobs:
           pip3 install --user -r ./sites/requirements.txt
           sudo gem install jekyll
       - name: Check style
-        run: |
-          cd sites
-          ./check_fmt.sh
+        run: sites/check_fmt.sh
       - name: Build
-        run: |
-          cd sites
-          ./build.sh
+        run: sites/build.sh
       - name: Publish
         if: github.ref == 'refs/heads/master'
         env:

sites/build.sh

@@ -1,6 +1,9 @@
 #!/usr/bin/env bash
 set -eu
 
+original_dir="$(pwd)"
+cd "$(dirname "$0")"
+
 function echo_step () {
   echo ''
   echo -e "\033[32m\033[1m$1...\033[0m"
@@ -39,8 +42,9 @@ python3 build_sitemap.py > hurl.dev/_site/sitemap.txt
 echo_step 'Generating feed.xml'
 cp hurl.dev/_posts/feed.xml hurl.dev/_site/blog/
 
+sites_dir=$(realpath --relative-to="$original_dir" "$(pwd)")
 echo ''
 echo 'Run local site'
 echo '-------------------'
-echo '    Build & watch: jekyll serve --source hurl.dev --destination hurl.dev/_site'
-echo '    Static: python3 -m http.server --dir hurl.dev/_site 4000'
+echo "    Build & watch: jekyll serve --source $sites_dir/hurl.dev --destination $sites_dir/hurl.dev/_site"
+echo "    Static: python3 -m http.server --dir $sites_dir/hurl.dev/_site 4000"

sites/build_home_samples.py

@@ -324,7 +324,7 @@ def make_home_samples():
 """,
         ),
         Sample(
-            name="SSL Certificate",
+            name="SSL/TLS Certificate",
             src="""\
 # Check attributes of the SSL certificate
 GET https://example.org
@@ -334,6 +334,9 @@ def make_home_samples():
 certificate "Issuer" == "C=US, O=Let's Encrypt, CN=R3"
 certificate "Expire-Date" daysAfterNow > 15
 certificate "Serial-Number" matches /[\\da-f]+/
+certificate "Subject-Alt-Name" contains "DNS:example.org"
+certificate "Subject-Alt-Name" split "," count == 2
+certificate "Value" startsWith "-----BEGIN CERTIFICATE-----"
 """,
         ),
         Sample(

sites/hurl.dev/_data/docs.yml

@@ -48,8 +48,13 @@
             items:
               - title: Capturing values
               - title: Asserts
-          - title: Options
-          - title: Environment
+          - title: Configuration
+          - title: All Options
+            items:
+              - title: HTTP options
+              - title: Output options
+              - title: Report options
+              - title: Other options
           - title: Exit Codes
           - title: WWW
           - title: See Also
@@ -211,6 +216,7 @@
             - title: Cookie capture
             - title: Body capture
             - title: Bytes capture
+            - title: RawBytes capture
             - title: XPath capture
             - title: JSONPath capture
             - title: Regex capture
@@ -250,6 +256,7 @@
             - title: Cookie assert
             - title: Body assert
             - title: Bytes assert
+            - title: RawBytes assert
             - title: XPath assert
             - title: JSONPath assert
             - title: Regex assert
@@ -272,11 +279,11 @@
             - title: base64Encode
             - title: base64UrlSafeDecode
             - title: base64UrlSafeEncode
+            - title: charsetDecode
             - title: count
             - title: dateFormat
             - title: daysAfterNow
             - title: daysBeforeNow
-            - title: decode
             - title: first
             - title: htmlEscape
             - title: htmlUnescape

sites/hurl.dev/_docs/asserting-response.md

@@ -26,8 +26,8 @@ jsonpath "$.cats[0].name" == "Felix"
 jsonpath "$.cats[0].lives" == 9
 ```
 
-Body responses can be encoded by server (see [`Content-Encoding` HTTP header]) but asserts in Hurl files are not 
-affected by this content compression. All body asserts (`body`, `bytes`, `sha256` etc...) work _after_ content decoding.
+Body responses can be encoded by server (see [`Content-Encoding` HTTP header][content-encoding]) but asserts in Hurl files are not 
+affected by this content compression. All body asserts (`body`, `bytes`, `sha256` etc...) except `rawbytes` work _after_ content decoding.
 
 Finally, body text asserts (`body`, `jsonpath`, `xpath` etc...) are also decoded to strings based on [`Content-Type` header] 
 so these asserts can be written with usual strings. 
@@ -92,6 +92,7 @@ The asserts order in a Hurl file is:
 </div>
 </div>
 
+
 ## Implicit asserts
 
 ### Version - Status
@@ -344,7 +345,7 @@ to specify the root directory of all file nodes.
 Optional list of assertions on the HTTP response within an `[Asserts]` section. Assertions can describe checks
 on status code, on the received body (or part of it) and on response headers.
 
-Structure of an assert:
+__Structure of an explicit assert:__
 
 <div class="schema-container schema-container u-font-size-1 u-font-size-2-sm u-font-size-3-md">
  <div class="schema">
@@ -375,6 +376,7 @@ can extract data from
 - body:
   - [`body`](#body-assert)
   - [`bytes`](#bytes-assert)
+  - [`rawbytes`](#rawbytes-assert)
   - [`xpath`](#xpath-assert)
   - [`jsonpath`](#jsonpath-assert)
   - [`regex`](#regex-assert)
@@ -681,6 +683,25 @@ header "Content-Length" == "12424"
 Like `body` assert, `bytes` assert works _after_ content encoding decompression (so the predicates values are not
 affected by `Content-Encoding` response header value).
 
+### RawBytes assert
+
+Check the value of the received HTTP response body as a raw bytestream. RawBytes assert consists of the keyword `rawbytes`
+followed by a predicate function and value.
+
+```hurl
+GET https://example.org/data.bin
+HTTP 200
+Content-Encoding: gzip
+[Asserts]
+header "Content-Length" == "32"
+rawbytes count == 32               # matches Content-Length (compressed size)
+bytes count == 100                 # decompressed size is larger
+rawbytes startsWith hex,1f8b;      # gzip magic bytes
+bytes startsWith hex,48656c6c6f;   # decompressed content starts with "Hello"
+```
+
+Unlike `bytes` assert, `rawbytes` returns the raw bytes _before_ any content decoding. For uncompressed responses, `rawbytes` and `bytes` return the same data.
+
 ### XPath assert
 
 Check the value of a [XPath] query on the received HTTP body decoded as a string (using the `charset` value in the
@@ -972,7 +993,8 @@ duration < 1000   # Check that response time is less than one second
 Check the SSL certificate properties. Certificate assert consists of the keyword `certificate`, followed by the 
 certificate attribute value.
 
-The following attributes are supported: `Subject`, `Issuer`, `Start-Date`, `Expire-Date` and `Serial-Number`.
+The following attributes are supported: `Subject`, `Issuer`, `Start-Date`, `Expire-Date`, `Serial-Number`, 
+`Subject-Alt-Name` and `Value`.
 
 ```hurl
 GET https://example.org
@@ -982,6 +1004,9 @@ certificate "Subject" == "CN=example.org"
 certificate "Issuer" == "C=US, O=Let's Encrypt, CN=R3"
 certificate "Expire-Date" daysAfterNow > 15
 certificate "Serial-Number" matches "[0-9af]+"
+certificate "Subject-Alt-Name" contains "DNS:example.org"
+certificate "Subject-Alt-Name" split "," count == 2
+certificate "Value" startsWith "-----BEGIN CERTIFICATE-----"
 ```
 
 [predicates]: #predicates
@@ -1008,7 +1033,7 @@ certificate "Serial-Number" matches "[0-9af]+"
 [`decode` filter]: {% link _docs/filters.md %}#decode
 [headers implicit asserts]: #headers
 [RFC 3339]: https://www.rfc-editor.org/rfc/rfc3339
-[`Content-Encoding` HTTP header]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding
+[content-encoding]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding
 [`Content-Type` header]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Type
 [`body` assert]: #body-assert
 [`location` filter]: {% link _docs/filters.md %}#location

sites/hurl.dev/_docs/capturing-response.md

@@ -36,14 +36,14 @@ HTTP 302
 {% endraw %}
 
 
-Body responses can be encoded by server (see [`Content-Encoding` HTTP header]) but captures in Hurl files are not
-affected by this content compression. All body captures (`body`, `bytes`, `sha256` etc...) work _after_ content decoding.
+Body responses can be encoded by server (see [`Content-Encoding` HTTP header][content-encoding]) but captures in Hurl files are not
+affected by this content compression. All body captures (`body`, `bytes`, `sha256` etc...) except `rawbytes` work _after_ content decoding.
 
 Finally, body text captures (`body`, `jsonpath`, `xpath` etc...) are also decoded to strings based on [`Content-Type` header]
 so these queries can be captures as usual strings.
 
 
-Structure of a capture:
+__Structure of a capture:__
 
 <div class="schema-container schema-container u-font-size-2 u-font-size-3-sm">
  <div class="schema">
@@ -71,6 +71,7 @@ A query can extract data from
 - body:
   - [`body`](#body-capture)
   - [`bytes`](#bytes-capture)
+  - [`rawbytes`](#bytes-capture)
   - [`xpath`](#xpath-capture)
   - [`jsonpath`](#jsonpath-capture)
   - [`regex`](#regex-capture)
@@ -198,6 +199,19 @@ my_data: bytes
 Like `body` capture, `bytes` capture works _after_ content encoding decompression (so the captured value is not
 affected by `Content-Encoding` response header).
 
+### RawBytes capture
+
+Capture the entire body as a raw bytestream from the received HTTP response, _before_ any content decoding.
+
+```hurl
+GET https://example.org/data.bin
+HTTP 200
+[Captures]
+raw_data: rawbytes
+```
+
+Unlike `bytes` capture, `rawbytes` returns the raw bytes before content encoding decompression. For uncompressed responses, `rawbytes` and `bytes` capture the same data.
+
 ### XPath capture
 
 Capture a [XPath] query from the received HTTP body decoded as a string.
@@ -423,7 +437,8 @@ duration_in_ms: duration
 
 Capture the SSL certificate properties. Certificate capture consists of the keyword `certificate`, followed by the certificate attribute value.
 
-The following attributes are supported: `Subject`, `Issuer`, `Start-Date`, `Expire-Date` and `Serial-Number`.
+The following attributes are supported: `Subject`, `Issuer`, `Start-Date`, `Expire-Date`, `Serial-Number`,
+`Subject-Alt-Name` and `Value`.
 
 ```hurl
 GET https://example.org
@@ -433,6 +448,8 @@ cert_subject: certificate "Subject"
 cert_issuer: certificate "Issuer"
 cert_expire_date: certificate "Expire-Date"
 cert_serial_number: certificate "Serial-Number"
+san: certificate "Serial-Number"
+value: certificate "Value"
 ```
 
 ## Redacting Secrets
@@ -469,3 +486,5 @@ pass: header "token" redact
 [`--secret` option]: {% link _docs/templates.md %}#secrets
 [MD5]: https://en.wikipedia.org/wiki/MD5
 [SHA-256]: https://en.wikipedia.org/wiki/SHA-2
+[content-encoding]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding
+[`Content-Type` header]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Type

sites/hurl.dev/_docs/entry.md

@@ -77,7 +77,8 @@ HTTP 200
 
 ### Cookie storage
 
-Requests in the same Hurl file share the cookie storage, enabling, for example, session based scenario.
+By default, requests in the same Hurl file share cookie storage, enabling session-based scenario. The shared cookie store can
+be disabled with [`--no-cookie-store`] option.
 
 ### Redirects
 
@@ -260,3 +261,4 @@ For complete reference, below is a diagram for the executed entries.
 [`repeat`]: {% link _docs/manual.md %}#repeat
 [`redirects` query]: {% link _docs/asserting-response.md %}#redirects-assert
 [`url` query]: {% link _docs/asserting-response.md %}#url-assert
+[`--no-coookie-store`]: {% link _docs/manual.md %}#no-cookie-store

sites/hurl.dev/_docs/filters.md

@@ -71,11 +71,11 @@ jsonpath "$.books" count == 12
 | [base64Encode](#base64encode)               | Encodes bytes into [Base64 encoded string].                                                                                            | bytes            | string |
 | [base64UrlSafeDecode](#base64urlsafedecode) | Decodes a Base64 encoded string into bytes (using [Base64 URL safe encoding]).                                                         | string           | bytes  |
 | [base64UrlSafeEncode](#base64urlsafeencode) | Encodes bytes into Base64 encoded string (using [Base64 URL safe encoding]).                                                           | bytes            | string |
+| [charsetDecode](#charsetdecode)             | Decodes bytes to string using a charset encoding.                                                                                      | bytes            | string |
 | [count](#count)                             | Counts the number of items in a collection.                                                                                            | collection       | number |
 | [dateFormat](#dateformat)                   | Formats a date to a string given [a specification format].                                                                             | date             | string |
 | [daysAfterNow](#daysafternow)               | Returns the number of days between now and a date in the future.                                                                       | date             | number |
 | [daysBeforeNow](#daysbeforenow)             | Returns the number of days between now and a date in the past.                                                                         | date             | number |
-| [decode](#decode)                           | Decodes bytes to string using encoding.                                                                                                | bytes            | string |
 | [first](#first)                             | Returns the first element from a collection.                                                                                           | collection       | any    |
 | [htmlEscape](#htmlescape)                   | Converts the characters `&`, `<` and `>` to HTML-safe sequence.                                                                        | string           | string |
 | [htmlUnescape](#htmlunescape)               | Converts all named and numeric character references (e.g. `&gt;`, `&#62;`, `&#x3e;`) to the corresponding Unicode characters.          | string           | string |
@@ -143,6 +143,23 @@ HTTP 200
 bytes base64UrlSafeEncode == "PDw_Pz8-Pg"
 ```
 
+### charsetDecode
+
+Decodes bytes to string using a charset encoding. Encoding labels are defined in [Encoding Standard].
+
+```hurl
+# The 'Content-Type' HTTP response header does not precise the charset 'gb2312'
+# so body must be decoded explicitly by Hurl before processing any text based assert
+GET https://example.org/hello_china
+HTTP 200
+[Asserts]
+header "Content-Type" == "text/html"
+# Content-Type has no encoding clue, we must decode ourselves the body response.
+bytes charsetDecode "gb2312" xpath "string(//body)" == "你好世界"
+```
+
+When the encoding is UTF-8 (i.e. `charsetDecode "utf-8"`), [an `utf8Decode` filter] can be used instead.
+
 ### count
 
 Counts the number of items in a collection.
@@ -189,21 +206,6 @@ HTTP 200
 certificate "Start-Date" daysBeforeNow < 100
 ```
 
-### decode
-
-Decodes bytes to string using encoding. Encoding labels are defined in [Encoding Standard].
-
-```hurl
-# The 'Content-Type' HTTP response header does not precise the charset 'gb2312'
-# so body must be decoded explicitly by Hurl before processing any text based assert
-GET https://example.org/hello_china
-HTTP 200
-[Asserts]
-header "Content-Type" == "text/html"
-# Content-Type has no encoding clue, we must decode ourselves the body response.
-bytes decode "gb2312" xpath "string(//body)" == "你好世界"
-```
-
 ### first
 
 Returns the first element from a collection.
@@ -475,7 +477,6 @@ HTTP 200
 bytes decode "gb2312" xpath "string(//body)" == "你好世界"
 ```
 
-
 [Captures]: {% link _docs/capturing-response.md %}
 [asserts]: {% link _docs/asserting-response.md %}
 [RFC3986]: https://www.rfc-editor.org/rfc/rfc3986
@@ -485,3 +486,5 @@ bytes decode "gb2312" xpath "string(//body)" == "你好世界"
 [Base64 encoded string]: https://datatracker.ietf.org/doc/html/rfc4648#section-4
 [Base64 URL safe encoding]: https://datatracker.ietf.org/doc/html/rfc4648#section-5
 [Encoding Standard]: https://encoding.spec.whatwg.org/#concept-encoding-get
+[an `utf8Decode` filter]: {% link _docs/filters.md %}#utf8decode
+[an `utf8Encode` filter]: {% link _docs/filters.md %}#utf8encode