Baseline test improvements: pretty-print snapshots, unified-diff failures, docstrings, local data

Sets a cleaner baseline for the test suite so future data and code PRs
are easier to develop, review, and triage. All changes confined to
`tests/` (plus a one-character typo fix in `database/etfs.csv`).
**All 32 tests pass.**

## Snapshot diffs that actually tell you what changed

The `test_show_options*.json` snapshots are JSON arrays of category labels (countries, sectors, currencies, …) — single-line strings of 1-2 KB. When a database update added/removed one entry, the GitHub diff was one giant string change with no way to see what differed.

```diff
- string_value = json.dumps(data, **kwargs)
+ string_value = json.dumps(data, indent=2, ensure_ascii=False, **kwargs)
```

`ensure_ascii=False` also means non-ASCII characters render natively (`Côte d'Ivoire` instead of escape sequences).

## Unified-diff on snapshot mismatch

`Recorder.assert_equal()` previously raised a `AssertionError: Change detected` with two truncated 500-char strings, often appearing identical in the visible part even when they actually differed later. We hit this exact issue while developing JerBouma#141 and JerBouma#143.

Now `assert_equal()` produces a proper `difflib.unified_diff` showing which line(s) changed; `assert_in_list` gained a proper assertion message.

## Tests now read the PR branch's data, not main

The library defaults to fetching `compression/*` from the GitHub `main` branch over HTTP. This meant tests on a PR validated the data on `main`, not the data in the PR — silently passing data-breaking PRs and failing on data-fixing PRs.

Two cooperating fixes:
- Every `tests/test_*.py` now instantiates with `use_local_location=True` so the library reads the local checkout.
- `tests/conftest.py` regenerates `compression/*.bz2` and `compression/categories/*.gzip` from the checked-out `database/*.csv` at import time, mirroring the production `database_update.yml` workflow. This must run *before* pytest collects the test modules (test-module imports instantiate `fd.X(use_local_location=True)`), so it is a top-level statement, not a fixture.

The compression artifacts themselves are not committed in this PR — they are deterministically derived from the CSVs and the test suite regenerates them on every run.

## Test module quality (incidental cleanup)

Since the regen touched every test file, took the opportunity to:
- Fix wrong module docstrings (`test_equities.py` and `test_moneymarkets.py` both started with `"""Currencies Test Module"""`).
- Replace `# pylint: disable=missing-function-docstring` with real docstrings on every test function in the 7 asset-class files and in `test_sec_enrichment_controller.py`.
- Add type hints `(recorder: Recorder) -> None`; `Recorder` import under `TYPE_CHECKING`, guarded by `from __future__ import annotations` — zero runtime cost.

## conftest.py cleanup

- Removed `# pylint: skip-file` so static analysis is no longer suppressed.
- Cast `request.config.getoption("--rewrite-expected")` return through `bool(...)` to satisfy the declared `-> bool` annotation (previously `Any | None`).

## Test infrastructure config (pyproject.toml)

- Dropped `pytest-recording` from dev deps — unused; the real dep used by `conftest.py` is `pytest-recorder` (which provides the `record_mode` / `disable_recording` fixtures).
- Declared `[tool.pytest.ini_options]` `testpaths = ["tests"]`, `addopts = "--strict-markers"`, and the `record_stdout` marker so future PRs can't silently introduce undeclared markers.

## Side fix: ASYMshsare → ASYMshares (etfs.csv + tests)

While running the new local-data tests we noticed `ASPY` (Leverage Shares ASYMmetric 500 ETF) had `family = "ASYMshsare"` — a typo in `database/etfs.csv`. Corrected to `ASYMshares` along with the references in `tests/test_etfs.py` and the affected fixture snapshots. The maintainer's `Update Compression Files` workflow will regenerate the binary artifacts on `main` post-merge.

## What this PR explicitly does NOT do

Deferred to focused follow-ups (out of scope here):
- `@pytest.mark.parametrize` / class-based grouping — would change pytest node IDs and break `Recorder.capture()` positional snapshot naming (renames 90+ files in one go).
- `pytest-xdist` parallel execution — adds a new dev dep; pure perf win.
- Wiring `pytest-cov` (already in dev deps) into CI for coverage reporting.
- Filling the 22% coverage gap on `financedatabase/helpers.py` (`FinanceFrame.to_toolkit`, `show_options` URL error path, `case_sensitive=True`).

Author: dokson

database/etfs.csv

@@ -2275,7 +2275,7 @@ ASIL-GB.AQ,MULTI UNITS LUXEMB,GBP,"MULTI UNITS LUXEMB is an investment fund inco
 ASIL.L,Multi Units Luxembourg - Lyxor China Enterprise (HSCEI) UCITS ETF,GBP,"Multi Units Luxembourg - Lyxor China Enterprise (HSCEI) UCITS ETF is an exchange-traded fund designed to track the performance of the Hang Seng China Enterprises Index (HSCEI). The ETF provides exposure to the largest companies in China that are listed on the Hong Kong Stock Exchange. Lyxor Asset Management, a subsidiary of Societe Generale, manages the ETF, which is domiciled in Luxembourg.",Equities,,Lyxor International Asset Management,LSE,
 ASIU.L,Lyxor China Enterprise (HSCEI) UCITS ETF - Acc-EUR,USD,"The investment objective of the MULTI UNITS LUXEMBOURG — Lyxor China Enterprise (HSCEI) UCITS ETF (the ""Sub-Fund"") is to track both the upward and the downward evolution of the Hang Seng China Enterprises Index Net Total Return index (the ""Index"") denominated in Hong Kong dollars (HKD), and representative of the major Chinese equities known as ""H-shares, while minimizing the volatility of the difference between the return of the Sub-Fund and the return of the Index (the ""Tracking Error"").",Equities,,Lyxor International Asset Management,LSE,LU1900068914
 ASPY,ASYMshares ASYMmetric 500 ETF,USD,"The investment seeks to track the total return performance, before fees and expenses, of the ASYMmetric 500 Index.
- The index is a rules-based, quantitative long/short hedging strategy that seeks to provide protection against bear market losses and to capture the majority of bull market gains with respect to exposure to the 500 largest capitalized equity securities publicly traded in the United States, which is referred to as the index's ""market"". Under normal market conditions, the fund will invest at least 80% of its total assets in securities and cash included in the index's Long Book.",Equities,Equities,ASYMshsare,PCX,
+ The index is a rules-based, quantitative long/short hedging strategy that seeks to provide protection against bear market losses and to capture the majority of bull market gains with respect to exposure to the 500 largest capitalized equity securities publicly traded in the United States, which is referred to as the index's ""market"". Under normal market conditions, the fund will invest at least 80% of its total assets in securities and cash included in the index's Long Book.",Equities,Equities,ASYMshares,PCX,
 ASR0.BE,BNPP ENERGY TRAN. CLCAP,EUR,"BNPP ENERGY TRAN. CLCAP is a fund managed by BNP Paribas Asset Management focused on investing in companies involved in the energy transition. The ""CLCAP"" indicates that it is an accumulation share class, meaning dividends are reinvested. Further information is required to understand the fund's specific investment strategy and the sectors within the energy transition it targets.",Energy,,BNP Paribas Asset Management,BER,
 ASR0.MU,Parvest SICAV - Energy Innovators,EUR,"Parvest SICAV - Energy Innovators is an investment fund focused on companies involved in the energy transition, including renewable energy, energy efficiency, and related technologies. The fund aims to provide long-term capital appreciation by investing in companies that are contributing to a more sustainable energy future. BNP Paribas Asset Management manages the fund, which is part of the Parvest SICAV fund range domiciled in Luxembourg.",Energy,,BNP Paribas Asset Management,MUN,
 ASR1.BE,BNPP GER.MF EQ. CL.CAP,EUR,"BNPP GER.MF EQ. CL.CAP is a fund managed by BNP Paribas Asset Management that invests in German equities (EQ). ""MF"" suggests it might be a multi-factor fund. ""CL.CAP"" indicates an capitalization-weighted accumulation share class, where dividends are reinvested.",Equities,,BNP Paribas Asset Management,BER,

poetry.lock

@@ -91,4 +91,9 @@ disable_error_code = "misc, valid-type, attr-defined, index"
 [tool.pytest.ini_options]
 filterwarnings = [
     "ignore::pytest.PytestAssertRewriteWarning:",
+]
+testpaths = ["tests"]
+addopts = "--strict-markers"
+markers = [
+    "record_stdout: capture and snapshot stdout output for the marked test",
 ]

pyproject.toml

@@ -1,3 +1,4 @@
+import difflib
 import json
 import os
 import pathlib
@@ -28,7 +29,88 @@
     "txt": [str],
 }
 
-# pylint: skip-file
+REPO_ROOT = pathlib.Path(__file__).resolve().parents[1]
+ASSET_CSVS = {
+    "cryptos": "summary",
+    "currencies": "summary",
+    "equities": "summary",
+    "etfs": "summary",
+    "funds": "summary",
+    "indices": "summary",
+    "moneymarkets": "summary",
+}
+
+
+def _regenerate_compression_artifacts() -> None:
+    """Mirror the Database-Update workflow so tests run against compression
+    artifacts derived from the *checked-out* CSV files.
+
+    The financedatabase library reads `compression/<asset>.bz2` and
+    `compression/categories/<asset>_categories.gzip`; without this step those
+    artifacts lag the `database/*.csv` on a PR branch and tests silently
+    validate against `main` instead of the PR change.
+    """
+    compression_dir = REPO_ROOT / "compression"
+    categories_dir = compression_dir / "categories"
+    for asset, name_col in ASSET_CSVS.items():
+        csv_path = REPO_ROOT / "database" / f"{asset}.csv"
+        if not csv_path.exists():
+            continue
+        df = pd.read_csv(csv_path)
+        df.to_csv(compression_dir / f"{asset}.bz2", index=False, compression="bz2")
+        indexed = pd.read_csv(csv_path, index_col=0)
+        categories: dict[str, Any] = {}
+        skip_cols = {"name", name_col, "manager_name", "manager_bio"}
+        for column in indexed.columns:
+            if column in skip_cols:
+                continue
+            vals = indexed[column].dropna().unique()
+            vals.sort()
+            categories[column] = vals
+        cat_df = pd.DataFrame.from_dict(categories, orient="index").reset_index()
+        cat_df.to_csv(
+            categories_dir / f"{asset}_categories.gzip",
+            index=False,
+            compression="gzip",
+        )
+
+
+def _snapshot_compression_artifacts() -> dict[pathlib.Path, bytes]:
+    """Capture current bytes of compression artifacts so they can be restored."""
+    compression_dir = REPO_ROOT / "compression"
+    snapshot: dict[pathlib.Path, bytes] = {}
+    for asset in ASSET_CSVS:
+        for path in (
+            compression_dir / f"{asset}.bz2",
+            compression_dir / "categories" / f"{asset}_categories.gzip",
+        ):
+            if path.exists():
+                snapshot[path] = path.read_bytes()
+    return snapshot
+
+
+def _restore_compression_artifacts(snapshot: dict[pathlib.Path, bytes]) -> None:
+    """Restore compression artifacts to their pre-test contents."""
+    for path, data in snapshot.items():
+        try:
+            path.write_bytes(data)
+        except Exception:
+            pass
+
+
+# Regenerate compression artifacts BEFORE pytest collects test modules. The
+# asset-class test modules instantiate `fd.X(use_local_location=True)` at
+# import time, so the artifacts must already be in sync with the checked-out
+# CSVs by then — a session-scoped fixture runs too late. A snapshot of the
+# original bytes is taken first; `pytest_sessionfinish` restores them so the
+# working tree stays clean for contributors.
+_compression_snapshot = _snapshot_compression_artifacts()
+_regenerate_compression_artifacts()
+
+
+def pytest_sessionfinish(session, exitstatus) -> None:  # noqa: ARG001
+    """Restore compression artifacts after the pytest session ends."""
+    _restore_compression_artifacts(_compression_snapshot)
 
 
 class Record:
@@ -43,7 +125,7 @@ def extract_string(data: Any, **kwargs) -> str:
                 **kwargs,
             )
         elif isinstance(data, tuple(EXTENSIONS_MATCHING["json"])):
-            string_value = json.dumps(data, **kwargs)
+            string_value = json.dumps(data, indent=2, ensure_ascii=False, **kwargs)
         else:
             raise AttributeError(f"Unsupported type : {type(data)}")
 
@@ -225,19 +307,39 @@ def assert_equal(self):
 
         for record in record_list:
             if record.record_changed:
+                expected = (record.recorded or "").splitlines()
+                actual = (record.captured or "").splitlines()
+                diff = "\n".join(
+                    difflib.unified_diff(
+                        expected,
+                        actual,
+                        fromfile="expected",
+                        tofile="actual",
+                        lineterm="",
+                        n=3,
+                    )
+                )
+                if not diff:
+                    diff = (
+                        f"(line-level diff empty — content differs only in trailing whitespace or EOL)\n"
+                        f"expected length={len(record.recorded or '')}, "
+                        f"actual length={len(record.captured or '')}"
+                    )
                 raise AssertionError(
-                    "Change detected\n"
-                    f"Record    : {record.record_path}\n"
-                    f"Expected  : {record.recorded[:self.display_limit]}\n"
-                    f"Actual    : {record.captured[:self.display_limit]}\n"
+                    f"Snapshot mismatch: {record.record_path}\n"
+                    f"Run pytest with --rewrite-expected to regenerate, or inspect the diff below.\n\n"
+                    f"{diff}"
                 )
 
     def assert_in_list(self, in_list: list[str]):
         record_list = self.__record_list
 
         for record in record_list:
             for string_value in in_list:
-                assert string_value in record.captured  # noqa
+                assert string_value in record.captured, (  # noqa: S101
+                    f"Expected substring {string_value!r} not found in "
+                    f"{record.record_path}"
+                )
 
     def persist(self):
         record_list = self.__record_list
@@ -345,7 +447,7 @@ def pytest_addoption(parser: Parser):
 @pytest.fixture(scope="session")  # type: ignore
 def rewrite_expected(request: SubRequest) -> bool:
     """Force rewriting of all expected data by : `record_stdout` and `recorder`."""
-    return request.config.getoption("--rewrite-expected")
+    return bool(request.config.getoption("--rewrite-expected"))
 
 
 @pytest.fixture

tests/conftest.py

@@ -1,6 +1,6 @@
 symbol,name,summary,currency,sector,industry_group,industry,exchange,market,country,state,city,zipcode,website,market_cap,isin,cusip,figi,composite_figi,shareclass_figi
-005860.KQ,"Hanil Feed Co., Ltd.","Hanil Feed Co., Ltd. produces and sells animal feed in South Korea. The company offers mixed feeds, including shelled corn, corn mix, green CF, and concentrates, as well as MET, a high-fiber custom-made feed. It also provides single grain feed products, such as soybean meal, cotton seed pulp, palm pulp, farina, sunflower meal, and wheat bran; and feed supplements comprising green cell, fodder, oat straw, reed, beet pulp, cotton seed, apple pulp, and alfalfa. The company was founded in 1963 and is based in Yongin, South Korea.",KRW,Consumer Staples,"Food, Beverage & Tobacco",,KOE,KONEX,South Korea,,Yongin-Si,,http://www.hanilfeed.com,Micro Cap,,,,,
+005860.KQ,"Hanil Feed Co., Ltd.","Hanil Feed Co., Ltd. produces and sells animal feed in South Korea. The company offers mixed feeds, including shelled corn, corn mix, green CF, and concentrates, as well as MET, a high-fiber custom-made feed. It also provides single grain feed products, such as soybean meal, cotton seed pulp, palm pulp, farina, sunflower meal, and wheat bran; and feed supplements comprising green cell, fodder, oat straw, reed, beet pulp, cotton seed, apple pulp, and alfalfa. The company was founded in 1963 and is based in Yongin, South Korea.",KRW,Consumer Staples,"Food, Beverage & Tobacco",,KOE,KONEX,South Korea,,Yongin-Si,,http://www.hanilfeed.com,Micro Cap,KR7005860002,,,,
 0236.HK,San Miguel Brewery Hong Kong Limited,"San Miguel Brewery Hong Kong Limited, together with its subsidiaries, manufactures and distributes bottled, canned, and draught beers under the San Miguel Pale Pilsen brand in Hong Kong, Mainland China, the Philippines, and internationally. It also sells beers under the San Miguel Pale Pilsen, San Mig Light, Dragon Beer, Red Horse, Guang's Pineapple Beer, etc. The company was founded in 1948 and is based in Shatin, Hong Kong. San Miguel Brewery Hong Kong Limited is a subsidiary of Neptunia Corporation Limited.",HKD,Consumer Staples,"Food, Beverage & Tobacco",Beverages,HKG,Hong Kong Stock Exchange,Hong Kong,,Sha Tin,,http://info.sanmiguel.com.hk,Nano Cap,,,,,
 0359.HK,"China Haisheng Juice Holdings Co., Ltd","China Haisheng Juice Holdings Co., Ltd., an investment holding company, manufactures and sells fruit juice concentrates and other related products. It is also involved in the plantation and sale of apple saplings, apples, and other fruits; production and sale of feed; and manufacture and sale of pectin and bottled fruit juice. The company operates in the People's Republic of China, the United States, Canada, South Africa, Saudi Arabia, Japan, Australia, Russia, Germany, the Netherlands, Turkey, and internationally. China Haisheng Juice Holdings Co., Ltd. is headquartered in Central, Hong Kong.",HKD,Consumer Staples,"Food, Beverage & Tobacco",Beverages,HKG,Hong Kong Stock Exchange,Hong Kong,,Central,,http://www.chinahaisheng.com,Nano Cap,KYG2111D1060,,,,
 042670.KS,"Doosan Infracore Co., Ltd.","Doosan Infracore Co., Ltd. engages in the infrastructure support business in South Korea and internationally. It manufactures and sells construction equipment, engines, attachments, and utility equipment. The company offers excavators, wheel loaders, portable air compressors, articulated dump trucks, lighting systems, and portable power equipment. It also provides cabin raisers, module units, tilting and elevating cabins, buckets, grapples, crushers, demolition equipment, and material handling equipment. In addition, the company offers off-highway and on-highway, and marine diesel engines, and gas engines for buses and trucks, power generators, and ships; and parts and services. It sells its products under the Doosan, Bobcat, and Geith brands. Doosan Infracore Co., Ltd. was founded in 1937 and is headquartered in Incheon, South Korea.",KRW,Industrials,Capital Goods,Machinery,KSC,KOSPI Stock Market,South Korea,,Incheon,,http://www.doosaninfracore.com,Mid Cap,,,,,
-070590.KQ,"Hansol Inticube Co., Ltd.","Hansol Inticube Co., Ltd. engages in setting up contact center infrastructure, and developing contact center solutions and wireless Internet solutions in South Korea. It offers digital contact center solutions, such as integrated routing, omnichannel, customer experience, outbound, self service, work force optimization, service control, and system control; AI communication solutions, including cloud type AI communication, robotic process automation, service orchestration engine, and voice gateway; and cloud-based communication service comprising contact center as a service and unified communication as a service. It also provides mobile solutions consisting of Internet protocol location server, MMSC, short message service center, cell broadcasting message service center, short message service gateway, media gateway control function, short message spam filtering system, push notification system, Internet protocol short message gateway, message sensing system, mobile signature service provider system, and remote applet management system, as well as mobile communication switching services and gateway switching services. The company was formerly known as Inticube Co., Ltd. and changed its name to Hansol Inticube Co., Ltd. in 2008. Hansol Inticube Co., Ltd. was founded in 2003 and is based in Seoul, South Korea.",KRW,Information Technology,Technology Hardware & Equipment,,KOE,KONEX,South Korea,,Seoul,,http://www.hansolinticube.com,Nano Cap,,,,,
+070590.KQ,"Hansol Inticube Co., Ltd.","Hansol Inticube Co., Ltd. engages in setting up contact center infrastructure, and developing contact center solutions and wireless Internet solutions in South Korea. It offers digital contact center solutions, such as integrated routing, omnichannel, customer experience, outbound, self service, work force optimization, service control, and system control; AI communication solutions, including cloud type AI communication, robotic process automation, service orchestration engine, and voice gateway; and cloud-based communication service comprising contact center as a service and unified communication as a service. It also provides mobile solutions consisting of Internet protocol location server, MMSC, short message service center, cell broadcasting message service center, short message service gateway, media gateway control function, short message spam filtering system, push notification system, Internet protocol short message gateway, message sensing system, mobile signature service provider system, and remote applet management system, as well as mobile communication switching services and gateway switching services. The company was formerly known as Inticube Co., Ltd. and changed its name to Hansol Inticube Co., Ltd. in 2008. Hansol Inticube Co., Ltd. was founded in 2003 and is based in Seoul, South Korea.",KRW,Information Technology,Technology Hardware & Equipment,,KOE,KONEX,South Korea,,Seoul,,http://www.hansolinticube.com,Nano Cap,KR7070590005,,,,