Add Polars lecture to complement existing Pandas lecture (#408)

* Initial plan

* Create polars lecture based on pandas lecture and add to TOC

Co-authored-by: mmcky <8263752+mmcky@users.noreply.github.com>

* Fix Polars .suffix() method to use .name.suffix() for compatibility

Co-authored-by: mmcky <8263752+mmcky@users.noreply.github.com>

* Fix column name from 'DATE' to 'observation_date' in FRED data filtering

Co-authored-by: mmcky <8263752+mmcky@users.noreply.github.com>

* Fix TypeError in Polars exercise and update pivot API usage

Co-authored-by: mmcky <8263752+mmcky@users.noreply.github.com>

* Fix DuplicateError in Polars exercise join logic by using concat and pivot approach

Co-authored-by: mmcky <8263752+mmcky@users.noreply.github.com>

* initial @mmcky review, in-work

* updates to series section

* edit round of DataFrames

* Update lectures/polars.md

Co-authored-by: Humphrey Yang <39026988+HumphreyYang@users.noreply.github.com>

* Update lectures/polars.md

Co-authored-by: Humphrey Yang <39026988+HumphreyYang@users.noreply.github.com>

* Update lectures/polars.md

Co-authored-by: Humphrey Yang <39026988+HumphreyYang@users.noreply.github.com>

* Update lectures/polars.md

Co-authored-by: Humphrey Yang <39026988+HumphreyYang@users.noreply.github.com>

* Update lectures/polars.md

Co-authored-by: Humphrey Yang <39026988+HumphreyYang@users.noreply.github.com>

* Update lectures/polars.md

Co-authored-by: Humphrey Yang <39026988+HumphreyYang@users.noreply.github.com>

* Update lectures/polars.md

Co-authored-by: Humphrey Yang <39026988+HumphreyYang@users.noreply.github.com>

* Update lectures/polars.md

Co-authored-by: Humphrey Yang <39026988+HumphreyYang@users.noreply.github.com>

* Update lectures/polars.md

Co-authored-by: Humphrey Yang <39026988+HumphreyYang@users.noreply.github.com>

* Update lectures/polars.md

Co-authored-by: Humphrey Yang <39026988+HumphreyYang@users.noreply.github.com>

* Update lectures/polars.md

Co-authored-by: Humphrey Yang <39026988+HumphreyYang@users.noreply.github.com>

* Update lectures/polars.md

Co-authored-by: Humphrey Yang <39026988+HumphreyYang@users.noreply.github.com>

* Update lectures/polars.md

Co-authored-by: Humphrey Yang <39026988+HumphreyYang@users.noreply.github.com>

* Update lectures/polars.md

Co-authored-by: Humphrey Yang <39026988+HumphreyYang@users.noreply.github.com>

* Update lectures/polars.md

Co-authored-by: Humphrey Yang <39026988+HumphreyYang@users.noreply.github.com>

* Update lectures/polars.md

Co-authored-by: Humphrey Yang <39026988+HumphreyYang@users.noreply.github.com>

* Update lectures/polars.md

Co-authored-by: Humphrey Yang <39026988+HumphreyYang@users.noreply.github.com>

* FIX: removed closing brackets

* add in cell fence removed from comments

* add in missing url

* Remove Exercises

* add exercises and solutions

* review of lecture and add section on Lazy evaluation

* FIX: issues with code

* Comprehensive improvements to Polars lecture

- Fix execution errors and deprecation warnings
- Add pyarrow dependency for Polars to pandas conversion
- Fix lazy evaluation method: replace describe_optimized_plan() with explain()
- Update deprecated join syntax: how='outer' to how='full'
- Fix yfinance integration with coalesce=True for different trading calendars
- Apply QuantEcon style guide compliance:
  - Convert headings from title case to sentence case
  - Split multi-sentence paragraphs per qe-writing-002 rule
  - Fix proper noun capitalization (polars -> Polars)
- Add lazy evaluation section with query optimization examples
- Expand exercises with comprehensive stock analysis examples
- Enhance plotting with markers, reference lines, and debugging info
- Fix replace() deprecation warning: use replace_strict()
- Add data validation and debugging output to exercises
- Improve visualization with better styling and error handling

All code cells now execute successfully with Polars 1.33.1

* Apply code review suggestions from @HumphreyYang and @jstac

Co-authored-by: mmcky <8263752+mmcky@users.noreply.github.com>

* Make Polars lecture PEP8 compliant with 80-character line limit

- Fixed long URL lines using proper string continuation
- Removed all trailing whitespace from code blocks
- Reformatted long method chains and function calls
- Improved docstring formatting for better readability
- Fixed exercise solutions with proper company names and color-coded plotting
- All Python code blocks now comply with PEP8 standards

* Update lectures/polars.md

Co-authored-by: Humphrey Yang <39026988+HumphreyYang@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Humphrey Yang <39026988+HumphreyYang@users.noreply.github.com>

* Fix review issues: cross-refs, URLs, headings, pip install, map_elements, duplicate display, legend order

* Address reviewer feedback: altair note, map_elements rationale, scan_csv tip

- Add note about Polars' built-in plotting API via Altair (per HumphreyYang)
- Add pedagogical note explaining why map_elements is shown (per HumphreyYang)
- Add tip about scan_csv for lazy file reading (per Shunsuke-Hori)

* Revise polars lecture: trim duplication, remove pandas plotting dependency, expand lazy eval

- Move polars after pandas_panel in TOC to keep pandas lectures together
- Remove pandas as runtime dependency; plot with matplotlib directly
- Replace map_elements code cell with concise note
- Use with_row_index() for missing value imputation
- Remove pd.to_datetime from read_data_polars helper
- Add performance comparison subsection with timing benchmark
- Merge redundant sections, cross-reference pandas lecture
- Rename pandas.md cross-ref label to pd-series for consistency
- Net reduction: 1000 -> 704 lines

* Improve polars lecture: pandas comparison, split code blocks, add prose

- Update benchmark link to official Polars TPC-H benchmarks
- Add pandas vs Polars timing comparison for small and large datasets
- Split monolithic code cells into focused cells with connecting prose
- Add connecting prose between all adjacent code cells
- Clean heading: use index directive instead of role syntax
- Remove redundant standalone index entry

* Clarify benchmark task, align Exercise 2 with pandas lecture

- Add prose explaining the grouped weighted-average computation
- Change Exercise 2 start date from 2000 to 1971 to match pandas
- Remove year >= 2001 filter from solution

* Add note about Polars Altair-based plotting API

* [polars] Sort joined frame by Date so yearly returns are correct

`read_data_polars` builds the combined price table with a chain of
`join(..., how='full', coalesce=True)` calls. Polars does not guarantee
join output row order: keys present on only one side are appended to the
end of the frame rather than slotted into place.

The Exercise 2 solution then takes `drop_nulls().first()/.last()` within
`group_by('year')`, which assumes rows are date-ordered. With the index
data that assumption breaks, because the Nikkei trades a different
holiday calendar from the US indices, so Nikkei-only dates land at the
end of the frame. The US columns are null on those appended rows and so
`drop_nulls()` skips them, which confines the damage to the Nikkei
column and let it pass review unnoticed.

Verified against Yahoo Finance data: the fix changes 51 of 204 yearly
return cells, all in the Nikkei column. Several were badly wrong and
some had the wrong sign (2017 read -6.42% against an actual +16.18%,
2020 read +3.79% against +18.27%). The code ran green throughout and
printed plausible numbers, which is the worst failure mode for a
published lecture.

Also in this commit:

- Add a note on join ordering, which reinforces the lecture's own
  "no index, no automatic alignment" theme and is a genuine gotcha
  when migrating from pandas.
- Drop an orphaned `[^mung]` footnote carried over from pandas.md. The
  word "munging" appears nowhere in this lecture, so the definition
  rendered as nothing at all.
- Cite the official Polars migration guide for the no-index discussion
  instead of a personal Medium post, for authority and longevity.
- Drop the unused `(pd-series)=` label from pandas.md. Nothing
  references it, so the PR now touches a single lecture.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: mmcky <8263752+mmcky@users.noreply.github.com>
Co-authored-by: mmcky <mamckay@gmail.com>
Co-authored-by: Matt McKay <mmcky@users.noreply.github.com>
Co-authored-by: Humphrey Yang <39026988+HumphreyYang@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: 4 people

lectures/_toc.yml

@@ -31,6 +31,7 @@ parts:
   chapters:
   - file: pandas
   - file: pandas_panel
+  - file: polars
 - caption: More Python Programming
   numbered: true
   chapters: