Major cleanup of DE Zoomcamp FAQ

- Reorganize sections: split general into general+environment,
  split module-1 (was 145 files) into module-1-{data,docker,
  postgres,gcp,terraform}, rename module-7 from "Streaming with
  Kafka" to just "Streaming"
- Move ~25 misplaced files to correct sections (Kestra→m2, dbt→m4,
  Spark→m6, Bruin→m5, GCP setup→general, etc.)
- Merge ~25 clusters of duplicate FAQs (biggest: 11 dbt+BigQuery
  region-mismatch FAQs into one; 10 Postgres connection failures
  into one; 9 Postgres data-folder permission errors into one)
- Rewrite outdated content: redirect cohort/deadline/playlist
  questions to the course repo instead of hardcoding years; drop
  references to Mage, Prefect, Faust, RisingWave (no longer in
  syllabus); generalize Terraform 1.1.3 download to current
- Drop 22 module-6 FAQs tied to the old Spark 3.x manual install
  (findspark, winutils, SPARK_HOME, PYTHONPATH/py4j zip,
  Spark Standalone, Hadoop on Windows). The course now uses
  Spark 4.x bundled via PySpark with uv, JDK 17/21 — those
  workarounds aren't needed
- Drop Anaconda-specific FAQs and replace with a short
  "use uv" pointer; strip incidental conda mentions throughout
- Renumber sort_order sequentially within every section; fix
  files with non-hex IDs and truncated filenames
- Total: 522 → 393 files (25% smaller)

de-faq-cleanup-plan.md at the repo root tracks every individual
change made, organized by stage.

Author: alexeygrigorev

_questions/data-engineering-zoomcamp/_metadata.yaml

@@ -3,8 +3,25 @@ course_name: "Data Engineering Zoomcamp"
 sections:
   - id: general
     name: "General Course-Related Questions"
-  - id: module-1
-    name: "Module 1: Docker and Terraform"
+    comment: "Course logistics: cohort schedule, certificate, deadlines, leaderboard, project rules, contributing"
+  - id: environment
+    name: "Environment & Setup"
+    comment: "Where/how to run the course: local vs Codespaces vs GCP VM, Python version, Windows/WSL/Mac, troubleshooting workflow, IDE/editor tips"
+  - id: module-1-data
+    name: "Module 1: Taxi Data (download & handling)"
+    comment: "How to download / unzip / handle the NY taxi datasets used throughout the course (CSV.GZ, Parquet, wget/curl issues)"
+  - id: module-1-docker
+    name: "Module 1: Docker"
+    comment: "Docker engine, Docker Compose, volumes, networking, WSL — anything containerization-related in module 1"
+  - id: module-1-postgres
+    name: "Module 1: Postgres, pgAdmin & Python ingestion"
+    comment: "Postgres, pgcli, pgAdmin, Python ingestion (pandas, SQLAlchemy, psycopg), and SQL"
+  - id: module-1-gcp
+    name: "Module 1: GCP setup & VM"
+    comment: "GCP account, billing, free trial, SDK install, service accounts, VM (Compute Engine) setup, SSH"
+  - id: module-1-terraform
+    name: "Module 1: Terraform"
+    comment: "Terraform IaC for GCP — provider, credentials, errors, state, teardown"
   - id: module-2
     name: "Module 2: Workflow Orchestration"
     comment: "Questions about Kestra and workflow orchestration go here" 
@@ -21,8 +38,8 @@ sections:
     name: "Module 6: Spark"
     comment: "Questions about Apache Spark, PySpark, and Dataproc go here"
   - id: module-7
-    name: "Module 7: Streaming with Kafka"
-    comment: "Questions about Kafka, streaming, and real-time data processing go here"
+    name: "Module 7: Streaming"
+    comment: "Questions about streaming and real-time data processing go here. Do not name specific tools (Kafka, PyFlink, Spark Streaming, Redpanda, etc.) in the section name — the course's streaming stack changes between cohorts."
   - id: project
     name: "Project"
   - id: workshop-1-dlthub

_questions/data-engineering-zoomcamp/environment/001_29e58c5c37_environment-is-python-39-still-the-recommended-ver.md

@@ -0,0 +1,11 @@
+---
+id: 29e58c5c37
+question: 'Environment: which Python version should I use?'
+sort_order: 1
+---
+
+Python 3.10 or 3.11 is a safe default — it works with the libraries used across the course (pandas, SQLAlchemy, dbt, dlt, PySpark with recent Spark releases, etc.).
+
+If you're following older recorded videos that use Python 3.9, that still works for everything except the very latest library versions; troubleshooting against the videos is easier on the version they use.
+
+If a specific module uses a stricter requirement, the course repo's module README will say so.

_questions/data-engineering-zoomcamp/environment/002_4f1fe161b1_environment-should-i-use-my-local-machine-gcp-or-g.md

@@ -0,0 +1,34 @@
+---
+id: 4f1fe161b1
+question: 'Environment: which OS / cloud / dev setup should I use? (local vs GCP VM
+  vs Codespaces, AWS alternative, OS support)'
+sort_order: 2
+---
+
+## OS support
+
+Linux is the smoothest, but the course works on macOS and Windows too. Students in the most recent cohorts have completed it on all three. Windows users typically need WSL2 to avoid friction with shell scripts in later modules.
+
+## Where to run the course
+
+You have three good options. Pick whichever suits you:
+
+1. Local machine (laptop / PC). Easiest if you're already comfortable with Docker locally. Windows users should use WSL2 from the start.
+2. GitHub Codespaces. A free Linux dev environment with Docker, Python, and many CLI tools pre-installed. Useful if your laptop is underpowered, or if you switch between home and office machines. Ports for things like Kestra/pgAdmin are exposed via Codespaces' forwarded URL — not `http://localhost`.
+3. Google Cloud VM. The course videos demonstrate this setup. Useful if you want a persistent remote environment to SSH into, especially while staying logged in across machines.
+
+You don't need both Codespaces and a GCP VM — pick one. You will need a GCP account regardless because the course uses BigQuery (in Module 3 and the project), but GCP for compute is optional.
+
+## Can I use AWS / Snowflake / Azure / a different stack?
+
+Yes. The capstone project is graded on creating a data pipeline and producing a visualization — it doesn't mandate any specific cloud. Considerations:
+
+- The lessons are recorded against GCP, so you'll need to translate steps yourself.
+- You may need to explain your choice during peer review.
+- Fewer fellow students will be using AWS/Azure, so help in Slack may be slower.
+
+If you only want to run the course locally without any cloud, you can do that for everything except Module 3's BigQuery homework, which requires GCP.
+
+## Is the course Windows / macOS / Linux friendly?
+
+All three work. Linux is best by default. On Windows, install WSL2 and run everything inside a WSL distro — Git Bash and MINGW64 are not always sufficient for shell scripts later in the course.

_questions/data-engineering-zoomcamp/environment/003_b92da7c113_environment-could-not-establish-connection-to-myse.md

@@ -0,0 +1,23 @@
+---
+id: b92da7c113
+question: 'Environment - Could not establish connection to "MyServerName": Got bad
+  result from install script'
+sort_order: 3
+---
+
+This issue occurs when attempting to connect to a GCP VM using VSCode on a Windows machine. You can resolve it by changing a registry value in the registry editor.
+
+Open the Run command window:
+- Use the shortcut keys `Windows + R`, or
+- Right-click "Start" and click "Run".
+
+Open the Registry Editor:
+- Type `regedit` in the Run command window, then press Enter.
+
+Change the registry value:
+- Navigate to `HKEY_CURRENT_USER\Software\Microsoft\Command Processor`.
+- Change the "Autorun" value from "if exists" to a blank.
+
+Alternatively, you can delete the saved fingerprint within the known_hosts file:
+
+In Windows, locate the file at `C:\Users\<your_user_name>\.ssh\known_hosts` and remove the entry for the server.

_questions/data-engineering-zoomcamp/environment/004_a8219681ec_environment-why-are-we-using-gcp-and-not-other-clo.md

@@ -0,0 +1,33 @@
+---
+id: a8219681ec
+question: 'GCP for the course: free trial vs sandbox, paying, country restrictions,
+  why GCP'
+sort_order: 4
+---
+
+## Why GCP and not AWS / Azure?
+
+For uniformity across the cohort. The course uses BigQuery, which is GCP-only, and most students already have a Google account that works for sign-up. The concepts (data warehouse, object storage, IaC) translate to AWS/Azure, but the lessons are recorded against GCP. You can use a different cloud — see [the environment FAQ](#4f1fe161b1) for tradeoffs.
+
+## Do I have to pay?
+
+No. GCP offers a free trial with $300 in credits for new accounts. The course materials fit comfortably within that budget if you destroy unused resources (VMs, datasets, buckets) after each module. Check your billing dashboard daily, especially after spinning up Compute Engine VMs.
+
+To sign up for the free trial you need a valid credit/debit card; GCP uses it to verify identity but doesn't charge it without your consent.
+
+## Free Trial vs Sandbox — which one?
+
+GCP has two free options. They are not equivalent for this course:
+
+- Free Trial ($300 credit, 90 days). Required for the course — gives you VMs, GCS buckets, and full BigQuery functionality.
+- Sandbox (free, no credit card). Limited services. It does not include VMs or GCS, and BigQuery features are restricted, so you cannot complete the course on Sandbox alone.
+
+Use the Free Trial.
+
+## My country isn't supported / my card isn't accepted
+
+GCP isn't available in some countries, and some cards are rejected even where it is. Workarounds students have used:
+
+- Try a different card. Cards from some banks (e.g. Kazakhstan-based Kaspi) sometimes don't work; cards from other banks/countries (e.g. TBC in Georgia) do.
+- Pyypl and similar virtual cards have worked for some.
+- If you can't get a GCP account at all, you can still complete most of the course locally — only Module 3's homework strictly requires BigQuery. See the environment FAQ for which parts have local alternatives.

_questions/data-engineering-zoomcamp/environment/005_8b0214d089_environment-roadblock-for-windows-users-in-modules.md

@@ -0,0 +1,7 @@
+---
+id: 8b0214d089
+question: 'Environment: shell scripts (*.sh) don''t work for Windows users without WSL'
+sort_order: 5
+---
+
+Several modules use shell scripts (`*.sh`) for setup or runtime tasks. Most Windows users running them outside WSL will hit issues — Git Bash and MINGW64 are not always sufficient. Set up a WSL environment from the start to avoid getting blocked partway through the course.

_questions/data-engineering-zoomcamp/environment/006_a83f047f52_how-to-troubleshoot-issues.md

@@ -0,0 +1,30 @@
+---
+id: a83f047f52
+question: How to troubleshoot issues and ask good questions
+sort_order: 6
+---
+
+## Try to solve it yourself first
+
+- Read the error message carefully — it usually includes a line number, a stack trace, and a description of what went wrong.
+- Search the message: copy the most specific part of the error (not the whole stack trace) into Google. The format `<tool> <error message>` works well, e.g. `pgcli error column c.relhasoids does not exist`.
+- Check the official documentation of the tool you're using.
+- Use Ctrl+F in this FAQ and in Slack channel pinned messages.
+- Restart the process / container / shell / VM and try once more — many transient errors resolve this way.
+- If you suspect the install is broken, uninstall first, then reinstall. Reinstalling on top of a broken install rarely helps.
+
+## Asking for help in Slack / forums
+
+When the troubleshooting steps don't help and you need another pair of eyes, include enough info that someone can actually help without going back and forth:
+
+- Operating system and version (e.g. Windows 11 + WSL Ubuntu 24.04, Mac M2, Linux Ubuntu 22.04).
+- Which lesson / video you're following, and which command failed.
+- The exact command and the exact error — paste both as text inside triple-backtick code blocks. Don't paste screenshots of text.
+- What you've already tried. If you skip this, helpers' first suggestions will be the things you already tried.
+- Stay in one thread. Reply to your own question; don't open a new post for a follow-up.
+
+If the same problem recurs, post in the same thread with what changed in your environment since last time.
+
+## Help others by contributing back
+
+If your problem isn't yet covered in this FAQ, consider [opening a PR](https://github.com/DataTalksClub/faq) so the next student doesn't have to debug it from scratch.

…slash-as-an-escape-character-in-git-b.md …slash-as-an-escape-character-in-git-b.md_questions/data-engineering-zoomcamp/module-1/007_1ba19ed6a0_git-bash-backslash-as-an-escape-character-in-git-b.md renamed to _questions/data-engineering-zoomcamp/environment/007_1ba19ed6a0_git-bash-backslash-as-an-escape-character-in-git-b.md _questions/data-engineering-zoomcamp/module-1/007_1ba19ed6a0_git-bash-backslash-as-an-escape-character-in-git-b.md renamed to _questions/data-engineering-zoomcamp/environment/007_1ba19ed6a0_git-bash-backslash-as-an-escape-character-in-git-b.md

@@ -0,0 +1,17 @@
+---
+id: e5dc51eac9
+question: 'VS Code: Tab using spaces'
+sort_order: 8
+---
+
+
+
+Error:
+
+```
+Makefile:2: *** missing separator.  Stop.
+```
+
+Solution:
+
+Tabs in documents should be converted to Tab instead of spaces. [Follow this stack](https://stackoverflow.com/questions/36814642/visual-studio-code-convert-spaces-to-tabs).

_questions/data-engineering-zoomcamp/environment/008_e5dc51eac9_vs-code-tab-using-spaces.md

@@ -0,0 +1,17 @@
+---
+id: 5b54567e89
+question: Opening an HTML file with a Windows browser from Linux running on WSL
+sort_order: 9
+---
+
+If you’re running Linux on Windows Subsystem for Linux (WSL) 2, you can open HTML files from the guest (Linux) with any Internet Browser installed on the host (Windows). Just install [wslu](https://wslutiliti.es/wslu/install.html) and open the page using `wslview`:
+
+```bash
+wslview index.html
+```
+
+You can customize which browser to use by setting the `BROWSER` environment variable first. For example:
+
+```bash
+export BROWSER='/mnt/c/Program Files/Firefox/firefox.exe'
+```