PR #3449: Move install_maxtext_extra_deps to dependencies directory

Imported from GitHub PR #3449

Move the `install_maxtext_extra_deps` deps directory to `dependencies` after `dependencies` was added to the PyPI package.

This command still works/runs the expected installation:
```
uv pip install -e .[tpu] --resolution=lowest
install_maxtext_tpu_github_deps

install_maxtext_tpu_post_train_extra_deps
```

CI also builds this command

Before submitting this PR, please make sure (put X in square brackets):
- [x] I have performed a self-review of my code. For an optional AI review, add the `gemini-review` label.
- [x] I have necessary comments in my code, particularly in hard-to-understand areas.
- [x] I have run end-to-end tests tests and provided workload links above if applicable.
- [x] I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

Copybara import of the project:

--
277b08e by Branden Vandermoon <bvandermoon@google.com>:

Move install_maxtext_extra_deps to dependencies directory

Merging this change closes #3449

COPYBARA_INTEGRATE_REVIEW=#3449 from AI-Hypercomputer:bvandermoon-github-deps 277b08e
PiperOrigin-RevId: 886463160

Author: bvandermoon

pyproject.toml

@@ -40,7 +40,7 @@ Repository = "https://github.com/AI-Hypercomputer/maxtext.git"
 allow-direct-references = true
 
 [tool.hatch.build.targets.wheel]
-packages = ["src/MaxText", "src/maxtext", "src/dependencies"]
+packages = ["src/MaxText", "src/maxtext", "src/dependencies/github_deps", "src/dependencies"]
 
 # TODO: Add this hook back when it handles device-type parsing
 # [tool.hatch.build.targets.wheel.hooks.custom]

src/dependencies/dockerfiles/maxtext_gpu_dependencies.Dockerfile

@@ -56,7 +56,7 @@ WORKDIR /deps
 COPY ${PACKAGE_DIR}/dependencies/github_deps/ src/dependencies/github_deps/
 COPY ${PACKAGE_DIR}/dependencies/requirements/ src/dependencies/requirements/
 COPY ${PACKAGE_DIR}/dependencies/scripts/ src/dependencies/scripts/
-COPY ${PACKAGE_DIR}/maxtext/integration/vllm/ src/maxtext/integration/vllm/
+COPY ${PACKAGE_DIR}/maxtext/integration/vllm/ src/MaxText/integration/vllm/
 
 # Install dependencies - these steps are cached unless the copied files change
 RUN echo "Running command: bash setup.sh MODE=$ENV_MODE JAX_VERSION=$ENV_JAX_VERSION DEVICE=${ENV_DEVICE}"

src/dependencies/github_deps/install_post_train_deps.py

@@ -34,12 +34,14 @@ def main():
   """
   os.environ["VLLM_TARGET_DEVICE"] = "tpu"
 
-  current_dir = os.path.dirname(os.path.abspath(__file__))
-  repo_root = os.path.abspath(os.path.join(current_dir, "..", ".."))
-  extra_deps_path = os.path.join(current_dir, "post_train_deps.txt")
-  if not os.path.exists(extra_deps_path):
-    raise FileNotFoundError(f"Dependencies file not found at {extra_deps_path}")
+  # Adjust this path if your post_train_deps.txt is in a different location,
+  # e.g., script_dir / "data" / "post_train_deps.txt"
+  extra_deps_file = script_dir / "post_train_deps.txt"
 
+  if not extra_deps_file.exists():
+    print(f"Error: '{extra_deps_file}' not found.")
+    print("Please ensure 'post_train_deps.txt' is in the correct location relative to the script.")
+    sys.exit(1)
   # Check if 'uv' is available in the environment
   try:
     subprocess.run([sys.executable, "-m", "pip", "install", "uv"], check=True, capture_output=True)

src/dependencies/github_deps/install_pre_train_deps.py

@@ -37,6 +37,14 @@ def main():
   if not os.path.exists(extra_deps_path):
     raise FileNotFoundError(f"Dependencies file not found at {extra_deps_path}")
 
+  # Adjust this path if your pre_train_deps.txt is in a different location,
+  # e.g., script_dir / "data" / "pre_train_deps.txt"
+  extra_deps_file = script_dir / "pre_train_deps.txt"
+
+  if not extra_deps_file.exists():
+    print(f"Error: '{extra_deps_file}' not found.")
+    print("Please ensure 'pre_train_deps.txt' is in the correct location relative to the script.")
+    sys.exit(1)
   # Check if 'uv' is available in the environment
   try:
     subprocess.run([sys.executable, "-m", "pip", "install", "uv"], check=True, capture_output=True)