Adds a new DAG to test "Jobset Ready Healthiness" metric

- Adds an entire new DAG to test this metric automatically
- Adds the "validate_replica_number" function
- Adds a "READY_TPU" Jobset in jobset_util.py

-Fix automatic cluster naming script

-Created "validate_jobset_replica_number" funtion in jobset_util.py
-Changed workload name to "IDLE_READY_TPU_20M"
-Changed schedule time and added statement to only run on PROD environment

Changed to new nodepool creation format

Remove unused DAG flow

Author: QuinnMMcGarry

dags/tpu_observability/utils/jobset_util.py

@@ -555,9 +555,12 @@ def wait_for_jobset_ttr_to_be_found(node_pool: node_pool_info) -> bool:
   return len(time_series) > 0
 
 
-@task.sensor(poke_interval=30, timeout=600, mode="reschedule")
-def wait_for_jobset_status_occurrence(
-    replica_type: str, job_name: str, node_pool: node_pool_info
+@task.sensor(poke_interval=30, timeout=900, mode="reschedule")
+def validate_jobset_replica_number(
+    node_pool: node_pool_info,
+    jobset_config: JobSet,
+    replica_type: str,
+    correct_replica_num: int,
 ):
   """
   A sensor which checks if are any jobset replicas in a status type.