Assortment of VortexSource improvements and fixes (#8718)

## Rationale for this change

General improvements of the `FileSource`-based Apache DataFusion
integration.

## What changes are included in this PR?

1. DF metrics are in use, so make better use of them both on reads and
writes, which is a step towards having the same default observability
that `ParquetSource` provides.
2. Fix a subtle issue where a dynamic expression only references
partition literals, which we currently don't handle correctly.
3. Drop `batch_size` from the source, by default DataFusion splits the
data coming out of `DataScanExec` nodes.
4. Make sure we keep schema and field metadata through all the
projections paths.

## What APIs are changed? Are there any user-facing changes?

More metrics will be reported in DF, and batch size needs to be done
through the `DataSourceExec` and `FileScanConfig` level.

---------

Signed-off-by: Adam Gutglick <adam@spiraldb.com>

Author: AdamGS

vortex-datafusion/src/convert/schema.rs

@@ -56,7 +56,10 @@ pub fn calculate_physical_schema(
         })
         .collect::<DFResult<Vec<_>>>()?;
 
-    Ok(Schema::new(fields))
+    Ok(Schema::new_with_metadata(
+        fields,
+        reference_logical_schema.metadata().clone(),
+    ))
 }
 
 /// Calculate the physical Arrow type for a field, preferring the logical type when the
@@ -246,6 +249,32 @@ mod tests {
         );
     }
 
+    #[test]
+    fn test_schema_metadata_preserved() -> DFResult<()> {
+        let logical_schema = Schema::new_with_metadata(
+            vec![Field::new("col", DataType::Int32, false)],
+            [("table".to_string(), "metadata".to_string())]
+                .into_iter()
+                .collect(),
+        );
+        let dtype = DType::Struct(
+            StructFields::from_iter([(
+                "col",
+                DType::Primitive(PType::I32, Nullability::NonNullable),
+            )]),
+            Nullability::NonNullable,
+        );
+
+        let physical_schema =
+            calculate_physical_schema(&dtype, &logical_schema, &ArrowSession::default())?;
+
+        assert_eq!(
+            physical_schema.metadata().get("table"),
+            Some(&"metadata".to_string())
+        );
+        Ok(())
+    }
+
     #[test]
     fn test_utf8_variants_preserved() {
         // Non-view string types become view types after roundtrip through DType,