[fix] handle parquet schema mismatch in dataset concatenation

[mwxely]

Motivation

When using YAML config to load multiple parquet files from different sources, concatenate_datasets() in DataUtilities.load_yaml() crashes with ArrowInvalid if the files have incompatible inferred Arrow schemas.

This is a common scenario in practice. For example, when one parquet file contains rows with image_url: {"url": "http://..."} (inferred as struct{url: string}) while another file has all-null values in the same column (inferred as null type), Arrow refuses to concatenate the two because their schemas don't match — even though the data is semantically compatible.

Reproduction:

datasets:
  - path: dataset_a.parquet   # has image_url with values → struct{url: string}
    data_type: parquet
  - path: dataset_b.parquet   # has image_url all null → null type
    data_type: parquet

ArrowInvalid: Schema at index 1 was different:
  image_url: null
vs
  image_url: struct<url: string>

Modifications

• Add _safe_concatenate_datasets() helper in data_utils.py that wraps concatenate_datasets() with a try/except fallback
• On success: zero overhead, same code path as before
• On schema mismatch: falls back to to_list() + from_list() row-wise concatenation with a warning log so users know the fallback was triggered
• Applied to both load_yaml() call sites (line 134 and line 239)

[kcz358]

This would be cause extremely slow and large memory needed under large scale data. Recommend to preprocess or cast dataset before training to ensure the dataset is in the same features.

[mwxely]

Good point — I've replaced the to_list() fallback with cast()-based schema alignment in e28d773. The new approach:

1. Tries cast() to the first dataset's features (fast, zero-copy for compatible schemas)
2. If that fails, builds a merged feature set preferring non-null types and casts all datasets to it

This keeps the fix zero-overhead on the happy path and avoids materializing the full dataset into memory on schema mismatch.