Skip to content

Commit d878af6

Browse files
authored
update ainode from 2093 (#1116)
1 parent f3b41d5 commit d878af6

16 files changed

Lines changed: 2472 additions & 776 deletions

File tree

src/UserGuide/Master/Table/AI-capability/AINode_Upgrade_apache.md

Lines changed: 144 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -164,8 +164,9 @@ It costs 1.615s
164164
1. AINode uses Transformers v4.56.2; avoid inheriting interfaces from older versions (<4.50).
165165
2. The model must inherit an AINode inference pipeline class (currently supports forecasting pipeline):
166166
* iotdb-core/ainode/iotdb/ainode/core/inference/pipeline/basic_pipeline.py
167-
168-
```python
167+
168+
**Before V2.0.9.3**
169+
```Python
169170
class BasicPipeline(ABC):
170171
def __init__(self, model_id, **model_kwargs):
171172
self.model_info = model_info
@@ -174,12 +175,16 @@ It costs 1.615s
174175

175176
@abstractmethod
176177
def preprocess(self, inputs, **infer_kwargs):
177-
"""Preprocess input data before inference, including shape validation and numerical conversion."""
178+
"""
179+
Preprocess the input data before the inference task starts, including shape validation and numerical conversion.
180+
"""
178181
pass
179182

180183
@abstractmethod
181184
def postprocess(self, output, **infer_kwargs):
182-
"""Postprocess model outputs after inference."""
185+
"""
186+
Postprocess the output results after the inference task is completed.
187+
"""
183188
pass
184189

185190

@@ -189,69 +194,176 @@ It costs 1.615s
189194

190195
def preprocess(self, inputs: list[dict[str, dict[str, torch.Tensor] | torch.Tensor]], **infer_kwargs):
191196
"""
192-
Preprocess and validate input data before model inference.
197+
Preprocess the input data before passing it to the model for inference, validating the shape and type of the input data.
193198
194199
Args:
195200
inputs (list[dict]):
196-
Input data as list of dicts containing:
197-
- 'targets': Tensor of shape (input_length,) or (target_count, input_length).
198-
- 'past_covariates': Optional dict of tensors, each of shape (input_length,).
199-
- 'future_covariates': Optional dict of tensors, each of shape (input_length,).
200-
infer_kwargs (dict, optional): Additional inference parameters, e.g.:
201-
- `output_length` (int): Required if 'future_covariates' provided for validation.
201+
Input data, a list of dictionaries, each dictionary contains:
202+
- 'targets': Tensor with shape (input_length,) or (target_count, input_length).
203+
- 'past_covariates': Optional, dictionary of tensors, each tensor with shape (input_length,).
204+
- 'future_covariates': Optional, dictionary of tensors, each tensor with shape (input_length,).
205+
206+
infer_kwargs (dict, optional): Additional keyword arguments for inference, such as:
207+
- `output_length`(int): Used to validate the validity of 'future_covariates' if provided.
202208
203209
Raises:
204-
ValueError: For invalid input formats (missing keys, invalid tensor shapes).
210+
ValueError: If the input format is invalid (e.g., missing keys, invalid tensor shapes).
205211
206212
Returns:
207-
Preprocessed and validated input ready for model inference.
213+
Preprocessed and validated input data that can be directly used for model inference.
208214
"""
209215
pass
210216

211217
def forecast(self, inputs, **infer_kwargs):
212218
"""
213-
Perform forecasting on given inputs.
219+
Perform forecasting on the given inputs.
214220
215221
Parameters:
216-
inputs: Input data for forecasting (type/structure model-specific).
217-
**infer_kwargs: Additional parameters, e.g.:
218-
- `output_length` (int): Number of future time points to generate.
222+
inputs: Input data for forecasting. The type and structure depend on the specific model implementation.
223+
**infer_kwargs: Additional inference parameters, e.g.:
224+
- `output_length`(int): The number of time points the model should generate.
219225
220226
Returns:
221-
Forecast output (format model-specific).
227+
Forecast output, the specific form depends on the specific model implementation.
222228
"""
223229
pass
224230

225231
def postprocess(self, outputs: list[torch.Tensor], **infer_kwargs) -> list[torch.Tensor]:
226232
"""
227-
Postprocess model outputs, validate shapes, and ensure expected dimensions.
233+
Postprocess the model outputs after inference, validating the shape of the output data and ensuring it meets the expected dimensions.
228234
229235
Args:
230-
outputs: List of 2D tensors, each of shape `[target_count, output_length]`.
236+
outputs:
237+
Model outputs, a list of 2D tensors, each tensor with shape `[target_count, output_length]`.
231238
232239
Raises:
233-
InferenceModelInternalException: For invalid output tensor shapes.
234-
ValueError: For incorrect output format.
240+
InferenceModelInternalException: If the output tensor shape is invalid (e.g., incorrect dimensions).
241+
ValueError: If the output format is incorrect.
235242
236243
Returns:
237-
List of postprocessed 2D tensors.
244+
list[torch.Tensor]:
245+
Postprocessed outputs, which will be a list of 2D tensors.
238246
"""
239247
pass
240248
```
241-
3. Modify `config.json` to include:
242-
```json
249+
250+
**From V2.0.9.3 onwards**
251+
```Python
252+
class BasicPipeline(ABC):
253+
def __init__(self, model_id, **model_kwargs):
254+
self.model_info = model_info
255+
self.device = model_kwargs.get("device", "cpu")
256+
self.model = load_model(model_info, device_map=self.device, **model_kwargs)
257+
258+
@abstractmethod
259+
def preprocess(self, inputs, **infer_kwargs):
260+
"""
261+
Preprocess the input data before the inference task starts, including shape validation and numerical conversion.
262+
"""
263+
pass
264+
265+
@abstractmethod
266+
def postprocess(self, output, **infer_kwargs):
267+
"""
268+
Postprocess the output results after the inference task is completed.
269+
"""
270+
pass
271+
272+
273+
class ForecastPipeline(BasicPipeline):
274+
def __init__(self, model_info, **model_kwargs):
275+
super().__init__(model_info, model_kwargs=model_kwargs)
276+
277+
def _preprocess(
278+
self,
279+
inputs: list[dict[str, dict[str, torch.Tensor] | torch.Tensor]],
280+
**infer_kwargs,
281+
):
282+
"""
283+
Preprocess the input data before passing it to the model for inference, validating the shape and type of the input data.
284+
285+
Args:
286+
inputs (list[dict[str, dict[str, torch.Tensor] | torch.Tensor]]):
287+
Input data, a list of dictionaries, each dictionary contains:
288+
- 'targets': Tensor with shape (input_length,) or (target_count, input_length).
289+
- 'past_covariates': Optional, dictionary of tensors, each tensor with shape (input_length,).
290+
- 'future_covariates': Optional, dictionary of tensors, each tensor with shape (input_length,).
291+
292+
infer_kwargs (dict, optional): Additional keyword arguments for inference, such as:
293+
- `output_length`(int): Used to validate the validity of 'future_covariates' if provided.
294+
295+
Raises:
296+
ValueError: If the input format is invalid (e.g., missing keys, invalid tensor shapes).
297+
298+
Returns:
299+
Preprocessed and validated input data that can be directly used for model inference.
300+
"""
301+
pass
302+
303+
def forecast(self, inputs, **infer_kwargs):
304+
"""
305+
Perform forecasting on the given inputs.
306+
307+
Parameters:
308+
inputs: Input data for forecasting. The type and structure depend on the specific model implementation.
309+
**infer_kwargs: Additional inference parameters, e.g.:
310+
- `output_length`(int): The number of time points the model should generate.
311+
312+
Returns:
313+
Forecast output, the specific form depends on the specific model implementation.
314+
"""
315+
pass
316+
317+
def _postprocess(self, outputs, **infer_kwargs) -> list[torch.Tensor]:
318+
"""
319+
Postprocess the model outputs after inference, validating the shape of the output data and ensuring it meets the expected dimensions.
320+
321+
Args:
322+
outputs:
323+
Model outputs, a list of 2D tensors, each tensor with shape `[target_count, output_length]`.
324+
325+
Raises:
326+
InferenceModelInternalException: If the output tensor shape is invalid (e.g., incorrect dimensions).
327+
ValueError: If the output format is incorrect.
328+
329+
Returns:
330+
list[torch.Tensor]:
331+
Postprocessed outputs, which will be a list of 2D tensors.
332+
"""
333+
pass
334+
```
335+
336+
3. Modify the model configuration file `config.json` to ensure it contains the following fields:
337+
338+
**Before V2.0.9.3**
339+
```JSON
340+
{
341+
"auto_map": {
342+
"AutoConfig": "config.Chronos2CoreConfig", // Specify the model Config class
343+
"AutoModelForCausalLM": "model.Chronos2Model" // Specify the model class
344+
},
345+
"pipeline_cls": "pipeline_chronos2.Chronos2Pipeline", // Specify the inference pipeline for the model
346+
"model_type": "custom_t5", // Specify the model type
347+
}
348+
```
349+
* The model Config class and model class **must** be specified via `auto_map`;
350+
* The inference pipeline class **must** be inherited and specified;
351+
* For built-in and user-defined models managed by AINode, `model_type` also serves as a unique non-duplicable identifier. That is, the model type to be registered must not duplicate any existing model types; models created via fine-tuning will inherit the model type of the original model.
352+
353+
**From V2.0.9.3 onwards**
354+
> The `model_type` parameter is **not required**
355+
```JSON
243356
{
244357
"auto_map": {
245-
"AutoConfig": "config.Chronos2CoreConfig",
246-
"AutoModelForCausalLM": "model.Chronos2Model"
358+
"AutoConfig": "config.Chronos2CoreConfig", // Specify the model Config class
359+
"AutoModelForCausalLM": "model.Chronos2Model" // Specify the model class
247360
},
248-
"pipeline_cls": "pipeline_chronos2.Chronos2Pipeline",
249-
"model_type": "custom_t5"
361+
"pipeline_cls": "pipeline_chronos2.Chronos2Pipeline", // Specify the inference pipeline for the model
250362
}
251363
```
252-
* Must specify model Config and model classes via `auto_map`.
253-
* Must implement and specify the inference pipeline class.
254-
* `model_type` serves as a unique identifier. Must not conflict with existing model types (for both `builtin` and `user_defined` models).
364+
* The model Config class and model class **must** be specified via `auto_map`;
365+
* The inference pipeline class **must** be inherited and specified;
366+
255367
4. Ensure the model directory contains:
256368
* `config.json` (model configuration)
257369
* `model.safetensors` (model weights)

0 commit comments

Comments
 (0)