Skip to content

Commit 536a261

Browse files
committed
document complete KB ETL config
1 parent 33b6c57 commit 536a261

1 file changed

Lines changed: 19 additions & 0 deletions

File tree

dashjoin-docs/docs/ai-ml-integration.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -279,6 +279,25 @@ which is a base64 encoded string. Consider the following expression:
279279

280280
It scans all files in the upload/kb folder, reads their contents using the openText function with as BASE_64.
281281
Optionally, you can provide tags.
282+
The ETL configuration is as follows:
283+
284+
```
285+
{
286+
"djClassName": "com.dashjoin.function.ETL",
287+
"ID": "etl",
288+
"expressions": {
289+
"expression": see above...
290+
},
291+
"database": "kb",
292+
"oldData": "Ignore",
293+
"createSchema": false
294+
}
295+
```
296+
297+
Create schema must be set to false, since the knowledge base tables Document and Tag are static.
298+
Handle existing data (oldData) is set the ignore. This is the recommended setting, since the system
299+
will detect whether a document was changed and only perform the embedding then.
300+
Setting it to "delete all" will delete all documents and re-ingest them again.
282301

283302
### API
284303

0 commit comments

Comments
 (0)