notebook_examples

ADS Expertise Notebooks

The Accelerated Data Science (ADS) SDK is maintained by the Oracle Cloud Infrastructure Data Science service team. It speeds up common data science activities by providing tools that automate and/or simplify common data science tasks, along with providing a data scientist friendly pythonic interface to Oracle Cloud Infrastructure (OCI) services, most notably OCI Data Science, Data Flow, Object Storage, and the Autonomous Database. ADS gives you an interface to manage the lifecycle of machine learning models, from data acquisition to model evaluation, interpretation, and model deployment.

The ADS SDK can be downloaded from PyPi, contributions welcome on GitHub

Topics

Contents

• AI Quick Action - Batch inferencing
• Audi Autonomous Driving Dataset Repository
• Bank Graph Example Notebook
• Building a Forecaster using AutoMLx
• Building and Explaining a Classifier using AutoMLx
• Building and Explaining a Regressor using AutoMLx
• Building and Explaining a Text Classifier using AutoMLx
• Building and Explaining an Anomaly Detector using AutoMLx - Experimental
• Connect to Oracle Big Data Service
• Data Flow Studio : Big Data Operations in Feature Store.
• Deploy LLM Models using BYOC
• Deploy LangChain Application as OCI Data Science Model Deployment
• Enhancing Real-time Capabilities: Streaming Use Cases in Feature Store.
• Fairness with AutoMLx
• Feature store handling querying operations
• Graph Analytics and Graph Machine Learning with PyPGX
• How to Read Data with fsspec from Oracle Big Data Service (BDS)
• Intel Extension for Scikit-Learn
• Introduction to Model Version Set
• Introduction to Streaming
• Introduction to the Oracle Cloud Infrastructure Data Flow Studio
• Model Evaluation with ADSEvaluator
• Natural Language Processing
• ONNX Integration with the Accelerated Data Science (ADS) SDK
• PySpark
• Retrieval Augmented Generative Question Answer Using OCI OpenSearch as Retriever
• Schema Enforcement and Schema Evolution in Feature Store
• Spark NLP within Oracle Cloud Infrastructure Data Flow Studio
• Text Classification and Model Explanations using LIME
• Text Classification with Data Labeling Service Integration
• Text Extraction Using the Accelerated Data Science (ADS) SDK
• Train, Register, and Deploy a Generic Model
• Train, Register, and Deploy a LightGBM Model
• Train, Register, and Deploy a PyTorch Model
• Train, Register, and Deploy a TensorFlow Model
• Train, Register, and Deploy an XGBoost Model
• Train, register, and deploy HuggingFace Pipeline
• Train, register, and deploy Sklearn Model
• Use feature store to perform PII data redaction, summarization, translation using openai
• Using Data Catalog Metastore with DataFlow
• Using Data Catalog Metastore with PySpark
• Using Livy on the Big Data Service
• Using feature store for feature ingestion and feature querying
• Using feature store for feature querying using pandas like interface for query and join
• Using feature store for feature querying using pandas like interface for query and join
• Using feature store for storage, retrieval, versioning and time travel of embeddings
• Using feature store for synthetic data generation using openai
• Working with Pipelines
• XGBoost with RAPIDS
• XGBoost for MySQL Heatwave
• XGBoost for OCI NoSQL

Notebooks

- Building and Explaining an Anomaly Detector using AutoMLx - Experimental

_{Updated: 05/29/2023}

automlx-anomaly_detection.ipynb

Build an anomaly detection model using the experimental, fully unsupervised anomaly detection pipeline in Oracle AutoMLx for the public Credit Card Fraud dataset.

This notebook was developed on the conda pack with slug: generalml_p312_cpu_x86_64_v1

automlx anomaly detection

_{Universal Permissive License v 1.0}

---

- Building and Explaining a Classifier using AutoMLx

_{Updated: 05/29/2023}

automlx-classifier.ipynb

Build a classifier using the Oracle AutoMLx tool and binary data set of Census income data.

This notebook was developed on the conda pack with slug: generalml_p312_cpu_x86_64_v1

automlx classification classifier

_{Universal Permissive License v 1.0}

---

- Fairness with AutoMLx

_{Updated: 05/29/2023}

automlx-fairness.ipynb

Develop a model and evaluate its fairness

This notebook was developed on the conda pack with slug: generalml_p312_cpu_x86_64_v1

automlx fairness

_{Universal Permissive License v 1.0}

---

- Building and Explaining a Regressor using AutoMLx

_{Updated: 05/29/2023}

automlx-regression.ipynb

Build a regressor using Oracle AutoMLx and a pricing data set. Training options will be explored and the resulting AutoMLx models will be evaluated.

This notebook was developed on the conda pack with slug: generalml_p312_cpu_x86_64_v1

automlx regression

_{Universal Permissive License v 1.0}

---

- Building and Explaining a Text Classifier using AutoMLx

_{Updated: 05/29/2023}

automlx-text_classification.ipynb

build a classifier using the Oracle AutoMLx tool for the public 20newsgroup dataset

This notebook was developed on the conda pack with slug: generalml_p312_cpu_x86_64_v1

automlx text classification text classifier

_{Universal Permissive License v 1.0.}

---

- Audi Autonomous Driving Dataset Repository

_{Updated: 03/30/2023}

audi-autonomous_driving-oracle_open_data.ipynb

Download, process and display autonomous driving data, and map LiDAR data onto images.

This notebook was developed on the conda pack with slug: pytorch28_p312_gpu_x86_64_v1

autonomous driving oracle open data

_{Universal Permissive License v 1.0}

---

- Using Livy on the Big Data Service

_{Updated: 03/26/2023}

big_data_service-(BDS)-livy.ipynb

Work interactively with a BDS cluster using Livy and two different connection techniques, SparkMagic (for a notebook environment) and with REST.

This notebook was developed on the conda pack with slug: pyspark35_p312_cpu_x86_64_v1

bds big data service livy

_{Universal Permissive License v 1.0}

---

- How to Read Data with fsspec from Oracle Big Data Service (BDS)

_{Updated: 03/29/2023}

read-write-big_data_service-(BDS).ipynb

Manage data using fsspec file system. Read and save data using pandas and pyarrow through fsspec file system.

This notebook was developed on the conda pack with slug: pyspark35_p312_cpu_x86_64_v1

bds fsspec

_{Universal Permissive License v 1.0}

---

- Deploy LLM Models using BYOC

_{Updated: 09/19/2024}

aqua-deploy-llm-byoc.ipynb

Deploy and perform inferencing using AI Quick Action models.

This notebook was developed on the conda pack with slug: generalml_p312_cpu_x86_64_v1

byoc llm quick action deploy

_{Universal Permissive License v 1.0}

---

- Retrieval Augmented Generative Question Answer Using OCI OpenSearch as Retriever

_{Updated: 12/13/2023}

use_of_cohere_embed_models_for_semantic_search_in_oci_opensearch.ipynb

Set up a retrieval-augmented generative QA using OCI OpenSearch as a retriever.

This notebook was developed on the conda pack with slug: generalml_p312_cpu_x86_64_v1

cohere OpenSearch RAG Retrieval Augmented Generative

_{Universal Permissive License v 1.0}

---

- Using Data Catalog Metastore with DataFlow

_{Updated: 03/26/2023}

pyspark-data_catalog-hive_metastore-data_flow.ipynb

Write and test a Data Flow batch application using the Oracle Cloud Infrastructure (OCI) Data Catalog Metastore. Configure the job, run the application and clean up resources.

This notebook was developed on the conda pack with slug: pyspark35_p312_cpu_x86_64_v1

data catalog metastore data flow

_{Universal Permissive License v 1.0}

---

- Text Classification with Data Labeling Service Integration

_{Updated: 03/30/2023}

data_labeling-text_classification.ipynb

Use the Oracle Cloud Infrastructure (OCI) Data Labeling service to efficiently build enriched, labeled datasets for the purpose of accurately training AI/ML models. This notebook demonstrates operations that can be performed using the Advanced Data Science (ADS) Data Labeling module.

This notebook was developed on the conda pack with slug: pytorch28_p312_gpu_x86_64_v1

data labeling text classification

_{Universal Permissive License v 1.0}

---

- Using Data Catalog Metastore with PySpark

_{Updated: 03/30/2023}

pyspark-data_catalog-hive_metastore.ipynb

Configure and use PySpark to process data in the Oracle Cloud Infrastructure (OCI) Data Catalog metastore, including common operations like creating and loading data from the metastore.

This notebook was developed on the conda pack with slug: pyspark35_p312_cpu_x86_64_v1

dcat data catalog metastore pyspark

_{Universal Permissive License v 1.0}

---

- Using feature store for feature ingestion and feature querying

_{Updated: 12/28/2023}

feature_store_quickstart.ipynb

Introduction to the Oracle Cloud Infrastructure Feature Store.Use feature store for feature ingestion and feature querying

This notebook was developed on the conda pack with slug: pyspark35_p312_cpu_x86_64_v1

feature store

_{Universal Permissive License v 1.0}

---

- Using feature store for storage, retrieval, versioning and time travel of embeddings

_{Updated: 12/28/2023}

feature_store_embeddings.ipynb

Feature store to store embeddings, version embeddings and time travel of embeddings.

This notebook was developed on the conda pack with slug: pyspark35_p312_cpu_x86_64_v1

feature store llm embeddings

_{Universal Permissive License v 1.0}

---

- Use feature store to perform PII data redaction, summarization, translation using openai

_{Updated: 12/28/2023}

feature_store_pii_redaction_and_transformation.ipynb

Use feature store to perform PII data redaction, summarization, translation using openai.

This notebook was developed on the conda pack with slug: pyspark35_p312_cpu_x86_64_v1

feature store querying

_{Universal Permissive License v 1.0}

---

- Schema Enforcement and Schema Evolution in Feature Store

_{Updated: 12/28/2023}

feature_store_schema_evolution.ipynb

Perform Schema Enforcement and Schema Evolution in Feature Store when materialising the data.

This notebook was developed on the conda pack with slug: pyspark35_p312_cpu_x86_64_v1

feature store querying schema enforcement schema evolution

_{Universal Permissive License v 1.0}

---

- Data Flow Studio : Big Data Operations in Feature Store.

_{Updated: 12/28/2023}

feature_store_spark_magic.ipynb

Run Feature Store on interactive Spark workloads on a long lasting Data Flow Cluster.

This notebook was developed on the conda pack with slug: pyspark35_p312_cpu_x86_64_v1

feature store querying spark magic data flow

_{Universal Permissive License v 1.0}

---

- Feature store handling querying operations

_{Updated: 12/28/2023}

feature_store_querying.ipynb

Using feature store to transform, store and query your data using pandas like interface to query and join

This notebook was developed on the conda pack with slug: pyspark35_p312_cpu_x86_64_v1

feature store querying

_{Universal Permissive License v 1.0}

---

- Using feature store for synthetic data generation using openai

_{Updated: 12/28/2023}

feature_store_medical_synthetic_data_openai.ipynb

Feature store quickstart guide to perform synthetic data generation using openai

This notebook was developed on the conda pack with slug: pyspark35_p312_cpu_x86_64_v1

feature store querying synthetic data generation

_{Universal Permissive License v 1.0}

---

- Using feature store for feature querying using pandas like interface for query and join

_{Updated: 12/28/2023}

feature_store_embeddings_openai.ipynb

Feature store quickstart guide to perform feature querying using pandas like interface for query and join.

This notebook was developed on the conda pack with slug: pyspark35_p312_cpu_x86_64_v1

feature store querying

_{Universal Permissive License v 1.0}

---

- Using feature store for feature querying using pandas like interface for query and join

_{Updated: 12/28/2023}

feature_store_ehr_data.ipynb

Feature store quickstart guide to perform feature querying using pandas like interface for query and join.

This notebook was developed on the conda pack with slug: pyspark35_p312_cpu_x86_64_v1

feature store querying

_{Universal Permissive License v 1.0}

---

- Enhancing Real-time Capabilities: Streaming Use Cases in Feature Store.

_{Updated: 01/02/2024}

feature_store_streaming_data_frame.ipynb

Carrying out schema enforcement and schema evolution on Feature Store.

This notebook was developed on the conda pack with slug: pyspark35_p312_cpu_x86_64_v1

feature store querying streaming

_{Universal Permissive License v 1.0}

---

- Train, Register, and Deploy a Generic Model

_{Updated: 03/26/2023}

train-register-deploy-other-frameworks.ipynb

Train, register, and deploy a generic model

This notebook was developed on the conda pack with slug: generalml_p312_cpu_x86_64_v1

generic model deploy model register model train model

_{Universal Permissive License v 1.0}

---

- Bank Graph Example Notebook

_{Updated: 06/05/2023}

graph_insight-autonomous_database.ipynb

Access

This notebook was developed on the conda pack with slug: pytorch28_p312_gpu_x86_64_v1

graph_insight autonomous_database

_{Universal Permissive License v 1.0}

---

- Train, register, and deploy HuggingFace Pipeline

_{Updated: 03/26/2023}

train-register-deploy-huggingface-pipeline.ipynb

Train, register, and deploy a huggingface pipeline.

This notebook was developed on the conda pack with slug: pytorch28_p312_gpu_x86_64_v1

huggingface deploy model register model train model

_{Universal Permissive License v 1.0}

---

- Intel Extension for Scikit-Learn

_{Updated: 03/26/2023}

accelerate-scikit_learn-with-intel_extension.ipynb

Enhance performance of scikit-learn models using the Intel(R) oneAPI Data Analytics Library. Train a k-means model using both sklearn and the accelerated Intel library and compare performance.

This notebook was developed on the conda pack with slug: tensorflow220_p312_gpu_x86_64_v1

intel intel extension scikit-learn scikit learn

_{Universal Permissive License v 1.0}

---

- Connect to Oracle Big Data Service

_{Updated: 03/27/2023}

big_data_service-(BDS)-kerberos.ipynb

Connect to Oracle Big Data services using Kerberos.

This notebook was developed on the conda pack with slug: pyspark35_p312_cpu_x86_64_v1

kerberos big data service bds

_{Universal Permissive License v 1.0}

---

- Deploy LangChain Application as OCI Data Science Model Deployment

_{Updated: 12/06/2023}

deploy-langchain-as-oci-data-science-model-deployment.ipynb

Deploy LangChain applications as OCI data science model deployment

This notebook was developed on the conda pack with slug: pytorch28_p312_gpu_x86_64_v1

langchain deploy model register model LLM

_{Universal Permissive License v 1.0}

---

- Building a Forecaster using AutoMLx

_{Updated: 05/29/2023}

automlx-forecasting.ipynb

Use Oracle AutoMLx to build a forecast model with real-world data sets.

This notebook was developed on the conda pack with slug: generalml_p312_cpu_x86_64_v1

language services string manipulation regex regular expression natural language processing NLP part-of-speech tagging named entity recognition sentiment analysis custom plugins

_{Universal Permissive License v 1.0}

---

- Natural Language Processing

_{Updated: 03/26/2023}

natural_language_processing.ipynb

Use the ADS SDK to process and manipulate strings. This notebook includes regular expression matching and natural language (NLP) parsing, including part-of-speech tagging, named entity recognition, and sentiment analysis. It also shows how to create and use custom plugins specific to your specific needs.

This notebook was developed on the conda pack with slug: pytorch28_p312_gpu_x86_64_v1

language services string manipulation regex regular expression natural language processing NLP part-of-speech tagging named entity recognition sentiment analysis custom plugins

_{Universal Permissive License v 1.0}

---

- Train, Register, and Deploy a LightGBM Model

_{Updated: 03/26/2023}

train-register-deploy-lightgbm.ipynb

Train, register, and deploy a LightGBM model.

This notebook was developed on the conda pack with slug: generalml_p312_cpu_x86_64_v1

lightgbm deploy model register model train model

_{Universal Permissive License v 1.0}

---

- Introduction to Model Version Set

_{Updated: 03/26/2023}

model_version_set.ipynb

A model version set is a way to track the relationships between models. As a container, the model version set takes a collection of models. Those models are assigned a sequential version number based on the order they are entered into the model version set.

This notebook was developed on the conda pack with slug: generalml_p312_cpu_x86_64_v1

model model experiments model version set

_{Universal Permissive License v 1.0}

---

- Model Evaluation with ADSEvaluator

_{Updated: 03/30/2023}

model_evaluation-with-ADSEvaluator.ipynb

Train and evaluate different types of models: binary classification using an imbalanced dataset, multi-class classification using a synthetically generated dataset consisting of three equally distributed classes, and a regression using a synthetically generated dataset with positive targets.

This notebook was developed on the conda pack with slug: generalml_p38_cpu_v1

model evaluation binary classification regression multi-class classification imbalanced dataset synthetic dataset

_{Universal Permissive License v 1.0}

---

- Text Classification and Model Explanations using LIME

_{Updated: 03/30/2023}

text_classification-model_explanation-lime.ipynb

Perform model explanations on an NLP classifier using the locally interpretable model explanations technique (LIME).

This notebook was developed on the conda pack with slug: pytorch28_p312_gpu_x86_64_v1

nlp lime model_explanation text_classification text_explanation

_{Universal Permissive License v 1.0}

---

- ONNX Integration with the Accelerated Data Science (ADS) SDK

_{Updated: 08/01/2023}

onnx-integration-ads.ipynb

Extract text from common formats (e.g. PDF and Word) into plain text. Customize this process for individual use cases.

This notebook was developed on the conda pack with slug: pytorch28_p312_gpu_x86_64_v1

onnx deploy model

_{Universal Permissive License v 1.0}

---

- Working with Pipelines

_{Updated: 03/26/2023}

pipelines-ml_lifecycle.ipynb

Create and use ML pipelines through the entire machine learning lifecycle

This notebook was developed on the conda pack with slug: generalml_p312_cpu_x86_64_v1

pipelines pipeline step jobs pipeline

_{Universal Permissive License v 1.0}

---

- Graph Analytics and Graph Machine Learning with PyPGX

_{Updated: 03/26/2023}

pypgx-graph_analytics-machine_learning.ipynb

Use Oracle's Graph Analytics libraries to demonstrate graph algorithms, graph machine learning models, and use the property graph query language (PGQL)

This notebook was developed on the conda pack with slug: pytorch28_p312_gpu_x86_64_v1

pypgx graph analytics pgx

_{Universal Permissive License v 1.0}

---

- PySpark

_{Updated: 06/02/2023}

pyspark-data_flow-application.ipynb

Develop local PySpark applications and work with remote clusters using Data Flow.

This notebook was developed on the conda pack with slug: pyspark35_p312_cpu_x86_64_v1

pyspark data flow

_{Universal Permissive License v 1.0}

---

- Introduction to the Oracle Cloud Infrastructure Data Flow Studio

_{Updated: 03/26/2023}

pyspark-data_flow_studio-introduction.ipynb

Run interactive Spark workloads on a long lasting Oracle Cloud Infrastructure Data Flow Spark cluster through Apache Livy integration. Data Flow Spark Magic is used for interactively working with remote Spark clusters through Livy, a Spark REST server, in Jupyter notebooks. It includes a set of magic commands for interactively running Spark code.

This notebook was developed on the conda pack with slug: pyspark35_p312_cpu_x86_64_v1

pyspark data flow

_{Universal Permissive License v 1.0}

---

- Spark NLP within Oracle Cloud Infrastructure Data Flow Studio

_{Updated: 03/26/2023}

pyspark-data_flow_studio-spark_nlp.ipynb

Demonstrates how to use Spark NLP within a long lasting Oracle Cloud Infrastructure Data Flow cluster.

This notebook was developed on the conda pack with slug: pyspark35_p312_cpu_x86_64_v1

pyspark data flow

_{Universal Permissive License v 1.0}

---

- Train, Register, and Deploy a PyTorch Model

_{Updated: 03/26/2023}

train-register-deploy-pytorch.ipynb

Train, register, and deploy a PyTorch model.

This notebook was developed on the conda pack with slug: pytorch28_p312_gpu_x86_64_v1

pytorch deploy model register model train model

_{Universal Permissive License v 1.0}

---

- AI Quick Action - Batch inferencing

_{Updated: 09/19/2024}

aqua-batch-inferencing.ipynb

Perform batch inferencing on LLMs using AI Quick Actions.

This notebook was developed on the conda pack with slug: generalml_p312_cpu_x86_64_v1

quick action batch inferencing llm

_{Universal Permissive License v 1.0}

---

- Train, register, and deploy Sklearn Model

_{Updated: 03/26/2023}

train-register-deploy-sklearn.ipynb

Train, register, and deploy an scikit-learn model.

This notebook was developed on the conda pack with slug: generalml_p312_cpu_x86_64_v1

scikit-learn deploy model register model train model

_{Universal Permissive License v 1.0}

---

- Introduction to Streaming

_{Updated: 03/30/2023}

streaming-service-introduction.ipynb

Connect to Oracle Cloud Insfrastructure (OCI) Streaming service with kafka.

This notebook was developed on the conda pack with slug: generalml_p312_cpu_x86_64_v1

streaming kafka

_{Universal Permissive License v 1.0}

---

- Train, Register, and Deploy a TensorFlow Model

_{Updated: 03/26/2023}

train-register-deploy-tensorflow.ipynb

Train, register, and deploy a TensorFlow model.

This notebook was developed on the conda pack with slug: tensorflow220_p312_gpu_x86_64_v1

tensorflow deploy model register model train model

_{Universal Permissive License v 1.0}

---

- Text Extraction Using the Accelerated Data Science (ADS) SDK

_{Updated: 03/26/2023}

document-text_extraction.ipynb

Extract text from common formats (e.g. PDF and Word) into plain text. Customize this process for individual use cases.

This notebook was developed on the conda pack with slug: pytorch28_p312_gpu_x86_64_v1

text extraction nlp

_{Universal Permissive License v 1.0}

---

- XGBoost with RAPIDS

_{Updated: 03/30/2023}

xgboost-with-rapids.ipynb

Compare training time between CPU and GPU trained models using XGBoost

This notebook was developed on the conda pack with slug: generalml_p311_cpu_x86_64_v3

xgboost rapids gpu machine learning classification

_{Universal Permissive License v 1.0}

---

- Train, Register, and Deploy an XGBoost Model

_{Updated: 03/26/2023}

train-register-deploy-xgboost.ipynb

Train, register, and deploy an XGBoost model.

This notebook was developed on the conda pack with slug: generalml_p312_cpu_x86_64_v1

xgboost deploy model register model train model

_{Universal Permissive License v 1.0}

---

- Train and Deploy an XGBoost Model for OCI MySQL Heatwave

_{Updated: 02/19/2025}

train-mysql-heatwave-deploy-xgboost.ipynb

Train and Deploy an XGBoost Model for OCI MySQL Heatwave

This notebook was developed on the conda pack with slug: generalml_p312_cpu_x86_64_v1

xgboost deploy model heatwave train model mysql

_{Universal Permissive License v 1.0}

---

---

- Train and Deploy an XGBoost Model for OCI NoSQL

_{Updated: 02/19/2025}

train-nosql-deploy-xgboost.ipynb

Train and Deploy an XGBoost Model for OCI NoSQL

This notebook was developed on the conda pack with slug: generalml_p312_cpu_x86_64_v1

xgboost deploy model heatwave train model mysql

_{Universal Permissive License v 1.0}

---