Skip to content

Commit 518fdf2

Browse files
authored
chore: update README with overview and reference to examples (#414)
1 parent 9f603dd commit 518fdf2

1 file changed

Lines changed: 248 additions & 1 deletion

File tree

README.rst

Lines changed: 248 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ With `virtualenv`_, it's possible to install this library without needing system
4545
install permissions, and without clashing with the installed system
4646
dependencies.
4747

48-
.. _`virtualenv`: https://virtualenv.pypa.io/en/latest/
48+
.. _virtualenv: https://virtualenv.pypa.io/en/latest/
4949

5050

5151
Mac/Linux
@@ -69,6 +69,253 @@ Windows
6969
<your-env>\Scripts\activate
7070
<your-env>\Scripts\pip.exe install google-cloud-aiplatform
7171
72+
73+
Overview
74+
~~~~~~~~
75+
This section provides a brief overview of the Vertex SDK for Python. You can also reference the notebooks in `vertex-ai-samples`_ for examples.
76+
77+
.. _vertex-ai-samples: https://github.com/GoogleCloudPlatform/ai-platform-samples/tree/master/ai-platform-unified/notebooks/unofficial/sdk
78+
79+
Importing
80+
^^^^^^^^^
81+
SDK functionality can be used from the root of the package:
82+
83+
.. code-block:: Python
84+
85+
from google.cloud import aiplatform
86+
87+
88+
Initialization
89+
^^^^^^^^^^^^^^
90+
Initialize the SDK to store common configurations that you use with the SDK.
91+
92+
.. code-block:: Python
93+
94+
aiplatform.init(
95+
# your Google Cloud Project ID or number
96+
# environment default used is not set
97+
project='my-project',
98+
99+
# the Vertex AI region you will use
100+
# defaults to us-central1
101+
location='us-central1',
102+
103+
# Googlge Cloud Stoage bucket in same region as location
104+
# used to stage artifacts
105+
staging_bucket='gs://my_staging_bucket',
106+
107+
# custom google.auth.credentials.Credentials
108+
# environment default creds used if not set
109+
credentials=my_credentials,
110+
111+
# customer managed encryption key resource name
112+
# will be applied to all Vertex AI resources if set
113+
encryption_spec_key_name=my_encryption_key_name,
114+
115+
# the name of the experiment to use to track
116+
# logged metrics and parameters
117+
experiment='my-experiment',
118+
119+
# description of the experiment above
120+
experiment_description='my experiment decsription'
121+
)
122+
123+
Datasets
124+
^^^^^^^^
125+
Vertex AI provides managed tabular, text, image, and video datasets. In the SDK, datasets can be used downstream to
126+
train models.
127+
128+
To create a tabular dataset:
129+
130+
.. code-block:: Python
131+
132+
my_dataset = aiplatform.TabularDataset.create(
133+
display_name="my-dataset", gcs_source=['gs://path/to/my/dataset.csv'])
134+
135+
You can also create and import a dataset in separate steps:
136+
137+
.. code-block:: Python
138+
139+
from google.cloud import aiplatform
140+
141+
my_dataset = aiplatform.TextDataset.create(
142+
display_name="my-dataset")
143+
144+
my_dataset.import(
145+
gcs_source=['gs://path/to/my/dataset.csv']
146+
import_schema_uri=aiplatform.schema.dataset.ioformat.text.multi_label_classification
147+
)
148+
149+
To get a previously created Dataset:
150+
151+
.. code-block:: Python
152+
153+
dataset = aiplatform.ImageDataset('projects/my-project/location/us-central1/datasets/{DATASET_ID}')
154+
155+
Vertex AI supports a variety of dataset schemas. References to these schemas are available under the
156+
:code:`aiplatform.schema.dataset` namespace. For more information on the supported dataset schemas please refer to the
157+
`Preparing data docs`_.
158+
159+
.. _Preparing data docs: https://cloud.google.com/ai-platform-unified/docs/datasets/prepare
160+
161+
Training
162+
^^^^^^^^
163+
The Vertex SDK for Python allows you train Custom and AutoML Models.
164+
165+
You can train custom models using a custom Python script, custom Python package, or container.
166+
167+
**Preparing Your Custom Code**
168+
169+
Vertex AI custom training enables you to train on Vertex AI datasets and produce Vertex AI models. To do so your
170+
script must adhere to the following contract:
171+
172+
It must read datasets from the environment variables populated by the training service:
173+
174+
.. code-block:: Python
175+
176+
os.environ['AIP_DATA_FORMAT'] # provides format of data
177+
os.environ['AIP_TRAINING_DATA_URI'] # uri to training split
178+
os.environ['AIP_VALIDATION_DATA_URI'] # uri to validation split
179+
os.environ['AIP_TEST_DATA_URI'] # uri to test split
180+
181+
Please visit `Using a managed dataset in a custom training application`_ for a detailed overview.
182+
183+
.. _Using a managed dataset in a custom training application: https://cloud.google.com/vertex-ai/docs/training/using-managed-datasets
184+
185+
It must write the model artifact to the environment variable populated by the traing service:
186+
187+
.. code-block:: Python
188+
189+
os.environ['AIP_MODEL_DIR']
190+
191+
**Running Training**
192+
193+
.. code-block:: Python
194+
195+
job = aiplatform.CustomTrainingJob(
196+
display_name="my-training-job",
197+
script_path="training_script.py",
198+
container_uri="gcr.io/cloud-aiplatform/training/tf-cpu.2-2:latest",
199+
requirements=["gcsfs==0.7.1"],
200+
model_serving_container_image_uri="gcr.io/cloud-aiplatform/prediction/tf2-cpu.2-2:latest",
201+
)
202+
203+
model = job.run(my_dataset,
204+
replica_count=1,
205+
machine_type="n1-standard-4",
206+
accelerator_type='NVIDIA_TESLA_K80',
207+
accelerator_count=1)
208+
209+
In the code block above `my_dataset` is managed dataset created in the `Dataset` section above. The `model` variable is a managed Vertex AI model that can be deployed or exported.
210+
211+
212+
AutoMLs
213+
-------
214+
The Vertex SDK for Python supports AutoML tabular, image, text, video, and forecasting.
215+
216+
To train an AutoML tabular model:
217+
218+
.. code-block:: Python
219+
220+
dataset = aiplatform.TabularDataset('projects/my-project/location/us-central1/datasets/{DATASET_ID}')
221+
222+
job = aiplatform.AutoMLTabularTrainingJob(
223+
display_name="train-automl",
224+
optimization_prediction_type="regression",
225+
optimization_objective="minimize-rmse",
226+
)
227+
228+
model = job.run(
229+
dataset=dataset,
230+
target_column="target_column_name",
231+
training_fraction_split=0.6,
232+
validation_fraction_split=0.2,
233+
test_fraction_split=0.2,
234+
budget_milli_node_hours=1000,
235+
model_display_name="my-automl-model",
236+
disable_early_stopping=False,
237+
)
238+
239+
240+
Models
241+
------
242+
243+
To deploy a model:
244+
245+
246+
.. code-block:: Python
247+
248+
endpoint = model.deploy(machine_type="n1-standard-4",
249+
min_replica_count=1,
250+
max_replica_count=5
251+
machine_type='n1-standard-4',
252+
accelerator_type='NVIDIA_TESLA_K80',
253+
accelerator_count=1)
254+
255+
256+
To upload a model:
257+
258+
.. code-block:: Python
259+
260+
model = aiplatform.Model.upload(
261+
display_name='my-model',
262+
artifact_uri="gs://python/to/my/model/dir",
263+
serving_container_image_uri="gcr.io/cloud-aiplatform/prediction/tf2-cpu.2-2:latest",
264+
)
265+
266+
To get a model:
267+
268+
.. code-block:: Python
269+
270+
model = aiplatform.Model('/projects/my-project/locations/us-central1/models/{MODEL_ID}')
271+
272+
Please visit `Importing models to Vertex AI`_ for a detailed overview:
273+
274+
.. _Importing models to Vertex AI: https://cloud.google.com/vertex-ai/docs/general/import-model
275+
276+
277+
Endpoints
278+
---------
279+
280+
To get predictions from endpoints:
281+
282+
.. code-block:: Python
283+
284+
endpoint.predict(instances=[[6.7, 3.1, 4.7, 1.5], [4.6, 3.1, 1.5, 0.2]])
285+
286+
287+
To create an endpoint
288+
289+
.. code-block:: Python
290+
291+
endpoint = endpoint.create(display_name='my-endpoint')
292+
293+
To deploy a model to a created endpoint:
294+
295+
.. code-block:: Python
296+
297+
model = aiplatform.Model('/projects/my-project/locations/us-central1/models/{MODEL_ID}')
298+
299+
endpoint.deploy(model,
300+
min_replica_count=1,
301+
max_replica_count=5
302+
machine_type='n1-standard-4',
303+
accelerator_type='NVIDIA_TESLA_K80',
304+
accelerator_count=1)
305+
306+
To undeploy models from an endpoint:
307+
308+
.. code-block:: Python
309+
310+
endpoint.undeploy_all()
311+
312+
To delete an endpoint:
313+
314+
.. code-block:: Python
315+
316+
endpoint.delete()
317+
318+
72319
Next Steps
73320
~~~~~~~~~~
74321

0 commit comments

Comments
 (0)