GCP Vertex AI

Vertex AI is a platform that enables the training and deployment of machine learning models and AI applications. It aims to streamline and expedite the development and deployment process for ML models, offering a variety of features and integrations tailored for enterprise-level workflows.

The integration with Google Cloud Platform (GCP) Vertex AI allows you to gather metrics such as token usage, latency, overall invocations, and error rates for deployed models. Additionally, it tracks resource utilization metrics for the model replicas as well as prediction metrics of endpoints.

The Vertex AI integration collects metrics and logs data.

The GCP Vertex AI includes Vertex AI Model Garden Publisher Model metrics under the publisher category, and the Vertex AI Endpoint metrics under the prediction category and audit logs under the logs.

You need Elasticsearch to store and search your data and Kibana to visualize and manage it. You can use our hosted Elasticsearch Service on Elastic Cloud, which is recommended or self-manage the Elastic Stack on your hardware.

Before using any GCP integration you will need:

First, you need to create a Service Account. Service Accounts (SAs) are principals in Google Cloud, enabling you to grant them access to resources through IAM policies.

The Elastic Agent uses the SA to access data on Google Cloud Platform using the Google APIs.

With your Service Account (SA) with access to Google Cloud Platform (GCP) resources, you need the credentials to associate with it: a Service Account Key.

From the list of SA:

1. Click the Service Account you just created to open the detailed view.
2. From the Keys section, click Add key > Create new key and select JSON as the type.
3. Download and store the generated private key securely. Note that the private key can't be recovered from GCP if lost.

There isn't a single, specific role required to view metrics for Vertex AI. Access depends on how the models are deployed and the permissions granted to your Google Cloud project and user account.

However, to summarize the necessary permissions and implied roles, you'll generally need a role that includes the following permissions:

• monitoring.metricDescriptor.list: Allows you to list available metric descriptors.
• monitoring.timeSeries.list: Allows you to list time series data for the metrics.

These permissions are included in many roles, but these are some of the most common ones:

• roles/monitoring.viewer: This role provides read-only access to Cloud Monitoring metrics.
• roles/aiplatform.user: This role grants broader access to Vertex AI, including model viewing and potentially metric access.
• More granular roles: For fine-grained control (recommended for security best practices), consider using a custom role built with the specific permissions needed. This would only include the necessary permissions to view model metrics, rather than broader access to all Vertex AI or Cloud Monitoring resources. This requires expertise in IAM (Identity and Access Management).
• Predefined roles with broader access: These roles provide extensive permissions within the Google Cloud project, giving access to metrics but granting much broader abilities than necessary for just viewing metrics. These are generally too permissive unless necessary for other tasks. Examples are roles/aiplatform.user or roles/editor.

For step-by-step instructions on how to set up an integration, refer to the Getting Started guide.

The next step is to configure the general integration settings used for logs and metrics from the supported services.

When you add the Google Cloud Platform VertexAI integration, you need to provide the Project ID and either the Credentials File or Credentials JSON.

The Project Id is the Google Cloud project ID where your resources exist.

Based on your preference, specify the information in either the Credentials File or the Credentials JSON field.

Save the JSON file with the private key in a secure location of the file system, and make sure that the Elastic Agent has at least read-only privileges to this file.

Specify the file path in the Elastic Agent integration UI in the "Credentials File" field. For example: /home/ubuntu/credentials.json.

Specify the content of the JSON file you downloaded from Google Cloud Platform directly in the Credentials JSON field in the Elastic Agent integration.

With a properly configured Service Account and the integration setting in place, it's time to start collecting the monitoring metrics.

No additional requirements to collect metrics.

Vertex AI offers two primary deployment types:

• Provisioned Throughput: Suitable for high-usage applications with predictable workloads and a premium on guaranteed performance.
• Pay-as-you-go: Ideal for low-usage applications, batch processing, and applications with unpredictable traffic patterns.

Now, you can track and monitor different deployment types (provisioned throughput and pay-as-you-go) in Vertex AI using the Model Garden Publisher resource.

With a properly configured Service Account and the integration setting in place, you can start collecting the logs.

Before you start, you need to create the following Google Cloud resources:

• Log Sink
• Pub/Sub Topic
• Subscription

Here's an example of collecting Vertex AI audit logs using a Pub/Sub topic, a subscription, and a Log Router. We will create the resources in the Google Cloud Console and then configure the Google Cloud Platform integration.

On the Google Cloud Console follow these steps:

At a high level, the steps required are:

1. Visit "Logging" > "Log Router" > "Create Sink" and provide a sink name and description.
2. In "Sink destination", select "Cloud Pub/Sub topic" as the sink service. Select an existing topic or "Create a topic". Note the topic name, as it will be provided in the Topic field in the Elastic agent configuration.
3. If you created a new topic, you must remember to go to that topic and create a subscription for it. A subscription directs messages on a topic to subscribers. Note the "Subscription ID", as it will need to be entered in the "Subscription name" field in the integration settings.
4. Under "Choose logs to include in sink", for example add resource.labels.service=aiplatform.googleapis.com and resource.type="audited_resource" in the "Inclusion filter" to include all audit logs.

This is just an example to create your filter expression to select the Vertex AI audit logs you want to export to the Pub/Sub topic.

Refer to Google Cloud Platform troubleshooting for more information about troubleshooting.

{
    "cloud": {
        "provider": "gcp",
        "account": {
            "name": "elastic-sa",
            "id": "elastic-sa"
        }
    },
    "agent": {
        "name": "docker-fleet-agent",
        "id": "f9c4beb9-c0c0-47ca-963a-a9dc00e2df5e",
        "ephemeral_id": "6c42a949-d522-44bf-818b-12c4a5908b90",
        "type": "metricbeat",
        "version": "8.15.2"
    },
    "@timestamp": "2024-11-07T05:50:40.000Z",
    "ecs": {
        "version": "8.0.0"
    },
    "gcp": {
        "vertexai": {
            "publisher": {
                "online_serving": {
                    "token_count": 13
                }
            }
        },
        "labels": {
            "resource": {
                "model_user_id": "gemini-1.5-flash-002",
                "model_version_id": "",
                "publisher": "google",
                "location": "us-central1"
            },
            "metrics": {
                "request_type": "shared",
                "type": "input"
            }
        }
    },
    "service": {
        "type": "gcp"
    },
    "data_stream": {
        "namespace": "default",
        "type": "metrics",
        "dataset": "gcp_vertexai.metrics"
    },
    "elastic_agent": {
        "id": "f9c4beb9-c0c0-47ca-963a-a9dc00e2df5e",
        "version": "8.15.2",
        "snapshot": false
    },
    "host": {
        "hostname": "docker-fleet-agent",
        "ip": [
            "172.25.0.7"
        ]
    },
    "metricset": {
        "period": 60000,
        "name": "metrics"
    },
    "event": {
        "duration": 913154084,
        "agent_id_status": "verified",
        "ingested": "2024-11-07T05:57:17Z",
        "module": "gcp",
        "dataset": "gcp_vertexai.metrics"
    }
}

ECS Field Reference

Check the ECS Field Reference for detailed information on ECS fields.

{
    "cloud": {
        "project": {
            "id": "elastic-abs"
        },
        "provider": "gcp"
    },
    "ecs": {
        "version": "8.11.0"
    },
    "event": {
        "action": "google.cloud.aiplatform.internal.PredictionService.CountTokens",
        "id": "1tif4epd6kb0",
        "kind": "event"
    },
    "gcp": {
        "vertexai": {
            "audit": {
                "authentication_info": {
                    "principal_email": "pc.cf@elastic.co"
                },
                "authorization_info": [
                    {
                        "granted": true,
                        "permission": "aiplatform.endpoints.predict",
                        "permission_type": "DATA_READ",
                        "resource": "projects/elastic-abs/locations/us-central1/publishers/google/models/gemini-2.0-flash-exp"
                    }
                ],
                "request": {
                    "@type": "type.googleapis.com/google.cloud.aiplatform.internal.CountTokensRequest",
                    "endpoint": "projects/elastic-abs/locations/us-central1/publishers/google/models/gemini-2.0-flash-exp"
                },
                "resource_name": "projects/elastic-abs/locations/us-central1/publishers/google/models/gemini-2.0-flash-exp",
                "resource_type": "audited_resource",
                "response": {
                    "@type": "type.googleapis.com/google.cloud.aiplatform.internal.CountTokensResponse"
                },
                "service_name": "aiplatform.googleapis.com",
                "type": "type.googleapis.com/google.cloud.audit.AuditLog"
            }
        }
    },
    "log": {
        "level": "INFO",
        "logger": "projects/elastic-abs/logs/cloudaudit.googleapis.com%2Fdata_access"
    },
    "source": {
        "ip": "175.16.199.0"
    },
    "user_agent": {
        "device": {
            "name": "Mac"
        },
        "name": "Chrome",
        "original": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36,gzip(gfe),gzip(gfe)",
        "os": {
            "full": "Mac OS X 10.15.7",
            "name": "Mac OS X",
            "version": "10.15.7"
        },
        "version": "135.0.0.0"
    }
}

ECS Field Reference

Check the ECS Field Reference for detailed information on ECS fields.