0% found this document useful (0 votes)
165 views24 pages

Generate Content With The Gemini Enterprise API - Generative AI On Vertex AI - Google Cloud

Uploaded by

gpt4.jomera
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
165 views24 pages

Generate Content With The Gemini Enterprise API - Generative AI On Vertex AI - Google Cloud

Uploaded by

gpt4.jomera
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 24

Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

Generate content with the Gemini Enterprise


API
Release Notes

Use or to generate content with Gemini.

The Gemini model family includes models that work with multimodal prompt requests. The
term multimodal indicates that you can use more than one modality, or type of input, in a
prompt. Models that aren't multimodal accept prompts only with text. Modalities can
include text, audio, video, and more.

Create a Google Cloud account to get started

To start using the Vertex AI API for Gemini, create a Google Cloud account
(https://console.cloud.google.com/freetrial?redirectPath=/marketplace/product/google/
cloudaicompanion.googleapis.com)
.

After creating your account, use this document to review the Gemini model request body
(#request), model parameters (#parameters), response body (#response), and some sample
requests (#sample-requests).

When you're ready, see the Vertex AI API for Gemini quickstart
(/vertex-ai/generative-ai/docs/start/quickstarts/quickstart-multimodal) to learn how to send a
request to the Vertex AI Gemini API using a programming language SDK or the REST API.

Supported models

Model Version

Gemini 1.5 Flash

Gemini 1.5 Pro

Gemini 1.0 Pro Vision

1 of 24 11/10/24, 01:37
Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

Model Version

Gemini 1.0 Pro

Note: Adding a lot of images to a request increases response latency.

Example syntax

Syntax to generate a model response.

Non-streaming

(#curl)
(#python)

gemini_model = GenerativeModel(MODEL_ID)
generation_config = GenerationConfig(...)

model_response = gemini_model.generate_content([...], generation_config,

Streaming

(#curl)
(#python)

gemini_model = GenerativeModel(MODEL_ID)
model_response = gemini_model.generate_content([...], generation_config,

Parameter list

See examples (#sample-requests) for implementation details.

2 of 24 11/10/24, 01:37
Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

Request body

{
"cachedContent": string,
"contents": [
{
"role": string,
"parts": [
{
∕∕ Union field data can be only one of the following:
"text": string,
"inlineData": {
"mimeType": string,
"data": string
},
"fileData": {
"mimeType": string,
"fileUri": string
},
∕∕ End of list of possible types for union field data.

"videoMetadata": {
"startOffset": {
"seconds": integer,
"nanos": integer
},
"endOffset": {
"seconds": integer,
"nanos": integer
}
}
}
]
}
],
"systemInstruction": {
"role": string,
"parts": [
{
"text": string
}
]
},
"tools": [
{
"functionDeclarations": [

3 of 24 11/10/24, 01:37
Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

{
"name": string,
"description": string,
"parameters": {
object ( (https://spec.openapis.org/oas/v3.0.3#schema) (
}
}
]
}
],
"safetySettings": [
{
"category": enum (HarmCategory),
"threshold": enum (HarmBlockThreshold)
}
],
"generationConfig": {
"temperature": number,
"topP": number,
"topK": number,
"candidateCount": integer,
"maxOutputTokens": integer,
"presencePenalty": float,
"frequencyPenalty": float,
"stopSequences": [
string
],
"responseMimeType": string,
"responseSchema": (/vertex-ai/docs/reference/rest/v1/Schema),
"seed": integer,
"responseLogprobs": boolean,
"logprobs": integer,
"audioTimestamp": boolean
},
"labels": {
string: string
}
}

The request body contains data with the following parameters:

Parameters

Optional:

The name of the cached content used as context to serve the


prediction. Format:

4 of 24 11/10/24, 01:37
Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

Required:

The content of the current conversation with the model.

For single-turn queries, this is a single instance. For multi-turn queries,


this is a repeated �eld that contains conversation history and the
latest request.

Optional:

Available for , , and


.

Instructions for the model to steer it toward better performance. For


example, "Answer as concisely as possible" or "Don't use technical
terms in your response".

The strings count toward the token limit.

The �eld of is ignored and doesn't affect


the performance of the model.

 Note: Only should be used in


should be in a separate paragraph.
and content in each

Optional. A piece of code that enables the system to interact with


external systems to perform an action, or set of actions, outside of
knowledge and scope of the model. See Function calling
(/vertex-ai/generative-ai/docs/model-reference/function-calling).

Optional. See Function calling


(/vertex-ai/generative-ai/docs/model-reference/function-calling).

Optional:

Per request settings for blocking unsafe content.

Enforced on .

Optional:

Generation con�guration settings.

Optional:

Metadata that you can add to the API call in the format of key-value
pairs.

5 of 24 11/10/24, 01:37
Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

The base structured data type containing multi-part content of a message.

This class consists of two main properties: and . The property denotes the
individual producing the content, while the property contains multiple elements, each
representing a segment of data within a message.

Parameters

Optional:

The identity of the entity that creates the message. The following
values are supported:

• : This indicates that the message is sent by a real person,


typically a user-generated message.

• : This indicates that the message is generated by the model.

The value is used to insert messages from the model into the
conversation during multi-turn conversations.

For non-multi-turn conversations, this �eld can be left blank or unset.

A list of ordered parts that make up a single message. Different parts


may have different IANA MIME types
(https://www.iana.org/assignments/media-types/media-types.xml).

For limits on the inputs, such as the maximum number of tokens or


the number of images, see the model speci�cations on the Google
models (/vertex-ai/generative-ai/docs/learn/models) page.

To compute the number of tokens in your request, see Get token count
(/vertex-ai/generative-ai/docs/multimodal/get-token-count).

A data type containing media that is part of a multi-part message.

Parameters

Optional:

A text prompt or code snippet.

6 of 24 11/10/24, 01:37
Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

Optional:

Inline data in raw bytes.

For , you can specify at most 1 image by


using .To specify up to 16 images, use .

Optional:

Data stored in a �le.

Optional: .

It contains a string representing the


�eld and a structured JSON object containing any parameters for the
function call predicted by the model.

See Function calling


(/vertex-ai/generative-ai/docs/model-reference/function-calling).

Optional: .

The result output of a that contains a string


representing the �eld and a
structured JSON object containing any output from the function call. It
is used as context to the model.

See Function calling


(/vertex-ai/generative-ai/docs/model-reference/function-calling).

Optional:

For video input, the start and end offset of the video in Duration
(https://protobuf.dev/reference/protobuf/google.protobuf/#duration)
format. For example, to specify a 10 second clip starting at 1:00, set
and
.

The metadata should only be speci�ed while the video data is


presented in or .

Content blob. If possible send as text rather than raw bytes.

Parameters

7 of 24 11/10/24, 01:37
Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

The media type of the �le speci�ed in the or �elds.


Acceptable values include the following:

 Click to expand MIME types


For , the maximum video length is 2


minutes.

For Gemini 1.5 Pro and Gemini 1.5 Flash, the maximum length of an
audio �le is 8.4 hours and the maximum length of a video �le (without
audio) is one hour. For more information, see Gemini 1.5 Pro media
requirements
(/vertex-ai/generative-ai/docs/multimodal/send-multimodal-
prompts#media_requirements)
.

Text �les must be UTF-8 encoded. The contents of the text �le count
toward the token limit.

There is no limit on image resolution.

8 of 24 11/10/24, 01:37
Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

The base64 encoding


(/vertex-ai/generative-ai/docs/image/base64-encode) of the image,
PDF, or video to include inline in the prompt. When including media
inline, you must also specify the media type ( ) of the data.

Size limit: 20MB

FileData

URI or web-URL data.

Parameters

IANA MIME type


(https://www.iana.org/assignments/media-types/media-types.xml) of
the data.

The URI or URL of the �le to include in the prompt. Acceptable values
include the following:

• Cloud Storage bucket URI: The object must either be publicly


readable or reside in the same Google Cloud project that's sending
the request. For and , the
size limit is 2 GB. For , the size limit is
20 MB.

• HTTP URL: The �le URL must be publicly readable. You can specify
one video �le and up to 10 image �les per request. Audio �les and
documents can't exceed 15 MB.

• YouTube video URL:The YouTube video must be either owned by


the account that you used to sign in to the Google Cloud console or
is public. Only one YouTube video URL is supported per request.

When specifying a , you must also specify the media type


( ) of the �le.

A predicted returned from the model that contains a string representing the

9 of 24 11/10/24, 01:37
Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

and a structured JSON object containing the parameters and


their values.

Parameters

The name of the function to call.

The function parameters and values in JSON object format.

See Function calling


(/vertex-ai/generative-ai/docs/model-reference/function-calling) for
parameter details.

The resulting output from a that contains a string representing the


. Also contains a structured JSON object with the output from
the function (and uses it as context for the model). This should contain the result of a
made based on model prediction.

Parameters

The name of the function to call.

The function response in JSON object format.

Metadata describing the input video content.

Parameters

Optional:

The start offset of the video.

10 of 24 11/10/24, 01:37
Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

Optional:

The end offset of the video.

Safety settings.

Parameters

Optional:

The safety category to con�gure a threshold for. Acceptable values


include the following:

Click to expand safety categories

Optional:

The threshold for blocking responses that could belong to the


speci�ed safety category based on probability.

Optional:

Specify if the threshold is used for probability or severity score. If not


speci�ed, the threshold is used for probability score.

11 of 24 11/10/24, 01:37
Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

Harm categories that block content.

Parameters

The harm category is unspeci�ed.

The harm category is hate speech.

The harm category is dangerous content.

The harm category is harassment.

The harm category is sexually explicit content.

Probability thresholds levels used to block a response.

Parameters

Unspeci�ed harm block threshold.

Block low threshold and higher (i.e. block more).

Block medium threshold and higher.

Block only high threshold (i.e. block less).

Block none.

Switches off safety if all categories are turned OFF

A probability threshold that blocks a response based on a combination of probability and


severity.

Parameters

The harm block method is unspeci�ed.

The harm block method uses both probability and severity


scores.

The harm block method uses the probability score.

12 of 24 11/10/24, 01:37
Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

Con�guration settings used when generating the prompt.

Parameters

Optional:

The temperature is used for sampling during response generation,


which occurs when and are applied. Temperature controls
the degree of randomness in token selection. Lower temperatures are
good for prompts that require a less open-ended or creative response,
while higher temperatures can lead to more diverse or creative results.
A temperature of means that the highest probability tokens are
always selected. In this case, responses for a given prompt are mostly
deterministic, but a small amount of variation is still possible.

If the model returns a response that's too generic, too short, or the
model gives a fallback response, try increasing the temperature.

• Range for : (default: )

• Range for : (default: )

• Range for : (default: )

• Range for : (default: )

• Range for : (default: )

For more information, see Content generation parameters


(/vertex-ai/generative-ai/docs/multimodal/content-generation-
parameters#temperature)
.

Optional:

If speci�ed, nucleus sampling is used.

Top-P
(/vertex-ai/generative-ai/docs/multimodal/content-generation-
parameters#top-p)
changes how the model selects tokens for output. Tokens are
selected from the most (see top-K) to least probable until the sum of
their probabilities equals the top-P value. For example, if tokens A, B,
and C have a probability of 0.3, 0.2, and 0.1 and the top-P value is ,
then the model will select either A or B as the next token by using
temperature and excludes C as a candidate.

Specify a lower value for less random responses and a higher value
for more random responses.

13 of 24 11/10/24, 01:37
Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

• Range:

• Default for :

• Default for :

• Default for :

• Default for :

Optional: Top-K
(/vertex-ai/generative-ai/docs/multimodal/content-generation-
parameters#top-k)
changes how the model selects tokens for output. A top-K of means
the next selected token is the most probable among all tokens in the
model's vocabulary (also called greedy decoding), while a top-K of
means that the next token is selected from among the three most
probable tokens by using temperature.

For each token selection step, the top-K tokens with the highest
probabilities are sampled. Then tokens are further �ltered based on
top-P with the �nal token selected using temperature sampling.

Specify a lower value for less random responses and a higher value
for more random responses.

Range:

Supported by only.

Default for :

Optional:

The number of response variations to return. For each request, you're


charged for the output tokens of all candidates, but are only charged
once for the input tokens.

Specifying multiple candidates is a Preview feature that works with


( is not supported).
The following models are supported:

• Gemini 1.5 Flash: - , default:

• Gemini 1.5 Pro: - , default:

• Gemini 1.0 Pro: - , default:

Optional: int

Maximum number of tokens that can be generated in the response. A


token is approximately four characters. 100 tokens correspond to

14 of 24 11/10/24, 01:37
Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

roughly 60-80 words.

Specify a lower value for shorter responses and a higher value for
potentially longer responses.

For more information, see Content generation parameters


(/vertex-ai/generative-ai/docs/multimodal/content-generation-
parameters#max-output-tokens)
.

Optional:

Speci�es a list of strings that tells the model to stop generating text if
one of the strings is encountered in the response. If a string appears
multiple times in the response, then the response truncates where it's
�rst encountered. The strings are case-sensitive.

For example, if the following is the returned response when


isn't speci�ed:

Then the returned response with set to


is:

Maximum 5 items in the list.

For more information, see Content generation parameters


(/vertex-ai/generative-ai/docs/multimodal/content-generation-
parameters#stop-sequences)
.

Optional:

Positive penalties.

Positive values penalize tokens that already appear in the generated


text, increasing the probability of generating more diverse content.

The maximum value for is up to, but not


including, . Its minimum value is .

Supported by and .

Optional:

Positive values penalize tokens that repeatedly appear in the


generated text, decreasing the probability of repeating content.

15 of 24 11/10/24, 01:37
Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

This maximum value for is up to, but not


including, . Its minimum value is .

Supported by and .

Optional:

Available for the following models:

The output response MIME type of the generated candidate text.

The following MIME types are supported:

• : JSON response in the candidates.

• (default): Plain text output.

• : For classi�cation tasks, output an enum value as


de�ned in the response schema.

Specify the appropriate response type to avoid unintended behaviors.


For example, if you require a JSON-formatted response, specify
and not .

Optional: schema (/vertex-ai/docs/reference/rest/v1/Schema)

The schema that generated candidate text must follow. For more
information, see Control generated output
(/vertex-ai/generative-ai/docs/multimodal/control-generated-output).

You must specify the parameter to use this


parameter.

Available for the following models:

Optional:

When seed is �xed to a speci�c value, the model makes a best effort
to provide the same response for repeated requests. Deterministic
output isn't guaranteed. Also, changing the model or parameter
settings, such as the temperature, can cause variations in the
response even when you use the same seed value. By default, a
random seed value is used.

16 of 24 11/10/24, 01:37
Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

This is a preview feature.

Available for the following models:

Optional:

If true, returns the log probabilities of the tokens that were chosen by
the model at each step. By default, this parameter is set to .

Available for the following models:

This is a preview feature.

Optional:

Returns the log probabilities of the top candidate tokens at each


generation step. The model's chosen token might not be the same as
the top candidate token at each step. Specify the number of
candidates to return by using an integer value in the range of - .

You must enable (#responseLogprobs) to use


this parameter.

This is a preview feature.

Optional:

Available for the following models:

Enables timestamp understanding for audio-only �les.

This is a preview feature.

Response body

{
"candidates": [

17 of 24 11/10/24, 01:37
Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

{
"content": {
"parts": [
{
"text": string
}
]
},
"finishReason": enum (FinishReason),
"safetyRatings": [
{
"category": enum (HarmCategory),
"probability": enum (HarmProbability),
"blocked": boolean
}
],
"citationMetadata": {
"citations": [
{
"startIndex": integer,
"endIndex": integer,
"uri": string,
"title": string,
"license": string,
"publicationDate": {
"year": integer,
"month": integer,
"day": integer
}
}
]
},
"avgLogprobs": double,
"logprobsResult": {
"topCandidates": [
{
"candidates": [
{
"token": string,
"logProbability": float
}
]
}
],
"chosenCandidates": [
{
"token": string,
"logProbability": float

18 of 24 11/10/24, 01:37
Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

}
]
}
}
],
"usageMetadata": {
"promptTokenCount": integer,
"candidatesTokenCount": integer,
"totalTokenCount": integer
},
"modelVersion": string
}

Response element Description

The model and version used for generation. For example:


.

The generated text.

The reason why the model stopped generating tokens. If empty, the model
has not stopped generating the tokens. Because the response uses the
prompt for context, it's not possible to change the behavior of how the model
stops generating tokens.

• : Natural stop point of the model or provided stop


sequence.

• : The maximum number of tokens as


speci�ed in the request was reached.

• : Token generation was stopped because the


response was �agged for safety reasons. Note that
is empty if content �lters block the output.

• : The token generation was stopped


because the response was �agged for unauthorized citations.

• : Token generation was stopped because


the response includes blocked terms.

• : Token generation was


stopped because the response was �agged for prohibited content, such as
child sexual abuse material (CSAM).

• : Token generation was stopped because the


response was �agged for sensitive personally identi�able information
(SPII).

• : Candidates were

19 of 24 11/10/24, 01:37
Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

blocked because of malformed and unparsable function call.

• : All other reasons that stopped the token

• : The �nish reason is unspeci�ed.

The safety category to con�gure a threshold for. Acceptable values include


the following:

 Click to expand safety categories


The harm probability levels in the content.

A boolean �ag associated with a safety attribute that indicates if the model's
input or output was blocked.

An integer that speci�es where a citation starts in the .

An integer that speci�es where a citation ends in the .

The URL of a citation source. Examples of a URL source might be a news


website or a GitHub repository.

The title of a citation source. Examples of source titles might be that of a


news article or a book.

The license associated with a citation.

The date a citation was published. Its valid formats are , , and
.

Average log probability of the candidate.

Returns the top candidate tokens ( ) and the actual chosen


tokens ( ) at each step.

20 of 24 11/10/24, 01:37
Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

Generative AI models break down text data into tokens for processing, which
can be characters, words, or phrases.

A log probability value that indicates the model's con�dence for a particular
token.

Number of tokens in the request.

Number of tokens in the response(s).

Number of tokens in the request and response(s).

Examples

Non-streaming text response

Generate a non-streaming model response from a text input.

(#rest) (#nodejs) (#java) (#go) (#c) (#rest-openai)


(#python)

import vertexai
from vertexai.generative_models import GenerativeModel

# TODO(developer): Update and un-comment below line


# PROJECT_ID = "your-project-id"
vertexai.init(project=PROJECT_ID, location="us-central1")

model = GenerativeModel("gemini-1.5-flash-002")

response = model.generate_content(
"What's a good name for a flower shop that specializes in selling bouquets of dr
)

print(response.text)
# Example response:
# **Emphasizing the Dried Aspect:**
# * Everlasting Blooms
# * Dried & Delightful
# * The Petal Preserve
# ...

21 of 24 11/10/24, 01:37
Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

Non-streaming multi-modal response

Generate a non-streaming model response from a multi-modal input, such as text and an
image.

(#rest) (#nodejs) (#java) (#go) (#c) (#rest-openai)


(#python)

import vertexai

from vertexai.generative_models import GenerativeModel, Part

# TODO(developer): Update and un-comment below line


# PROJECT_ID = "your-project-id"
vertexai.init(project=PROJECT_ID, location="us-central1")

model = GenerativeModel("gemini-1.5-flash-002")

response = model.generate_content(
[
Part.from_uri(
"gs:∕∕cloud-samples-data∕generative-ai∕image∕scones.jpg",
mime_type="image∕jpeg",
),
"What is shown in this image?",
]
)

print(response.text)
# That's a lovely overhead shot of a rustic-style breakfast or brunch spread.
# Here's what's in the image:
# * **Blueberry scones:** Several freshly baked blueberry scones are arranged on par
# They look crumbly and delicious.
# ...

Streaming text response

Generate a streaming model response from a text input.

(#rest) (#nodejs) (#java) (#go) (#rest-openai)


(#python)

import vertexai

22 of 24 11/10/24, 01:37
Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

from vertexai.generative_models import GenerativeModel

# TODO(developer): Update and un-comment below line


# PROJECT_ID = "your-project-id"
vertexai.init(project=PROJECT_ID, location="us-central1")

model = GenerativeModel("gemini-1.5-flash-002")
responses = model.generate_content(
"Write a story about a magic backpack.", stream=True
)

for response in responses:


print(response.text)
# Example response:
# El
# ara wasn't looking for magic. She was looking for rent money.
# Her tiny apartment, perched precariously on the edge of Whispering Woods,
# ...

Streaming multi-modal response

Generate a streaming model response from a multi-modal input, such as text and an image.

(#rest) (#nodejs) (#java) (#go) (#rest-openai)


(#python)

import vertexai

from vertexai.generative_models import GenerativeModel, Part

# TODO(developer): Update and un-comment below line


# PROJECT_ID = "your-project-id"

vertexai.init(project=PROJECT_ID, location="us-central1")

model = GenerativeModel("gemini-1.5-flash-002")
responses = model.generate_content(
[
Part.from_uri(
"gs:∕∕cloud-samples-data∕generative-ai∕video∕animals.mp4", "video∕mp4"
),
Part.from_uri(
"gs:∕∕cloud-samples-data∕generative-ai∕image∕character.jpg",
"image∕jpeg",

23 of 24 11/10/24, 01:37
Generate content with the Gemini Enterprise API | ... https://cloud.google.com/vertex-ai/generative-ai/docs...

),
"Are these video and image correlated?",
],
stream=True,
)

for response in responses:


print(response.candidates[0].content.text)
# Example response:
# No, the video and image are not correlated. The video shows a Google Photos
# project where animals at the Los Angeles Zoo take selfies using modified cameras.
# The image is a simple drawing of a wizard.

Model versions

To use the auto-updated version


(/vertex-ai/generative-ai/docs/learn/model-versioning#auto-updated-version), specify the model
name without the trailing version number, for example instead of
.

For more information, see Gemini model versions and lifecycle


(/vertex-ai/generative-ai/docs/learn/model-versioning#gemini-model-versions).

What's next

• Learn more about the Gemini API (/vertex-ai/generative-ai/docs/model-reference/gemini).

• Learn more about Function calling


(/vertex-ai/generative-ai/docs/multimodal/function-calling).

• Learn more about Grounding responses for Gemini models


(/vertex-ai/generative-ai/docs/multimodal/ground-gemini).

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0
License (https://creativecommons.org/licenses/by/4.0/), and code samples are licensed under the Apache
2.0 License (https://www.apache.org/licenses/LICENSE-2.0). For details, see the Google Developers Site
Policies (https://developers.google.com/site-policies). Java is a registered trademark of Oracle and/or its
a�liates.

Last updated 2024-11-08 UTC.

24 of 24 11/10/24, 01:37

You might also like