Skip to content

Commit ffeaa42

Browse files
feat(dataflow): update the api
#### dataflow:v1b3 The following keys were added: - schemas.Job.properties.runtimeUpdatableParams.$ref (Total Keys: 1) - schemas.PubsubLocation.properties.dynamicDestinations.type (Total Keys: 1) - schemas.RuntimeUpdatableParams (Total Keys: 6)
1 parent acd9205 commit ffeaa42

8 files changed

+114
-3
lines changed

docs/dyn/dataflow_v1b3.projects.jobs.html

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -433,6 +433,10 @@ <h3>Method Details</h3>
433433
&quot;replaceJobId&quot;: &quot;A String&quot;, # If this job is an update of an existing job, this field is the job ID of the job it replaced. When sending a `CreateJobRequest`, you can update a job by specifying it here. The job named here is stopped, and its intermediate state is transferred to this job.
434434
&quot;replacedByJobId&quot;: &quot;A String&quot;, # If another job is an update of this job (and thus, this job is in `JOB_STATE_UPDATED`), this field contains the ID of that job.
435435
&quot;requestedState&quot;: &quot;A String&quot;, # The job&#x27;s requested state. `UpdateJob` may be used to switch between the `JOB_STATE_STOPPED` and `JOB_STATE_RUNNING` states, by setting requested_state. `UpdateJob` may also be used to directly set a job&#x27;s requested state to `JOB_STATE_CANCELLED` or `JOB_STATE_DONE`, irrevocably terminating the job if it has not already reached a terminal state.
436+
&quot;runtimeUpdatableParams&quot;: { # Additional job parameters that can only be updated during runtime using the projects.jobs.update method. These fields have no effect when specified during job creation. # This field may ONLY be modified at runtime using the projects.jobs.update method to adjust job behavior. This field has no effect when specified at job creation.
437+
&quot;maxNumWorkers&quot;: 42, # The maximum number of workers to cap autoscaling at. This field is currently only supported for Streaming Engine jobs.
438+
&quot;minNumWorkers&quot;: 42, # The minimum number of workers to scale down to. This field is currently only supported for Streaming Engine jobs.
439+
},
436440
&quot;satisfiesPzs&quot;: True or False, # Reserved for future use. This field is set only in responses from the server; it is ignored if it is set in any requests.
437441
&quot;stageStates&quot;: [ # This field may be mutated by the Cloud Dataflow service; callers cannot mutate it.
438442
{ # A message describing the state of a particular execution stage.
@@ -769,6 +773,10 @@ <h3>Method Details</h3>
769773
&quot;replaceJobId&quot;: &quot;A String&quot;, # If this job is an update of an existing job, this field is the job ID of the job it replaced. When sending a `CreateJobRequest`, you can update a job by specifying it here. The job named here is stopped, and its intermediate state is transferred to this job.
770774
&quot;replacedByJobId&quot;: &quot;A String&quot;, # If another job is an update of this job (and thus, this job is in `JOB_STATE_UPDATED`), this field contains the ID of that job.
771775
&quot;requestedState&quot;: &quot;A String&quot;, # The job&#x27;s requested state. `UpdateJob` may be used to switch between the `JOB_STATE_STOPPED` and `JOB_STATE_RUNNING` states, by setting requested_state. `UpdateJob` may also be used to directly set a job&#x27;s requested state to `JOB_STATE_CANCELLED` or `JOB_STATE_DONE`, irrevocably terminating the job if it has not already reached a terminal state.
776+
&quot;runtimeUpdatableParams&quot;: { # Additional job parameters that can only be updated during runtime using the projects.jobs.update method. These fields have no effect when specified during job creation. # This field may ONLY be modified at runtime using the projects.jobs.update method to adjust job behavior. This field has no effect when specified at job creation.
777+
&quot;maxNumWorkers&quot;: 42, # The maximum number of workers to cap autoscaling at. This field is currently only supported for Streaming Engine jobs.
778+
&quot;minNumWorkers&quot;: 42, # The minimum number of workers to scale down to. This field is currently only supported for Streaming Engine jobs.
779+
},
772780
&quot;satisfiesPzs&quot;: True or False, # Reserved for future use. This field is set only in responses from the server; it is ignored if it is set in any requests.
773781
&quot;stageStates&quot;: [ # This field may be mutated by the Cloud Dataflow service; callers cannot mutate it.
774782
{ # A message describing the state of a particular execution stage.
@@ -1089,6 +1097,10 @@ <h3>Method Details</h3>
10891097
&quot;replaceJobId&quot;: &quot;A String&quot;, # If this job is an update of an existing job, this field is the job ID of the job it replaced. When sending a `CreateJobRequest`, you can update a job by specifying it here. The job named here is stopped, and its intermediate state is transferred to this job.
10901098
&quot;replacedByJobId&quot;: &quot;A String&quot;, # If another job is an update of this job (and thus, this job is in `JOB_STATE_UPDATED`), this field contains the ID of that job.
10911099
&quot;requestedState&quot;: &quot;A String&quot;, # The job&#x27;s requested state. `UpdateJob` may be used to switch between the `JOB_STATE_STOPPED` and `JOB_STATE_RUNNING` states, by setting requested_state. `UpdateJob` may also be used to directly set a job&#x27;s requested state to `JOB_STATE_CANCELLED` or `JOB_STATE_DONE`, irrevocably terminating the job if it has not already reached a terminal state.
1100+
&quot;runtimeUpdatableParams&quot;: { # Additional job parameters that can only be updated during runtime using the projects.jobs.update method. These fields have no effect when specified during job creation. # This field may ONLY be modified at runtime using the projects.jobs.update method to adjust job behavior. This field has no effect when specified at job creation.
1101+
&quot;maxNumWorkers&quot;: 42, # The maximum number of workers to cap autoscaling at. This field is currently only supported for Streaming Engine jobs.
1102+
&quot;minNumWorkers&quot;: 42, # The minimum number of workers to scale down to. This field is currently only supported for Streaming Engine jobs.
1103+
},
10921104
&quot;satisfiesPzs&quot;: True or False, # Reserved for future use. This field is set only in responses from the server; it is ignored if it is set in any requests.
10931105
&quot;stageStates&quot;: [ # This field may be mutated by the Cloud Dataflow service; callers cannot mutate it.
10941106
{ # A message describing the state of a particular execution stage.
@@ -1416,6 +1428,10 @@ <h3>Method Details</h3>
14161428
&quot;replaceJobId&quot;: &quot;A String&quot;, # If this job is an update of an existing job, this field is the job ID of the job it replaced. When sending a `CreateJobRequest`, you can update a job by specifying it here. The job named here is stopped, and its intermediate state is transferred to this job.
14171429
&quot;replacedByJobId&quot;: &quot;A String&quot;, # If another job is an update of this job (and thus, this job is in `JOB_STATE_UPDATED`), this field contains the ID of that job.
14181430
&quot;requestedState&quot;: &quot;A String&quot;, # The job&#x27;s requested state. `UpdateJob` may be used to switch between the `JOB_STATE_STOPPED` and `JOB_STATE_RUNNING` states, by setting requested_state. `UpdateJob` may also be used to directly set a job&#x27;s requested state to `JOB_STATE_CANCELLED` or `JOB_STATE_DONE`, irrevocably terminating the job if it has not already reached a terminal state.
1431+
&quot;runtimeUpdatableParams&quot;: { # Additional job parameters that can only be updated during runtime using the projects.jobs.update method. These fields have no effect when specified during job creation. # This field may ONLY be modified at runtime using the projects.jobs.update method to adjust job behavior. This field has no effect when specified at job creation.
1432+
&quot;maxNumWorkers&quot;: 42, # The maximum number of workers to cap autoscaling at. This field is currently only supported for Streaming Engine jobs.
1433+
&quot;minNumWorkers&quot;: 42, # The minimum number of workers to scale down to. This field is currently only supported for Streaming Engine jobs.
1434+
},
14191435
&quot;satisfiesPzs&quot;: True or False, # Reserved for future use. This field is set only in responses from the server; it is ignored if it is set in any requests.
14201436
&quot;stageStates&quot;: [ # This field may be mutated by the Cloud Dataflow service; callers cannot mutate it.
14211437
{ # A message describing the state of a particular execution stage.
@@ -1801,6 +1817,10 @@ <h3>Method Details</h3>
18011817
&quot;replaceJobId&quot;: &quot;A String&quot;, # If this job is an update of an existing job, this field is the job ID of the job it replaced. When sending a `CreateJobRequest`, you can update a job by specifying it here. The job named here is stopped, and its intermediate state is transferred to this job.
18021818
&quot;replacedByJobId&quot;: &quot;A String&quot;, # If another job is an update of this job (and thus, this job is in `JOB_STATE_UPDATED`), this field contains the ID of that job.
18031819
&quot;requestedState&quot;: &quot;A String&quot;, # The job&#x27;s requested state. `UpdateJob` may be used to switch between the `JOB_STATE_STOPPED` and `JOB_STATE_RUNNING` states, by setting requested_state. `UpdateJob` may also be used to directly set a job&#x27;s requested state to `JOB_STATE_CANCELLED` or `JOB_STATE_DONE`, irrevocably terminating the job if it has not already reached a terminal state.
1820+
&quot;runtimeUpdatableParams&quot;: { # Additional job parameters that can only be updated during runtime using the projects.jobs.update method. These fields have no effect when specified during job creation. # This field may ONLY be modified at runtime using the projects.jobs.update method to adjust job behavior. This field has no effect when specified at job creation.
1821+
&quot;maxNumWorkers&quot;: 42, # The maximum number of workers to cap autoscaling at. This field is currently only supported for Streaming Engine jobs.
1822+
&quot;minNumWorkers&quot;: 42, # The minimum number of workers to scale down to. This field is currently only supported for Streaming Engine jobs.
1823+
},
18041824
&quot;satisfiesPzs&quot;: True or False, # Reserved for future use. This field is set only in responses from the server; it is ignored if it is set in any requests.
18051825
&quot;stageStates&quot;: [ # This field may be mutated by the Cloud Dataflow service; callers cannot mutate it.
18061826
{ # A message describing the state of a particular execution stage.
@@ -2178,6 +2198,10 @@ <h3>Method Details</h3>
21782198
&quot;replaceJobId&quot;: &quot;A String&quot;, # If this job is an update of an existing job, this field is the job ID of the job it replaced. When sending a `CreateJobRequest`, you can update a job by specifying it here. The job named here is stopped, and its intermediate state is transferred to this job.
21792199
&quot;replacedByJobId&quot;: &quot;A String&quot;, # If another job is an update of this job (and thus, this job is in `JOB_STATE_UPDATED`), this field contains the ID of that job.
21802200
&quot;requestedState&quot;: &quot;A String&quot;, # The job&#x27;s requested state. `UpdateJob` may be used to switch between the `JOB_STATE_STOPPED` and `JOB_STATE_RUNNING` states, by setting requested_state. `UpdateJob` may also be used to directly set a job&#x27;s requested state to `JOB_STATE_CANCELLED` or `JOB_STATE_DONE`, irrevocably terminating the job if it has not already reached a terminal state.
2201+
&quot;runtimeUpdatableParams&quot;: { # Additional job parameters that can only be updated during runtime using the projects.jobs.update method. These fields have no effect when specified during job creation. # This field may ONLY be modified at runtime using the projects.jobs.update method to adjust job behavior. This field has no effect when specified at job creation.
2202+
&quot;maxNumWorkers&quot;: 42, # The maximum number of workers to cap autoscaling at. This field is currently only supported for Streaming Engine jobs.
2203+
&quot;minNumWorkers&quot;: 42, # The minimum number of workers to scale down to. This field is currently only supported for Streaming Engine jobs.
2204+
},
21812205
&quot;satisfiesPzs&quot;: True or False, # Reserved for future use. This field is set only in responses from the server; it is ignored if it is set in any requests.
21822206
&quot;stageStates&quot;: [ # This field may be mutated by the Cloud Dataflow service; callers cannot mutate it.
21832207
{ # A message describing the state of a particular execution stage.
@@ -2492,6 +2516,10 @@ <h3>Method Details</h3>
24922516
&quot;replaceJobId&quot;: &quot;A String&quot;, # If this job is an update of an existing job, this field is the job ID of the job it replaced. When sending a `CreateJobRequest`, you can update a job by specifying it here. The job named here is stopped, and its intermediate state is transferred to this job.
24932517
&quot;replacedByJobId&quot;: &quot;A String&quot;, # If another job is an update of this job (and thus, this job is in `JOB_STATE_UPDATED`), this field contains the ID of that job.
24942518
&quot;requestedState&quot;: &quot;A String&quot;, # The job&#x27;s requested state. `UpdateJob` may be used to switch between the `JOB_STATE_STOPPED` and `JOB_STATE_RUNNING` states, by setting requested_state. `UpdateJob` may also be used to directly set a job&#x27;s requested state to `JOB_STATE_CANCELLED` or `JOB_STATE_DONE`, irrevocably terminating the job if it has not already reached a terminal state.
2519+
&quot;runtimeUpdatableParams&quot;: { # Additional job parameters that can only be updated during runtime using the projects.jobs.update method. These fields have no effect when specified during job creation. # This field may ONLY be modified at runtime using the projects.jobs.update method to adjust job behavior. This field has no effect when specified at job creation.
2520+
&quot;maxNumWorkers&quot;: 42, # The maximum number of workers to cap autoscaling at. This field is currently only supported for Streaming Engine jobs.
2521+
&quot;minNumWorkers&quot;: 42, # The minimum number of workers to scale down to. This field is currently only supported for Streaming Engine jobs.
2522+
},
24952523
&quot;satisfiesPzs&quot;: True or False, # Reserved for future use. This field is set only in responses from the server; it is ignored if it is set in any requests.
24962524
&quot;stageStates&quot;: [ # This field may be mutated by the Cloud Dataflow service; callers cannot mutate it.
24972525
{ # A message describing the state of a particular execution stage.

docs/dyn/dataflow_v1b3.projects.jobs.workItems.html

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -609,6 +609,7 @@ <h3>Method Details</h3>
609609
},
610610
&quot;pubsubLocation&quot;: { # Identifies a pubsub location to use for transferring data into or out of a streaming Dataflow job. # The stream is a pubsub stream.
611611
&quot;dropLateData&quot;: True or False, # Indicates whether the pipeline allows late-arriving data.
612+
&quot;dynamicDestinations&quot;: True or False, # If true, then this location represents dynamic topics.
612613
&quot;idLabel&quot;: &quot;A String&quot;, # If set, contains a pubsub label from which to extract record ids. If left empty, record deduplication will be strictly best effort.
613614
&quot;subscription&quot;: &quot;A String&quot;, # A pubsub subscription, in the form of &quot;pubsub.googleapis.com/subscriptions//&quot;
614615
&quot;timestampLabel&quot;: &quot;A String&quot;, # If set, contains a pubsub label from which to extract record timestamps. If left empty, record timestamps will be generated upon arrival.
@@ -641,6 +642,7 @@ <h3>Method Details</h3>
641642
},
642643
&quot;pubsubLocation&quot;: { # Identifies a pubsub location to use for transferring data into or out of a streaming Dataflow job. # The stream is a pubsub stream.
643644
&quot;dropLateData&quot;: True or False, # Indicates whether the pipeline allows late-arriving data.
645+
&quot;dynamicDestinations&quot;: True or False, # If true, then this location represents dynamic topics.
644646
&quot;idLabel&quot;: &quot;A String&quot;, # If set, contains a pubsub label from which to extract record ids. If left empty, record deduplication will be strictly best effort.
645647
&quot;subscription&quot;: &quot;A String&quot;, # A pubsub subscription, in the form of &quot;pubsub.googleapis.com/subscriptions//&quot;
646648
&quot;timestampLabel&quot;: &quot;A String&quot;, # If set, contains a pubsub label from which to extract record timestamps. If left empty, record timestamps will be generated upon arrival.

docs/dyn/dataflow_v1b3.projects.locations.flexTemplates.html

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -493,6 +493,10 @@ <h3>Method Details</h3>
493493
&quot;replaceJobId&quot;: &quot;A String&quot;, # If this job is an update of an existing job, this field is the job ID of the job it replaced. When sending a `CreateJobRequest`, you can update a job by specifying it here. The job named here is stopped, and its intermediate state is transferred to this job.
494494
&quot;replacedByJobId&quot;: &quot;A String&quot;, # If another job is an update of this job (and thus, this job is in `JOB_STATE_UPDATED`), this field contains the ID of that job.
495495
&quot;requestedState&quot;: &quot;A String&quot;, # The job&#x27;s requested state. `UpdateJob` may be used to switch between the `JOB_STATE_STOPPED` and `JOB_STATE_RUNNING` states, by setting requested_state. `UpdateJob` may also be used to directly set a job&#x27;s requested state to `JOB_STATE_CANCELLED` or `JOB_STATE_DONE`, irrevocably terminating the job if it has not already reached a terminal state.
496+
&quot;runtimeUpdatableParams&quot;: { # Additional job parameters that can only be updated during runtime using the projects.jobs.update method. These fields have no effect when specified during job creation. # This field may ONLY be modified at runtime using the projects.jobs.update method to adjust job behavior. This field has no effect when specified at job creation.
497+
&quot;maxNumWorkers&quot;: 42, # The maximum number of workers to cap autoscaling at. This field is currently only supported for Streaming Engine jobs.
498+
&quot;minNumWorkers&quot;: 42, # The minimum number of workers to scale down to. This field is currently only supported for Streaming Engine jobs.
499+
},
496500
&quot;satisfiesPzs&quot;: True or False, # Reserved for future use. This field is set only in responses from the server; it is ignored if it is set in any requests.
497501
&quot;stageStates&quot;: [ # This field may be mutated by the Cloud Dataflow service; callers cannot mutate it.
498502
{ # A message describing the state of a particular execution stage.

0 commit comments

Comments
 (0)