[Doc] OpenAPI spec #17292

kaisecheng · 2025-03-08T01:33:13Z

Release notes

Migrate API doc to OpenAPI spec

What does this PR do?

break down the share part to components
add the missing fields for Plugins info, Node stats, Node Info, Hot threads and Health report
expand Node Info /_node/<types> to /_node/jvm, /_node/pipelines and /_node/os
expand Node Stats to /_node/stats, /_node/stats/pipelines, /_node/stats/pipelines/{pipeline_name}, /_node/stats/jvm, /_node/stats/flow, /_node/stats/events, /_node/stats/process, /_node/stats/reloads

Why is it important/What is the impact to the user?

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files (and/or docker env variables)
I have added tests that prove my fix is effective or that my feature works

Author's Checklist

it passed Swagger Editor checks https://editor.swagger.io/

How to test this PR locally

Related issues

CLoses: [Logstash API] Complete API doc migration to OpenAPI spec #16967

Use cases

Screenshots

Logs

mergify · 2025-03-08T01:33:50Z

This pull request does not have a backport label. Could you fix it @kaisecheng? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit.
backport-8.x is the label to automatically backport to the 8.x branch.
If no backport is necessary, please add the backport-skip label

github-actions · 2025-03-08T01:56:34Z

📃 DOCS PREVIEW ✨ https://logstash_bk_17292.docs-preview.app.elstc.co/diff

github-actions · 2025-03-10T18:07:58Z

📃 DOCS PREVIEW ✨ https://logstash_bk_17292.docs-preview.app.elstc.co/diff

karenzone · 2025-03-10T20:18:32Z

Related: #16594
@kaisecheng, we're working on getting the validation PR working for Logstash, too. 🤞
Thanks for your work on this.

Also tagging @lcawl as our resident API guru.

github-actions · 2025-03-12T16:46:07Z

📃 DOCS PREVIEW ✨ https://logstash_bk_17292.docs-preview.app.elstc.co/diff

karenzone

@kaisecheng, I generated a BUMP.SH preview, and this is looking good! Thanks for fleshing out so much of the content. Please keep it going!
Bump links expire after a short time, but I'm happy to pull down new content and generate a new link
anytime you'd like for me to.

docs/static/spec/openapi/logstash-api.yaml

lcawl · 2025-03-12T22:44:26Z

docs/static/spec/openapi/logstash-api.yaml

+          type: object
+          additionalProperties:
+            description: "pipeline name"


I don't think that description makes sense in this context, so IMO it either needs to be improved or removed.

Suggested change

description: "pipeline name"

In OpenAPI, additionalProperties means a map structure where the key / field name is dynamic generated

{ additionalProperty1: { ... } additionalProperty2: { ... } }

Description is to tell user the dynamic part additionalProperty1 is a pipeline name, where the pipeline name is a user defined value.

Take a static field example from _node/os

{ "os": { "name": "Mac OS X", "arch": "x86_64", "version": "10.12.4", "available_processors": 8 } }

os, name, arch, version, available_processors never change

updated the description to explain the key and value

docs/static/spec/openapi/logstash-api.yaml

lcawl · 2025-03-12T23:00:39Z

Hi! When I run npx @redocly/cli lint logstash-api.yaml --config redocly.yaml there are a few messages that I think are worth fixing. For example:

[1] logstash-api.yaml:343:5 at #/paths/~~1_node~~1stats~1events/get/operationId
Operation object should contain operationId field.

The operationId uniquely identifies the operation. It is case sensitive and must be unique in the OpenAPI document. It is recommended to use camel case (for example, createConnector).
Our docs use this value when generating the URL, therefore it's better to explicitly define it than rely on something automated.

The rest of my suggestions are just style-related and shouldn't be considered blockers. Thanks for creating this content!

Co-authored-by: Lisa Cawley <[email protected]>

docs/static/spec/openapi/logstash-api.yaml

Co-authored-by: Lisa Cawley <[email protected]>

- add links and description - add infinity to flow window - fix flow stats component

kaisecheng · 2025-03-13T23:24:03Z

@karenzone @lcawl Many thanks for helping to build the doc. I think this is ready for another review.

copy the description and links from the current doc to markdown
comment out the repeated common response for a cleaner presentation

#                  - $ref: '#/components/schemas/Common'
#                  - $ref: '#/components/schemas/PipelineSettings'

add examples for all APIs
some fixes of components
update authentication to basic

github-actions · 2025-03-13T23:37:09Z

📃 DOCS PREVIEW ✨ https://logstash_bk_17292.docs-preview.app.elstc.co/diff

docs/static/spec/openapi/logstash-api.yaml

lcawl · 2025-03-14T00:17:46Z

docs/static/spec/openapi/logstash-api.yaml

+                                            example: ["pipeline_execution"]
+                                            items:
+                                              type: string
+                                    diagnosis:


diagnosis:

There's a linting error related to this definition:

[9] logstash-api.yaml:1455:33 at #/paths/~1_health_report/get/responses/200/content/application~1json/examples/backpressureCase/value/indicators/pipelines/indicators/blocked-pipeline/diagnosis/0/id Example value must conform to the schema: `0` property must NOT have unevaluated properties `id`. 1453 | symptom: "The pipeline is unhealthy; 1 area is impacted and 1 diagnosis is available" 1454 | diagnosis: 1455 | - id: "logstash:health:pipeline:flow:worker_utilization:diagnosis:5m-blocked" 1456 | cause: "pipeline workers have been completely blocked for at least five minutes" 1457 | action: "address bottleneck or add resources" referenced from logstash-api.yaml:1313:15 at #/paths/~1_health_report/get/responses/200/content/application~1json Warning was generated by the no-invalid-media-type-examples rule.

I think this means the id value that appears in the example is missing from this schema definition.

andsel

Checking with https://editor-next.swagger.io/ I got this warnings:

severity	line	code	Message
Warning	1319	10069	property "example" is deprecated, use "examples" instead
Warning	1342	10069	property "example" is deprecated, use "examples" instead
Warning	1379	10069	property "example" is deprecated, use "examples" instead
Hint	1506	10071	nullable has no special meaning, if not set on purpose use `type="null"` instead
Hint	2010	10071	nullable has no special meaning, if not set on purpose use `type="null"` instead
Hint	2017	10071	nullable has no special meaning, if not set on purpose use `type="null"` instead
Hint	2021	10071	nullable has no special meaning, if not set on purpose use `type="null"` instead

- add id to health report - fix typo - add comment for the usage of allOf

github-actions · 2025-03-14T10:33:23Z

📃 DOCS PREVIEW ✨ https://logstash_bk_17292.docs-preview.app.elstc.co/diff

kaisecheng · 2025-03-14T10:46:37Z

@andsel example and examples are both valid in openAPI 3.1
I saw the same warning but found examples structure is overkilled. examples requires a single example to be in array form, so I choose to ignore the warning.

robbavey · 2025-03-14T13:29:13Z

@andsel - If @kaisecheng has addressed your concerns, can you please remove the requests changes status from your PR

karenzone

@kaisecheng, thank you for all of the thought and effort you put into this project. LGTM!

andsel

LGTM

github-actions · 2025-03-14T17:06:08Z

📃 DOCS PREVIEW ✨ https://logstash_bk_17292.docs-preview.app.elstc.co/diff

lcawl

LGTM2, thanks for addressing all my suggestions!

kaisecheng · 2025-03-14T17:27:46Z

Thank you all for your suggestions! It looks good

github-actions · 2025-03-14T17:41:03Z

📃 DOCS PREVIEW ✨ https://logstash_bk_17292.docs-preview.app.elstc.co/diff

karenzone · 2025-03-14T19:34:35Z

PUBLISHED: https://www.elastic.co/docs/api/doc/logstash/
Thanks for publishing, @lcawl

kaisecheng added 5 commits March 7, 2025 15:48

health report openAPI spec

ffe3843

add node info, extract common response, fix hot thread

93d93e5

extract common pipeline info

ce1d84f

rename flow state

ae9c64d

add node stats

619ade1

add missing field to node plugins

985009a

karenzone assigned karenzone and kaisecheng and unassigned karenzone Mar 10, 2025

karenzone requested review from lcawl and karenzone March 10, 2025 20:51

karenzone added the docs label Mar 10, 2025

rearrange APIs alphabetically and add _node/stats/os API

5d671d4

karenzone reviewed Mar 12, 2025

View reviewed changes

lcawl reviewed Mar 12, 2025

View reviewed changes

docs/static/spec/openapi/logstash-api.yaml Outdated Show resolved Hide resolved

Update docs/static/spec/openapi/logstash-api.yaml

d69c240

Co-authored-by: Lisa Cawley <[email protected]>

kaisecheng commented Mar 13, 2025

View reviewed changes

docs/static/spec/openapi/logstash-api.yaml Outdated Show resolved Hide resolved

kaisecheng and others added 6 commits March 13, 2025 10:07

Update docs/static/spec/openapi/logstash-api.yaml

6dfa5f7

Update docs/static/spec/openapi/logstash-api.yaml

0dbe35f

Co-authored-by: Lisa Cawley <[email protected]>

Update docs/static/spec/openapi/logstash-api.yaml

f856715

Co-authored-by: Lisa Cawley <[email protected]>

Update docs/static/spec/openapi/logstash-api.yaml

ad5ea70

Co-authored-by: Lisa Cawley <[email protected]>

Update docs/static/spec/openapi/logstash-api.yaml

b0830e5

Co-authored-by: Lisa Cawley <[email protected]>

Update docs/static/spec/openapi/logstash-api.yaml

c7a4e9f

Co-authored-by: Lisa Cawley <[email protected]>

- move pretty parameter to component

9fc749d

- add links and description - add infinity to flow window - fix flow stats component

kaisecheng changed the title ~~[WIP] [Doc] OpenAPI spec~~ [Doc] OpenAPI spec Mar 13, 2025

kaisecheng marked this pull request as ready for review March 13, 2025 23:16

update Authentication to basic

8c2eac3

lcawl reviewed Mar 13, 2025

View reviewed changes

docs/static/spec/openapi/logstash-api.yaml Show resolved Hide resolved

lcawl reviewed Mar 14, 2025

View reviewed changes

docs/static/spec/openapi/logstash-api.yaml Outdated Show resolved Hide resolved

lcawl reviewed Mar 14, 2025

View reviewed changes

docs/static/spec/openapi/logstash-api.yaml Outdated Show resolved Hide resolved

lcawl reviewed Mar 14, 2025

View reviewed changes

docs/static/spec/openapi/logstash-api.yaml Outdated Show resolved Hide resolved

lcawl reviewed Mar 14, 2025

View reviewed changes

docs/static/spec/openapi/logstash-api.yaml Outdated Show resolved Hide resolved

lcawl reviewed Mar 14, 2025

View reviewed changes

andsel requested changes Mar 14, 2025

View reviewed changes

- remove wrong comma

1c5db10

- add id to health report - fix typo - add comment for the usage of allOf

karenzone approved these changes Mar 14, 2025

View reviewed changes

andsel self-requested a review March 14, 2025 14:30

andsel approved these changes Mar 14, 2025

View reviewed changes

break down flow stats to three parts to fix MQ flow stats

50108e4

lcawl approved these changes Mar 14, 2025

View reviewed changes

add example to _node/stats

f7d35bb

kaisecheng merged commit 4779d5e into elastic:main Mar 14, 2025
7 of 8 checks passed

lcawl mentioned this pull request Mar 14, 2025

OpenAPI deployment warning #17323

Open

yaauie mentioned this pull request Mar 17, 2025

Openapi followup #17341

Open

2 tasks

[Doc] OpenAPI spec #17292

[Doc] OpenAPI spec #17292

Uh oh!

Conversation

kaisecheng commented Mar 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Release notes

What does this PR do?

Why is it important/What is the impact to the user?

Checklist

Author's Checklist

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

Uh oh!

mergify bot commented Mar 8, 2025

Uh oh!

github-actions bot commented Mar 8, 2025

Uh oh!

github-actions bot commented Mar 10, 2025

Uh oh!

karenzone commented Mar 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 12, 2025

Uh oh!

karenzone left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lcawl Mar 12, 2025

Choose a reason for hiding this comment

Uh oh!

kaisecheng Mar 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kaisecheng Mar 13, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lcawl commented Mar 12, 2025

Uh oh!

Uh oh!

kaisecheng commented Mar 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 13, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lcawl Mar 14, 2025

Choose a reason for hiding this comment

Uh oh!

andsel left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Mar 14, 2025

Uh oh!

kaisecheng commented Mar 14, 2025

Uh oh!

robbavey commented Mar 14, 2025

Uh oh!

karenzone left a comment

Choose a reason for hiding this comment

Uh oh!

andsel left a comment

kaisecheng commented Mar 8, 2025 •

edited

Loading

karenzone commented Mar 10, 2025 •

edited

Loading

kaisecheng Mar 13, 2025 •

edited

Loading

kaisecheng commented Mar 13, 2025 •

edited

Loading