Skip to content

Conversation

alexreal1314
Copy link
Contributor

@alexreal1314 alexreal1314 commented Mar 4, 2025

Summary

Purpose of this PR is to update UI components which display vulnerabilities in order to support 3rd party integrations and in particularly Qualys, changes include:

  • New column in Default view in table- "Vulnerability Title"
  • Renaming "Vulnerability" column to "CVE ID"
  • Introducing "Title" (Vulnerability Title) and "Subtitle" (CVE ID) in vulnerability flyout while also supporting multiple CVEs.
  • Update severity handling to be case insensitive across relevant components - data grid, flyout, contextual flyout and insights tab data grid.

Checklist

Vulnerabilities data grid and Flyout

  • Add new column with header 'Vulnerability Title' and the underlying field should be 'vulnerability.title' field.
  • change existing column with header 'Vulnerability' to 'CVE ID' and underlying field should be 'vulnerability.id'.
  • aggregation query when grouping by CVE should fetch severities to be case-insensitive - since 3rd party integrations can introduce severities not in UPPER CASE.
  • Components related to vulnerability severity handle supported severity levels disregarding the case - vulnerabilities flyout and contextual flyout, data grid and insights tab data table.
  • Multiple CVEs under vulnerability.id is a supported case in Vulnerability Findings in Kibana.
  • missing vulnerability.id is handled gracefully in the UI - data grid
  • missing vulnerability.id is handled gracefully in the UI - flyout

Insights tab data grid and contextual Flyout

  • Add new column with header 'Vulnerability Title' and the underlying field should be 'vulnerability.title' field.
  • change existing column with header 'Vulnerability' to 'CVE ID' and underlying field should be 'vulnerability.id'.
  • fix redirect link when clicking on a vulnerabilty row that redirect to the vulnerabilities tab - make sure search params and filters are working with vulnerability.id array of strings.
  • contextual flyout should support severities maps to be case insensitive.
  • Unit or functional tests were updated or added to match the most common scenarios
  • Flaky Test Runner was used on any tests changed
  • The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines

The following topics will be merged as part of another PR

Vulnerabilities data grid and Flyout

  • arrays in package-related vulnerability fields are rendered correctly in the flyout
  • arrays in package-related vulnerability fields are rendered correctly in the data grid.

Insights tab data grid and contextual Flyout

  • arrays in package-related vulnerability fields are rendered correctly in the contextual flyout
  • arrays in package-related vulnerability fields are rendered correctly in the insights tab data grid.
    Vulnerabilities data grid grouping

Grouping by multi-value fields

  • grouping by CVE should be handled properly in the UI.
  • each field which might have multiple values - package.name, package.fixed_version and package.version should be flattened by each value separately so user have the max value and can group by each value.

Identify risks

The UI changes in this PR address new integrations which has different structure to vulnerability fields which might cause UI bugs. Tests were modified and added to mitigate the risk of encountering issues.

Screenshots

Screenshot 2025-03-10 at 16 04 25

Screenshot 2025-03-10 at 16 07 04

Screenshot 2025-03-10 at 16 07 32

Screenshot 2025-03-10 at 16 04 40

Grouping before fix by multiple values
image

Flatten by each value
image

@alexreal1314 alexreal1314 linked an issue Mar 4, 2025 that may be closed by this pull request
2 tasks
@alexreal1314 alexreal1314 force-pushed the 198669-handle-vulnerabilityid-flyout branch 4 times, most recently from fcc5d86 to adf8c30 Compare March 9, 2025 08:20
@alexreal1314 alexreal1314 added Team:Cloud Security Cloud Security team related release_note:feature Makes this part of the condensed release notes ci:cloud-deploy Create or update a Cloud deployment backport:skip This commit does not require backporting labels Mar 9, 2025
@alexreal1314 alexreal1314 changed the title 198669 handle vulnerabilityid flyout [Cloud Security] 198669 handle vulnerabilityid flyout Mar 10, 2025
@kibanamachine
Copy link
Contributor

Cloud deployment initiated, see credentials at: https://buildkite.com/elastic/kibana-deploy-cloud-from-pr/builds/102

@alexreal1314 alexreal1314 added release_note:enhancement and removed release_note:feature Makes this part of the condensed release notes labels Mar 11, 2025
@alexreal1314 alexreal1314 changed the title [Cloud Security] 198669 handle vulnerabilityid flyout [Cloud Security] 198669 add support for multiple CVEs and improve vulnerability data grid, flyout and contextual flyout UI Mar 11, 2025
@alexreal1314 alexreal1314 marked this pull request as ready for review March 11, 2025 15:47
@alexreal1314 alexreal1314 requested a review from a team as a code owner March 11, 2025 15:47
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-cloud-security-posture (Team:Cloud Security)

Copy link
Contributor

@JordanSh JordanSh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added some comments for now, still need to continue the review but i think some changes you can already start working on. I'm also adding paulo and max as reviewers since paulo is most the experienced with the table and max has the most knowledge with 3Ps

Comment on lines 10 to 20
const isVulnSeverity = (severity: string): severity is VulnSeverity =>
VALID_SEVERITIES.includes(severity as VulnSeverity);

export const getNormalizedSeverity = (severity?: string): VulnSeverity => {
if (!severity) {
return 'UNKNOWN';
}

return (
isVulnSeverity(severity.toUpperCase()) ? severity.toUpperCase() : 'UNKNOWN'
) as VulnSeverity;
};
Copy link
Contributor

@JordanSh JordanSh Mar 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I think you can remove the casting by doing this:
const isVulnSeverity = (severity: string): severity is VulnSeverity => 
     VALID_SEVERITIES.some(validSeverity => validSeverity === severity);
  1. regarding
  if (!severity) {
    return 'UNKNOWN';
  }

are we sure we want to flag missing data is UKNOWN? maybe it should be left blank?

  1. the casting at the end pretty much negates the entire process that we are doing, I'm suggesting some changes:
  const upperCaseSeverity = severity.toUpperCase();

  if (isVulnSeverity(upperCaseSeverity)) {
    return upperCaseSeverity;
  }

  return 'UNKNOWN';
  1. regarding the return 'UNKNOWN'; at the end. since we meant to be agnostic to the data, maybe we want to support 3P types of severities? changing customer's severity types to something else might cause them problems. should verify with product.

@smriti0321 @tinnytintin10 @nick-alayil, can one of you please confirm point 2 and 4?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JordanSh For #2 Yes "Unknown" severity is recognised category in vulnerability detection space. We should ideally show Severity- Unknown and not leave it blank like other fields. For #4 I am not very clear about ask, overall we should not assign "UNKNOWN" severity on our side if source is not providing this value.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @smriti0321!
So just to summarize
We only assign severity UNKNOWN if the actual value is indeed "UNKNOWN", or if the severity value is undefined. For any other case we either assign our default values or any other value that the 3P has saved in this field.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TL;DR: We should enumerate 3P data as-is. We can make UI/UX adjustments for proper display, but no data transformations.

Thanks for the discussion everyone. Let me clarify a few points:

are we sure we want to flag missing data is UKNOWN? maybe it should be left blank?

I agree we should leave it blank. Here's why:

  1. Our primary goal is to enumerate 3P vuln findings data without enrichment at this stage. Making enrichment decisions now could create inconsistencies with future 3P data sources.
  2. UNKNOWN is actually a legitimate severity value that some tools output explicitly. We shouldn't override or assign it when the original source doesn't provide this value.

changing customer's severity types to something else might cause them problems.

Completely agree here. 👍

It's worth noting this code was originally built for CNVM specific use cases, where we made certain adjustments to handle missing data for better table presentation. Now that we're expanding to enumerate all 3P vuln data in the findings page, we should maintain the original 3P data integrity.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alexreal1314 since we will need to support empty severity and other random values, we will also need to address it in the UI. we will need to verify we can still group search and display it in the flyout

Copy link
Contributor Author

@alexreal1314 alexreal1314 Mar 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nick-alayil @smriti0321 @JordanSh thanks for the inputs.
just to make sure with some examples regarding severity:

  1. low/medium/high/critical/unknown -> in any case we convert to upper case so we can use our colors for marking severities.
  2. missing severity field/any other value -> stays the same.
    image

image

image

do we want to display it like that in the data grid? also need to think what to show in the flyout.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • low/medium/high/critical/unknown -> in any case we convert to upper case so we can use our colors for marking severities.

Yes, that sounds good 👍

  • missing severity field/any other value -> stays the same.

Correct.

do we want to display it like that in the data grid?

Yes please.

also need to think what to show in the flyout.

If there is no severity, we don't need to display anything as shown in your 2nd flyout screenshot.

@@ -161,6 +161,25 @@ export const CloudSecurityDataTable = ({
};
}, [persistedSettings, columnHeaders]);

// This is a mapper that adds vulnerability.id field as an empty string - needed for the flyout to open
const rowsWithVulnerabilityId = useMemo(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can this be done on the flyout level instead? i dont think we should run over all the items in the table. we can just have a default value for the id when the flyout is opening

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JordanSh the issue is that the flyout doesn't open at all if 'vulnerability.id' key is missing.

Copy link
Contributor

@JordanSh JordanSh Mar 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thats what im suggesting to change. lets make the flyout be able to open even without vuln id. thats what happening anyway. we just need to remove this condition from the flyout rather than add empty ids to all items (which is also data manipulation which we want to avoid as much as possible)

note that will change will require us to add tests for this scenario, make sure nothing breaks in the flyout without vuln id

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JordanSh in that case what should be the check instead of this one?
const isCspVulnerabilityFinding = ( source: Record<string, any> | undefined ): source is CspVulnerabilityFinding => { return source?.vulnerability?.id !== undefined; };

do we want some kind of other check maybe based on vendor? or we allow to open the flyout from any type of vulnerability?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

waiting for a decision on how should we check if we should open the vulnerability flyout - based on observer.vendor field or another field.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👀

@JordanSh JordanSh requested review from maxcold and opauloh March 13, 2025 13:27
Copy link
Contributor

@seanrathier seanrathier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! I left a few comments.

@@ -161,6 +161,25 @@ export const CloudSecurityDataTable = ({
};
}, [persistedSettings, columnHeaders]);

// This is a mapper that adds vulnerability.id field as an empty string - needed for the flyout to open
const rowsWithVulnerabilityId = useMemo(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👀

@alexreal1314 alexreal1314 requested a review from a team as a code owner March 18, 2025 10:41
@alexreal1314 alexreal1314 force-pushed the 198669-handle-vulnerabilityid-flyout branch 3 times, most recently from d2452af to 3f3bfc2 Compare March 18, 2025 14:30
Copy link
Contributor

@seanrathier seanrathier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving my concerns, thanks for making the changes. 👍

@alexreal1314 alexreal1314 added ci:cloud-deploy Create or update a Cloud deployment ci:cloud-redeploy Always create a new Cloud deployment and removed ci:cloud-deploy Create or update a Cloud deployment ci:cloud-redeploy Always create a new Cloud deployment labels Mar 18, 2025
@alexreal1314 alexreal1314 force-pushed the 198669-handle-vulnerabilityid-flyout branch 2 times, most recently from 712fc14 to 236cfa7 Compare March 25, 2025 11:09
@alexreal1314 alexreal1314 force-pushed the 198669-handle-vulnerabilityid-flyout branch from 236cfa7 to 32694ed Compare March 25, 2025 13:21
@alexreal1314 alexreal1314 removed the request for review from a team March 25, 2025 13:49
@elasticmachine
Copy link
Contributor

elasticmachine commented Mar 25, 2025

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #65 / EQL execution logic API @ess @serverless @serverlessQA EQL type rules parses shard failures for EQL event query
  • [job] [logs] Investigations - Security Solution Cypress Tests #3 / KPI visualizations in Alerts Page Histogram legend hover actions should should add a filter in to KQL bar should should add a filter in to KQL bar

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
cloudSecurityPosture 683 687 +4
securitySolution 7095 7096 +1
total +5

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/cloud-security-posture 97 99 +2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
cloudSecurityPosture 488.8KB 493.7KB +4.9KB
securitySolution 8.9MB 8.9MB +655.0B
total +5.5KB
Unknown metric groups

API count

id before after diff
@kbn/cloud-security-posture 97 99 +2

History

@alexreal1314 alexreal1314 merged commit c7e829e into main Mar 25, 2025
9 checks passed
@alexreal1314 alexreal1314 deleted the 198669-handle-vulnerabilityid-flyout branch March 25, 2025 15:23
cqliu1 pushed a commit to cqliu1/kibana that referenced this pull request Mar 31, 2025
…nerability data grid, flyout and contextual flyout UI (elastic#213039)

## Summary

Purpose of this PR is to update UI components which display
vulnerabilities in order to support 3rd party integrations and in
particularly Qualys, changes include:

- New column in Default view in table- "Vulnerability Title"
- Renaming "Vulnerability" column to "CVE ID"
- Introducing "Title" (Vulnerability Title) and "Subtitle" (CVE ID) in
vulnerability flyout while also supporting multiple CVEs.
- Update severity handling to be case insensitive across relevant
components - data grid, flyout, contextual flyout and insights tab data
grid.


### Checklist

**Vulnerabilities data grid and Flyout**

- [x] Add new column with header 'Vulnerability Title' and the
underlying field should be 'vulnerability.title' field.
- [x] change existing column with header 'Vulnerability' to 'CVE ID' and
underlying field should be 'vulnerability.id'.
- [x] aggregation query when grouping by CVE should fetch severities to
be case-insensitive - since 3rd party integrations can introduce
severities not in UPPER CASE.
- [x] Components related to vulnerability severity handle supported
severity levels disregarding the case - vulnerabilities flyout and
contextual flyout, data grid and insights tab data table.
- [x] Multiple CVEs under `vulnerability.id` is a supported case in
Vulnerability Findings in Kibana.
- [x] missing `vulnerability.id` is handled gracefully in the UI - data
grid
- [x] missing `vulnerability.id` is handled gracefully in the UI -
flyout

**Insights tab data grid and contextual Flyout**

- [x] Add new column with header 'Vulnerability Title' and the
underlying field should be 'vulnerability.title' field.
- [x] change existing column with header 'Vulnerability' to 'CVE ID' and
underlying field should be 'vulnerability.id'.
- [x] fix redirect link when clicking on a vulnerabilty row that
redirect to the vulnerabilities tab - make sure search params and
filters are working with vulnerability.id array of strings.
- [x] contextual flyout should support severities maps to be case
insensitive.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

**The following topics will be merged as part of another PR**
### Vulnerabilities data grid and Flyout
- arrays in package-related vulnerability fields are rendered correctly
in the flyout
- arrays in package-related vulnerability fields are rendered correctly
in the data grid.

### Insights tab data grid and contextual Flyout
- arrays in package-related vulnerability fields are rendered correctly
in the contextual flyout
- arrays in package-related vulnerability fields are rendered correctly
in the insights tab data grid.
**Vulnerabilities data grid grouping**

### Grouping by multi-value fields
- grouping by CVE should be handled properly in the UI.
- each field which might have multiple values - package.name,
package.fixed_version and package.version should be flattened by each
value separately so user have the max value and can group by each value.

### Identify risks
The UI changes in this PR address new integrations which has different
structure to vulnerability fields which might cause UI bugs. Tests were
modified and added to mitigate the risk of encountering issues.

### Screenshots


![Screenshot 2025-03-10 at 16 04
25](https://github.com/user-attachments/assets/8da0974e-dfb3-4aa7-a40f-daaeae53d8f5)

![Screenshot 2025-03-10 at 16 07
04](https://github.com/user-attachments/assets/5d079075-3774-46a1-9098-6e437fbd8e1e)

![Screenshot 2025-03-10 at 16 07
32](https://github.com/user-attachments/assets/5f68bebc-f532-4647-9671-40f41c4b039b)

![Screenshot 2025-03-10 at 16 04
40](https://github.com/user-attachments/assets/8bcba728-f513-434c-b764-5995737216d7)


Grouping before fix by multiple values

![image](https://github.com/user-attachments/assets/21756172-819c-4836-ba3a-79ae9ed6cbad)

Flatten by each value

![image](https://github.com/user-attachments/assets/d329d3f7-b499-4abb-8e40-6c8580be9202)
alexreal1314 added a commit that referenced this pull request Apr 1, 2025
## Summary

Purpose of this PR is to handle grouping of multi-value fields which are
introduced by Qualys VDMR integrations.
This PR adds the capability to flatten grouping results of the following
fields - vulnerability.id, package.name, package.version and
package.fixed_version, which are all ECS fields.

It continues the changes of this
[PR](#213039).

### Checklist

**The following topics will be merged as part of another PR**
### Vulnerabilities data grid and Flyout
- [x] grouping by CVE should be handled properly in the UI.
- [x] multi value fields are flattened - each value is counted as
separate group key.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct release_note:* label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)


### Screenshots

Grouping before fix by multiple values

![image](https://github.com/user-attachments/assets/21756172-819c-4836-ba3a-79ae9ed6cbad)

Flatten by each value

![image](https://github.com/user-attachments/assets/d329d3f7-b499-4abb-8e40-6c8580be9202)
maxcold added a commit that referenced this pull request May 27, 2025
…1302)

## Summary

This PR backports changes related to Qualys VMDR added integration
support that were merged to version 9.1.0.

PRs backported:

- #213039
- #216411


### Identify risks

Thorough sanity check need to be made before merging since a lot of
conflicts were fixed manually.

---------

Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: Maxim Kholod <[email protected]>
maxcold pushed a commit to maxcold/kibana that referenced this pull request May 27, 2025
## Summary

Purpose of this PR is to handle grouping of multi-value fields which are
introduced by Qualys VDMR integrations.
This PR adds the capability to flatten grouping results of the following
fields - vulnerability.id, package.name, package.version and
package.fixed_version, which are all ECS fields.

It continues the changes of this
[PR](elastic#213039).

### Checklist

**The following topics will be merged as part of another PR**
### Vulnerabilities data grid and Flyout
- [x] grouping by CVE should be handled properly in the UI.
- [x] multi value fields are flattened - each value is counted as
separate group key.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct release_note:* label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

### Screenshots

Grouping before fix by multiple values

![image](https://github.com/user-attachments/assets/21756172-819c-4836-ba3a-79ae9ed6cbad)

Flatten by each value

![image](https://github.com/user-attachments/assets/d329d3f7-b499-4abb-8e40-6c8580be9202)

(cherry picked from commit 1b9bf80)

# Conflicts:
#	src/platform/packages/shared/kbn-grouping/src/components/grouping.tsx
#	src/platform/packages/shared/kbn-grouping/src/containers/query/index.test.ts
#	src/platform/packages/shared/kbn-grouping/src/containers/query/index.ts
#	src/platform/packages/shared/kbn-grouping/src/containers/query/types.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting ci:cloud-deploy Create or update a Cloud deployment release_note:enhancement Team:Cloud Security Cloud Security team related v9.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Handle missing vulnerability.id gracefully
9 participants