-
Notifications
You must be signed in to change notification settings - Fork 8.4k
[Cloud Security] 198669 add support for multiple CVEs and improve vulnerability data grid, flyout and contextual flyout UI #213039
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
fcc5d86
to
adf8c30
Compare
Cloud deployment initiated, see credentials at: https://buildkite.com/elastic/kibana-deploy-cloud-from-pr/builds/102 |
Pinging @elastic/kibana-cloud-security-posture (Team:Cloud Security) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added some comments for now, still need to continue the review but i think some changes you can already start working on. I'm also adding paulo and max as reviewers since paulo is most the experienced with the table and max has the most knowledge with 3Ps
x-pack/platform/packages/shared/kbn-cloud-security-posture/common/types/vulnerabilities.ts
Outdated
Show resolved
Hide resolved
const isVulnSeverity = (severity: string): severity is VulnSeverity => | ||
VALID_SEVERITIES.includes(severity as VulnSeverity); | ||
|
||
export const getNormalizedSeverity = (severity?: string): VulnSeverity => { | ||
if (!severity) { | ||
return 'UNKNOWN'; | ||
} | ||
|
||
return ( | ||
isVulnSeverity(severity.toUpperCase()) ? severity.toUpperCase() : 'UNKNOWN' | ||
) as VulnSeverity; | ||
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- I think you can remove the casting by doing this:
const isVulnSeverity = (severity: string): severity is VulnSeverity =>
VALID_SEVERITIES.some(validSeverity => validSeverity === severity);
- regarding
if (!severity) {
return 'UNKNOWN';
}
are we sure we want to flag missing data is UKNOWN? maybe it should be left blank?
- the casting at the end pretty much negates the entire process that we are doing, I'm suggesting some changes:
const upperCaseSeverity = severity.toUpperCase();
if (isVulnSeverity(upperCaseSeverity)) {
return upperCaseSeverity;
}
return 'UNKNOWN';
- regarding the
return 'UNKNOWN';
at the end. since we meant to be agnostic to the data, maybe we want to support 3P types of severities? changing customer's severity types to something else might cause them problems. should verify with product.
@smriti0321 @tinnytintin10 @nick-alayil, can one of you please confirm point 2 and 4?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@JordanSh For #2 Yes "Unknown" severity is recognised category in vulnerability detection space. We should ideally show Severity- Unknown and not leave it blank like other fields. For #4 I am not very clear about ask, overall we should not assign "UNKNOWN" severity on our side if source is not providing this value.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @smriti0321!
So just to summarize
We only assign severity UNKNOWN if the actual value is indeed "UNKNOWN", or if the severity value is undefined
. For any other case we either assign our default values or any other value that the 3P has saved in this field.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TL;DR: We should enumerate 3P data as-is. We can make UI/UX adjustments for proper display, but no data transformations.
Thanks for the discussion everyone. Let me clarify a few points:
are we sure we want to flag missing data is UKNOWN? maybe it should be left blank?
I agree we should leave it blank. Here's why:
- Our primary goal is to enumerate 3P vuln findings data without enrichment at this stage. Making enrichment decisions now could create inconsistencies with future 3P data sources.
UNKNOWN
is actually a legitimate severity value that some tools output explicitly. We shouldn't override or assign it when the original source doesn't provide this value.
changing customer's severity types to something else might cause them problems.
Completely agree here. 👍
It's worth noting this code was originally built for CNVM specific use cases, where we made certain adjustments to handle missing data for better table presentation. Now that we're expanding to enumerate all 3P vuln data in the findings page, we should maintain the original 3P data integrity.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@alexreal1314 since we will need to support empty severity and other random values, we will also need to address it in the UI. we will need to verify we can still group search and display it in the flyout
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@nick-alayil @smriti0321 @JordanSh thanks for the inputs.
just to make sure with some examples regarding severity:
- low/medium/high/critical/unknown -> in any case we convert to upper case so we can use our colors for marking severities.
- missing severity field/any other value -> stays the same.
do we want to display it like that in the data grid? also need to think what to show in the flyout.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- low/medium/high/critical/unknown -> in any case we convert to upper case so we can use our colors for marking severities.
Yes, that sounds good 👍
- missing severity field/any other value -> stays the same.
Correct.
do we want to display it like that in the data grid?
Yes please.
also need to think what to show in the flyout.
If there is no severity, we don't need to display anything as shown in your 2nd flyout screenshot.
x-pack/solutions/security/packages/kbn-cloud-security-posture/public/src/utils/query_utils.ts
Show resolved
Hide resolved
@@ -161,6 +161,25 @@ export const CloudSecurityDataTable = ({ | |||
}; | |||
}, [persistedSettings, columnHeaders]); | |||
|
|||
// This is a mapper that adds vulnerability.id field as an empty string - needed for the flyout to open | |||
const rowsWithVulnerabilityId = useMemo( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can this be done on the flyout level instead? i dont think we should run over all the items in the table. we can just have a default value for the id when the flyout is opening
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@JordanSh the issue is that the flyout doesn't open at all if 'vulnerability.id' key is missing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thats what im suggesting to change. lets make the flyout be able to open even without vuln id. thats what happening anyway. we just need to remove this condition from the flyout rather than add empty ids to all items (which is also data manipulation which we want to avoid as much as possible)
note that will change will require us to add tests for this scenario, make sure nothing breaks in the flyout without vuln id
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@JordanSh in that case what should be the check instead of this one?
const isCspVulnerabilityFinding = ( source: Record<string, any> | undefined ): source is CspVulnerabilityFinding => { return source?.vulnerability?.id !== undefined; };
do we want some kind of other check maybe based on vendor? or we allow to open the flyout from any type of vulnerability?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
waiting for a decision on how should we check if we should open the vulnerability flyout - based on observer.vendor field or another field.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👀
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good! I left a few comments.
x-pack/platform/packages/shared/kbn-cloud-security-posture/common/index.ts
Outdated
Show resolved
Hide resolved
x-pack/solutions/security/packages/kbn-cloud-security-posture/public/index.ts
Outdated
Show resolved
Hide resolved
x-pack/solutions/security/packages/kbn-cloud-security-posture/public/src/utils/hooks_utils.ts
Outdated
Show resolved
Hide resolved
@@ -161,6 +161,25 @@ export const CloudSecurityDataTable = ({ | |||
}; | |||
}, [persistedSettings, columnHeaders]); | |||
|
|||
// This is a mapper that adds vulnerability.id field as an empty string - needed for the flyout to open | |||
const rowsWithVulnerabilityId = useMemo( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👀
...urity_posture/public/pages/vulnerabilities/utils/create_detection_rule_from_vulnerability.ts
Show resolved
Hide resolved
...olutions/security/plugins/cloud_security_posture/public/pages/vulnerabilities/utils/index.ts
Outdated
Show resolved
Hide resolved
d2452af
to
3f3bfc2
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approving my concerns, thanks for making the changes. 👍
…tagrid and flyout
…ability to CVE ID column header
update flyout overview tab to support multiple cves, published date move to be above alerts section
…alues update insights tab vulnerabilities column names fix redirect from insights tab to vulnerabilties findings page
712fc14
to
236cfa7
Compare
236cfa7
to
32694ed
Compare
💛 Build succeeded, but was flaky
Failed CI StepsTest Failures
Metrics [docs]Module Count
Public APIs missing comments
Async chunks
History
|
…nerability data grid, flyout and contextual flyout UI (elastic#213039) ## Summary Purpose of this PR is to update UI components which display vulnerabilities in order to support 3rd party integrations and in particularly Qualys, changes include: - New column in Default view in table- "Vulnerability Title" - Renaming "Vulnerability" column to "CVE ID" - Introducing "Title" (Vulnerability Title) and "Subtitle" (CVE ID) in vulnerability flyout while also supporting multiple CVEs. - Update severity handling to be case insensitive across relevant components - data grid, flyout, contextual flyout and insights tab data grid. ### Checklist **Vulnerabilities data grid and Flyout** - [x] Add new column with header 'Vulnerability Title' and the underlying field should be 'vulnerability.title' field. - [x] change existing column with header 'Vulnerability' to 'CVE ID' and underlying field should be 'vulnerability.id'. - [x] aggregation query when grouping by CVE should fetch severities to be case-insensitive - since 3rd party integrations can introduce severities not in UPPER CASE. - [x] Components related to vulnerability severity handle supported severity levels disregarding the case - vulnerabilities flyout and contextual flyout, data grid and insights tab data table. - [x] Multiple CVEs under `vulnerability.id` is a supported case in Vulnerability Findings in Kibana. - [x] missing `vulnerability.id` is handled gracefully in the UI - data grid - [x] missing `vulnerability.id` is handled gracefully in the UI - flyout **Insights tab data grid and contextual Flyout** - [x] Add new column with header 'Vulnerability Title' and the underlying field should be 'vulnerability.title' field. - [x] change existing column with header 'Vulnerability' to 'CVE ID' and underlying field should be 'vulnerability.id'. - [x] fix redirect link when clicking on a vulnerabilty row that redirect to the vulnerabilities tab - make sure search params and filters are working with vulnerability.id array of strings. - [x] contextual flyout should support severities maps to be case insensitive. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) **The following topics will be merged as part of another PR** ### Vulnerabilities data grid and Flyout - arrays in package-related vulnerability fields are rendered correctly in the flyout - arrays in package-related vulnerability fields are rendered correctly in the data grid. ### Insights tab data grid and contextual Flyout - arrays in package-related vulnerability fields are rendered correctly in the contextual flyout - arrays in package-related vulnerability fields are rendered correctly in the insights tab data grid. **Vulnerabilities data grid grouping** ### Grouping by multi-value fields - grouping by CVE should be handled properly in the UI. - each field which might have multiple values - package.name, package.fixed_version and package.version should be flattened by each value separately so user have the max value and can group by each value. ### Identify risks The UI changes in this PR address new integrations which has different structure to vulnerability fields which might cause UI bugs. Tests were modified and added to mitigate the risk of encountering issues. ### Screenshots     Grouping before fix by multiple values  Flatten by each value 
## Summary Purpose of this PR is to handle grouping of multi-value fields which are introduced by Qualys VDMR integrations. This PR adds the capability to flatten grouping results of the following fields - vulnerability.id, package.name, package.version and package.fixed_version, which are all ECS fields. It continues the changes of this [PR](#213039). ### Checklist **The following topics will be merged as part of another PR** ### Vulnerabilities data grid and Flyout - [x] grouping by CVE should be handled properly in the UI. - [x] multi value fields are flattened - each value is counted as separate group key. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) ### Screenshots Grouping before fix by multiple values  Flatten by each value 
…1302) ## Summary This PR backports changes related to Qualys VMDR added integration support that were merged to version 9.1.0. PRs backported: - #213039 - #216411 ### Identify risks Thorough sanity check need to be made before merging since a lot of conflicts were fixed manually. --------- Co-authored-by: kibanamachine <[email protected]> Co-authored-by: Maxim Kholod <[email protected]>
## Summary Purpose of this PR is to handle grouping of multi-value fields which are introduced by Qualys VDMR integrations. This PR adds the capability to flatten grouping results of the following fields - vulnerability.id, package.name, package.version and package.fixed_version, which are all ECS fields. It continues the changes of this [PR](elastic#213039). ### Checklist **The following topics will be merged as part of another PR** ### Vulnerabilities data grid and Flyout - [x] grouping by CVE should be handled properly in the UI. - [x] multi value fields are flattened - each value is counted as separate group key. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) ### Screenshots Grouping before fix by multiple values  Flatten by each value  (cherry picked from commit 1b9bf80) # Conflicts: # src/platform/packages/shared/kbn-grouping/src/components/grouping.tsx # src/platform/packages/shared/kbn-grouping/src/containers/query/index.test.ts # src/platform/packages/shared/kbn-grouping/src/containers/query/index.ts # src/platform/packages/shared/kbn-grouping/src/containers/query/types.ts
Summary
Purpose of this PR is to update UI components which display vulnerabilities in order to support 3rd party integrations and in particularly Qualys, changes include:
Checklist
Vulnerabilities data grid and Flyout
vulnerability.id
is a supported case in Vulnerability Findings in Kibana.vulnerability.id
is handled gracefully in the UI - data gridvulnerability.id
is handled gracefully in the UI - flyoutInsights tab data grid and contextual Flyout
release_note:*
label is applied per the guidelinesThe following topics will be merged as part of another PR
Vulnerabilities data grid and Flyout
Insights tab data grid and contextual Flyout
Vulnerabilities data grid grouping
Grouping by multi-value fields
Identify risks
The UI changes in this PR address new integrations which has different structure to vulnerability fields which might cause UI bugs. Tests were modified and added to mitigate the risk of encountering issues.
Screenshots
Grouping before fix by multiple values

Flatten by each value
