Skip to content

Agentless: remove organization field in ingest pipeline for Agentless integrations #14172

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 10 commits into from
Jun 16, 2025
Merged

Agentless: remove organization field in ingest pipeline for Agentless integrations #14172

merged 10 commits into from
Jun 16, 2025

Conversation

chemamartinez
Copy link
Contributor

@chemamartinez chemamartinez commented Jun 6, 2025 •
edited
Loading

Proposed commit message

Fixes a potential ingestion error in integrations that support Agentless
because Agentless agents include a global processor to add the
organization field as a string, which collides with the ECS organization field
the integration expects.

This is a temporary workaround that removes the fields added by
Agentless from the ingested documents to avoid collision in integrations 
that populate that fields, as well as type conflict in searchs.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Related issues

@chemamartinez chemamartinez self-assigned this Jun 6, 2025
@chemamartinez chemamartinez added Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] bugfix Pull request that fixes a bug issue Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] Team:obs-ds-hosted-services Observability Hosted Services team [elastic/obs-ds-hosted-services] agentless Agentless related issues labels Jun 6, 2025
@chemamartinez chemamartinez marked this pull request as ready for review June 6, 2025 11:35
@chemamartinez chemamartinez requested review from a team as code owners June 6, 2025 11:35
@elasticmachine
Copy link

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@chemamartinez chemamartinez force-pushed the 14142-agentless-organization-field branch from 9ea574c to 0f1c509 Compare June 6, 2025 12:00
@elastic-vault-github-plugin-prod
Copy link

Package aws - 3.7.1 containing this change is available at https://epr.elastic.co/package/aws/3.7.1/

@elastic-vault-github-plugin-prod
Copy link

Package beyondtrust_pra - 0.2.1 containing this change is available at https://epr.elastic.co/package/beyondtrust_pra/0.2.1/

@elastic-vault-github-plugin-prod
Copy link

Package claroty_xdome - 0.1.1 containing this change is available at https://epr.elastic.co/package/claroty_xdome/0.1.1/

@elastic-vault-github-plugin-prod
Copy link

Package cloud_security_posture - 2.0.0-preview04 containing this change is available at https://epr.elastic.co/package/cloud_security_posture/2.0.0-preview04/

@elastic-vault-github-plugin-prod
Copy link

Package crowdstrike - 1.75.2 containing this change is available at https://epr.elastic.co/package/crowdstrike/1.75.2/

@elastic-vault-github-plugin-prod
Copy link

Package ess_billing - 1.4.2 containing this change is available at https://epr.elastic.co/package/ess_billing/1.4.2/

@elastic-vault-github-plugin-prod
Copy link

Package google_scc - 1.10.1 containing this change is available at https://epr.elastic.co/package/google_scc/1.10.1/

@elastic-vault-github-plugin-prod
Copy link

Package google_secops - 1.1.1 containing this change is available at https://epr.elastic.co/package/google_secops/1.1.1/

@elastic-vault-github-plugin-prod
Copy link

Package google_workspace - 2.41.1 containing this change is available at https://epr.elastic.co/package/google_workspace/2.41.1/

@elastic-vault-github-plugin-prod
Copy link

Package m365_defender - 3.9.1 containing this change is available at https://epr.elastic.co/package/m365_defender/3.9.1/

@elastic-vault-github-plugin-prod
Copy link

Package microsoft_defender_endpoint - 2.38.1 containing this change is available at https://epr.elastic.co/package/microsoft_defender_endpoint/2.38.1/

@elastic-vault-github-plugin-prod
Copy link

Package microsoft_sentinel - 1.1.1 containing this change is available at https://epr.elastic.co/package/microsoft_sentinel/1.1.1/

@elastic-vault-github-plugin-prod
Copy link

Package o365 - 2.18.2 containing this change is available at https://epr.elastic.co/package/o365/2.18.2/

@elastic-vault-github-plugin-prod
Copy link

Package okta - 3.10.1 containing this change is available at https://epr.elastic.co/package/okta/3.10.1/

@elastic-vault-github-plugin-prod
Copy link

Package panw_cortex_xdr - 2.3.1 containing this change is available at https://epr.elastic.co/package/panw_cortex_xdr/2.3.1/

@elastic-vault-github-plugin-prod
Copy link

Package prisma_cloud - 3.2.1 containing this change is available at https://epr.elastic.co/package/prisma_cloud/3.2.1/

@elastic-vault-github-plugin-prod
Copy link

Package proofpoint_itm - 0.1.2 containing this change is available at https://epr.elastic.co/package/proofpoint_itm/0.1.2/

@elastic-vault-github-plugin-prod
Copy link

Package qualys_vmdr - 6.7.2 containing this change is available at https://epr.elastic.co/package/qualys_vmdr/6.7.2/

@elastic-vault-github-plugin-prod
Copy link

Package sentinel_one - 1.35.1 containing this change is available at https://epr.elastic.co/package/sentinel_one/1.35.1/

@elastic-vault-github-plugin-prod
Copy link

Package splunk - 0.3.2 containing this change is available at https://epr.elastic.co/package/splunk/0.3.2/

@elastic-vault-github-plugin-prod
Copy link

Package sublime_security - 1.9.1 containing this change is available at https://epr.elastic.co/package/sublime_security/1.9.1/

@elastic-vault-github-plugin-prod
Copy link

Package tenable_io - 4.0.2 containing this change is available at https://epr.elastic.co/package/tenable_io/4.0.2/

@elastic-vault-github-plugin-prod
Copy link

Package ti_abusech - 2.10.1 containing this change is available at https://epr.elastic.co/package/ti_abusech/2.10.1/

@elastic-vault-github-plugin-prod
Copy link

Package ti_recordedfuture - 2.0.1 containing this change is available at https://epr.elastic.co/package/ti_recordedfuture/2.0.1/

@elastic-vault-github-plugin-prod
Copy link

Package ti_threatq - 1.34.2 containing this change is available at https://epr.elastic.co/package/ti_threatq/1.34.2/

@elastic-vault-github-plugin-prod
Copy link

Package vectra_rux - 0.1.1 containing this change is available at https://epr.elastic.co/package/vectra_rux/0.1.1/

@elastic-vault-github-plugin-prod
Copy link

Package wiz - 3.2.1 containing this change is available at https://epr.elastic.co/package/wiz/3.2.1/

@elastic-vault-github-plugin-prod
Copy link

Package zscaler_zia - 3.12.1 containing this change is available at https://epr.elastic.co/package/zscaler_zia/3.12.1/

This was referenced Jun 17, 2025
Merged
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kcreddy kcreddy kcreddy left review comments

@zmoog zmoog zmoog approved these changes

@seanrathier seanrathier seanrathier approved these changes

@3kt 3kt 3kt approved these changes

@efd6 efd6 efd6 approved these changes

Assignees

@chemamartinez chemamartinez

Labels
agentless Agentless related issues bugfix Pull request that fixes a bug issue Integration:armis Armis Integration:aws AWS Integration:beyondtrust_pra BeyondTrust PRA Integration:claroty_xdome Claroty xDome Integration:cloud_security_posture Security Posture Management Integration:crowdstrike CrowdStrike Integration:ess_billing Elasticsearch Service Billing (Community supported) Integration:google_scc Google Security Command Center Integration:google_secops Google SecOps Integration:google_workspace Google Workspace Integration:m365_defender Microsoft Defender XDR Integration:microsoft_defender_endpoint Microsoft Defender for Endpoint Integration:microsoft_sentinel Microsoft Sentinel Integration:o365 Microsoft Office 365 Integration:okta Okta Integration:panw_cortex_xdr Palo Alto Cortex XDR Integration:prisma_cloud Palo Alto Prisma Cloud Integration:proofpoint_itm Proofpoint ITM Integration:qualys_vmdr Qualys VMDR Integration:sentinel_one SentinelOne Integration:splunk Splunk Integration:sublime_security Sublime Security Integration:tenable_io Tenable Vulnerability Management Integration:ti_abusech AbuseCH Integration:ti_recordedfuture Recorded Future Integration:ti_threatq ThreatQuotient (Partner supported) Integration:vectra_rux Vectra RUX Integration:wiz Wiz Integration:zscaler_zia Zscaler Internet Access Team:Cloud Security Cloud Security team [elastic/cloud-security-posture] Team:obs-ds-hosted-services Observability Hosted Services team [elastic/obs-ds-hosted-services] Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Agentless Integrations] Agentless integrations 'organization' field pipeline failure
9 participants
@chemamartinez @elasticmachine @ishleenk17 @seanrathier @zmoog @kcreddy @3kt @efd6 @andrewkroh