Skip to content

ES|QL: pruning of JOINs leads to missing fields #127467

@luigidellaquila

Description

@luigidellaquila

Probably related to #126017, but the symptoms are different:

With CSV dataset:

from books,message_*,ul* 
| enrich languages_policy on status 
| drop `language_name`, `bytes_out`, `id`, id 
| dissect book_no "%{type}" 
| dissect author.keyword "%{HZicfARaID}" 
| mv_expand `status` 
| sort HZicfARaID, year DESC NULLS LAST, publisher DESC NULLS FIRST, description DESC, type NULLS LAST, message ASC NULLS LAST, title NULLS FIRST, status NULLS LAST 
| enrich languages_policy on book_no 
| grok message "%{WORD:DiLNyZKNDu}" 
| limit 7972
| rename year as language_code 
| lookup join languages_lookup on language_code 
| limit 13966 
| stats  rcyIZnSOb = min(language_code), `ratings` = min(@timestamp), dgDxwMeFYrD = count(`@timestamp`), ifyZfXigqVN = count(*), qTXdrzSpY = min(language_code) by author.keyword
| rename author.keyword as message 
| lookup join message_types_lookup on message 
| stats  `ratings` = count(*) by type 
| stats  `type` = count(type), `ratings` = count(*) 
| keep `ratings`, ratings	
{
    "error": {
        "root_cause": [
            {
                "type": "verification_exception",
                "reason": "Found 1 problem\nline 1:832: Unknown column [type]"
            }
        ],
        "type": "verification_exception",
        "reason": "Found 1 problem\nline 1:832: Unknown column [type]"
    },
    "status": 400
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    :Analytics/ES|QLAKA ESQL>bugTeam:AnalyticsMeta label for analytical engine team (ESQL/Aggs/Geo)medium-riskAn open issue or test failure that is a medium risk to future releases

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions